UK Pollsters lie
On this Substack, which I’ve called the “Poll Data Series” I mostly talk about US stuff because… American.
But in my research I’ve also found some interesting things in foreign elections. True, while the US mostly has two-party elections and other countries often have three or more competitive parties, it goes much deeper than that. And it starts with how their poll data is reported.
In simplest terms, US pollsters tell the truth about their data and UK pollsters lie about it. That’s not to say the data UK pollsters collect is bad, it is just as good or better than US data in most cases. But how it’s reported is deeply troublesome, from both a scientific perspective and as it relates to misinforming the public.
US data is reported via what I call the “spread method.” As anyone who follows me knows, spread is a junk metric, but at least the number isn’t derived from a lie, I guess? The bar is low in this field.
Spread Method: The difference between the top two candidates. 44 to 40? +4. 50 to 46? Also +4. The accuracy of polls, according to this method, is detected by how well the “spread” of the result matches the “spread” poll. This method says that even if a poll were provably perfect, unless undecideds split 50/50, it was wrong.
UK data (and most non-US data) is reported via the “proportional method.”
Proportional Method: “allocates” people who say they “don’t know” (or refused to answer) who they will vote for, but are likely to vote, proportionally to those who are decided. It reports this “allocation” as if it were the poll. The accuracy of polls, according to this method, is detected by how well the “vote share” of each major party candidate is predicted.
The proportional method is nothing more than lying about poll data. Pollsters - by tradition - lie and say “this is what our poll said.” No, it wasn’t. It’s what you guessed.
If your poll was accurate, but undecideds split in a way differently from what you assumed (which is both testable and tested) your method would say your poll (not the shitty assumption) was wrong. US pollsters do this too, but by the proportional method’s requirement to lie, there’s no way to detect how many undecideds there are (which directly relates to the range and variability of possible outcomes).
A 40-30 poll with 30% undecided has a much wider range of outcomes than a 50-45 poll with 5% undecided. Analysts and experts claim to understand this, but their work proves they don't.
This “allocation” of undecideds is nothing more than the shittiest possible forecast, and really proves the point that experts in this field have no idea what they’re doing. Them don’t understand the difference between polls and forecasts.
In an effort to make these lying pollcasts more “accurate” some pollsters - with the blessing of experts in the field - instead of allocating proportionally, decided to allocate based on their past vote.
What’s another lie to a liar?
Here's how that, um, “worked”
If you asked someone who they plan to vote for, they say “I don’t know,” but they reported voting for Labour in the last election, the pollster reports that answer as “they said they plan to vote for Labour.”
In a proper scientific field, it would be considered misconduct. No, they didn’t say that, liar.
I’ve conducted research in Exercise Physiology labs, and if I have an athlete hooked up to a machine and it gives me a reading that I deem not reliable or useful, replacing that reading with the one from their most recent test - by all means a perfectly reasonable guess of what the reading “should be” - it would get me literally banned from performing research.
The only reason this “method” hasn’t been banned, is because “we’ve always done it this way.”
In 2015, as in many elections previous, Conservatives had somehow outperformed their pollcast, so pollsters in 2017 started aggressively “aLLoCatinG” undecided voters to favor conservatives.
Here's how the “allocate based on how they voted in the last election” polls came out:
And then, the result.
Whaaat
How could this happen?
All we did was take the undecided voters (which happened to be around 15%) and aLLoCaTe them to who they voted for last time, largely conservatives.
So did they admit their allocations were wrong, when it was effectively proven as such?
Remember, undecided voters were supposed to overwhelmingly favour Conservatives but then ended up largely favouring Labour.
So did the media tell pollsters to stop making assumptions? Nope.
Did pollsters and experts say “our assumptions were really bad”? Also no.
This was the report:
No one can call them on their bullocks because they don't understand that it's bullocks!
Properly calculated, the polls were very accurate (I did the math, it's too much for this post, but I'll share it elsewhere)
But pollsters who utilize the proportional method defend it by saying, “well, don’t know isn’t an option at the ballot box…”
Yeah, no shit.
But guess what?
It’s an option in the poll.
And if someone says they “don’t know” in the poll, and you say they didn’t, that makes you a liar.
Data is under no obligation to be useful to you
Lying about data in a way that makes it appear more useful than it is, is a textbook example of bad science, why this field is an absolute trainwreck, and almost no one respects or trusts it. They shouldn’t. Even people in the field don’t.
The differing “methods”
The whole “public opinion” thing is a big topic to some people, along with “public trust.”
So let me do a quick demonstration why the public shouldn’t trust these liars.
A poll that observes:
Candidate A: 44%
Candidate B: 40%
Candidates C-E: 6%
Undecided/Don’t Know: 10%
Would be reported by the “spread method” as +4. That’s really bad. Simple arithmetic could easily get Candidate B to a victory with just 7% of the undecided vote, even if the poll were perfect. Candidate A would win in an “unexpected landslide” with the same. This overlooks the possibility of a very small error (e.g. Candidate A 43%, Candidate B 41%) leading to an even smaller number of undecideds appearing to “flip” the result…and ignores the partisan lean of Candidates C-E, who could change their minds closer to the election.
These are all variables that can be accounted for, it’s not hard, but people currently working in the field are just too lazy or stupid to do it. If that sounds mean, good, do better math I guess.
Nonetheless, this “+4” reporting in the spread method is not outright lying. Misleading, junk science, but not lying.
Here’s how the same data would be reported in the proportional method:
Candidate A: 49%
Candidate B: 44%
Candidate C-E: 7%
Here, Candidate A has secured nearly 50% of the public’s support (lie) and hilariously would also be reported as “+5” by these pollsters and the media.
The same exact data can be reported with a different “lead” according to your country’s tradition. Very healthy scientific field.
By this method, unless Candidate A eventually receives 49%, and B 44%, the poll was bad.
This example was an easy one that featured a small number of third parties and a moderate number of undecideds, just to illustrate the point.
Here’s a real-world example.
This poll, taken by YouGov, observed about 31% support for Labour. They reported 47%.
Now, accounting for people who say they “would not vote” is plenty sensible.
But allocating people who you determine are likely to vote - based on your best guess - is not. It's lying.
That's how they got from 31% to 47%.
Yes, this poll was and is reported as “Labour has 27-point lead”
Not because the poll said that, but because the assumption did.
UK is heading for another General Election sometime, likely, in 2024.
Things are not getting better there, in terms of poll reporting.
All poll aggregators I could find report the “allocated” lies of the proportional method, giving the unscientific precision to their forecast (which they lie and call the poll) and say there are no undecideds when they are well into the teens.
It also provides a ton more evidence that these people don't know the most basic statistic of the field - the margin of error - because it has nothing to do with their guesses.
For that reason, I'll be starting a “UK poll average” post updating probably once or twice per month. All other poll aggregators lie and report zero undecided, I won't.