Nate Silver
A pollster that doesn't understand aggregate polls can increase errors and decreases confidence.
~ Aristotle Sabouni
Created: 2017-05-13 |
History
A little backstory: Nate came along and created an aggregate polling system where he sampled other polls (on a state-by-state basis), then put a model on top (a series of biases), and then tried to get a better result than the individual polls were getting. And he tried this during Obama elections. He failed miserably, but not so miserably than Democrats couldn't be duped by it.
What happened is that if you looked at his individual state by state projections, he was off by quite a bit. And if you looked as his rationale for turn-out, it was way off. But if you ignored all the reasoning behind it, and look at his final projections -- he was closer than any of the other major pollsters. Tada, I got closer to the target, I win. And the left went nuts: he's a fricken genius.
The rest were scratching their heads. Being right for the wrong reasons doesn't make you right, it makes you lucky. So while he was closer (for all the wrong reasons), than anyone else, during Barack 2008, and again in 2012. It's way off that he was that good. He missed quite a bit on mid-terms (2010 and 2014).
His model had only been proven (if you call error ridden, but his errors averaged out to close enough on final outcome), in cases where you had a charming first-black outsider President, running against an unlikable veteran or easy to attack Mormon Billionaire. I kept telling my friends that's still a one trick pony, until he tries another trick. How will it hold against an unlikable corrupt crony establishment vote against an slightly more charming loudmouthed bombastic billionaire? We'd have to wait and see.
I also pointed out:
- That aggregate polls can be worse than individual polls, if the average (or many of them) are introducing the same errors/bias. In averaging many polls, you're increasing the odds of errors being introduced, but trying to make up for that by injective diversity (with competing diverse samples to reduce the errors back down). But that only works if there's diversity in the way you're doing the sampling and assumptions made. If they're all making the same assumptions, you just introduced more bias.
- That Democrats had never successfully passed the baton from one Democrat President to another. (JFK-Johnson doesn't count, because JFK caught a bullet. And FDR-Truman was because it was during war and FDR died in office). And all the polls were assuming +11 Democrat turn-out, which is another way of saying that Hillary (the least likable candidate the Democrats put up since Wallace) was going to get the same turn-out as Prince Charming Obama, with minorities and women -- thus she was going to be the exception. I called bullshit on their sampling methods.
- In transition years, especially when the Press is being pro-Democrat/anti-Republican (which is most years), there's less reason for Republicans to take polls, or answer honestly. Democrats are more consensus, so attempts by the Press to energize them or demoralize the base, seems to work better on them ("I'm with the herd"). But on Republicans, attempts to demoralize tends to frustrate and anger, and drive up their turn-out -- and their willingness to avoid pollsters or even lie to them. So Republicans are underpolled in years where the press is overly supportive of the opposition (or overly negative towards theirs). With Reagan the polls were off by up to 11 for these reasons.
Mock, mock, mock. Who was I compared to the great 538? They'd show me when he knocked it out of the park in 2016 and he was predicting a Hillary landslide (and ignore that he said Trump would never win the primary). And splat. Nate failed completely, for exactly the reasons I'd predicted.
NOTE: I never predicted by how much they were off. While I've taken a few classes on it, and I'm pretty smart, I'm not a statistician. It's a lot easier to see different flaws in the gross assumptions they're using to weight the equations, than to know by how much those errors are going to matter in final outcome. I only knew the former. But I couldn't tell if it was going to be enough to swing the election or not: just that they were way undercounting Trump support.
Present[edit | edit source]
Nate's latest act of prestidigitation is trying Svengali his gullible base into thinking he was the least wrong, just because he fell on his face, and was completely and disastrously wrong, with stuff that many of his detractors were claiming:
Now a mature person, that learned from his mistakes, would say something like, "I was disastrously wrong. Here's exactly how and why, and here's what I'm going to do to fix it".
An immature/egocentric one will claim, "Look over there: they were wrong too". Or, "I wasn't wrong by THAT much... and here's some distractions and hand waving to try to make the complete failure look like less of a failure, or even somewhat a success". But that is like the doctor trying to claim in his malpractice suit that while the patient died from sepsis caused by a forgotten sponge, that at least his sutures were nice and evenly spaced so there will be no scarring". Kinda beside the point to his kids, Dr. Nate.
So let's decompose this corpse of excuses:
"I was within the margin of error" or "I was only off by 2 points".[edit | edit source]
That flim-flam is hysterical, in both senses of the word. He does that by claiming, "well national average was only a few points off". But his models weren't based on national averages -- they were based on a State-by-state model (as it should be). How close was he on individual States? Off by up to 16 points. That's nowhere near the predictive margin of error. And he missed on both sides. In the 11 States (and DC) Hillary won, they were off by well above the margin of error, including +5 in California and +20 in Hawaii. And in the midwest/rustbels (which seems to include states like Maine, Florida, Louisiana and Florida), they were off by +16 points. Far more states were outside the margin of error, than inside it. And the only reason it looks like Nate was even close, was because he was so off on BOTH sides, that it averaged out to look like it was closer than it was. This is like throwing 3 darts at a dartboard, and never hitting the board once... then claiming if you average the 3 misses it results in 3 bullseyes.
But in the final polls, over 50 years, the polls are historically off by 2%[edit | edit source]
This is complete flim-flam again. He's playing a game where he's cooking the numbers and averaging out failure. This wasn't an average year -- this was a democrat Transition year. On the last two of those were off by 7.2 and 3.4% because the pollsters don't take the transition factor (public is sick of Democrat excuses) into account. And that reduces the error by another bit of flim-flam, "let's compared the week before the election". Let's not, if you want to look at averages, then look at averages throughout the election process -- where for 90% they're projecting huge Democrat leads (to try to motivate their base), and then only tighten them in the last week so when they're off, they won't be off by as much. If you look at the broader average, they were off by more.
Anyways, instead of looking at Nate's failure as a de-noised (averaged) national percent, let's do it by outcome. What was his prediction or by delegates. In the former, he was predicting Hillary by a 70/30% split. That's a lot more than off by a few percent. And in delegates/states, he was off by hundreds (or almost every key swing state, where he focused his attention). So in the metrics that matter, it was a fucking disaster. Thus, he's trying to change the topic to something else.
Then there's the "surprise" to almost no one that most late breakers broke for Trump.[edit | edit source]
Yeah, undecideds usually don't break for the incumbent, that what makes them undecided. If the known entity can't seal the deal, and the economy isn't that good, the undecideds always break for the change agent and outsider. If the economy is good or at least trending better, they're more likely to break 50/50 or marginally towards the status quo. So this wasn't a shock, "who knew" thing. This was a model failure, that many had warned them about. Own the mistake to convince us that you've learned something.
But Nate predicted Trump had a better chance that the others did?[edit | edit source]
Even his 35% chance thing is bullshit. Like many partisan pollsters he didn't have that for most of the election, and just threw it in the last couple week as political ass covering. So the facts are still that he's saying Hillary was at least twice as likely to win as Trump, still means he was wrong. His little hedge/insurance policy which is more political than mathematical (since he doesn't really publish his models) - still was way off, because Trump won. He's just trying to compare to bigger idiots to make himself look good. But that's like being the smartest kid on the short-bus.
Then, to cook the facts more, he only compares his poll to NYT aggregate (a far left DNC front site). But the NYT is a partisan hack site that was only looked at polls telling them what they wanted to hear. If you look any broader you have:
- The USC/L.A. times poll was much better than Nate's aggregate polling, but not included in NYT or Nate's comparisons. And even that was off 4 points in popular vote, just more dead on in Trump support and where.
- The IBD/TIPP poll, continued their most accurate poll in the last 4 elections, and had nailed this one in declaring it was going to be a lot closer than Nate had. But it mis-predicted Clinton as the winner, and was wrong as well. (Just not as wrong as 538).
- The Republican Trafalgar Group turned out to be the most accurate in some of the states that Trump won that gave him the electoral college win. And it said why (the hidden voter phenomenon). And even it missed on a few states.
So Nate's article plays flim-flam and distract by looking at averages. Instead of being honest and admitting that many individual polls, beat his aggregate model poll, by a lot in percent, and even more in outcome -- especially in key states, which made the difference in the electoral college. Which is all that counts.
Conclusion[edit | edit source]
So the polls are often models (constructs) that have major logic flaws. The most common is the assumption that his year will be a lot like the last. That assumption works fine for re-elections, where unless the economy is going to shit, the people opt for the status quo and little changes. But it's lousy in transition years. It also ignores that no two elections are alike, because people are often voting for personalities and human factors that are well outside their historical norms. McCain had the charm of sandpaper being used as toilet tissue: so his policies weren't going to matter. And Romney was an ultra-white Mormon billionaire that was going to have unusually large problems against a minority -- because minorities are more racist (identity politics), and will break for the person of color. That's unusual compared to elections where one of the candidates wasn't half black.
Thus, you have to take into account all the unusual variables into your models, and what Nate is trying to distract from, is he didn't do that. He didn't pick Obama for the right reason, but he got lucky on his numbers predictions. He said that Trump wouldn't win the primary, and he did. Then he predicted he wouldn't win the election, and almost ever one of Nate's state models were way off (well outside the margin of error), even if he can try to hide that failure with averages that say, "I wasn't off by as much as this other cherry-picked list of worse failures". Nate knows this or he isn't good at what he does -- and if he knows this, then he's intentionally spinning and hiding how bad the failure was.
I think he should stop trying to salvage credibility from that shit show -- and just own his mistake honestly. And explain how he's going to fix it next time. Because if he's trying to spin a failure into "almost a complete success", then he's more interested in ego and immediate reputation than the math and honesty/integrity of what he does.
🔗 More
| |
| |
| |
| |
🔗 Links
- Orange is the new black: Trump 2016
- https://www.currentaffairs.org/2016/12/why-you-should-never-ever-listen-to-nate-silver
Tags: People Polemics TBD FakeNewsmen Other Pollster
