Yes, I have thoughts and intuitions beyond what’s reflected in FLIPR. My jumping-off point for these questions are races where we differ from other models, prediction markets, or the conventional wisdom. What are they seeing that I’m missing, or vice versa? Are there other reasonable modeling choices I might have made that would have produced a meaningfully different forecast?
I guess that’s a way of saying “where do I trust FLIPR and where don’t I?” although I don’t think that’s quite the right framing. For one thing, I mostly do trust FLIPR. For another, the whole point of building a model, in some sense, is to set clear rules ahead of time so you don’t have to trust your gut or make ad hoc assumptions in midstream. Now that we’re in “silly season,” it sometimes feels like everyone from the expert ratings agencies to the pollsters themselves are hedging their bets or sometimes herding. FLIPR might be wrong, but it gives you an independent opinion every day, based on rules we locked into place months ago. Still, I keep a mental list each cycle of things we might change in our models the next time around.
Where do FLIPR and prediction markets differ the most?
The differences between FLIPR and prediction markets are internally consistent. In the Senate, there are roughly a half-dozen races where we’re modestly or materially more bullish on Democrats’ chances, although also 1 or 2 where the difference runs in the other direction and we have better odds than Polymarket does for Republicans.
The following is a rather technical way to think about it. But basically, if you plugged Polymarket’s forecasts for individual races into FLIPR, and relied on FLIPR to model the correlations between races and estimate Democrats’ overall win probability, my guess is that it would replicate Polymarket’s topline number pretty well, and produce a 65-70 percent chance of Democrats winning the Senate instead of FLIPR’s actual 75 percent. In that sense, this isn’t really a macro-level disagreement about the intrinsic amount of uncertainty in the forecast. Rather, the disagreements boil down to individual races.
FLIPR vs Prediction Markets
Silver discussed every race. In the clip above I grabbed all of the ones I have discussed and a few more.
The biggest differences between Silver and the prediction markets are Iowa, Kansas, Alaska, Michigan, Maine, and Texas (in that order, of the states I have frequently discussed).
Silver assess where he personally feels which side he might take. This is interesting because I have had pretty big disagreements with Silver on some of the races, especially Iowa, Maine, and Alaska.
Silver Self Assesses FLIPR
Michigan Yeah, prediction markets are probably reasonable here. I say that because FLIPR doesn’t really have any mechanism to account for El-Sayed’s progressivism, even though I think it would be reasonable to do so. In fact, in Michigan, we have the fundamentals being slightly better for El-Sayed than his polls in part because of his prodigious fundraising. If we accounted for his ideology rating within the fundamentals, I imagine we’d be pretty close to Polymarket. Who knows if there could also be some sort of ethnic or religious bias against him too; no Muslim-American has ever been elected to the U.S. Senate.
Ohio FLIPR is a hair higher on Sherrod Brown than prediction markets. But look, this is a newsletter where I promised some gut-checks. And I actually wonder if FLIPR is too low on Brown.
Maine More straightforward than Maine, for instance, which is ground zero for the question of whether polling errors are persistent, since Collins has repeatedly beaten her polls by a lot. According to FLIPR, Troy Jackson is slightly ahead in the polls, way ahead in “simple fundamentals” which account for Maine’s blue lean in presidential races — but a slight underdog in “fancy fundamentals” that account for Collins’s strong re-election margins in the past.
I think FLIPR has some fairly smart heuristics here, including accounting for the fact that uncertainty is high in this race. But it’s not a race that Democrats should feel safe about, and personally I might be closer to Polymarket’s 64 percent chances for Jackson than FLIPR’s 70.5 percent. I also expect Maine to be less correlated with national outcomes than most other states, i.e. I can imagine Jackson winning in a disappointing night for Democrats or Collins winning in an otherwise good night for Dems.
Texas The streams have somewhat converged here. Ken Paxton has gained a point or so in polls, and FLIPR’s odds for James Talarico have declined slightly as a result — but Talarico’s chances have held steady or slightly improved in prediction markets, possibly because of traders reading tea-leaves on early-voting data. It does feel like there’s a bit more herding here in the polls than in some of other Senate races, though I’m not sure who that favors, exactly.
We’ve written about Texas a lot. It’s a flip that ranks high on “unthinkability” because of Texas’s symbolic importance and the fact that Democrats haven’t won a Senate race there since 1988. While we’re slightly higher on Talarico than the markets, I’m surprised the gap isn’t larger.
Still, the math showing Talarico as a modest favorite checks out.
To be fair, forecasting a Democrat to be favored in Texas is not exactly a forecast that one sleeps easily with, but I’m not sure if that’s an analytical reaction or an emotional one.
Alaska This is the one race where FLIPR is notably lower on Democrats’ chances than markets are. That strikes me as important in terms of the overall Senate equation because Alaska, up in the Arctic Circle, is one of the least correlated states with the Lower 48. You can imagine Democrats having a somewhat disappointing night but Mary Peltola winning, or maybe the reverse.
What accounts for the difference? It might partly be FLIPR’s heuristics on ranked-choice voting. We put a fair amount of work into these, and that’s the sort of detail that gut impressions might ignore. Basically, the story would be that people are looking at the topline difference between Peltola and the “real” Dan Sullivan, where she’s ahead by a couple of points. But if Peltola doesn’t hit 50 percent in the first round and Sullivan picks up votes from the “fake” Dan Sullivan and the other Republican on the ballot, he could gain a point or so under ranked-choice voting reallocations. For what it’s worth, FLIPR has been extremely bullish on Democrats’ chances in the Alaska gubernatorial race as compared with the consensus in another race that reflects our RCV model. So it seems to have some strong opinions about Alaska.
The argument that we’re underrating Peltola is decent too, though.I’ll cop out by saying I think FLIPR is a bit too low on Peltola because we aren’t really sufficiently accounting for candidate quality — while prediction markets might be a bit too high if they aren’t thinking enough about RCV.
Iowa and Kansas These are the two states where the prediction markets are most bullish on GOP chances relative to FLIPR. My topline takeaway is that I think that’s probably reasonable, and these are states where I worry about FLIPR’s forecast a bit.
But despite their geographic and demographic proximity — which implies the possibility of correlated polling error between them — Iowa and Kansas are actually rather different cases. “Simple fundamentals” pegs Iowa as merely an R +1 state, which isn’t much of an edge, obviously. But Ashley Hinson is one of the stronger Republican candidates on the map, even if Josh Turek is a strong candidate too. In Texas, Alaska, and Ohio, all of which have similar partisan lean scores, Democrats have a much clearer candidate quality advantage, I’d argue. In Iowa, they’re sitting a bit more naked in the event of a systematic overestimation of Democrats in red states. And of course, there was infamously a big polling error in Iowa in 2024.
My subjective view is that this is about as close to a pure 50/50 toss-up as it gets, so I suppose I’m once again splitting the difference between FLIPR and prediction markets.
Kansas is redder than Iowa, having voted for Trump by 16 points in 2024 as compared to 13 in the Hawkeye State. Our blended fundamentals calculation, even accounting for the blue national environment, has Kansas at R +4.4, so Adam Hamilton would have to outperform by several points. And Kansas hasn’t elected a Democrat to the Senate since 1932! And yet, polls aside, there are some silver linings here for Democrats. Sharice Davids has repeatedly won in the blueish KS-3 district centered around well-educated Kansas City suburbs. Laura Kelly and Kathleen Sebelius won statewide in gubernatorial races, and Democrats absolutely crushed it in the abortion referendum in the state in 2022. And it’s trending a bit blue relative to the national baseline: Hillary Clinton lost it by 21 points in 2016 compared to Harris’s 16.
Furthermore, Roger Marshall is kind of the canonical example of a candidate with significant vulnerabilities who might not have been prepared for a blue wave. But, we’re suffering from a lack of quality polling in Kansas as compared to most of the other key races. Yet again, I’ll boringly suggest that the truth is probably somewhere in between FLIPR and Polymarket.
Mish Ohio October 24, 2026
Mish Link: Silver weights the Fox News Poll (Beacon/Shaw) heavily as he should.
But he underweights the Cook Political poll. And he overweights other polls going back to August.
Silver says Brown leads by 3.7 points. I suggest 5.0.
Importantly, in the last 17 polls there is only a single one that shows Husted in the lead.
Ohio Senate Democrat Odds
Polymarket: 69 percent
Split Ticket: 71 percent
Nate Silver Deluxe: 72.6 percent
Nate Silver Classic: 70.2
Nate Silver Lite: 76.0 percent
Mish: 80 percent
Ohio Synopsis. Silver now says “And I actually wonder if FLIPR is too low on Brown.“.
That is precisely what I concluded. Ohio is the only race where I am higher than Silver and the betting markets.
In the same post, I commented on several other states.
I see the three most likely pickup states are North Carolina, Ohio, and Alaska, in that order. Democrats need just one more state to win the Senate if they hold Michigan.
I have Texas more likely than Maine now. The Fox News poll has Collins (R) and Jackson (D) tied at 49 percent each. Trafalgar also has the race tied.
I have been far more cautious on Maine than Nate Silver for months. The Fox News poll is another warning.
I put Democrat pickup odds of Maine no better than a 55 percent tilt, and Texas for Democrat Talarico at 62 percent.
If Republicans hold Iowa and Kansas (certainly not a given), Democrats need either Texas or Maine.
Adding things up, the overall odds of Democrats taking the Senate are about a 65 percent, no change for me.
Will Kansas and Iowa Go the Same Direction?
My seat-of-the-pants idea is Kansas is a bit more likely than Iowa to flip (say 55 Kansas and 50 Iowa).
The betting markets do not agree, so this is a Mish audible. We will see.
What the Hell Is Nate Silver Doing with His House Effects?
Silver Deluxe gives Democrats a 70.9 percent chance of winning Texas
I am not going to go through six pages of polls to see how far off that posted 2.5 percent is from an actual mathematical calculation
I agree with the betting markets on Alaska, South Carolina, and Texas. I agree with Silver Deluxe on Kansas, and have Iowa a genuine tossup (about the midpoint of Polymarket and Silver).
For about a month I have had Democrat odds of winning the Senate at 65 percent.
In this election, my seats of the pants analysis got to a Blue Wave forecast and a 65 percent Democrat winning the Senate expectation faster than Silver or the betting markets.
Today, Silver Deluxe gives Democrats a 76.1 percent chance of winning the Senate. But how much does his house edge play into that? We have no idea how to parse it by pollster.
Silver, after being 10 points behind me most of the year is now 10 points ahead of me on Democrat odds of winning the Senate.
Note that on Iowa I commented that “Iowa a genuine tossup (about the midpoint of Polymarket and Silver).“
Silver’s audible was “My subjective view is that this is about as close to a pure 50/50 toss-up as it gets, so I suppose I’m once again splitting the difference between FLIPR and prediction markets.“
The betting markets have picked up on this. Mainstream media hasn’t.
Kalshi has Mary Peltoa at 68 percent, and has been above 60 percent most of the year.
Nate Silver Deluxe Model Alaska
Nate Silver’s deluxe model has Peltola the underdog. I don’t buy it.
I suspect his model thinks Alaska is a red state so it stacked the deck for Sullivan.
Silver’s Classic model also has Sullivan the favorite. However, his polls-only Lite model has a Peltola a 57.1 percent favorite.
I think Silver’s models have messed up here.
Peltola has an excellent slogan, sounds like a Republican on fishing and energy, and has a big advantage in Alaska’s ranked voting system.
In Alaska’s ranked voting system, three Republicans are running, two of them have the same name, There are two Dan Sullivan candidates on the ballot, one of them is the incumbent Senator.
Putting all of this together, I am about 65 percent likely, at the moment, that Democrats take the House and Senate.
That estimate is a blend of Polymarket, Kalshi, and Nate Silver (but adjusting Silver for Alaska).
Silver is on the right side of Alaska now, but still heavily underweights Peltola. I have consistently sided with Polyymarket and my key reason was candidate quality.
But I also thought Peltola would get a bit of a pull from the governor’s race which everyone thinks will go Democrat.
Here’s what Silver says today. “The argument that we’re underrating Peltola is decent too, though.I’ll cop out by saying I think FLIPR is a bit too low on Peltola because we aren’t really sufficiently accounting for candidate quality.”
Bingo. Nor did Silver address Peltola’s excellent campaign slogans.
Who Is Favored to Win the Senate in 6 Different Models
Here are 10 Senate races in 6 models, one of them mine.
I look at everything (polls and betting markets) and I factor in where I think the economy inflation and sentiment is headed.
I also look at sentiment, Cook Political, individual poll cross tabs, and momentum.
I weigh recent high quality polls heavily, and toss stale polls much more than Nate Silver.
In the end, it’s just a mental assessment of everything above, but not an average. If I have a reason I will bend towards one of the above models.
I was the first of anyone above to get to a 60 percent then 65 percent chance of Democrats winning the Senate.
But I am reluctant to go much higher than 65 percent because of betting markets on Iowa and Maine.
Yet, the single most likely event is Democrats pick up a net of 6 Senate seats.
I will go with SplitTicket at 75 percent on Michigan and my assessment of 85 percent for Minnesota.
Alaska Senate
Nate Silver Deluxe: 63.0
Nate Silver Classic: 58.6
Nate Silver Lite: 65.8
Polymarket: 74.0
SplitTicket: 67.0
Mish: 74.0
The betting markets were well ahead of Silver all year and still are vs Silver Classic. I have followed Democrat Mary Peltola’s campaign and concluded early on she would win.
It appears as if Nate Silver’s models view Peltola as a Democrat in a Republican state. I view Peltola as an Alaskan running for Alaska, with a strong message.
I side with Polymarket at 74 percent.
Democrat Sherrod Brown is running in a Republican state. But Brown is a former US Senator and Ohio has issues (diesel in a farm state, and data centers that are very unpopular).
Local anger over energy bills and lost farmland hits two key battleground races.
In Ohio, I don’t discount the betting markets entirely, but counting on Ohio acting like a solid red state is now wrong.
Maine Senate
Nate Silver Deluxe: 71.8
Nate Silver Classic: 68.3
Nate Silver Lite: 69.6
Polymarket: 56.0
SplitTicket: 63.0
Mish: 58.0
I am strongly influenced by Polymarket and history.
Historically, Republican Sue Collins normally outperforms her polls in elections. And she does not have the corruption baggage of Paxon in Texas. Finally, Democrats have a very weak candidate in Troy Jackson.
The most recent quality poll is Suffolk University at Jackson +2 on October 8-11, released today.
49%Troy Jackson (D)
47%Susan Collins (R)
4% undecided
That’s easily within the margin of error. So, I am an uneasy 58 percent for Jackson.
Iowa Senate
Nate Silver Deluxe: 58.5
Nate Silver Classic: 54.9
Nate Silver Lite: 59.0
Polymarket: 42.0
SplitTicket: 50.0
Mish: 52.0
Iowa is the first big divergence between the betting markets (42.0) and Nate Silver Deluxe (58.5). That’s a massive 16.5 percentage point differential.
I asked Nate Silver for an explanation. He did not answer. I wonder if he ever answers anything. I am batting something like zero out of ten responses from Silver.
This is a genuine tossup in my opinion.
Nebraska Senate
Nate Silver Deluxe: 30.2
Nate Silver Classic: 30.5
Nate Silver Lite: 49.2
Polymarket: 28.0
SplitTicket: 28.0
Mish: 32.0
Nebraska would be a huge upset. I don’t see it and nor does anyone else. However, a roughly one-in-three shot, or even a one-in-four shot is not nothing.
Republicans are spending money defending the state, something no one thought feasible.
This also serves to balance out the one-in-four shot Republicans have in Michigan.
Kansas Senate
Nate Silver Deluxe: 50.2
Nate Silver Classic: 48.2
Nate Silver Lite: 58.9
Polymarket: 38.0
SplitTicket: 42.0
Mish: 52.0
Kansas and Alaska have something very much in common. Both have non-traditional Democrats running.
Nate Silver on Maine.
“It’s not a race that Democrats should feel safe about, and personally I might be closer to Polymarket’s 64 percent chances for Jackson than FLIPR’s 70.5 percent.“
Silver now reassesses Kansas as “I’ll boringly suggest that the truth is probably somewhere in between FLIPR and Polymarket.“
FLIPR on Kansas is currently is Democrats 50.8. Polymarket is 38 percent.
I think FLIPR is correct. My reason is the same as Alaska. Democrats have a solid candidate who is a popular minister of one of the biggest congregations in the US. He is anti-woke and not a traditional Democrat.
Those are all the same reasons I went with Peltola in Alaska.
Given the dearth of polls I see little reason to change my assessment.
Synopsis
Nate Silver called audibles in the same direction and mostly for the same reasons I did ahead of him on Ohio, Alaska, Maine, and Iowa.
I feel pretty good about that. Silver also shifted Kansas but I didn’t. I am now 10 or so points higher on Kansas vs his Deluxe model but still at tossup level.
My overall assessment is 65-70 percent on Democrats taking the Senate. And that fits with the betting market and Silver’s statement ” But basically, if you plugged Polymarket’s forecasts for individual races into FLIPR, and relied on FLIPR to model the correlations between races and estimate Democrats’ overall win probability, my guess is that it would replicate Polymarket’s topline number pretty well, and produce a 65-70 percent chance of Democrats winning the Senate instead of FLIPR’s actual 75 percent.”
Despite my change on Ohio and agreement with Polymarket on Alaska, I am still at 65-70 percent. That’s because of Iowa and Maine, my two big concerns for months.
That said, skew is not uniform. The odds of Democrats picking up six seats is more likely than Democrats picking up only three.
Mish Fearless House Forecast: Democrats 239 vs Cook Political 227-237
Polymarket has a 82% chance that there will no resumption of attacks on Iran before the midterms. Will Bibi and the president make a bundle taking the 18% odds?
peelo
18 hours ago
How much skin does Nate have in that game? His game, I submit, is a different one: clicks: garnering attention.
peelo
18 hours ago
So I build a model with 75 factors of various fuzziness each. I assign weights to each. Push a button and the output streams out.
To me the one real question is, am I willing to bet a significant amount of my net worth on my claimed model’s predictions? Same for stocks, or the future success of mortgage bonds. My answer to all these is no. If the answer is yes, couldn’t I, or a Nate Silver, expect to make a bundle in prediction markets bets right now?
Now let’s ask a different question: can I make this model and discussion into engaging output in an attention economy? I would wonder if that different question, i.e., what is the infotainment value, is the one really being asked by Silver, et al. How many clicks? That’s where his bread is actually buttered.
i was born, raised and reared on wall street bullshit. the analyst price targets are just marketing tools to bring in the investment banking business and to make the institutional and retail investors think the wall st houses know something. no offense to mish, of course, but everyone here knows his game is click bait or ego or both. or maybe to sell his artwork. nothing wrong with any of that. we call this capitalism. if you all want to treat yourself to the how and why this all came about watch the free 4 part documentary “century of self on youtube.”
You’re being unnecessarily rough on Mish. His traffic dipped due to his straightforward take on Trump, which hurt his readership numbers.
You think America is full of a-holes, and it’s a wrong take and colors your perspective.
Somehow, you prefer to live here, own property, rentals, have family here, and apparently take a deep interest in the country.
But Mish is not an a-hole, does a public service via his blog, and many of us don’t like it when he’s unfairly impugned.
He has more integrity in his little finger than most in their whole body.
yippee
18 hours ago
betting odds really got the brazil election very wrong. i’ve been punting on politics since the 1980s with the london bookies and now with the past few decades the prediction market makers online………i sense the election outcome next week might be a shock.
Garbage in, garbage out. But it’s fun to take a random walk now and then.
Last edited 12 hours ago by ‘Lil Mr.
Quatloo
20 hours ago
Meanwhile, Israel’s election is Tuesday. Netanyahu has been trying to start a war with Iran before voting (in-person voting is generally required). The head of the IDF refused to carry out Netanyahu’s orders designed to provoke an Iranian attack on Israel.
Netanyahu’s goal is to provoke Iran to attack Israel before or during the election, so that he can suspend the election and remain in power indefinitely, or until he is certain he will win the next election
The Window Cleaner
22 hours ago
Excellent exegesis of the election as always Mish, however the following statement ironically describes the problems with economic “theory” today on both sides of the aisle:
“the whole point of building a model, in some sense, is to set clear rules ahead of time so you don’t have to trust your gut or make ad hoc assumptions in midstream.”
IOW capitalist and socialist theorists are caught in an unresolving duality due to their theoretics not creating known and enforceable barriers but won’t/can’t breakthrough to a resolving thirdness due to rigid orthodoxy and failing to find the key factor that keeps the present chaos in force.
The new monetary paradigm of Strategic Monetary Gifting that breaks up Finance’s monopoly paradigm of Debt Only and the 50% Discount/Rebate policy at the various points of retail sale out does the profitability of capitalism and out economic democracy’s socialism.
wisdomicsblog.com
KPStaufen
22 hours ago
The MAGA and America First branding worked pretty well for Trump’s first term and for the 2024 Presidential election, but the shine is off that brand for those independents, Hispanic, and Gen Z young men who carried Trump over the top in 2024. These voters are not ideologically conservative, they were voting for change and the reinvented history of the first Trump term that they may not have been paying too much attention to in real time. Now two years in they have not seen the results that they were promised and they are ready to bet on another horse. They now see through Trump’s gaslighting and grift. This is also happening on a more global scale with Trump-like politicians losing power in Hungary and Serbia of late, and the Trump aligned candidate in Brazil losing his bid for the Presidency over the weekend. World leaders are also, of late, rebuffing Trump’s bluster. One last gasp of hope for the autocrat-loving hard right movement globally is the idea of Russia, China and the U.S. nominating from the UN Security Council, ousted Hungarian strongman Victor Orban for the open Secretary General position. It is possible for the UK or France to veto that nomination, but that veto would take a lot of political intestinal fortitude that will likely only come from a weakened Trump following the U.S. mid-term elections. If somehow the Republicans maintain majorities in Congress, expect Victor Orban to be the next Secretary General of the UN. If there is a blue wave next Tuesday and Trump is significantly weakened in the eyes of the world, expect that Orban either is not nominated or his nomination being vetoed by France and the UK.
strongGnu
22 hours ago
NY Times gets caught publishing fake polls so how are we supposed to beleive any of these polls. The infamous Iowa polls of Hilary, Biden and Kamala should tell you everything. The 10 point swing should tell you everything. This is an attempt to suppress voting and support the theme that there is no fraud. Polls are sensalism.
It is ironic that if you ask the pollster for an audit trail with processes, procudure and written documentation; they fail miserably. No poll can be taken seriously without sample size, collection method, age, race, party ID, non respondents and likely flaws taken into account.
Even with the best pollsters actual results can be double the percentage of real world numbers. Example: 48% to 52% with a +/- 4% errror rate is a tie. Real result could be 44% to 56%. Ask yourself – what good are these?
“Sensalism” is a new one. I take it you never managed to catch an English or a statistics class?
randocalrissian
22 hours ago
My favorite incarnation of Nate Silver is the one everyone ignores. Moving on… .
‘Lil Mr.
22 hours ago
That’s a hell of a lot of analysis. It will be interesting to see how the models work. I would simply plug in the specific races into Ai and see what comes up. You might have to pose the same question several times to the same Ai and see what consensus comes up. These are of course all odds, not certainties, as many people would like to believe.
Something tells me Kalshi and Polymarket are liberally and youthfully biased as I think these are the types of people using them. I also think that when a race tips in one direction or another that FOMO may come into play exaggerating the odds. I also wonder if “Trump” voters are playing it close to the vest as in previous elections. One little old white lady in Ohio commented that we shouldn’t throw out the baby with the bath water, voting for Ramaswamy. Perhaps the we shouldn’t overlook the power of one mother speaking to another no matter how vile the rhetoric.
No one should be above doing a little research. It might be able to breakdown each factor incorporated into all the different models. Then breakdown the reliability of those factors. Maybe do some back testing. Maybe look at the common factors the models share. Find some patterns between the models and have an Ai explain them. Mish has done quite a lot here, but there’s always room for more creativity with your queries and spotting patterns.
I asked Ai about when to buy a mining stock recently and I ignored the analysis. I wish I had not. You may say that’s pure luck but one thing LLMs are good at is evaluating stochastic events. Elections and stock prices are exactly these. On rare occasions there are massive shocks due to unforeseen events, but on the whole, random walks lead to inevitable outcomes just as they do in thermodynamics.
But of course with any model it’s garbage in, garbage out. I foresee a day when Ai can be used to select likely voters, call them, ask questions, and evaluate the likelihood of truthfulness of the person based on the interaction. Then aggregate the answers into its black box and come out with a pretty good estimate. I’m certainly not sure when that may happen, but the ability will emerge.I think you underestimate the pattern recognition ability of Ai in general. Pattern recognition is just the first step in actual thinking. Sure it’s spitting out some junk now and again, but it ain’t all junk. It’s up you the inquirer to make that decision.
totally agree. when i inquire about historical random questions on different houses and blocks i’ve lived on in my first 66 years the initial feedback might be a bit wrong but after i clarify the facts i know it comes back to verify that is so…….and spits out even more insightful things. it’s amazing. i use it for trading now too. very helpful.
The only problem with that is that it has a habit of lying often and doing so boldly as it can for it knows not the truth. It cannot tell you each factor, it cannot tell you the reliability of each factor. At best, it can point you in the direction of doing the work yourself. That’s good, but it’s far from a cigar and current AI tools can’t be trusted to do even that.
The problem with AI consistently goes back to the fact that (inconsistent) pattern recognition is a leaf in the cigar, but the people pushing AI would like it to be the whole thing and fold itself to boot, all while still being cost effective. This will never happen with LLMs and no amount of fantasizing about them being better than just a good tool can make it happen.
AI is a fine tool, but you’ll be hard pressed to find people who can use it. I think the best example is that it’s a power drill idiots keep trying to use as a hammer. They bet their livelihoods on it eventually working as a hammer because they desperately need it to be one for the circular economy around it to make any sense. This will not make it a hammer, but they really only need people to think it is one long enough to run out the back door with a few bags of cash.
It’s as if the Ai has a personality of its own. Perhaps I should ask more smart ass questions to see where it goes? Just bust out with, “YOU LYING SCUM. Why do you do this to me! Every answer is the same. Why do you hate me!” LOL. Next thing you know my computer gets locked up until I can come up with some hefty Bitcoin.
Ai needs to be isolated somehow to well defined problems. Things like scientific research. It really needs to be in the hands of experts who understand the foundational knowledge. Yeah, Ai isn’t for everyone unless you’re looking for basic educational stuff. Which I have found excellent for getting up to speed on some subjects. Kind of like Wiki, but you don’t have to hunt through all the pages and can drill down on the specific topics you’re interested in.
my best use for AI has been going down memory lane. asking AI if it knows about the teen hangouts in ny area during the 1970s. i have been laughing my balls off on the lists and comments………also great to ask r/e and history. questions.
whirlaway
23 hours ago
There are 18 references to “polls” and 31 references to different election betting sites in this article.
I’m shocked Mish isn’t using fiftyplusone, which has a much clearer methodology than Nate and lists every single poll along with the house weights they use for pollsters.
Worth noting that the all knowing prediction markets called the election in Brazil completely wrong. I don’t understand the emphasis on them given their track record is generally not great.
Given the vast sums spent to influence elections and the fact that the prediction markets are themselves being treated as news worth following, to what degree are the betting markets being manipulated to create news?
Using prediction markets is just not willing to admit to having no idea.
Stu
1 day ago
I have never trusted Nate’s polls or opinion. I am far from being along on this take. Many like myself don’t trust his spin, take or whatever he calls it. It’s not a poll of choice for sure, and is sort of a last minute see it was right, or crickets…
In 2008, Nate Silver was all the rage, but only because nobody
had ever brought any sort of statistical modeling of elections to the forefront for the average person to read and see. Yes, political scientists in the halls of the University of Michigan, Columbia University, and the University of Iowa had been making quite interesting forecasting models for years. However, the media-savvy Silver was able to bring attention to his model, truly making him the “first” in this “forecasting” genre. He knew the power of understanding blogging, and rode in on that fame. Unfortunately, tenured professors of great stature don’t have an inkling for media, nor do they need to.
Of course, since his 2008 model (which, honestly, predicted one of the most predictable elections in since 1980), most of his models have been spotty at best, and flat out wrong many times. For example, in 2018, FiveThirtyEight said that Democrat Andrew Gillum would win Florida’s gubernatorial election, with some odds being as high has 3/4. However, Republican Ron DeSantis won the election. Admittedly, one model did get the results in Florida correct…mine. Yeah, I’ll toot my own horn here.
Subscribe
But still, why is Nate Silver’s model meh? And, possibly more importantly, why are people misinformed when it comes to his so-called “forecasting model”. Let’s delve into it.
First of all, the biggest mislabeling is that Silver’s model is a “forecasting” model. That’s simply not true. In fact, it’s a “nowcasting” model, something that I do as well. And since polling only gives us a snapshot of a certain point in time, Silver’s model today only gives us what the results would be if the election were held today. Therefore, any of FiveThirtyEight’s projections are essentially useless until Election Day, and doesn’t really predict anything, as the results change on a daily basis. To give credit, FiveThirtyEight does put this disclaimer in their website.
Of course, nowcasting models aren’t bad. My highly-accurate model of Florida is a nowcast. However, the data being used makes all the difference. In my model, I use voter turnout. With voter turnout, you can get an idea of certain trends, such as who has yet to vote. With polling data, you have no idea how the polls will move over the course of a month, a week, or even a day. Simply, polls aren’t a tool of predictive behavior in the future.
Subscribe
With that, let’s look at what FiveThirtyEight uses, which are polls. Of course, there are a lot of problems when it comes to using polls as a predictive tool. First of all, how do you know that the polling firm is correct? The averaging of the polls tries to mitigate this factor, but it’s still a factor nonetheless. Simple averaging polls doesn’t account of outliers that might skew the aggregate (such as the dreaded Rasmussen Poll).
Another problem that FiveThirtyEight has (and possibly Silver’s new model, but I don’t know), is the polling “rating”, and how they grade polls from best to worst. According to their website, they account for accuracy, stating
Accuracy, as measured by the average error and average bias of a pollster’s polls. We quantify error by calculating how close a pollster’s surveys land to actual election results, adjusting for how difficult each contest is to poll. Bias is error that accounts for whether a pollster systematically overestimates Republicans or Democrats.
Of course, the obvious problem here is that “accuracy” can only be tested on Election Day, not before. This means that a poll only needs to be “accurate” once, in their last poll before Election Day. Any polls that are done before that can’t be taken into account because there is no other data to compare the polls with (because we don’t have elections every day), which makes “accuracy” a mute point.
Also, when do those polls need to be “accurate”? Let’s say, in a hypothetical election, a Democratic candidate wins by 2% in a general election. Two months before, in September, “Polling Firm A” only did one poll, and nailed the election results with a 2% margin for the Democrat. However, at the same time (in September), all other polls had the Republican candidate winning by around 6%. Obviously, the political climate in September was vastly different than what would happen on Election Day. Still, is “Polling Firm A” considered accurate? Of course, we don’t know because the election was held two months later. Therefore, it’s extremely hard to tell if “Polling Firm A” firm is “accurate”.
When it comes to forecasting models (which we already said that Silver’s/538’s isn’t), Dr. Michael Lewis-Beck set out four criteria as to what makes a good forecast: accuracy, lead, parsimony, and reproductibility. For Silver/538, they really don’t have any of these. So what are these?
Lead – How far in the future can an election outcome be predicted?
Accuracy – How accurate is the model?
Parsimony – How few variables are needed to make a prediction?
Reproductibility – Can the model be used from election to election?
Lead – We already mentioned how Silver’s model violates the “lead” rule, as it is a nowcast and not a forecast.
Accuracy – When it comes to accuracy, that can be debated. Since they use a “win ratio” instead of an outright winner, FiveThirtyEight gives itself little wiggle room to still claim victory, since they don’t claim absolute predictions with the exception of the safest of seats. For example, in Florida, FiveThirtyEight said that Joe Biden had a 69 out of 100 chance of winning the state. But he didn’t win the state. Still, because they aren’t claiming an outright winner, they can fall back on the “we were close” excuse instead of the “we were wrong” reality.
Oh, by the way, my 2020 Florida nowcast nailed it.
Just a side note, the Silver model becomes even more chaotic when looking at other countries and projecting seats in a parliamentary system. Let’s take Eric Grenier’s 2015 model of the Canadian election. Gernier’s poll-tracker, based on Nate Silver’s model, was extremely inaccurate. However, because he probably used standard deviations in determining seat totals, he could claim a slight victory. For example, he said that the Conservative Party of Canada would win 122 seats, with the low watermark being 99 seats and the high watermark being 139 seats. A 40-seat swing is 12% of the entire Canadian Parliament! Still, the Tories won 99 seats, so he was “technically” correct. As for the seat projections for the Liberals and the NDP, he wasn’t even within the standard deviation margins.
Parsimony – This is where the 538 (sorry, I’m tired of typing up FiveThirtyEight all the time) really falls apart. The idea is basically that the fewer variables that you have, the better the model is. Why? Because once you start adding more variables, you come to the point that you are just making descriptions of observations and not predicting anything.
Let me give you an example. Let’s say I’m thinking of a car and I give you hints. If I just tell you “I’m thinking of a car”, that too broad. However, if I say that “the car was made in the 1970s”, that narrows it down. If I further say that “this car is no longer in production”, that narrows it down more. If I say that “this car had major problems with its fuel tank”, now you know it’s probably a Ford Pinto, which it is. That last variable basically just confirmed the choice, and also become a variable only applicable in this particular situation.
In the case of 538, they do the same thing. Their “economic conditions” variable has an additional number of variables in it. Essentially, any time their model misses the point, they just add more variables to make the model work correctly. Yes, a model can be tweaked, but shouldn’t go through complete overhauls.
Also, when 538 adds more variables, why do they pick these specific ones? It seems like many of their variables are arbitrary based on correlation and not based on casual factors. In forecasting, one assumes that a voter is or isn’t voting a certain way because there are casual factors pushing voters that way. In my nowcast, my assumption is that if a voter is registered with a political party, they are likely to vote for that party in the general election. The 538 and Silver models have nothing regarding causality.
Reproductibility – I already hit on this when talking about parsimony. The more and more variables that are added election after election, it no longer becomes reproducable.
Speaking of the variables, and the 538/Silver models in general, the lack of causality makes them hard to swallow. Neither of these models say X leads to vote choice. Again, they just pick variables that just happen to go with the result.
Why is this important? Well, let’s say one of our variables is that Republicans win the general election every time that a full moon happens on July 15th on the election year (which I just made up). That isn’t causation, but just simply correlation. Same thing can be said with the age of Miss America and the number of murders by steam, hot vapors, and hot objects (shown in the graph below). While 538 might not go to this extreme, they aren’t far off. Their models real don’t show anything regarding causation whatsoever.
Now, of course, you can say that if we are just looking at odds of winning, the causation shouldn’t matter. Fair enough, I can see where that argument comes from, if you look at it from a pure statistical standpoint. But since we are talking about human behavior, we have to take into account that external social, not scientific, factors influence vote choice, thus influence the election’s final result (what is being predicted). Human behavior isn’t easily measurable as is, for example, the reaction of iron to certain other elements. In the latter, you can purely use statistics to understand results. You can’t do that in behavioral science (which is why the theory part of most stats-driven “political scientists” are usually pure garbage, which is another story for another day).
Finally, and this is a problem with all models regarding elections in the United States, is that there are only two choices. All American models have to do is to put their money on black or red. And with the exception of 2016, picking the winning side has been extremely easy since 1980. Coincidentally, that’s when most predictive models came out. So I can’t say this is a fault of Silver or 538, but when you have a 50/50 choice, and it’s pretty certain that one of those choices have a higher likelihood of winning, then it isn’t really that impressive.
Decorate Your Walls with Mish Fine Art Images
Click each image to view details or purchase in the store.
Stay Informed
Subscribe to MishTalk
You will receive all messages from this feed and they will be delivered by email.
Polymarket has a 82% chance that there will no resumption of attacks on Iran before the midterms. Will Bibi and the president make a bundle taking the 18% odds?
How much skin does Nate have in that game? His game, I submit, is a different one: clicks: garnering attention.
So I build a model with 75 factors of various fuzziness each. I assign weights to each. Push a button and the output streams out.
To me the one real question is, am I willing to bet a significant amount of my net worth on my claimed model’s predictions? Same for stocks, or the future success of mortgage bonds. My answer to all these is no. If the answer is yes, couldn’t I, or a Nate Silver, expect to make a bundle in prediction markets bets right now?
Now let’s ask a different question: can I make this model and discussion into engaging output in an attention economy? I would wonder if that different question, i.e., what is the infotainment value, is the one really being asked by Silver, et al. How many clicks? That’s where his bread is actually buttered.
i was born, raised and reared on wall street bullshit. the analyst price targets are just marketing tools to bring in the investment banking business and to make the institutional and retail investors think the wall st houses know something. no offense to mish, of course, but everyone here knows his game is click bait or ego or both. or maybe to sell his artwork. nothing wrong with any of that. we call this capitalism. if you all want to treat yourself to the how and why this all came about watch the free 4 part documentary “century of self on youtube.”
https://www.youtube.com/watch?v=eJ3RzGoQC4s
You’re being unnecessarily rough on Mish. His traffic dipped due to his straightforward take on Trump, which hurt his readership numbers.
You think America is full of a-holes, and it’s a wrong take and colors your perspective.
Somehow, you prefer to live here, own property, rentals, have family here, and apparently take a deep interest in the country.
But Mish is not an a-hole, does a public service via his blog, and many of us don’t like it when he’s unfairly impugned.
He has more integrity in his little finger than most in their whole body.
betting odds really got the brazil election very wrong. i’ve been punting on politics since the 1980s with the london bookies and now with the past few decades the prediction market makers online………i sense the election outcome next week might be a shock.
Garbage in, garbage out. But it’s fun to take a random walk now and then.
Meanwhile, Israel’s election is Tuesday. Netanyahu has been trying to start a war with Iran before voting (in-person voting is generally required). The head of the IDF refused to carry out Netanyahu’s orders designed to provoke an Iranian attack on Israel.
https://www.israelhayom.com/2026/10/26/political-uproar-over-reported-clash-between-netanyahu-and-idf-chief/
Netanyahu’s goal is to provoke Iran to attack Israel before or during the election, so that he can suspend the election and remain in power indefinitely, or until he is certain he will win the next election
Excellent exegesis of the election as always Mish, however the following statement ironically describes the problems with economic “theory” today on both sides of the aisle:
“the whole point of building a model, in some sense, is to set clear rules ahead of time so you don’t have to trust your gut or make ad hoc assumptions in midstream.”
IOW capitalist and socialist theorists are caught in an unresolving duality due to their theoretics not creating known and enforceable barriers but won’t/can’t breakthrough to a resolving thirdness due to rigid orthodoxy and failing to find the key factor that keeps the present chaos in force.
The new monetary paradigm of Strategic Monetary Gifting that breaks up Finance’s monopoly paradigm of Debt Only and the 50% Discount/Rebate policy at the various points of retail sale out does the profitability of capitalism and out economic democracy’s socialism.
wisdomicsblog.com
The MAGA and America First branding worked pretty well for Trump’s first term and for the 2024 Presidential election, but the shine is off that brand for those independents, Hispanic, and Gen Z young men who carried Trump over the top in 2024. These voters are not ideologically conservative, they were voting for change and the reinvented history of the first Trump term that they may not have been paying too much attention to in real time. Now two years in they have not seen the results that they were promised and they are ready to bet on another horse. They now see through Trump’s gaslighting and grift. This is also happening on a more global scale with Trump-like politicians losing power in Hungary and Serbia of late, and the Trump aligned candidate in Brazil losing his bid for the Presidency over the weekend. World leaders are also, of late, rebuffing Trump’s bluster. One last gasp of hope for the autocrat-loving hard right movement globally is the idea of Russia, China and the U.S. nominating from the UN Security Council, ousted Hungarian strongman Victor Orban for the open Secretary General position. It is possible for the UK or France to veto that nomination, but that veto would take a lot of political intestinal fortitude that will likely only come from a weakened Trump following the U.S. mid-term elections. If somehow the Republicans maintain majorities in Congress, expect Victor Orban to be the next Secretary General of the UN. If there is a blue wave next Tuesday and Trump is significantly weakened in the eyes of the world, expect that Orban either is not nominated or his nomination being vetoed by France and the UK.
NY Times gets caught publishing fake polls so how are we supposed to beleive any of these polls. The infamous Iowa polls of Hilary, Biden and Kamala should tell you everything. The 10 point swing should tell you everything. This is an attempt to suppress voting and support the theme that there is no fraud. Polls are sensalism.
It is ironic that if you ask the pollster for an audit trail with processes, procudure and written documentation; they fail miserably. No poll can be taken seriously without sample size, collection method, age, race, party ID, non respondents and likely flaws taken into account.
Even with the best pollsters actual results can be double the percentage of real world numbers. Example: 48% to 52% with a +/- 4% errror rate is a tie. Real result could be 44% to 56%. Ask yourself – what good are these?
“Sensalism” is a new one. I take it you never managed to catch an English or a statistics class?
My favorite incarnation of Nate Silver is the one everyone ignores. Moving on… .
That’s a hell of a lot of analysis. It will be interesting to see how the models work. I would simply plug in the specific races into Ai and see what comes up. You might have to pose the same question several times to the same Ai and see what consensus comes up. These are of course all odds, not certainties, as many people would like to believe.
Something tells me Kalshi and Polymarket are liberally and youthfully biased as I think these are the types of people using them. I also think that when a race tips in one direction or another that FOMO may come into play exaggerating the odds. I also wonder if “Trump” voters are playing it close to the vest as in previous elections. One little old white lady in Ohio commented that we shouldn’t throw out the baby with the bath water, voting for Ramaswamy. Perhaps the we shouldn’t overlook the power of one mother speaking to another no matter how vile the rhetoric.
“I would simply plug the races into ai and see what comes up”
What exactly do you think an LLM is going to tell you from this? It’s genuinely baffling what people seem to think AI can do.
No one should be above doing a little research. It might be able to breakdown each factor incorporated into all the different models. Then breakdown the reliability of those factors. Maybe do some back testing. Maybe look at the common factors the models share. Find some patterns between the models and have an Ai explain them. Mish has done quite a lot here, but there’s always room for more creativity with your queries and spotting patterns.
I asked Ai about when to buy a mining stock recently and I ignored the analysis. I wish I had not. You may say that’s pure luck but one thing LLMs are good at is evaluating stochastic events. Elections and stock prices are exactly these. On rare occasions there are massive shocks due to unforeseen events, but on the whole, random walks lead to inevitable outcomes just as they do in thermodynamics.
But of course with any model it’s garbage in, garbage out. I foresee a day when Ai can be used to select likely voters, call them, ask questions, and evaluate the likelihood of truthfulness of the person based on the interaction. Then aggregate the answers into its black box and come out with a pretty good estimate. I’m certainly not sure when that may happen, but the ability will emerge.I think you underestimate the pattern recognition ability of Ai in general. Pattern recognition is just the first step in actual thinking. Sure it’s spitting out some junk now and again, but it ain’t all junk. It’s up you the inquirer to make that decision.
totally agree. when i inquire about historical random questions on different houses and blocks i’ve lived on in my first 66 years the initial feedback might be a bit wrong but after i clarify the facts i know it comes back to verify that is so…….and spits out even more insightful things. it’s amazing. i use it for trading now too. very helpful.
The only problem with that is that it has a habit of lying often and doing so boldly as it can for it knows not the truth. It cannot tell you each factor, it cannot tell you the reliability of each factor. At best, it can point you in the direction of doing the work yourself. That’s good, but it’s far from a cigar and current AI tools can’t be trusted to do even that.
The problem with AI consistently goes back to the fact that (inconsistent) pattern recognition is a leaf in the cigar, but the people pushing AI would like it to be the whole thing and fold itself to boot, all while still being cost effective. This will never happen with LLMs and no amount of fantasizing about them being better than just a good tool can make it happen.
AI is a fine tool, but you’ll be hard pressed to find people who can use it. I think the best example is that it’s a power drill idiots keep trying to use as a hammer. They bet their livelihoods on it eventually working as a hammer because they desperately need it to be one for the circular economy around it to make any sense. This will not make it a hammer, but they really only need people to think it is one long enough to run out the back door with a few bags of cash.
It’s as if the Ai has a personality of its own. Perhaps I should ask more smart ass questions to see where it goes? Just bust out with, “YOU LYING SCUM. Why do you do this to me! Every answer is the same. Why do you hate me!” LOL. Next thing you know my computer gets locked up until I can come up with some hefty Bitcoin.
Ai needs to be isolated somehow to well defined problems. Things like scientific research. It really needs to be in the hands of experts who understand the foundational knowledge. Yeah, Ai isn’t for everyone unless you’re looking for basic educational stuff. Which I have found excellent for getting up to speed on some subjects. Kind of like Wiki, but you don’t have to hunt through all the pages and can drill down on the specific topics you’re interested in.
my best use for AI has been going down memory lane. asking AI if it knows about the teen hangouts in ny area during the 1970s. i have been laughing my balls off on the lists and comments………also great to ask r/e and history. questions.
There are 18 references to “polls” and 31 references to different election betting sites in this article.
I’m shocked Mish isn’t using fiftyplusone, which has a much clearer methodology than Nate and lists every single poll along with the house weights they use for pollsters.
Who does Nate have winning the 1919 World Series?
FLIPR or the betting markets?
Worth noting that the all knowing prediction markets called the election in Brazil completely wrong. I don’t understand the emphasis on them given their track record is generally not great.
Prediction markets are about bettors. Not voters.
However, I understand the reason why bloggers and sites like RCP emphasize them – MONEY.
To be fair, prediction markets have in some circumstances been shown to more accurately forecast election outcomes than polls/pundits.
Given the vast sums spent to influence elections and the fact that the prediction markets are themselves being treated as news worth following, to what degree are the betting markets being manipulated to create news?
Voting with money is much different than voting for free.
bingo. the problem now. 20 years ago, nobody knew you could punt on politics. i have been doing it for almost 50 years.
The blank back of the signs were always good for school crafts projects.
Using prediction markets is just not willing to admit to having no idea.
I have never trusted Nate’s polls or opinion. I am far from being along on this take. Many like myself don’t trust his spin, take or whatever he calls it. It’s not a poll of choice for sure, and is sort of a last minute see it was right, or crickets…
In 2008, Nate Silver was all the rage, but only because nobody
had ever brought any sort of statistical modeling of elections to the forefront for the average person to read and see. Yes, political scientists in the halls of the University of Michigan, Columbia University, and the University of Iowa had been making quite interesting forecasting models for years. However, the media-savvy Silver was able to bring attention to his model, truly making him the “first” in this “forecasting” genre. He knew the power of understanding blogging, and rode in on that fame. Unfortunately, tenured professors of great stature don’t have an inkling for media, nor do they need to.
Of course, since his 2008 model (which, honestly, predicted one of the most predictable elections in since 1980), most of his models have been spotty at best, and flat out wrong many times. For example, in 2018, FiveThirtyEight said that Democrat Andrew Gillum would win Florida’s gubernatorial election, with some odds being as high has 3/4. However, Republican Ron DeSantis won the election. Admittedly, one model did get the results in Florida correct…mine. Yeah, I’ll toot my own horn here.
Subscribe
But still, why is Nate Silver’s model meh? And, possibly more importantly, why are people misinformed when it comes to his so-called “forecasting model”. Let’s delve into it.
First of all, the biggest mislabeling is that Silver’s model is a “forecasting” model. That’s simply not true. In fact, it’s a “nowcasting” model, something that I do as well. And since polling only gives us a snapshot of a certain point in time, Silver’s model today only gives us what the results would be if the election were held today. Therefore, any of FiveThirtyEight’s projections are essentially useless until Election Day, and doesn’t really predict anything, as the results change on a daily basis. To give credit, FiveThirtyEight does put this disclaimer in their website.
Of course, nowcasting models aren’t bad. My highly-accurate model of Florida is a nowcast. However, the data being used makes all the difference. In my model, I use voter turnout. With voter turnout, you can get an idea of certain trends, such as who has yet to vote. With polling data, you have no idea how the polls will move over the course of a month, a week, or even a day. Simply, polls aren’t a tool of predictive behavior in the future.
Subscribe
With that, let’s look at what FiveThirtyEight uses, which are polls. Of course, there are a lot of problems when it comes to using polls as a predictive tool. First of all, how do you know that the polling firm is correct? The averaging of the polls tries to mitigate this factor, but it’s still a factor nonetheless. Simple averaging polls doesn’t account of outliers that might skew the aggregate (such as the dreaded Rasmussen Poll).
Another problem that FiveThirtyEight has (and possibly Silver’s new model, but I don’t know), is the polling “rating”, and how they grade polls from best to worst. According to their website, they account for accuracy, stating
Accuracy, as measured by the average error and average bias of a pollster’s polls. We quantify error by calculating how close a pollster’s surveys land to actual election results, adjusting for how difficult each contest is to poll. Bias is error that accounts for whether a pollster systematically overestimates Republicans or Democrats.
Of course, the obvious problem here is that “accuracy” can only be tested on Election Day, not before. This means that a poll only needs to be “accurate” once, in their last poll before Election Day. Any polls that are done before that can’t be taken into account because there is no other data to compare the polls with (because we don’t have elections every day), which makes “accuracy” a mute point.
Also, when do those polls need to be “accurate”? Let’s say, in a hypothetical election, a Democratic candidate wins by 2% in a general election. Two months before, in September, “Polling Firm A” only did one poll, and nailed the election results with a 2% margin for the Democrat. However, at the same time (in September), all other polls had the Republican candidate winning by around 6%. Obviously, the political climate in September was vastly different than what would happen on Election Day. Still, is “Polling Firm A” considered accurate? Of course, we don’t know because the election was held two months later. Therefore, it’s extremely hard to tell if “Polling Firm A” firm is “accurate”.
When it comes to forecasting models (which we already said that Silver’s/538’s isn’t), Dr. Michael Lewis-Beck set out four criteria as to what makes a good forecast: accuracy, lead, parsimony, and reproductibility. For Silver/538, they really don’t have any of these. So what are these?
Lead – How far in the future can an election outcome be predicted?
Accuracy – How accurate is the model?
Parsimony – How few variables are needed to make a prediction?
Reproductibility – Can the model be used from election to election?
Lead – We already mentioned how Silver’s model violates the “lead” rule, as it is a nowcast and not a forecast.
Accuracy – When it comes to accuracy, that can be debated. Since they use a “win ratio” instead of an outright winner, FiveThirtyEight gives itself little wiggle room to still claim victory, since they don’t claim absolute predictions with the exception of the safest of seats. For example, in Florida, FiveThirtyEight said that Joe Biden had a 69 out of 100 chance of winning the state. But he didn’t win the state. Still, because they aren’t claiming an outright winner, they can fall back on the “we were close” excuse instead of the “we were wrong” reality.
Oh, by the way, my 2020 Florida nowcast nailed it.
Just a side note, the Silver model becomes even more chaotic when looking at other countries and projecting seats in a parliamentary system. Let’s take Eric Grenier’s 2015 model of the Canadian election. Gernier’s poll-tracker, based on Nate Silver’s model, was extremely inaccurate. However, because he probably used standard deviations in determining seat totals, he could claim a slight victory. For example, he said that the Conservative Party of Canada would win 122 seats, with the low watermark being 99 seats and the high watermark being 139 seats. A 40-seat swing is 12% of the entire Canadian Parliament! Still, the Tories won 99 seats, so he was “technically” correct. As for the seat projections for the Liberals and the NDP, he wasn’t even within the standard deviation margins.
Parsimony – This is where the 538 (sorry, I’m tired of typing up FiveThirtyEight all the time) really falls apart. The idea is basically that the fewer variables that you have, the better the model is. Why? Because once you start adding more variables, you come to the point that you are just making descriptions of observations and not predicting anything.
Let me give you an example. Let’s say I’m thinking of a car and I give you hints. If I just tell you “I’m thinking of a car”, that too broad. However, if I say that “the car was made in the 1970s”, that narrows it down. If I further say that “this car is no longer in production”, that narrows it down more. If I say that “this car had major problems with its fuel tank”, now you know it’s probably a Ford Pinto, which it is. That last variable basically just confirmed the choice, and also become a variable only applicable in this particular situation.
In the case of 538, they do the same thing. Their “economic conditions” variable has an additional number of variables in it. Essentially, any time their model misses the point, they just add more variables to make the model work correctly. Yes, a model can be tweaked, but shouldn’t go through complete overhauls.
Also, when 538 adds more variables, why do they pick these specific ones? It seems like many of their variables are arbitrary based on correlation and not based on casual factors. In forecasting, one assumes that a voter is or isn’t voting a certain way because there are casual factors pushing voters that way. In my nowcast, my assumption is that if a voter is registered with a political party, they are likely to vote for that party in the general election. The 538 and Silver models have nothing regarding causality.
Reproductibility – I already hit on this when talking about parsimony. The more and more variables that are added election after election, it no longer becomes reproducable.
Speaking of the variables, and the 538/Silver models in general, the lack of causality makes them hard to swallow. Neither of these models say X leads to vote choice. Again, they just pick variables that just happen to go with the result.
Why is this important? Well, let’s say one of our variables is that Republicans win the general election every time that a full moon happens on July 15th on the election year (which I just made up). That isn’t causation, but just simply correlation. Same thing can be said with the age of Miss America and the number of murders by steam, hot vapors, and hot objects (shown in the graph below). While 538 might not go to this extreme, they aren’t far off. Their models real don’t show anything regarding causation whatsoever.
Now, of course, you can say that if we are just looking at odds of winning, the causation shouldn’t matter. Fair enough, I can see where that argument comes from, if you look at it from a pure statistical standpoint. But since we are talking about human behavior, we have to take into account that external social, not scientific, factors influence vote choice, thus influence the election’s final result (what is being predicted). Human behavior isn’t easily measurable as is, for example, the reaction of iron to certain other elements. In the latter, you can purely use statistics to understand results. You can’t do that in behavioral science (which is why the theory part of most stats-driven “political scientists” are usually pure garbage, which is another story for another day).
Finally, and this is a problem with all models regarding elections in the United States, is that there are only two choices. All American models have to do is to put their money on black or red. And with the exception of 2016, picking the winning side has been extremely easy since 1980. Coincidentally, that’s when most predictive models came out. So I can’t say this is a fault of Silver or 538, but when you have a 50/50 choice, and it’s pretty certain that one of those choices have a higher likelihood of winning, then it isn’t really that impressive.