What the Hell Is Nate Silver Doing with His House Effects?

Whatever it is, he should explain.

Using Nate Silver’s poll weights and poll percentages for all South Carolina polls I calculate Republican Darline Graham has a 2.86 percentage point lead over Democrat Annie Andrews.

Graham Leads by How Much?

Fortunately, there is only one page and seven polls to analyze.

Just looking at the above page one can wonder “How the heck did Silver get Graham 1.8 points ahead out of that?”

My lead chart calculates the actual results.

House Effects

Nate Silver’s House Effects explains the discrepancy. If he thinks a pollster is biased then the applies house effects but never says what they are.

I believe this applies to Rasmussen Reports with reason, and Trafalgar with no legitimate reason.

If he posted his house effects we would know precisely what he believes the biases to be.

Revised Margin

Legitimate Transparency Issue

When an aggregator applies systematic adjustments to individual pollsters (shifting Rasmussen or Trafalgar left, Impact Research right, etc.) before weighting, the final published numbers will diverge from a simple weighted mean of the unadjusted percentages.

The influence weights can still be accurate and the arithmetic internally consistent once those adjustments are applied; the problem is that the adjustments themselves are invisible. This is a legitimate transparency issue.

If the influence figures and the raw poll numbers are displayed, readers can (and will) recompute the weighted average. When the result doesn’t match the headline figure, the only way to resolve the discrepancy is for the site to publish the house-effect adjustments (or the adjusted percentages) that were used.

Without that, the posted average cannot be fully audited from the data shown.The same point applies regardless of whether any particular pollster’s adjustment is large or small: the calculation should either add up from the numbers on the page or the page should show the numbers that do add up.

Dear Nate Silver Let’s Discuss Trafalgar

Flashback November 6, 2024: Dear Nate Silver, Might I Suggest “High Quality” Pollsters Be Based on Accuracy

Once again, AtlasIntel and Trafalgar were the two most accurate pollsters based on margins.

You-Gov, allegedly high-quality was anything but. The same can be said for New York Times/Siena with Democrats winning North Carolina and Nevada.

Both NC and NV will be hugely out of the NYT/Siena stated margins of error.

Accuracy Assessment

Calling the winning state accurately is not the best measure. Rather, the best measure is how close a call is vs the final result.

AtlasIntel was closest on 4. Traflagar was closest on 3.

I seriously considered adopting the average of AtlasIntel and Trafalgar as my model. It is why I included that line in the first place.

But I didn’t. Instead I averaged in Nate Silver and Real Clear Politics. A big mistake.

Silver can [brag] about being within margins of error because +-3 he was within margins of error in most states. And did manage to beat “High Quality” NYT/Siena in Wisconsin, Pennsylvania, Nevada, and Georgia.

Congrats?

Seat-of-the-Pants vs Silver

My seat-of-the-pants method had one advantage over Silver.

From February on, I repeatedly stated Trump would dramatically outperform on the youth vote and black vote vs 2020.

He did, by amazing margins.

And that is why I put all the battleground states in Trump’s column except for Michigan which I got wrong as did Trafalgar.

On a win-loss basis, I beat Trafalgar but Trafalgar posted more accurate numbers on average, and that’s what’s really important.

Silver has on many occasions blasted Trafalgar because he doesn’t know what they are doing.

But what the hell is Silver doing?

Maine vs South Carolina

Deluxe vs Lite South Carolina

Are Democrat chances in South Carolina almost as good as Republican chances in Maine?

Really?

In Silver’s Lite Model, Democrats have a 38.7 percent chance of winning South Carolina whereas Republicans only have a 29.9 percent chance of winning Maine.

Q: How do you go from 38.7 percent in the Lite Model to 24.9 percent in the Deluxe Model?
A: More house impacts.

I do not object to this, but I do object to the lack of transparency on pollsters and poll margins.

Nebraska Is a Hoot

Based off polls, Nate Silver gives Democrats a 49.4 percent chance of winning Nebrasksa.

Silver does not believe that, and neither do I. But it’s possible.

Regardless, Silver translates those polls into 69.9 percent chance Republicans win in his Deluxe Model.

Don’t get me started on Iowa, but I think Silver is too optimistic at 59 percent. And I think Silver is too low on Alaska at only 63.5 percent. Silver is 52.1 percent on Kansas.

I am reluctant to go higher due to lack of polls and specific concerns over Maine, Iowa, and Michigan.

Texas Senate Nate Silver

Silver Deluxe gives Democrats a 70.9 percent chance of winning Texas

I am not going to go through six pages of polls to see how far off that posted 2.5 percent is from an actual mathematical calculation.

For starters, page four goes back to June which I find ridiculous, but so does Silver given the June weights are 0.1 percent.

The earliest poll with a weight of 0.10 or higher is from August. I would toss it. But this is largely nitpicking. Silver’s weights overall seem reasonable.

The most encouraging things for Democrats are the Traflagar, Fox News (Beacon Research, and Marist polls.

The most discouraging thing is the more recent InsiderAdvantage poll that shows Paxton in the lead.

Betting Markets

Alaska: 75 percent Democrat
Iowa: 59 percent Republican
South Carolina: 84 percent Republican
Kansas: 64 percent Republican
Texas: 62 percent Democrat
Democrats Win Senate: 63 percent

I agree with the betting markets on Alaska, South Carolina, and Texas. I agree with Silver Deluxe on Kansas, and have Iowa a genuine tossup (about the midpoint of Polymarket and Silver).

Seat of the Pants

I use betting markets, polls, and my own economic expectations. It was my inflation expectations, tariff expectations, and labor market expectations that I figured would give Republicans grief.

Trump’s idiotic war sealed the deal.

For about a month I have had Democrat odds of winning the Senate at 65 percent.

In this election, my seats of the pants analysis got to a Blue Wave forecast and a 65 percent Democrat winning the Senate expectation faster than Silver or the betting markets.

Today, Silver Deluxe gives Democrats a 76.1 percent chance of winning the Senate. But how much does his house edge play into that? We have no idea how to parse it by pollster.

Silver, after being 10 points behind me most of the year is now 10 points ahead of me on Democrat odds of winning the Senate.

With a dearth of quality polls I am reluctant to go over 65 percent.