On Sunday, the New York Times introduced the NYT Upshot/Siena College Poll, a “new way” of conducting polls.
Upshot/Siena’s first poll was released yesterday, but first let’s dive into the methodology.
On Monday, The Upshot and Siena College will release their first survey of likely voters in Florida.
It’s a little different from most polls sponsored by major media organizations. Most such surveys contact voters by dialing random telephone numbers. The New York Times Upshot/Siena College poll was conducted using Florida records from a voter registration file, a data set covering every registered voter in the country. Our data set came from L2, a nonpartisan political data vendor.
Today, virtually all campaigns conduct their surveys off voter registration files.
Why hasn’t the media done the same thing? An important reason is that polls often aspire to measure the entire public’s views on an issue, not just likely voters. A recent New York Times/CBS News survey on gender attitudes, for instance, is a great example of the sort of poll that would suffer from interviewing only registered voters. Registered voters may decide our elections, but they’re not representative of the broader public’s views and attitudes.
When it comes to election polling, though, the voter registration file has real advantages. Its rich data can be used to make sure the sample is representative.
The Upshot/Siena survey is weighted by the race, party registration, age, gender, region and past vote history of active voters.
The vote history of respondents can also be used to improve predictions of who’s likely to vote, one of the biggest challenges in polling.
Most public pollsters use the self-reported vote intention of respondents to determine who is likely to vote. They then exclude those deemed unlikely to vote. There are a lot of limitations to this type of voter screen. Study after study has shown that self-reported vote intention is not a very good predictor of turnout. The “likely” or “unlikely” determination could cause polls to overrepresent the likeliest voters. And, for good measure, the people who respond to public polls tend to be likelier to vote than the general population.
Campaign pollsters instead rely more on the past behavior of voters. Studies have shown that vote history is a more accurate predictor of turnout than self-reported vote intention.
The Upshot/Siena survey averages the two approaches, 50-50. Half of the likely voter measure comes from how voters report their vote intention; the other half is based on a model predicting the probability that they vote. The result is that all respondents are still considered to have some chance of voting, as they would in reality.
Random digit dialing gives public pollsters little control over their respondents — they don’t know whom they’re calling in advance. The result is that they can often get samples, for instance, that are far too old and white, requiring them to undertake herculean weighting efforts to bring underrepresented populations up to their share of the electorate. Many public polls weight some respondents 15 or 20 times more than their least-weighted respondent.
But pollsters using the voter file know whom they’re calling in advance, so they can simply call more of the underrepresented voters, reducing the need for weighting.
The Upshot/Siena poll called a younger and more diverse set of voters, knowing that many of them would not respond, to try to get a more representative sample of respondents in the end. It yielded a younger and more diverse sample than in most polls, and it considerably reduced the amount of weighting.
Are there downsides? Yes, as there are with any method.
A poll conducted off the voter file could miss new voters who have registered since the voter file was last updated. The effect is marginal — but at this point of the cycle, just ahead of the rush to register before deadlines, it can make the biggest difference.
Not everyone has a telephone number in the voter file. And in this election, voter file polls can struggle to weight by education (although the importance of this is reduced, to some extent, by weighting by party registration — an extremely stronger predictor of vote choice).
Nonetheless, the advantages of using a voter file sample outweigh the disadvantages for just about every major campaign pollster.
The Downside
In the article Nate Cohn mentioned two downsides. Are there others? Should the weight be 50-50, 60-40, 75-25 or 90-10?
Is it possible that over-sampling younger voters, less likely to vote, is the wrong thing to do?
Bear in mind that all the pollsters bump up (or down) weights if they do not have enough results in a particular demographic, so oversampling or undersampling to a modest degree should not be an issue, in theory.
Bumper Sticker Analysis
The youth vote not only went overwhelmingly for Obama, but they turned out en masse. Don’t expect that to happen again. How many Hillary bumper stickers do you see vs. the last election?
You don’t see many bumper stickers for Trump either, but voters in his age demographic are more apt to vote.
I suspect all the polls will be off by quite a bit this year in regards to the youth vote. Hillary simply cannot rally the youth in a way Obama did. Many disenfranchised voters really want Bernie Sanders.
I am skeptical of “likely to vote” numbers, but especially regarding the youth vote.
How Reliable are the Polls?
Inquiring minds may wish to consider the Acting Man blog article US Presidential Election – How Reliable are the Polls?
Independents
An NBC poll shows How Third-Party Candidates Could Affect the 2016 Race.
The results show that Clinton loses slightly more support nationally with the existence of third-party candidates. Libertarian Gary Johnson and Green Party candidate Jill Stein share about 15 percent of the vote nationally in a four-way race that includes Clinton and Trump. But when the third-party candidates’ supporters must choose between only Clinton and Trump, they split for Clinton at slightly higher rates than the GOP nominee.
The question is whether those who say they will vote for a third-party candidate are really non-voters who are just expressing dissatisfaction with the major party choices. Past survey research has shown that explicitly naming third-party candidates on questionnaires considerably overstates their actual support on Election Day.
How do you weight that?
Clearly this is a problem for all polls, not just the New York Times poll.
It comes downs to estimating turnout. I suspect all the polls overweight the likelihood of millennials and independents to show up.
Upshot/Siena Florida Survey Results
Please consider results of the New York Times Upshot/ Siena College Florida Poll.
Former Secretary of State Hillary Clinton and businessman Donald Trump are nearly tied in a four-way race for Florida’s key electoral votes, according to a New York Times Upshot/Siena College Research Institute poll of likely Florida voters released today. Clinton currently has the support of 41 percent of likely voters to Trump’s 40 percent with former Governor Gary Johnson garnering 9 percent and Green Party Candidate Jill Stein with 2 percent.
“Trump has as large a lead among Republicans (78 points) as Clinton does with Democrats (77 points) and independents are evenly split at 34 percent for Trump and 32 percent for Clinton with 18 percent for Johnson. Women lean towards Clinton but men tend to support Trump,” said Siena College Poll Director Don Levy. “Trump leads in the North, Bay Area and Central portions of the state, while Clinton leads in the vote rich Southeast and the Southwest is a toss-up.
“There is not only a significant gender gap in this race, but also large racial divides,” Levy said. “Trump is up 51 to 30 percent among white voters, while Clinton has a commanding 82-4 percent lead with African-Americans and 61-21 percent among Hispanics/Latinos.”
“Both candidates suffer from a majority of Florida voters having an unfavorable opinion of them. Clinton is viewed favorably by 40 percent and unfavorably by 53 percent while Trump’s numbers are 39 positive and 55 percent negative.
This New York Times Upshot/Siena College survey was conducted September 10-14, 2016 by telephone calls to 867 likely voters. Calls were made to a stratified weighted sample of voters from the L-2 Voter list via both land and cell phones. Interviews were conducted in English and Spanish. A likely-to-vote probability was computed for each respondent based on both their stated likelihood to vote as well as by virtue of the imputation of a turnout probability score based on past voting behavior applied to their specific voting history. This probability to vote was applied as a weight along with a weight that considered party registration, age, region, gender and race. This poll has a margin of error of +/- 3.3 percentage points.
White Voters Keep Trump’s Hopes Alive in Must-Win Florida
New York Times writer Nate Cohn says White Voters Keep Trump’s Hopes Alive in Must-Win Florida.
A new poll, by The New York Times Upshot/Siena College, suggests that Mr. Trump is keeping his hopes alive in Florida, the largest and most diverse of the crucial battleground states. The reason: White voters favor him by a large margin.
Mrs. Clinton leads by a single point, 41 to 40 percent, among likely voters in a four-way race that includes Gary Johnson and Jill Stein. The race is tied in the head-to-head race, 43-43.
It’s a story that’s playing out across the country. National polls suggest that the bottom has fallen out for Mrs. Clinton among white voters without a degree, causing her substantial lead in national surveys to all but evaporate.
We used the responses to our poll to build a statistical model of the vote preferences of every registered voter, based on the information available in the L2 voter file. It’s the same basic approach taken by the major campaigns’ data analytics and targeting teams.
Clinton’s Turnout Challenge
Mrs. Clinton may have a narrow edge among likely voters, but the race isn’t quite so close among registered voters, who support her by a four-point margin.
Her challenge is straightforward: to get less likely voters to the polls. Mr. Trump has a considerable lead among the likeliest voters, the older, generally whiter voters who regularly turn out in primaries and midterm elections. He has a five-point lead, for instance, among voters who participated in the 2014 midterm election. The model, similarly, finds that Mr. Trump has a seven-point lead among registered voters with a greater than 90 percent chance of turning out.
The presidential election will inevitably draw millions of additional voters from the pool of less regular voters, who are younger and more diverse. Mrs. Clinton has a sizable lead among these less regular voters. The poll, for instance, gives Mrs. Clinton a 10-point lead among registered voters who did not participate in the 2014 midterm elections. The model gives her a lead among every group of voters who are less than 90 percent likely to vote.
Candidate Support vs. Likelihood to Vote
Candidate Support vs. Likelihood to Vote Scaled to Size of Electorate
Where the Model Breaks Down
Those charts are where the model likely breaks down. I strongly suspect youth will not turn out in the expected percentages. I also expect those on the fence will turn out in less than the expected percentages.
That is my guess, and to Cohn’s credit he discusses just that.
The potential upside for Mrs. Clinton is obvious. If everyone in the state turned out and chose between one of the two major candidates, the model suggests that Mrs. Clinton might lead by six points.
But these are not people with a robust track record of voting, and they’re not yet ready to indicate their support for Mrs. Clinton, let alone turn out and vote for her. A lack of enthusiasm among younger voters wouldn’t just mean an older electorate; it might also mean a whiter electorate.
How Confident Are We?
No poll is perfect. As a result, it’s generally better to look at an average of recent surveys, which currently shows a very close race in Florida.
Our likely-voter screen averages two methods: asking voters whether they’ll vote, and using a statistical model to estimate the probability that voters will participate in the election.
Mrs. Clinton fared worse than she did among registered voters under both measures. But if we had used only self-reported vote intention, as many public polls do, Mrs. Clinton would have had a two-point lead. If we had used our model based on vote history, the race would have been tied.
With the result so close, there are different choices we could have made that could have given either Mr. Trump or Mrs. Clinton the lead. In seven weeks, we’ll have a decisive answer.
Nate Silver Grades Upshot/Siena
Nate Silver gave “grade A” to the poll and assigned it the highest weight yet of any Florida Poll.
Mish Comments on Election Odds
I struck the bottom two polls because they are simply too dated. Both favored Trump.
I have never questioned Silver’s assigned grades. I also accept the overall methodology of the survey.
But I do question turnout assumptions. So does Silver (more accurately, he questions something).
Silver took the +1 Clinton results from the Upshot/Siena survey and said it was really +1 Trump.
I do not know Silver’s precise reason. Here are six factors that I see when calculating election odds.
- Momentum is clearly in favor of Trump.
- I suspect polls in general overestimate turnout in the wrong places.
- Overall dislike of both candidates favors Trump.
- Hillary’s health is in question.
- Democrats and Independents who strongly favored Bernie are more disenfranchised that Republican voters who favored someone other than Trump.
- Trump’s ability to rouse enthusiasm and Hilllary’s lack of ability to do so suggests hidden trump support.
Some of those factors may show up in polls, but some may not. The most likely place the polls are wrong is in the likeliness of someone to vote.
Social Standpoint
From a social standpoint, Trump is in a winning spot. I believe conventional wisdom is wrong in at least two states.
For details, please see Is Trump Really Behind in Colorado? New Hampshire? Meme of the Campaign in Reverse.
This may come down to the debates starting next week. If Trump does well, he will win.
Mike “Mish” Shedlock
