What to Expect From Senate Polls in the Final Days
An analysis of polling from last three weeks of recent elections
October 16th, 2014,
A few weeks ago, FiveThirtyEight’s Nate Silver released a dataset of over 6,000 polls, all conducted within 21 days of the election. This trove of data dates back to 1998, and covers all manner of races: Senate, House, gubernatorial, and presidential.
This allows us to build some expectations for what we might expect in the closing days of this election. With more time, we could probably be a bit more precise in our analysis, but the shortened time frame still gives us an opportunity to pluck, at the very least, some rules of thumb out of the data.
For my basic analysis, I ordered Senate polls in each race by date. I simply made a series of three-poll averages (if there were not yet three polls, I took two-poll or one-poll averages as appropriate). I then sorted those averages by date and margin: If a poll concluded 10 days before an election and the average showed a candidate down two percentage points, it was categorized appropriately.
These data are summarized on the following table, which can best be thought of as the answer to the following question: At any given number of days out, if a candidate has a lead of a certain size, how often does that candidate win?
Table 1: Percentage of candidates holding on to leads over closing days of election
Note: Leads are rounded off, so a lead of 1% actually represents an average lead of 0.5-1.49 points.
An example of how we would read this chart with respect to current polls: Assume that as we get polls from the North Carolina race that concluded Tuesday, Wednesday, or today, they confirm Sen. Kay Hagan’s (D) lead of roughly two points. Candidates who lead by two points 19-21 days out win about 65% of the time. So we’d conclude that Hagan is the favorite, but her lead is not insurmountable.
On the other hand, let’s assume that polls confirm Dan Sullivan (R) of Alaska’s current lead of around five points. Candidates who lead by five points at that time win 90% of the time over the past eight cycles. In fact, candidates who lead by five points at any point in the third week before the election are 25 for 29 (86%). Put differently, under that circumstance, we’d have a pretty high degree of confidence that Sullivan would win — probably higher than even the formal models are suggesting.
In fact, to predict a win for Sen. Mark Begich (D-AK), you’d basically have to predict something fundamentally dissimilar about 2014 vis-à-vis 1998 to 2012. Those arguments have been made — the potential success of the Bannock Street project to boost Democratic turnout or a potential collapse in the accuracy of polls due to declining response rates. But those sorts of objections to polls are made every cycle: We don’t really have a good basis for doing anything more than shrugging and acknowledging that this time could be different.
Regardless, we might want to know a bit more about the sorts of candidates who do lose. In particular, this cycle is notable for the large number of incumbents who find themselves vulnerable. For quite some time, analysis of incumbent races was guided by the so-called “incumbent rule.” This rule, which grew out of a thorough analysis by Nick Panagakis in 1989, postulated that late undecideds tended to break for the challenger, and that an incumbent who fell below 50% was vulnerable and could potentially lose.
The rule has fallen out of favor, in part because subsequent research has eroded its credibility and in part because people interpreted the rule too literally, with analysts hyperventilating every time an incumbent fell to 48-49% in the polls.
At the same time, there do seem to be some broad conclusions that we can draw from an incumbent’s vote share:
Table 2: Win record by incumbent poll share in final 21 days
As you can see, only one Senate candidate that ever went above 50% in a three-poll average at any point in the last 21 days of a campaign ended up losing: Sen. Al D’Amato (R-NY) in 1998. Only one candidate who hit 50% lost: Sen. Spencer Abraham (R-MI) in 2000. Candidates hitting 49% do lose at times: Sen. Tom Daschle (D-SD), George Allen (R-VA), Jim Talent (R-MO), and Slade Gorton (R-WA) all went on to lose — and perhaps significantly, all did so in years when the fundamentals were aligned against their party. But as a general matter, candidates who are at 48-49% in the last 21 days rarely lose. (For the purposes of this analysis, appointees running in their first election are not considered incumbents.)
There seems to be a cut-point at 46-47%, where an incumbent at that level really does find himself or herself in a Toss-Up race. Below that point, incumbents are in deep trouble. At 45%, only Sens. Bill Nelson (D-FL) in 2012, Barbara Boxer (D-CA) and Harry Reid (D-NV) in 2010, Saxby Chambliss (R-GA) in 2008, and Tim Johnson (D-SD) and Mary Landrieu (D-LA) in 2002 (in Landrieu’s case, this was technically for the primary, not the eventual runoff) went on to win in November. The only incumbents who routinely put up numbers in the low 40s yet won in the fall were Sens. Harry Reid (D-NV) in 2010 and Wayne Allard (R-CO) in 2002.
Assuming that current poll numbers are more-or-less repeated in the next few days, this is not good news this cycle for Sens. Begich, Landrieu, or Mark Pryor (D-AR), who all find themselves in positions where almost all similarly-positioned incumbents have lost. It suggests that Sens. Hagan and Mark Udall (D-CO) are really in Toss-up races right now, while Sen. Jeanne Shaheen’s (D-NH) positioning is probably firmer than many Republicans would like to admit, yet not quite so firm as Democrats believe. It also confirms that Sen. Pat Roberts’ (R-KS) position is precarious — although the unique dynamics of that race potentially set him up to be one of the exceptions. Senate Minority Leader Mitch McConnell (R-KY) is also probably more vulnerable than analysts suggest right now, although the fundamentals in that state may likewise position him somewhat similarly to Boxer in 2010.
On the other hand, it might be that being below 46% might not be a symbol of vulnerability. As Silver has suggested, candidates below 46% might lose not because they are below 46%, but rather because they trail their opponents. That is certainly consistent with the “Allard exception:” Allard was below 45%, but he didn’t trail in the closing days.
To try to get at this angle, here is a list of incumbents who have trailed in any three-poll average at any point in the final 21 days and gone on to win. I also note the largest margin by which they’ve trailed, and the number of times they trailed in a poll average:
Table 3: Incumbents who won but trailed in at least one three-poll average in final 21 days
Note: Deficits are rounded off, so a deficit of one point actually represents an average deficit of 0.5-1.49 points. “Tie” means the margin was less than 0.5 points.
Here, we see what a true outlier Reid’s win in 2010 represents. He is the only Senate candidate in the last eight cycles to fall behind his opponent by more than two points in the closing days of the campaign who then went on to win in the fall. Again, these are not promising data for Pryor, Landrieu, or Begich. On the other hand, it is a useful reminder that poll failures do occur, although they are rare.
What about incumbents who lead in the closing days of the election, but then go on to lose?
Table 4: Incumbents who lost but led in at least one three-poll average in final 21 days
Note: Leads are rounded off, so a lead of one point actually represents an average lead of 0.5-1.49 points.
Spencer Abraham, the most recent Republican senator from Michigan, stands out as something of the ultimate cautionary tale: A candidate who led by as many as 13 points in the closing days of a race, who led consistently, in 12 poll averages, and yet who lost. But as a general matter, it is fully possible for candidates with multiple leads of reasonably large size to go on to lose in November.
If there is a unifying theme among these incumbents who led, but lost, it is that they were running in environments that were overall unfavorable for their parties: Republicans in 1998, 2000, 2006 and 2008, Democrats in 2002 and 2004. All of them represented states that were either aligned against their party in presidential elections, or purple (as Missouri was in 2004 and 2008).
There are again limitations to this methodology, and doubtless follow-up work will be needed for future elections. In particular, we would want some cross-analysis: How did incumbents who were both below 50% and trailed fare versus those who led, and so forth. We’d also want some sort of analysis for open races. But unfortunately, we don’t have a well-worn theory to test there.
But based on this analysis, I think we’d say as follows: Pryor, Landrieu, and Begich are in deep trouble. It would probably require a fundamental shift in historic patterns for them to win. This could absolutely happen, but it is extremely difficult to predict these sorts of shifts. Analysts trying to predict these shifts might be correct every now and again, but over the long run, they will be wrong more often than not.
The Senate race in Colorado and even North Carolina really should be considered Toss-ups. We would probably put a thumb on the scale for Hagan, and a thumb on the scale against Udall (although the introduction of all-mail balloting really does create some added uncertainty about the polls here). On the Republican side, we’d see McConnell and Roberts as Toss-ups as well, although the national environment and orientation of their states probably counsels putting a thumb on the scale for them (a very light one in Roberts’ case). Finally, Shaheen isn’t out of the woods, although the forest line is in sight.
This doesn’t mean that Senate control is a done deal for Republicans by any stretch. But it should be sobering for the analyst attempting to sketch out how the next three weeks will go for Democrats. To predict Democrats retaining Senate control, you basically have to bet on (a) Democrats sweeping South Dakota, Iowa, Colorado, and North Carolina; (b) picking off enough Republican seats in very red states like Kentucky, Kansas, or Georgia to offset any losses in (a) or; (c) systemic polling failure. You can make a plausible case for each of those scenarios, with (b) probably being the most likely. Regardless, given the current state of polling and knowing how races have behaved over the past few cycles, those really do appear to be the options left for Democrats.
|Sean Trende is the senior elections analyst for RealClearPolitics and a senior columnist for the Crystal Ball. He is the author of The Lost Majority: Why the Future of Government Is Up for Grabs and Who Will Take It, and co-author of the Almanac of American Politics 2014. Follow Sean on Twitter @SeanTrende.|