Sabato's Crystal Ball

Never Say Die Senate Candidates: Don Blankenship Lost His Primary but Plans to Run in November Anyway

If he does, the former coal magnate will be just the latest in a long line of Senate primary losers to run in a general election

Geoffrey Skelley, Associate Editor, Sabato's Crystal Ball June 21st, 2018

facebooktwittergoogle_plusredditpinterestmail

KEY POINTS FROM THIS ARTICLE

— Although he lost in West Virginia’s Republican primary for U.S. Senate, Don Blankenship (R) is actively seeking to run in the general election as the Constitution Party’s nominee. His attempt to run in November will likely involve a legal challenge to “sore loser” election rules that prevent a losing primary candidate from running in the general election. Most states have such laws today.

— Blankenship’s primary vote was more concentrated in the southern and central parts of the Mountain State, particularly the southern congressional district, WV-3. Blankenship’s best district in the GOP primary also happened to be Sen. Joe Manchin’s (D) best district in his 2010 and 2012 Senate contests, and the district also was Trump’s best in his 2016 presidential victory. If the race between Manchin and state Attorney General Patrick Morrisey (R) is close and Blankenship does indeed run in the general election, the former coal baron could affect the outcome.

— If Blankenship runs as the Constitution Party’s nominee or as a write-in candidate, he will join a short list of candidates who lost their primary contests but then ran in the general election. In all, 39 Senate elections have featured candidates who won at least 1% in a major-party primary and then won at least 1% in the general election as a third-party, independent, or write-in candidate. In at least four cases, those “never say die” candidates may have influenced which party won a Senate race, and two of them won: Sen. Lisa Murkowski (R-AK) in 2010 and now-former Sen. Joe Lieberman (D-CT) in 2006.

West Virginia: Wild and wonderful political machinations

In most instances, losing a party primary signals the end of a candidate’s run for office. However, in West Virginia’s 2018 U.S. Senate contest, Republican primary loser Don Blankenship wants to join a select list of candidates who lost their primary bids in a Senate race but ran in the general election regardless. In a few cases, these candidates notably influenced the outcomes of their respective Senate elections.

On May 8, Blankenship finished third in the GOP primary for the Mountain State’s Senate seat, seemingly ending his bid for a place in Congress’ upper chamber. Blankenship won 20.0% of the primary vote, trailing state Attorney General Patrick Morrisey (34.9%) and Rep. Evan Jenkins (29.2%). Morrisey advanced to the general election, where he will face Sen. Joe Manchin (D-WV), one of the most endangered Democratic Senate incumbents in the 2018 cycle (the Crystal Ball currently rates the West Virginia race as a Toss-up).

Yet Blankenship announced on May 21 that he would attempt to run in the general election under the banner of the Constitution Party. The conservative minor party occasionally nominates candidates who cause a splash — for example, former Rep. Tom Tancredo (R, CO-6) finished second in the 2010 Colorado gubernatorial contest as the Constitution nominee — but never win statewide contests (or most other elections, too). A possible third-party bid by Blankenship obviously worries Republican leaders because it might siphon some conservative votes away from Morrisey in an election that could be very close. On June 6, Blankenship began a petition drive to qualify for the November ballot; West Virginia’s rules require him to collect around 4,500 valid signatures by Aug. 1 to mount his third-party bid.

However, it remains to be seen if Blankenship will qualify for the general election ballot. Like most states, West Virginia’s election code has statutory elements that either expressly or effectively prevent candidates who lost a party primary from running as the nominee of another party or as an independent. According to a 2011 paper by Michael Kang of the Emory University School of Law, 39 out of 50 states have either: a “sore loser” law that specifically prohibits losing primary candidates from running in the general election; or a ban on cross-filing as a candidate of multiple parties in a primary, which prevents a candidate from running as a third-party or independent candidate after losing a party primary (at the time of Kang’s paper, West Virginia’s election law fell into the latter category). Five other states have partial prohibitions on cross-filing that can prevent candidates from running in general elections after losing primaries (Vermont falls in this category though there have been recent examples of losing primary candidates running in the general). Three states (California, Louisiana, and Washington) operate nonpartisan primaries that differentiate them from the rest of the country (Kang’s paper predates California’s switch to a top-two primary system). Lastly, three states (Connecticut, Iowa, and New York) have no sore loser laws or bans on cross-filing with different parties that specifically bar a candidate from running in the general election after losing a party primary.

Blankenship, a former coal executive who served time in jail because of his role in the 2010 Upper Big Branch Mine disaster, promises a legal challenge to West Virginia’s election laws in order to qualify for the November ballot. As mentioned above, West Virginia’s pre-2018 rules effectively prevented a “sore loser” run, but earlier this year the state legislature passed a “sore loser” law that specifically prohibits such a bid. While it is unclear if the legislation’s authors gave thought to Blankenship while crafting the legislation, the new law prohibits losing primary candidates from changing their party registration to a minor party or unaffiliated in order to become a general election candidate via the later filing deadline for third-party and independent office-seekers. Complicating matters, however, is the fact this new law did not go into effect until June 5, about a month after the West Virginia primary. The legal situation is murky, but it is possible that Blankenship could make the general election ballot as the Constitution Party nominee. Otherwise, he definitely has the wherewithal (read: money) to mount a credible write-in bid. Blankenship is very unlikely to win if he is a candidate — recognized or write-in — but he has made it clear that his disdain for the GOP establishment, especially Senate Majority Leader Mitch McConnell (R-KY), could provide him with sufficient motivation to carry out his threat to run.

So far, only two nonpartisan polls of the Mountain State race have included Blankenship as a third-party candidate (though there have been few horserace polls in West Virginia or other competitive Senate contests). A May 22 Gravis survey showed Manchin ahead of Morrisey 53%-40% in a head-to-head matchup. When the pollster included Blankenship, it did not find a notable difference in the Manchin-Morrisey margin, finding the Democratic incumbent up 51%-39% and Blankenship at 5%. A Monmouth poll released on Wednesday found Manchin ahead of Morrisey 49%-42% in the head-to-head question. Manchin’s margin improved slightly to 48%-39% when Monmouth included Blankenship, who took 4%. While the conventional wisdom is that Blankenship would take more from Morrisey, these two polls do not prove that conclusively.

If the Manchin-Morrisey matchup is close and Blankenship does indeed run in the general election, the former coal baron could affect the outcome. Blankenship’s vote was more concentrated in the southern and central parts of the Mountain State, particularly the southern congressional district, WV-3. Map 1 shows his performance by county with the state’s three congressional districts outlined in black. While Blankenship won 20.0% statewide, he did better in WV-3 than elsewhere, garnering about 22% of the vote there and finishing ahead of Morrisey, the overall winner. As some observers predicted in the lead-up to the primary, Jenkins may have been hurt by the large number of Democratic registrants in WV-3, his home district, who might have voted for President Donald Trump in 2016 but were not eligible to cast ballots in a Republican primary (Democrats maintain a voter registration advantage statewide even though Republicans have come to dominate the state at the ballot box). WV-3 was Trump’s best district in his second-best state: he won the Mountain State 67.9% to 26.2% over Hillary Clinton and carried WV-3 by 72.5% to 23.3%, a 51.1-point margin.

Map 1: Blankenship percentage by county in the 2018 Republican primary for U.S. Senate

Note: Current congressional district lines outlined on map

Source: Official results from West Virginia Secretary of State

Blankenship’s relative strength in the south, at least compared to other parts of the Mountain State, is important because that area has also been Manchin’s highest-performing region in his two Senate wins in 2010 and 2012 against John Raese (R). Maps 2 and 3 show Manchin’s margin in the state’s three congressional districts in the 2010 and 2012 races. West Virginia’s districts barely changed in redistricting after the 2010 census, with only Mason County in the far west of the state shifting from WV-2 to WV-3 on the post-2010 map.

Map 2: Manchin margin in 2010 U.S. Senate race, by congressional district

Source: Dave Leip’s Atlas of U.S. Presidential Elections; West Virginia Secretary of State

Map 3: Manchin margin in 2012 U.S. Senate race, by congressional district

Source: Dave Leip’s Atlas of U.S. Presidential Elections; West Virginia Secretary of State

In other words, Blankenship’s best district in the GOP primary also happened to be Manchin’s best district in his 2010 and 2012 Senate contests, and the district also was Trump’s best in his 2016 presidential victory. Additionally, WV-3 was Morrisey’s worst-performing congressional district in his two general election wins for state attorney general: In 2012, he lost it by around six points while winning statewide by a little more than two points, and in 2016, he won it by four points while winning statewide by about 10 points. It was also Morrisey’s worst district in the 2018 GOP primary, though it was also Jenkins’ home base. Should he be an active candidate in November, Blankenship could exacerbate Morrisey’s pattern of relative weakness in WV-3 by winning over some potential GOP voters, which would help Manchin. However, West Virginia is an unusual state politically-speaking, so it is possible that Blankenship could win over some disaffected voters who traditionally cast Democratic ballots and might otherwise vote for Manchin against Morrisey, who Manchin allies are attacking as a carpetbagger (born in New York, Morrisey grew up in New Jersey and ran for Congress in NJ-7 in 2000 prior to winning West Virginia’s attorney generalship). With little polling to go on, it is hard to say. Nonetheless, some of Blankenship’s strongest areas overlap with the Democratic incumbent’s traditionally highest-performing counties and the Republican challenger’s weakest.

“Never say die” candidates: Past primary losers who ran in the general election

Of course, Blankenship actually has to run in the general election, and it is impossible to know whether he will successfully surmount the legal hurdles seemingly blocking his path. But if he does run as the Constitution Party’s nominee or as a write-in candidate, he will join a short list of candidates who lost their primary contests but then ran in the general election. I made a list of every candidate who lost a major-party primary while winning at least 1% and then won at least 1% in the general election as a third-party, independent, or write-in candidate. In all, 39 candidacies (some candidates are on the list more than once) met the criteria for inclusion in Table 1.

Table 1: Candidates who lost a major-party U.S. Senate primary but ran in the general election

Symbols and abbreviations: An “*” indicates a special election. A “^” indicates that Alaska held an open primary that included candidates from all parties, so Gruening’s primary percentage is calculated based on the overall votes cast for just Democratic candidates. Candidates with “(i)” by their names were incumbents seeking reelection. Party abbreviations: “Con.” for Conservative Party, “Const.” for Constitution Party, “Ind.” for independent, “Lib.” for Liberal Party, “Marij.” for U.S. Marijuana Party, “Nat’l” for National Party, “Prog.” for Progressive Party, “Proh.” for Prohibition Party.

Notes: Candidates are listed according to their finish in general elections and then primaries (the “rank” columns). This list only includes candidates who lost a major-party primary and ran as a third-party, independent, or write-in candidate in the general election (this includes one candidate — Ernest Gruening — who lost a nomination in a type of all-party primary). The list excludes candidates who lost a major party’s primary while either winning another major party’s primary or while winning write-in votes in a losing primary effort. For example, Sen. George Norris (R-NE) won write-in votes in both the Democratic and Republican primaries in 1936 but did not officially enter the primaries, instead seeking reelection as an independent rather than as a Republican. The list also excludes primary winners who declined a major-party nomination. For example, Sen. Bernie Sanders (I-VT) has twice won the Democratic primary for Senate in Vermont but maintains his independent affiliation.

Footnotes: 1) Incumbent won as third-party or write-in candidate but caucused with same party in next Congress; 2) Hutchens lost at the Democratic state convention after no candidate won a majority of delegates based on the Georgia primary result; 3) Shuler also won 16.8% in the Democratic primary; 4) O’Brien also won 1.6% in the Republican primary; 5) Cerney also won 0.9% in the Republican primary; 6) Bates also won 35.3% in the Farmer-Labor primary; 7) Candidate cross-filed to run in the primaries of both major parties.

Sources: Archived election results from state election websites, Hathi Trust, and archive.org; CQ Press Guide to U.S. Elections, vol. II (6th ed.); Newspapers.com

Some of the “never say die” candidates in Table 1 cross-filed to run in the primaries of multiple parties, sometimes both major parties. This was particularly easy in California, where from 1914 to 1959 candidates could cross-file to run in as many party primaries as they wanted, and candidates did not have to reveal their actual party registration on the ballot until 1953. For most of that period, however, if a candidate lost the primary of the party with which he was registered, he could not win the nomination of another party even if he won that other party’s primary. Among the candidacies listed in Table 1, the most notable California case was Reverend Bob Shuler, a prohibitionist who ran in both the Democratic and Republican primaries in 1932. Although he lost in both major-party primaries, he won the Prohibition Party’s primary; as a registered member of that minor party, Shuler advanced to the general election despite losing the major-party primaries, and his campaign may have cost the GOP a seat in the U.S. Senate. Winning 23% of the Republican primary vote, Shuler had finished a very close third in a five-way race for the GOP nomination, trailing defeated incumbent Sen. Samuel Shortridge (24%) and victor Tallant Tubbs (25%), who opposed Prohibition. In November, the typically Republican Golden State backed Democratic nominee William Gibbs McAdoo (43%) over Tubbs (31%) and Shuler (26%), with Shuler’s prohibitionist stance attracting many “dry” Republican voters opposed to Tubbs’ status as a “wet.” Of course, Franklin Roosevelt’s 21-percentage point edge over incumbent President Herbert Hoover at the top of the ticket likely boosted McAdoo as well.

Besides Shuler’s 1932 campaign, three other cases where a major-party primary loser may have influenced which party won a Senate seat include the 1918 Montana, 1944 North Dakota, and 1980 New York races.

In Montana, Rep. Jeannette Rankin (R, MT-AL) ran for her party’s Senate nomination in 1918 rather than reelection due in part to redistricting, which replaced the state’s two at-large seats with geographically-based districts and created a seat in her part of the state that was heavily Democratic. Rankin famously became the first woman elected to Congress when she won a seat in 1916, and she gained further notoriety as a pacifist by casting votes against the declarations of war for American entry into both World War I and II (she had won election to the House again in 1940, enabling her to vote on the latter declaration). With the First World War raging, she lost the August 1918 GOP primary by about four percentage points to Oscar Lanstrum (R) for the right to face incumbent Sen. Thomas Walsh (D-MT) in November. But in mid-September she was nominated by the National Party to run in the general election against Walsh and Lanstrum. In a favorable environment for Republicans — the GOP gained five net seats in 1918 Senate contests — Rankin won 23% of the vote and may have helped Walsh win reelection in Big Sky Country with just 41% of the vote to Lanstrum’s 36%.

In North Dakota, intraparty friction on the Republican side helped enable a Democratic takeover in a 1944 contest. The factional strife between the agrarian and progressive Nonpartisan League and the conservative Independent Voters Association (later renamed the Republican Organizing Committee in 1942) had long been a fact of life in North Dakota GOP politics, often leaving the Democratic Party as a third wheel in the Peace Garden State.[1] In 1944, the ROC endorsed incumbent Sen. Gerald Nye (R) in his reelection bid at its convention, while the NPL supported Rep. Usher Burdick (R, ND-AL) against Nye in the Republican primary. But Lynn Stambaugh, a third major Republican candidate who had been mooted as a potential choice for the ROC instead of Nye, entered the GOP race independently of both camps. In an incredibly close three-way primary battle, Nye narrowly won renomination with 34%, defeating Stambaugh (33%) by only about one percentage point (less than 1,000 votes) and finishing ahead of Burdick’s 32% (a fourth candidate won 1%).[2] Stambaugh opted to run in the general election as an independent, something that a previous GOP primary loser to Nye — William “Wild Bill” Langer — had unsuccessfully attempted in Nye’s 1938 reelection bid (Langer is included in Table 1; he later won the state’s other Senate seat in 1940). In November 1944, Gov. John Moses (D) defeated both Nye and Stambaugh while garnering 45% of the statewide vote. Nye won 33% and Stambaugh 21%, helping Moses become the first Democrat to win a Senate election in North Dakota since the start of popular elections for U.S. senators following the ratification of the 17th Amendment in 1913. In a postscript, Moses did not hold the seat for very long, dying just three months into his term. His appointed replacement, Sen. Milton Young (R), won a 1946 special election and held the seat until retiring before the 1980 election.

In that 1980 election, Sen. Jacob Javits’ (R-NY) third-party candidacy in New York probably helped a more conservative Republican retain Javits’ seat. A Rockefeller Republican, Javits was probably the most liberal Republican in the Senate — his DW-Nominate score for the 96th Congress of 1979-1981, his final one, placed Javits to the left of all other GOP senators and four Democrats (plus independent former Democrat Harry Byrd Jr. of Virginia). In a New York Republican Party shifting to the right, conservative Hempstead Town Presiding Supervisor Al D’Amato (R) of Nassau County won sufficient support at the June state GOP convention to take on Javits in a primary without needing to gather petition signatures. Ahead of the September primary, D’Amato received the nominations of the influential Conservative and Right-to-Life parties while Javits earned the ballot line of the Liberal Party. (New York remains one of three states in the country with no ban on cross-filing, and its multiparty system has long been a staple of Empire State politics.) With opposition to the Republican incumbent unified behind D’Amato, Javits lost the Republican primary 56%-44%. Bad headlines about the 76-year-old Javits’ health probably hurt the incumbent, too. However, as the Liberal Party nominee, Javits was on the November ballot, and his candidacy still attracted significant support, including backing from the state teachers’ union. On Election Day, Javits won 11% of the vote, and had Javits instead abandoned his reelection bid, many of his 665,000 voters might have backed Rep. Elizabeth Holtzman (D, NY-16), the Democratic nominee. D’Amato narrowly defeated Holtzman 44.9%-43.5%, needing a relatively small plurality to claim victory. As the Republican nominee, D’Amato probably earned an additional boost from Ronald Reagan’s three-point edge in New York over incumbent President Jimmy Carter at the top of the ticket (analogous to Javits’ role in the Senate race, independent liberal Republican John Anderson’s 7.5% vote share may have cost Carter the Empire State’s 41 electoral votes).

Outside of these four cases, most general election bids by major-party primary losers do not appear to have notably affected the party-winner outcomes in U.S. Senate general elections. Notably, the two candidates to win a general election after losing a primary were both incumbent senators who accomplished this feat in recent times. In 2006, Sen. Joe Lieberman (D-CT) lost renomination to an anti-Iraq War primary challenger on his left, Ned Lamont (who is the frontrunner for the Nutmeg State’s Democratic gubernatorial nomination this year). But Lieberman mounted a general election campaign in one of the only states that still does not have any legal prohibitions against primary losers qualifying for the general election ballot. Lieberman won the general by garnering many Democratic and Republican votes, with the Republican nominee relegated to third-wheel status in the race. In 2010, Sen. Lisa Murkowski (R-AK) lost renomination to Tea Party challenger Joe Miller. However, she managed to win reelection without even making the official November ballot, successfully running a write-in campaign to defeat Miller. A previous primary loser in Alaska, Sen. Ernest Gruening (D-AK), had attempted to do this in 1968 after losing renomination to Mike Gravel (D), but the incumbent finished third with 17% in the general election.[3] Two other incumbent senators, Sen. Robert Stanfield (R-OR) in 1926 and Sen. Smith Brookhart (R-IA) in 1932, lost renomination in their party primaries but won at least 1% in losing general election bids.

Some of the other candidates in Table 1 were not especially influential on general election outcomes, but many were interesting figures. For example, John Neal of Tennessee is on the list five times, but he never came close to winning the Democratic nomination for U.S. Senate or harming Democrats’ chances of retaining a Senate seat. Neal, a progressive who helped lead the defense team during the famous Scopes Trial regarding the teaching of evolution, may have been the archetypal perennial candidate: He sought election at least 20 times in the Volunteer State, mainly for Senate and governor, over a 40-year period between 1920 and 1960. Having previously run unsuccessfully for various state and local offices in Massachusetts, Thomas O’Brien joined demagogic crypto-fascist Charles Coughlin’s Union Party in the mid-1930s. In 1936, O’Brien ran in both the Democratic and Republican primaries for Senate in Massachusetts, losing both races. He then ran in November as the Union Party’s Senate candidate — where he won 7% — while also serving as his party’s vice presidential nominee in that year’s presidential contest. The Union ticket, led by Rep. William Lemke (R, ND-AL), won about 2% of the vote nationally. Louis Ward, another Coughlin acolyte, finished a close second to Rep. Prentiss Brown (D, MI-11) in Michigan’s 1936 Democratic primary for Senate and then won a little more than 4% in the general election, which Brown won with an outright majority.

A handful of sitting and former officeholders ran competitive primary campaigns but had little effect on the November outcome. Rep. Joseph Monaghan (D, MT-1) eschewed a reelection bid in 1936 to challenge incumbent Sen. James Murray (D-MT) in Montana’s Democratic primary for Senate. After losing the primary by just two percentage points, Monaghan ran in the general as an independent, but Murray won reelection with 55% in November. Rep. Fred Aandahl (R, ND-AL) opted to challenge the aforementioned Sen. William Langer (R-ND) in the 1952 Republican primary in North Dakota. Previously North Dakota’s governor for six years, Aandahl lost to Langer in the primary and then finished a distant third in the general election with 10% as a write-in candidate. Former Sen. Glen Taylor (D-ID) won a Senate seat in 1944, then ran as the Progressive Party’s vice presidential nominee in 1948, and then lost renomination to the Senate in 1950. “The Singing Cowboy” ran for the Senate again in 1956 but lost by 200 votes to Frank Church (D) in Idaho’s Democratic primary. Taylor ran as a write-in candidate in the general, but his 5% vote share mattered little as Church won 56% and routed incumbent Sen. Herman Welker (R). Ex-Rep. Charles Randall (Proh., CA-9), one of the only Prohibition Party candidates to ever win a seat in Congress, did not seriously threaten Sen. Hiram Johnson (R-CA) in the Golden State’s 1928 GOP primary for Senate, but won 6% in the general election as the Prohibition nominee.

Some “never say die” candidates later found some degree of electoral success after their failed primary-general election bids. In 1950, Wesley Powell (R) narrowly lost to incumbent Sen. Charles Tobey (R-NH) in New Hampshire’s GOP primary and then won 6% as a write-in candidate in November; Powell would later win two terms as the Granite State’s governor. In 2006, Christine O’Donnell (R) finished a distant third in the Republican primary for Senate in Delaware. But she attracted 4% of the general election vote as a write-in, presaging her infamous 2010 Senate bid in which she upset Rep. Mike Castle (R, DE-AL) in the GOP primary but then lost badly in the general election to now-Sen. Chris Coons (D-DE).

Still others did not have much of an electoral impact but were notable socio-political figures. A notorious bigot and isolationist, evangelist Gerald Smith lost with 31% in Michigan 1942’s GOP Senate primary and then won around 3% as a write-in candidate in November. Ella Boole ran in the 1920 New York Senate Republican primary, trailing far behind incumbent Sen. James Wadsworth Jr. (R-NY). Boole, a major leader of the Women’s Christian Temperance Union in New York, then ran as the Prohibition nominee in the general election, where Wadsworth easily won reelection.

In West Virginia’s 2018 Senate election, there is little reason to think Blankenship can win the general election if he makes the ballot as the Constitution Party nominee or runs as a write-in candidate. Candidates who are not major-party nominees rarely win elections. Nevertheless, if the Manchin-Morrisey contest is close, a few percent of the vote for a Blankenship bid could affect which party controls the Senate seat for the next six years. In turn, with Republicans holding a narrow 51-49 advantage in the Senate, the West Virginia seat could be pivotal for overall control of the upper chamber in the next Congress. Every vote counts, but particularly so in a close election in a closely divided legislative body.

Footnotes

1. Today, the North Dakota state Democratic Party is officially the “North Dakota Democratic-Nonpartisan League Party,” as the NPL joined forces with the Democrats in the 1950s.

2. A sitting U.S. House member, Usher Burdick opted to run for the House again after losing in the U.S. Senate primary but lost as an independent candidate. Burdick’s son, Quentin Burdick, would go on to become a Democratic member of the House and then the Senate.

3. Gravel served two terms before losing renomination in a primary himself in 1980, finishing behind his former rival Ernest Gruening’s son Clark Gruening (D). The younger Gruening lost to Frank Murkowski (R) in the 1980 general election; today Murkowski’s daughter Lisa Murkowski holds that Senate seat. Gravel later ran a quixotic campaign for the Democratic presidential nomination in 2008.


Senate 2018: Two Rust Belt Ratings Move in the Democrats’ Direction

Pennsylvania and Wisconsin look more and more like Republican reaches

Geoffrey Skelley, Associate Editor, Sabato's Crystal Ball June 21st, 2018

facebooktwittergoogle_plusredditpinterestmail

KEY POINTS FROM THIS ARTICLE

— The Crystal Ball has new ratings in the Pennsylvania and Wisconsin U.S. Senate contests, both in Democrats’ direction. In Pennsylvania, Sen. Bob Casey’s (D) reelection bid moves from Likely Democratic to Safe Democratic, while in Wisconsin, Sen. Tammy Baldwin’s (D) race goes from Leans Democratic to Likely Democratic.

— Overall, we now rate 49 Senate seats as at least leaning toward the GOP, 44 as at least leaning toward the Democrats (and the two independents who caucus with them), and seven seats as Toss-ups. Republicans remain favorites to hold the Senate after the 2018 election in November, although there is a narrow path for Democrats to take the majority.

— Finally, Democrats have a real chance at an upset in one of the Trumpiest districts in the country, WV-3. The Crystal Ball is shifting its rating there from Likely Republican to Leans Republican.

Table 1: Crystal Ball Senate ratings changes

Senator Old Rating New Rating
Bob Casey (D-PA) Likely Democratic Safe Democratic
Tammy Baldwin (D-WI) Leans Democratic Likely Democratic

Table 2: Crystal Ball House ratings change

Member/District Old Rating New Rating
WV-3 Open (Jenkins, R) Likely Republican Leans Republican

Many Rust Belt races look like Republican reaches

In 2016, President Donald Trump narrowly carried Pennsylvania and Wisconsin by a combined 67,000 or so votes to defeat Hillary Clinton and carry the two states’ combined 30 electoral votes. Those victories, along with Trump’s 10,000-vote win in Michigan, seemingly signaled an end to the “Blue Wall” that supposedly gave Democrats a built-in advantage in the Electoral College.

Fast-forward 18 months, and Democrats find themselves the beneficiaries of a midterm environment where a Republican incumbent president has an approval rating in the low 40s. While Democrats are defending about three-fourths of the Senate seats up in 2018 and could very well lose net seats in November, the Rust Belt seats that they worried about at the start of the cycle seem to be increasingly out of reach for Republicans. Just last week, the Crystal Ball moved the Ohio race between Sen. Sherrod Brown (D) and Rep. Jim Renacci (R, OH-16) from Leans Democratic to Likely Democratic.

Now Sen. Tammy Baldwin (D-WI) is also joining the Likely Democratic category, moving from Leans Democratic. Baldwin spent much of the cycle looking like a top-tier target for the GOP, and along with Sens. Brown of Ohio and Bill Nelson (D-FL), she appeared to be one of the only truly endangered Democratic senators in a state that Barack Obama carried twice. However, the environment in the Badger State appears to be sufficiently favorable for her reelection chances to receive an upgrade. Election results over the past six months in the state suggest that Republicans will have trouble this November. These results scared Gov. Scott Walker (R-WI), too, to the point that the Wisconsin GOP sought a legislative workaround to overcome a court order to hold additional special elections. In a January special election, Democrats won a state senate special election by 10 percentage points in a seat that Trump won by 17 points. Then in a Wisconsin Supreme Court election in April, liberal Rebecca Dallet defeated conservative Michael Screnock in a statewide race by 11.5 points. These results had played a part in the Crystal Ball’s previous Leans Democratic rating for Baldwin, but then the pro-Democratic environment in Wisconsin received further confirmation on June 12, when Democrats captured another state senate seat in Trump territory, winning by three points in a district Trump carried by 18 points in 2016. In four special elections for the Wisconsin state legislature in 2018, Democrats are outperforming Clinton’s district margin in the 2016 election by 20 points, on average. That is, if Clinton lost a district by 17 points, the average Democratic improvement would result in a Democratic victory by three points. To be sure, this is a small sample, but along with the state judicial election, the results seem to warrant the concern expressed by Walker and other Wisconsin Republicans.

On top of these election outcomes, Baldwin’s potential GOP opponents are not engendering much confidence from GOP leaders. Senate Majority Leader Mitch McConnell (R-KY) recently made headlines when he did not include Wisconsin and other purported Republican Senate targets on a list of key contests that he thought would determine control of the Senate. The GOP primary in Wisconsin appears to be a two-way race between veteran and businessman Kevin Nicholson (R) and state Sen. Leah Vukmir (R). Vukmir received the state party endorsement in the Aug. 14 primary at the GOP’s May convention and has the backing of many Walker allies. Nicholson has caught flak from some Republicans for having been a Democrat, and not just any Democrat — he was president of the College Democrats of America and even spoke at the 2000 Democratic National Convention. But conservative groups like the Club for Growth are backing Nicholson’s candidacy, and he had outraised Vukmir $2.3 million to $1.2 million as of March 31. However, the winner of the Nicholson-Vukmir battle will likely be in a major financial hole against Baldwin, who had $8.1 million in her war chest at of the end of March, almost 10 times Nicholson’s cash on hand ($840,000) and about 13 times Vukmir’s reserves ($640,000). Given the late primary and Baldwin’s large financial edge, it is little wonder that conservative outside groups have already spent $3.1 million against Baldwin to soften her up for the eventual Republican nominee. Still, while Democratic groups have spent $1 million to support Baldwin, Senate Majority PAC — the main Senate Democratic Super PAC — notably did not include Wisconsin in its initial nine-state, $80 million ad reservation. While such reservations can and will change, the lack of bookings may show confidence in Baldwin’s position. On Wednesday, the Marquette University Law School Poll released its latest survey, which found Baldwin comfortably ahead of both Vukmir and Nicholson — 49%-40% and 50%-39%, respectively. This survey reinforced our view that Baldwin is now a stronger bet to win reelection.

Meanwhile, we’re keeping Scott Walker’s own bid for a third gubernatorial term rated as Leans Republican. But that’s mainly because of uncertainty over the large Democratic primary field angling to take him on. If that primary produces a credible nominee, Walker could be in a Toss-up race by Labor Day. Walker has won three statewide victories, winning full terms as governor by about six points apiece in 2010 and 2014 and a recall election in 2012 by about seven points. Those clear but close margins suggest that he could be in serious danger in this sort of environment. The Marquette poll showed him leading each of 10 potential opponents but by small margins in some instances.

In addition to the new rating in Wisconsin, the Crystal Ball is also shifting its rating in Pennsylvania from Likely Democratic to Safe Democratic, effectively taking the Keystone State off the board. Sen. Bob Casey (D-PA) had double-digit leads over Rep. Lou Barletta (R, old PA-11) in recent nonpartisan polls from both Franklin & Marshall College and Muhlenberg College. Pennsylvania, like Wisconsin, went undeclared by McConnell and Senate Majority PAC in recent listings of competitive races, and Casey too held a lopsided financial edge over his general election opponent as of April 25, with $10.1 million in his war chest to Barletta’s $1.3 million cash on hand. Back in April, the Washington Examiner reported that GOP leaders had “all but written off” Barletta, and they remain frustrated with the Republican nominee.

Map 1: Crystal Ball Senate ratings

With these two ratings changes, the Crystal Ball now rates five contests as Likely Democratic and 11 races as Safe Democratic. Baldwin joins Sens. Debbie Stabenow (D-MI), Tina Smith (D-MN), Bob Menendez (D-NJ), and the aforementioned Brown in the former category. Overall, we rate 49 seats as at least leaning toward the GOP, 44 as at least leaning toward the Democrats (and the two independents who caucus with them), and seven seats as Toss-ups (five currently held by Democrats and two held by Republicans). Republicans remain favorites to hold the Senate after the 2018 election in November, although there is a narrow path for Democrats to take the majority.

We’ve got one other change this week, which is in the House. West Virginia, as noted in this week’s lead piece, is a confounding state politically. Despite voting for Donald Trump by more than 40 points, Democrats still retain a voter registration edge in the state, and the state could very well send Sen. Joe Manchin (D-WV) back to the Senate. Manchin seems likely to carry WV-3, the state’s southernmost district, whether he wins or loses statewide, even though that coal-dominated district also gave Trump his biggest margin of the state’s three districts in 2016. Former Rep. Nick Rahall (D) held this district for decades before losing to Rep. Evan Jenkins (R) in 2014; Jenkins lost the Senate primary in May, thus opening his seat. We flagged WV-3 more than a year ago as a dark horse Democratic target owing to the interesting candidacy of state Sen. Richard Ojeda (D), who has become something of political cult hero in the district and who has attracted national attention. He will face state Del. Carol Miller (R), who won a competitive primary in May. A Monmouth University poll released Wednesday showed a close race — Ojeda was up 43%-41%, with an even bigger lead in the pollster’s likely voter models — and we’re adjusting our rating accordingly, moving the race from Likely Republican to Leans Republican. For all the focus this cycle on Democratic House targets in affluent, well-educated, and traditionally Republican suburbs, the party has some promising targets in places that strongly supported the president but are more open to voting Democratic down the ballot. WV-3 is perhaps the most extreme and unusual example: Trump won the district by 49.2 points in 2016, his 13th-best district in the country according to Daily Kos Elections’ calculations. And yet this district could send a Democrat to Congress.


Two Ways of Thinking About Election Predictions and What They Tell Us About 2018

There are differences in method, accuracy, and probability between quantitative forecasting and ratings-based handicapping

G. Elliott Morris, Guest Columnist June 14th, 2018

facebooktwittergoogle_plusredditpinterestmail

KEY POINTS FROM THIS ARTICLE

— Two approaches to forecasting — one formally statistical, one rigorous yet flexible handicapping — produce different tools that we can use to evaluate the battle for control of the U.S. House in the 2018 midterms.

— The Crystal Ball and other political handicappers use a “qualitative” method to generate ratings of individual seats using election news, candidate evaluation, and some hard data. Others use quantitative modeling to produce probabilities of how likely it is for one party or the other to win each House seat.

— The quantitative model described below is more bullish on the Democrats’ House prospects than the Crystal Ball’s race ratings, but both indicate considerable uncertainty about which party will win a House majority this November.

— Those following this year’s House elections would be wise to take into account both qualitative race ratings, like those done by the Crystal Ball, as well as quantitative models, like the model described below, when assessing the race for the House.

Introduction

To understand the differences between quantitative, data-driven predictions and those made from traditional, data-influenced handicapping, one should direct their attention to the names of two websites: Sabato’s Crystal Ball at the University of Virginia Center for Politics, and my blog, The Crosstab. One is a reference to the soothsayer, a fortune-teller who stares into their glass ball and derives the fate of an event by evaluating some known and unknown factors. The other is a reference to the contingency table, a common tool in survey research that breaks down responses to a question by subsets of responses to another. The names of mine and this website are coincidentally descriptive of the ways in which our predictive methods differ.

In forecasting outcomes there are both harms and benefits to these two approaches. This piece evaluates those differences in the context of the 2018 midterm elections and delivers some much-needed attention to that which is common, not just contrasting, between the two.

Before I begin, allow me to highlight a guiding principle of this article. The contrast between my quantitative and the Center for Politics’ “qualitative” handicapping of the 2018 House elections is mostly the difference between continuous predictions (those that assign a chance to outcomes between 100% win or 100% loss) and binary predictions (those that assign either “win” or “lose” to a party). Whereas the method I employ uses data to generate a probability of victory for Democrats in all 435 U.S. congressional districts and their chance of winning the majority of seats, the Crystal Ball’s method tells you that either Democrats or Republicans are favored to win a particular race (technically, the Crystal Ball’s method is a discrete method, with set degrees of certainty on both sides, but is more proximate to binary than continuous prediction). Keep this difference between continuous (even better: distributional) and binary/discrete projections in mind.

This piece is broken up in three sections. In the first, I go through what the two methods take into account when projecting race outcomes. I then divulge the differences between what they tell us, and in the final section I break down the differences in the models’ past and current forecasts.

Inputs

Both my and the Crystal Ball’s methods of predicting the 2018 midterm elections to the United States House of Representatives are processes that (1) take in information, called inputs; (2) do something with that information; and (3) spit out other information, called outputs. If you remember ninth grade mathematics, these are both called functions (Dr. Seuss had a youthful explanation of functions that I remember from high school pre-calculus). However, after this rough categorization, the two functions diverge considerably.

My model to forecast the 2018 United States house midterms is a probabilistic statistical model that takes in a variety of input and, through four stages, produces outputs. The overall approach was developed by political scientists Joseph Bafumi of Dartmouth College, Robert Erikson of Columbia University, and Christopher Wlezien at the University of Texas at Austin. You can read their paper here. The model performs its estimation in four stages (note that the technical details of my model differ slightly from Bafumi et. al.’s methodology):

  1. Calculate an estimate of the national environment today:
    1. Compute a weighted average of all congressional generic ballot polls taken for the 2018 cycle so far.
    2. Compute the average change in post-2016 special elections from the previous Democratic margin in a seat to the margin in the special election.
    3. Repeat this for every day of every year going back to 1992.
  2. Predict the national environment on Nov. 6, 2018:
    1. Use generic ballot polling averages at this point in past cycles…
    2. …combined with the average special election swing, again at this point in past cycles …
    3. …to generate a prediction of the national vote on election day. The final projection has around a six-point margin of error today.
  3. Use a variety of inputs to predict results at the district level:
    1. Create a baseline projection for every district by combining:
      1. The partisan lean of a district (a method developed at FiveThirtyEight that averages a seat’s 2016 democratic presidential win/loss margin with its 2012 presidential margin, weighted 75%/25% to put more emphasis on the more recent cycle);
      2. The previous candidate’s margin in the district;
      3. Candidate-specific variables, like whether an incumbent is running or if one candidate is significantly qualitatively “worse” than the other.
    2. Swing this baseline projection of the district the appropriate amount left/right, determined by the projected Democratic margin in the national vote from step 2.3.
  4. Simulate 50,000 election outcomes:
    1. For each trial, vary the estimated national popular vote randomly according to the margin of error of past predictions of the national vote. Add that error to each seat uniformly (NY-15, the seat where Hillary Clinton did the best in 2016, gets swung just as much as TX-13, the seat where Donald Trump did the best).
    2. Vary the forecast Democratic margin in each seat according to error that is correlated between districts. This accounts for the chance that our forecasts have more error in red than blue districts, white than minority districts, educated than uneducated districts, etc.
    3. Add up the number of seats Democrats win.
    4. Repeat this 50,000 times. The percentage chance that Democrats have of winning the election is simply the number of times they win 218 seats or more (a bare majority) divided by the total number of trials. Each seat has its own win probability generated the exact same way (by keeping a list of seats won/lost in each trial).

On any given day, the numbers generated by my forecasting model represent the best predictions we have of Democratic win margins at the national level and in each House district, and the chance that those projections will err. Remember, these projections are continuous, with outcomes occurring along a distribution of possibilities and each seat having a specific probability of victory. In the end, I produce a dataset of continuous vote shares and win probabilities, ranging from 0% to 100%, for every House seat in the nation and the nation itself.

The process by which the UVA Center for Politics generates its race ratings is different, however, and does not follow such a strict, formal statistical methodology.

The analysts at UVA consider myriad data, some quantitative and some not — often on different scales, e.g. how do you compare previous Democratic win margin to the following headline: “A gay Republican, the child abuse he sanctioned, and the homophobia used to defend him” — to come up with their projections. The analysts use a number of factors, including electoral history, polling, candidate quality, modeling, and district news in the method they use. The ratings ultimately reflect their judgment about the likelihood of one side or the other prevailing in a given contest.

Outputs

As discussed, these two approaches to forecasting — one formally statistical, one rigorous yet flexible handicapping — produce different tools that we use to understand upcoming elections. Whereas I rate each seat on a scale from 0% to 100% for the likelihood that it is won by Democrats, the team at UVA produce ratings that lie on a discrete scale: from Safe, Likely, and Lean Republican, to Toss-up, to Lean, Likely, and Safe Democratic. To evaluate what these differences might mean in November 2018, it is useful to explore what they meant last time around.

The Past: Accuracy of seat ratings and forecasting models

The big question everyone wants answered is: How likely is it, say, for a “Lean Republican” seat to be won by a Democrat? What about a Likely, or better yet, Safe, Republican seat? One would hope that races rated differently would convey different win probabilities for Democrats and Republicans. Indeed, they do.

To determine how well the UVA Center for Politics election ratings matched election outcomes over time, I combined their historical ratings going back to 2004 with actual results in House districts (made available by the MIT Election and Data Science Lab). The results are shown below.

Figure 1: Accuracy of race ratings by category

Notes: This figure stacks each district over Democrats’ actual November vote margin in the seat depending on its race rating from Sabato’s Crystal Ball. Ratings for all elections since 2004 are included where available.

You can see that there are certainly differences between UVA’s Safe, Likely, and Lean categories on both sides of the aisle; safer districts are less likely to see large upsets, and seats that lean toward either part are sometimes, though not frequently, won by the opposition. Overall, the ratings are relatively well calibrated, and 89% of rated seats not categorized as Toss-ups end up being won by the party that is favored to win.

If I want to compare the Center for Politics rating with my own ratings, however, I need to put them on the same continuous scale. I do so by simply taking the average Democratic win margin and raw probability of victory for every category of race rating. The figure below shows the results of this analysis.

Figure 2: Converting House race ratings to probabilities of victory

Notes: This figure graphs the implied Democratic win margin and win probability for each race rating category. To get an implied forecast of Democratic win margin and win probability, I calculated the average win margin, standard deviation, and percent of the time Democrats win for each race-rating category over all House elections since 2004. Points on the graph are sized by the number of contests in that category.

In the left panel of the graphic, I show that each category has an identifiable point estimate and band of uncertainty (or confidence interval) surrounding it. Lean Democratic seats are won, on average, with a six-point Democratic margin, for example; Lean Republican seats give GOP candidates a seven-point average margin; Likely Democratic seats give Democratic candidates a 14-point average margin, and so forth.

Each of these categories also has a corresponding Democratic probability of victory for the seats placed within. In Lean Democratic seats, Democrats win the elections 78% of the time; Lean Republican: 18%; Likely D: 95%; Likely R: 3%; Safe D: 99%; Safe R 0%, and Toss-up districts: 59%. These values are plotted on the right of the preceding figure, with the size of each point showing the number of seats earning that designation over the years.

It is apparent that qualitative seat ratings have provided good forecasts of Democratic win margins and probabilities in the past, but how do they compare to the predictions generated by my formal statistical model? Below, I recreate the probability-by-ratings figures for seat ratings generated by re-running my 2018 U.S. House midterms model for the 2016 House elections. Specifically, seat ratings are assigned for each seat according to the forecast win probabilities for each seat: if both parties have a win probability below 60%, the seat is considered a Toss-up; Lean Democratic/Republican seats are those with a win probability below 80%. Likely seats are those with win probabilities below 95%. Seats rated as greater than 95% likely for either party are considered Safe R/D.

Figure 3: Probabilities of Democratic victory based on House race ratings

Notes: This figure shows the actual Democratic probability of winning for seats rated as Safe R/D, Likely R/D, Lean R/D, or Toss-up, with the rating derived from its forecast win probability. Points sized by the number of contests in that category.

What first stands out is how pro-Democratic the Toss-up category is. However, as there are only six seats in this category, this error is caused by the Democrats winning one more seat than they ought to (four out of six instead of three out of six) — a likely insignificant difference in the long term.

What is more important is the much higher proportion of Safe to Lean/Likely seats in the quantitative forecast than in the qualitative ratings. It should be noted that this could be partly due to the Center for Politics’ omission of ratings for some lopsided seats.

Of the 384 House elections that took place in states without redistricting their congressional boundaries prior to the 2016 election, my forecast predicted them with 98.7% accuracy, getting just five non-Toss-up seats incorrect three fewer seats in aggregate than they actually did in November. Two of 28 Likely Republican seats were won by Democrats, one of eight Lean Republican seats was won by Democrats, and two of 12 Lean Democratic seats were won by Republican candidates. The predictions for all 435 seats erred 10 times, making total error 2%.

It’s worth noting that the seat with the biggest (10 percentage points) error in my re-run 2016 forecast was AZ-1, which the Center for Politics correctly predicted would be won by now-Rep. Tom O’Halleran (D) instead of Paul Babeu (R) — he’s the candidate referenced in “A gay Republican…” headline cited above. The UVA projections picked other Republican seats as Democratic pickups that I did not, and ended up over-shooting the Democrats’ number of seats by seven last cycle, while I low-balled them by three seats. I pick the AZ-1 example as it displays the biggest weakness of quantitative forecasting: the difficulty of accounting for deficits in candidate quality in a data-driven fashion. However, this is not as large an issue as one might think, given the overall record of the modified Bafumi et. al. method.

To be sure, what if neither method alone is the correct answer? For the sake of completeness, if you had combined the forecasts with a method that accounts for the uncertainty in both projections (using a Bayesian update to the normal distribution of outcomes — certainly a good, but not the most sophisticated way, to do so), you would have predicted the 2016 elections spot on, with Democrats being projected to win 194 seats in the U.S. House, though six individual projections were wrong but canceled each other out.

The figure and table below depicts the quantitative forecast, ratings-based forecast, blend of the two, and final result in each of the top 20 closest districts in the 2016 U.S. House elections.

Figure 4 and Table 1: Blended House forecasts in 20 closest 2016 House races

Notes: This figure shows estimates for the 2016 House elections according to different methods. The “blended” forecast is a Bayesian update to the normal distribution with the quantitative forecast being used as the prior, the seat rating being used as the likelihood, and the resulting posterior estimate and credible interval being used as the final prediction and margin of error.

Above, seats with lines that cross zero are the ones where the blended prediction “missed” the result, though it should be noted that all the outcomes fell within the margin of error.

It should be noted that although the combination of both ratings and the forecasting model are a useful tool for understanding U.S. House elections, the predictiveness of the blended predictions is worse earlier in the cycle. This is due to seat ratings moving less predictably than other indicators (like national congressional polling) and producing more noise in the estimates in, say, June of the election year, rather than late October or November. In other words, this method only works better than the forecast alone when the final race ratings for House seat are made available.

So, what do we know at this point in the piece — and in the 2018 midterms cycle?

First, the data are clear that discrete seat ratings perform ever-so-slightly worse than the data-driven quantitative forecasts, though both did correctly predict the outcome of the House majority in 2016. Second, it’s evident that probabilities are slightly more certain in the quantitative forecast than in the qualitative ratings and that the seat ratings give less room for flexible probability within categories. Third, I find that errors in the quantitative analysis are sometimes controlled for in the discrete ratings, though errors exist elsewhere to cancel out some of these gains. Finally, a blend of both measurements provides the best seat-by-seat understanding of the midterm elections.

Given the track record of both approaches, it is worthwhile to consider the following: what are the differences in the outputs of both models today?

The Present: Different forecasts for different seats

Given the different methods employed by myself and the team at the University of Virginia Center for Politics and the differences in forecasts in 2016, one should expect that there are variations in the predictions for November 2018 as well.

Indeed, there are (some) large differences in our forecasts. Though a portion of the discrepancies can be explained by the inability of qualitative ratings today to adjust for movement in the national environment — which my forecast does, and which pushes expectations toward the party out of power — and some can be explained by my quantitative method not taking well into account the quality of some district’s nominees, other differences can reflect real disagreement among the methodology.

The table below details the 15 districts where my forecasts and the UVA qualitative forecasts disagree — in other words, where one of us say the Democrats/Republicans are more likely to pick up a seat, and the other says Republicans/Democrats are the more probable victors.

Table 2: Differences between Crystal Ball and Crosstab House forecast Democratic win probabilities

However, many of these differences arise in seats that either of us rate as Toss-ups, accentuating differences between forecasts for contests where we’re actually quite uncertain about the outcome of the races. Here’s what that table looks like without Toss-up seats.

Table 3: Differences between Crystal Ball and Crosstab House forecast Democratic win probabilities, excluding Toss-ups

As you can see, where it matters most (in calling districts for either party), the methods are arriving at roughly the same conclusions. It is in Toss-up seats where the biggest discrepancies arise.

However, what really counts is the probability of victory assigned to either party — in each seat and in the nation as a whole — and the range of outcomes in the upcoming midterm elections.

Next, I answer the question of how many seats Democrats are likely to win according to the seat probabilities assigned by both methods. What’s going to happen in November?

The Future: Who’s going to win the majority?

Above all else, the main advantage of the quantitative model is the ability to simulate thousands of possible elections — some where Democrats do better, some where Republicans do, within the statistical range of error we’ve observed in the past — and generate a final probability for the chance that Democrats win a majority of seats. In its current form, the House ratings here at Sabato’s Crystal Ball (and elsewhere) are not able to compete with that (and perhaps, neither should they!).

However, the cool thing about the quantitative abstraction of the UVA seat ratings detailed above is that it comes with everything we need to be able to plug it into the simulation phase of the formal model. This way, we can account for the inherent error in the ratings while producing a nationwide probability that Democrats may win the majority of seats in the U.S. House of Representatives.

Better yet, instead of doing exactly what my model does (which produced some strange-looking seat outcomes due to inflated margins of error transitioning from the seat rating to average win margin), I can skip that step and work directly with the win probabilities derived from the past accuracy of ratings. For every trial, I model the expected change in probability from varying a district’s forecast vote margin according to (1) national error and (2) correlated seat error. I use the inverse normal distribution to make sure that probabilities are adjusted properly (seats rated as Toss-ups will see larger shifts in probability than Safe seats, for example).

Akin to the visualizations on my forecasting homepage, below I graph the range of possible seat outcomes for both models. The taller the line, the more likely it is that the Democrats win that number of seats.

Figure 5: Differing House forecasts

Notes: This figure shows the distribution of possible number of Democratic seats after the 2018 U.S House midterms according to the same simulation method applied to two sets of seat forecasts. Forecasts according to ratings and probabilities generated on June 7, 2018.

After simulating 50,000 trial elections with the seat-level win probabilities assigned by the UVA Center for Politics, we have our answer: Democrats are much more favored in the quantitative forecast (close to a 60% chance of winning the majority of seats) than in the discrete ratings (about a 40% chance). The expected number of districts won by Democrats is 10 seats larger in my own forecast than in the ratings at this website.

Why the difference? The discrepancy is explained by two major factors:

First, the quantitative method has the advantage of being able to look into the future, estimating where the national environment is likely to be in November this year by tracing movement in past election cycles from June until election day.

Second, there is a higher number of Lean and Likely Republican seats in my continuous, probabilistic data than in the UVA ratings. This explains the large right tail of possible Democratic seats graphed in blue above; there are more seats that Democrats can pick up in a large blue “wave” in my data, pushing expectations to the right.

You can see these differences in the cross tabulation (see what I did there?) below. Although mine and the UVA ratings agree on 118 Safe Republican seats, I rate 38 GOP-held districts as Likely or Lean that are rated as Safe here. Nine of their Lean/Likely ratings I give a Toss-up (between 40 and 60% chance of Democratic victory) designation.

Table 4: Comparing forecasts

This again shows the main limitation of the ratings-based approach, at least in trying to plug it into a purely quantitative forecasting method: the discrete scale of probabilistic ratings. Since each seat is put into a category that has a historical probability of voting for a Democrat or Republican, seats are not allowed to vary in the probabilities assigned to them. Even if one Lean Republican seats looks more competitive than another nearly Lean Republican seats, they are both placed in a bucket that elects Republicans 18% of the time.

On the other hand, my forecast generates a specific win margin and win probability for each individual seat, so NC-02 and PA-10 can have their own specific probabilities assigned to them (36% and 22%, respectively) on a continuous scale. All of these differences add up to produce a forecast that is more optimistic about Democrats’ future in the U.S. House of Representatives.

Closing thoughts

This piece has reviewed the methods, history, and current projections of two different prediction methods — a crystal ball and a statistical model — for this fall’s midterm elections to the U.S. House of Representatives. I have reviewed differences in probabilities between the measures, accuracy in the 2016 elections, and even prognosticated about the future of the House according to the two varying processes. What is clear is that the two vary considerably in some parts, and are similar in others.

Not any one approach is god’s gift to election handicapping, however. Both mine and the UVA Center for Politics forecasts have erred in the past, some of which cancel each other out and some of which do not, and a combination of both projections performs best in predicting the final partisan breakdown of seats. Indeed, even within methods, there is variation; the Cook Political Report and Inside Elections race ratings, as well as those published by media outlets like CNN all have disagreements — sometimes large ones — about ratings in some key districts. A new statistical forecasting model published by my soon-to-be colleagues at The Economist also has differences with my own personal method. While these differences can individually sometimes mislead, the truth frequently lies between them all. Apart from the technical details, there are also important differences in how we conceptually utilize continuous and discrete forecasts (some scientific, some journalistic) that this article does not discuss.

As we head into the heat of the summer of this 2018 midterm cycle pundits, politicos, and voters alike should take note of the past, present, and future differences between quantitative forecasting methods and the typical race-based handicapping. If the past holds true, the former will do well at producing precision probabilities for each U.S. House seat based on its individual characteristics, and the latter will do well at reducing large deviation from the forecast that typically arises from issues with candidate quality and rapidly changing districts.

Whatever method you pick (if you’ve learned anything from this piece, you ought to pick both), rest assured that the two are well-tested methods that will get us 90-98% of the way to foreseeing what will happen on November 6, 2018. It’s the remaining 2-10% that will make or break House forecasting this fall.

G. Elliott Morris is a data journalist who specializes in elections, political science, and predictive analytics. Elliott has previously crunched numbers for Decision Desk HQ and the Pew Research Center. Elliott graduated from the University of Texas at Austin in May and he joins The Economist in July. Follow him on Twitter and at his blog, TheCrosstab.com.