Expert Predictions

This is blog post #4 in a series of analytical posts about the 2022 midterms. This post is affiliated with Gov 1347: Election Analytics, a course at Harvard University in the Government department.


The Plan This Week

Last week, I took a close look at The Economist’s and FiveThirtyEight’s House forecast models and their methodology, and compared them. With these models in mind, I updated my own 2022 midterm forecast and model to include national economic conditions (Gross Domestic Product, Real Disposable Income, and unemployment), generic ballot polls (partisan preference), and the midterm-president’s-party effect.

One of the most interesting differences between the two models was that FiveThirtyEight explicitly mentions their use of expert forecasts (Cook Political Report, Inside Elections, and Sabato’s Crystal Ball) to predict future elections. Expert forecasts are intuitively a strong predictor of the race outcomes, as they are the best predictions elections experts have given all they data they gathered. This week, I wanted to investigate how accurate these expert predictions really are, specifically for the 2018 midterms.

How Accurate Are Expert Predictions?

To conduct this week’s investigation, I compare the actual results of the 2018 midterm elections to the average election ratings of experts. The 2018 results shown below are Democrat vote shares in each district. The 2018 expert election ratings come from Ballotpedia, with the election ratings taking on integer values 1 through 7 – 1 represents a Solid/Safe Democrat prediction and 7 represents a Solid/Safe Republican average expert prediction for each district.

Actual Vote Share in 2018, by District

Below is the actual (D) vote share in the 2018 midterm elections at the district level.

Actual 2018 (D) Vote Share.

Most of the map is either red or white, which seems to suggest that most district vote share results were in the Republican party’s favor or close between the two major parties. However, it is important to remember that for the House of Representatives, land is not proportional to representation, population is. Also, the president’s party often performs poorly in midterm years, as discussed in detail in the first blog post of this series. Those two reminders considered, it then makes sense that the Democrats actually took the House in the 2018 midterms, with 235 seats going to Democrats and 199 going to Republicans (New York Times 2018).

Expert Predictions for Vote Share in 2018, by District

Below are the expert ratings for the 2018 midterms, which I call predictions because the ratings reflect what the experts believed would happen in each congressional district election. Red areas represent high expert confidence that the region will go to Republicans, while blue areas represent high confidence that the region will go to Democrats. Expert Ratings for 2018 Midterms.

Evaluation

For the most part, the predictions are pretty accurate via visual investigation. Most of the areas that actually had majority-Republican vote shares in the 2018 elections were predicted to be likely or solidly Republican by the experts on average. The blue districts of the 2018 midterms, like the ones along the west coast of Washington, Oregon, and California, in Arizona and New Mexico, in southern Texas, and scattered throughout the Rust Belt, were impressively predicted by the experts and their ratings, reflected in the standout blue coloring in the plot above. The experts were also able to pin down the close races, like the ones in southern New Mexico and in Maine. Overall, I think the average expert ratings predicted the congressional races at the district level very well, despite being just one predictor (that is a culmination of many predictors considered by the experts) in my model.

My 2022 Model and Forecast, Updated

My New Model

My national model remains the same as it was last week, and I cannot fully extend my national model to the district level because it uses national variables like GDP change, as well as national responses to the generic ballot polls. That said, I can incorporate this week’s investigation into a district-level model that predicts the House races by district using the most recent expert ratings data I have access to. These district-level models are of course imperfect because they only use expert ratings, but across all of them, they have an average R-squared of 0.78.

My New Forecast

Below are my models’ predictions for the major vote share taken by Democrats for the 94 congressional districts where the expert ratings were available.

statedistrictpred
AlaskaAL48.03
Arizona151.30
Arizona249.10
Arizona648.19
California2148.30
California2260.08
California2558.31
California2654.77
California349.02
California4551.94
California4758.68
California4950.06
California956.05
Colorado348.72
Colorado755.14
Connecticut449.58
Connecticut551.77
Florida1349.10
Florida1548.92
Florida1642.36
Florida252.57
Florida2247.55
Florida2748.87
Florida751.79
Georgia1253.79
Georgia652.09
Illinois1150.91
Illinois1349.98
Illinois1449.43
Illinois1749.32
Illinois653.55
Illinois851.17
Iowa147.23
Iowa248.58
Iowa347.41
Kansas348.34
Maine247.16
Maryland670.30
Michigan1149.35
Michigan347.56
Michigan745.56
Michigan848.92
Minnesota149.60
Minnesota249.52
Minnesota349.31
Minnesota850.25
Missouri246.53
Nebraska250.03
Nevada350.07
Nevada452.93
New Hampshire149.69
New Hampshire251.32
New Jersey1160.44
New Jersey246.43
New Jersey349.77
New Jersey550.37
New Jersey749.27
New Mexico150.81
New Mexico248.27
New York148.92
New York1145.26
New York1851.87
New York1948.23
New York246.52
New York2248.60
New York2549.11
New York346.90
New York453.23
North Carolina1348.93
North Carolina653.43
North Carolina751.02
North Carolina949.77
Ohio146.03
Ohio1020.01
Ohio1358.82
Ohio1547.43
Ohio762.47
Oregon552.66
Pennsylvania150.84
Pennsylvania1046.13
Pennsylvania1248.24
Pennsylvania1741.72
Pennsylvania651.10
Pennsylvania748.00
Pennsylvania847.94
South Carolina149.78
Texas2349.28
Virginia1048.25
Virginia250.30
Virginia548.37
Virginia750.98
Washington350.43
Washington850.44
Wisconsin344.73

References

[1] Ballotpedia. (2022). https://ballotpedia.org/Main_Page

[2] Cook Political Report. (2002). House Race Ratings. https://www.cookpolitical.com/ratings/house-race-ratings

[3] Inside Elections. (2022). House Ratings. https://insideelections.com/ratings/house

[4] Sabato’s Crystal Ball. (2022). House Race Ratings. https://centerforpolitics.org/crystalball/2022-house/

[5] The New York Times. (2018, November 6). U.S. House election results 2018. The New York Times. https://www.nytimes.com/interactive/2018/11/06/us/elections/results-house-elections.html