Nate Silver and the Stylish Statistics of Predicting Elections

Guest Author: Michael Dimock

Nate Silver speaking at SXSW in 2009. Photo by Randy Stewart, available under CC BY-SA 2.0 license.

Nate Silver is a statistics guru whose claim to fame has come from correctly predicting 49 of the 50 states in the 2008 presidential election as well as all 35 senate races. He then showed that this performance was far from a fluke as he gave another outstanding prediction for the 2012 U.S. election. A leader in the field of political statistics, he is the founder and editor-in-chief of his popular website FiveThirtyEight. Extending far beyond major political elections, the website also works with the statistics of sports, science, health, economics, and culture. Nate Silver can be referred to as both a star and a statistician—a combination that sounds more like a contradiction than a reality.

Being a mathematics major, I have learned to never blindly trust the statistics that show up in news reports, and Nate Silver is often one of the first to warn others to be wary of statistics presented in the media. A good intro piece to Silver’s statistical style and ability is the following video where Silver explores the question, “Does racism affect how you vote?”

In the video, Nate Silver breaks down the 2008 presidential race to find out if the ethnicity of a candidate was a big factor in the election outcome. While each person may have their own knee-jerk reaction as to whether or not there is an effect, Nate Silver does a fine job of objectively breaking down each piece of statistical evidence that is presented. By understanding the complexity of the scenario, Silver is very open about the level of accuracy of the data collected. Silver cleverly looks at many factors and has a good grasp of how various statistics should be used to understand the raw data. (And the final conclusion is that racism does in fact play a role in how the nation votes for the president.)

In another talk, Nate Silver explores the unique challenges and goals of political modeling and how to do it effectively.

Silver discusses the difficulty of using statistics as a means to predict the outcome of political campaigns. The first and most understandable complication is the lack of data. There have been few presidential races in recent history, so there is not a lot of past data to work with. Beyond that, the data collected for each campaign changes; for example, rapidly advancing technology continually affects the way we poll.

As a political statistician, it is difficult to make the determination of which polls will be most effective, considering each may inadvertently attract a different subset of the population to participate. As Silver looks at all of this data, he must analyze statistics on statistics to identify the best polls to use. This is where Silver’s job becomes particularly difficult and where experience plays a large role: he needs to determine how valuable each piece of information is and properly weigh each bit accordingly.

Nate Silver builds safeguards and rules into his statistical models which protect against intentional or unintentional biases or assumptions in creating the statistics. These rules consist mainly of how his models will weigh different kinds of data that is collected, making Silver a kind of numbers scientist who is very careful to not allow his hypotheses to affect the conclusions of his careful experiments. Instead of trying to make the data match a certain result, he lets the data speak for itself and he listens to what it has to say. The recent growth and success of quantitative analysis of social issues is nicely written up in The Guardian’s piece Nate Silver: it’s the numbers, stupid.

Beyond that, we have the issue of giving consumers context for the data in order to assist with, for instance, the distinction between causation and correlation. Silver gives an example of correlation with no proven causation: in 12 of the last 14 elections, if the Washington Redskins won the game immediately before the election, then the incumbent won the election. Taken out of context, this raw data could be fleshed out to produce plenty of misleading reports in the media. Along the same vein, the fascinating classic 1954 book How to Lie with Statistics by Darrell Huff helps to open the readers’ eyes to how easy it can be to produce a desired conclusion from a given set of raw data by using carefully chosen statistics.

Nate Silver’s high popularity stemming from his success in predicting political campaigns and social issues shows just how much the public appears to appreciate his objective style of presenting statistics amid a sea of overblown hype in the mainstream media.