Of all the speakers chosen for the panel on “Race and the American Voter,” Nate Silver is possibly least qualified for the role. The other members of the discussion are, unlike Silver, mostly minorities; all add more to the conversation than Silver, and when he does speak, he doesn’t really bring up the detailed statistical analyses that made the 2008 presidential campaign predictions on his blog, FiveThirtyEight.com, so remarkably accurate. Nonetheless, Silver is almost assuredly the most well-known speaker in attendance; to the thousands of political junkies who frequented his website every day, multiple times a day, until November 4, his name is as familiar as Obama’s, Clinton’s, or Sarah Palin’s.
It would be cynical to say that his star power is the only reason this slightly uncomfortable-looking number cruncher has been invited to the day’s event, though. Nor is it due solely to the fact that the University of Chicago—which hosted the talk in Ida Noyes Hall on December 11—is his alma mater. As Silver is demonstrating, detailed quantitative data evaluation can be really quite useful in creating prediction models that apply to a startling variety of fields. His work with FiveThirtyEight is just one example; before that, he caught the attention of fantasy baseball players nationwide with his development of PECOTA, or the “Player Empirical Comparison and Optimization Test Algorithm,” a sabermetric (baseball analysis) system developed to forecast the performances of Major League Baseball players. Silver came up with it while working as a consultant in the years following his graduation from the UofC in 2000; it first appeared in a book published by sabermetrics think tank Baseball Prospectus in 2003. He joined the organization’s staff one year later.
“I was always a baseball fan, and I needed something to occupy my time, I was kind of bored…so I could do this [work on the PECOTA analyses] and it looked like I was doing work,” Silver explains, recalling the system’s origins. “But basically, I thought a lot about how baseball players would do, and a lot of [existing projections] weren’t as good as they could be…basically I get frustrated enough and it’s like ‘Fuck it, I’m going to do it myself.’”
The same mentality applied when Silver shifted his gaze from the baseball diamond to the political arena. “I was getting into the primaries in part because I was trying to procrastinate from doing other work—it’s kind of a theme I guess,” he tosses off with a laugh. “And I was looking at a lot of blogs and they weren’t very analytical, they weren’t very quantitative, and with politics, there’s a lot of data. [The blogs were] all kind of wishy-washy,” he explains, “and if you go into a campaign office…[the analyses] are very quantitative.”
So Silver began analyzing politics through the same type of quantitative lens, and started posting for the liberal blog Daily Kos under the pseudonym “Poblano” in late 2007. But the limitations of this approach soon took their toll. “I wanted to track how Barack Obama and Hillary Clinton were doing against John McCain, [and] I wanted to do this on a daily basis…I figured I’d start this [FiveThirtyEight], and periodically start to cross-post” between the two websites, he explains. “I didn’t want to hog too much attention [from the Daily Kos],” he laughs.
In March 2008 Silver—still posting under his pseudonym—launched FiveThirtyEight.com; a month later he was joined by Sean Quinn, a fellow Daily Kos poster who contacted him, eager to help. Quinn brought his own strengths to the fledgling endeavor. “He’d worked on campaigns before, and he’d worked on the ground game,” Silver says. “I think that’s something not covered very well by the mainstream press…that stuff tends to be neglected, I think.”
Stemming from the popularity of both Quinn’s and “Poblano’s” postings at the Daily Kos, FiveThirtyEight attracted a significant fanbase from the start, but it wasn’t until after the North Carolina and Indiana primaries on May 6 that its popularity really exploded. Silver recalls the scenario: “I know the polls show it’s really tight in NC, but we think Obama is going to win by thirteen, fourteen points, and he did…Anytime you make a prediction like that people give you probably too much credit for it,” he muses. “But after that it”—both the website’s renown and his own—“started to really take off. It’s pretty nonlinear, once you get one mention in the mainstream media, other people [quickly follow suit],” Silver explains. On May 30, he revealed his true identity, and the interview requests and late-night television appearances quickly began to pile up.
So what’s the secret behind Silver’s predictive success? “There’s no one magic thing,” Silver comments, “but with anything like this, when you want to be very thorough about creating a model, you get ninety percent of the way there the first time through. People say, why should I make this effort to get a little bit of gain at the margin?” But it’s that little bit that makes Silver’s predictions so much more accurate. When creating his models, he looks at “every stat imaginable pretty much”—then, if he finds that “these projections seem wrong, they seem off somehow…you go back and you try to make an improvement.”
Of course, many other diligent analysts follow the same advice, and for those who aspire to Silver’s success, his approach can be somewhat readily adopted. “The methodology’s fairly well disclosed,” he states. “I’m not going to provide a total playbook to people, but most of it should be fairly clear…You can’t really patent [your technique],” he admits, “but you have an edge because you’re always working on it more and everyone’s caught up, while you’re making changes…so you know, you can’t just kind of create something and just sort of rest on your laurels. With the political model, even as late as two or three weeks before the election we were making adjustments to it,” he explains. “If you don’t have that advantage of getting to something first, you’re always playing catch-up.”
Resting on his laurels is the last accusation anyone could make against Silver. Besides his continued work with Baseball Prospectus and the maintenance of FiveThirtyEight, Silver is also writing a book for Penguin Press, slated for release in summer 2010. With it, Silver plans to take his quantitative analyses and his knack for producing startlingly accurate prediction models and applying them well beyond the realms of baseball and politics. It’s going to be “a big kind of Freakonomics book, but I’m not sure [how it will go] because I haven’t started it yet,” he laughs. “Basically, I want to talk to forecasters and fashion [experts] and see what these people have in common.” Like fellow UofC celebrity Steven Levitt, Silver plans to analyze “stuff you don’t expect to be in a quantitatively-oriented field.”
But political junkies, rest assured: Silver has no intention of relinquishing his role in the Beltway brouhaha. He’s got lots on the agenda for FiveThirtyEight, including the possibility of bringing on new writers. ”It’d be nice to add someone a little right of center…we don’t really tend to be a partisan site, but at the same time, if I have an opinion, I have a right to make it known,” he asserts—something devout followers have no doubt picked up upon already.
He also plans to start training his analytical eagle’s eye on Congress. “One thing we’re going to do is when there’s an important vote in the Congress, we’re going to create a box score, to see how people voted…but we also want to look at what drove people to vote a certain way. How did people vote from Texas? How did people vote who get money from the oil industry? Who needed to get re-elected? Because a lot of things that passed like sixty-three votes to thirty-five or something… people don’t really stop and think why did people vote in certain ways,” he reasons. In this way, he hopes readers will “start to understand what motivates certain congressmen—are they more concerned with their constituents or who’s giving them money?”
With coverage like that, Silver’s sure to keep his name in the spotlight, even as names like Sarah Palin (hopefully) begin to fade out. It’s an enviable position to be in, but not one without its potential pitfalls. “With the baseball stuff, there’s less pressure in a way, people don’t get mad…people don’t take it so personal—if you insult the Pittsburgh Pirates it’s not the same as insulting Sarah Palin,” he muses. But if controversy is the price of the broad relevance of Silver’s statistical analysis, he’s not about to shy away from it. Besides, in the end, you can’t argue with the numbers.