Nate Silver has received LOTS of attention in the mainstream media and among those I follow on Twitter. Forget the pre-election bashing of Silver, the post-election range of opinions on Silver’s work is unbelievably dramatic. Either “America’s Chief Wizard Nate Silver Had the Best Election Night of Anybody…” or “Obama’s big win does not mean Nate Silver is a towering electoral genius.” And, everything in between. All of this has caused lots of discussion about “big data” and the merits of quantitative approaches to political analysis.
On top of the conversation of Silver’s work comes an article from TIME called “Inside the Secret World of the Data Crunchers Who Helped Obama Win.” It’s a fascinating inside look at how the Obama administration embraced data mining and data modeling techniques to gain advantages in the campaigning process. After sharing this article on Twitter, I faced a little resistance, including from sociologist Zeynep Tufekci who stated that this sort of reliance on data-mining “…isn’t great for democracy.” Others, including David Parry, Assistant Professor of Emerging Media and Communications at the University of Texas at Dallas, echoed this sentiment.
From that TIME article it’s own, or even in combination with the hype around Nate Silver’s work, one might conclude that our democracy is in trouble of being overrun by “number crunchers.” Problem is, it’s not that simple. It never is.
Surf the web for a tiny bit longer and you’ll find this fascinating article in the New York Times about President Obama’s meetings with presidential historians. Over a few dinners with these historians, he asked pointed questions that would yield narrative responses from which he could learn.
That was why the president seemed to relish those dinners, the historians surmised: they were an antidote to the cable television news shows and moment to moment political wrangling he disparaged. Each time, he would go around the table, asking the largely left-tilting group how he was doing and what he could learn from the men they had studied, according to interviews with eight biographers who attended.
Yes, President Obama’s team learned a whole lot from the quantitative data they were collecting and analyzing to help his re-election campaign. At the same time, though, the President was collecting qualitative data in the form of historical narratives to help him help our country. He wanted to know, among other things, what lessons could be learned from the past about communicating a vision for our country.
It feels to me like there is a lot of either/or thinking in my worlds these days. This seems particularly true around issues related to data. I wrote about this 6 years ago (!), when applying the ideas in Michael Lewis’ book (now movie) Moneyball to education. Then, I wrote:
Most baseball purists and old school baseball people fervently opposed this sabermetric orientation. They argued that you couldn’t judge a player by crunching numbers. You had to watch the players play, get to know them as people, etc.; in other words, make value determinations by scouting the old fashioned way. The numbers were cold and unreliable, they’d say.
Sounds familiar, right? I went on to write:
The popular sports media would have us believe that sabermetric analysis is an opposing paradigm to traditional baseball scouting methods. But, the fact is that sabermetric analysis has been used by the A’s (and now many other teams as well) as a complement to more traditional methods of scouting and player valuation. It is not as if the A’s have fired all of their scouts and hired all statisticians; their scouting department includes a few number crunchers in addition to all of the scouts who do what they’ve always done… Like sabermetrics in baseball, (statistically oriented) DDDM is a complementary approach to professional judgment in education. They are epistemologically different approaches, but they are not mutually exclusive.
Folks, Nate Silver is not the face of a soulless future where decisions are made exclusively by robo-analysts. His methods and those used by the Obama administration during the re-election campaign, are merely additive to our decision-making repertoire. Let’s please stop this “either/or” thinking around data and think more in “both/and” terms.