Datafication and You

“This overturns centuries of established practices and challenges our most basic understanding of how to make decisions and comprehend reality.” — Big Data: A Revolution That Will Transform How We Live, Work, and Think

The rise of big data has been a rich vein to tap in nearly every discipline and industry. Accompanying this explosion of data and (perhaps more significantly) the computing power to utilize it is a wealth of discussion on the potential effects on nothing less than human understanding and the future of civilization. Latest in the genre is Big Data: A Revolution That Will Transform How We Live, Work, and Think, by Viktor Mayer-Schönberger, professor of Internet governance and regulation at Oxford University, and Kenneth Cukier, data editor at The Economist. The duo offers some familiar cautionary alarm, but is supported by new insight into how data analysis has changed and why it matters.

Mayer-Schönberger and Cukier describe the “datafication” of society—taking something that has never before been treated as data and turning it into a numerically quantified format (they point to research at the Advanced Institute of Industrial Technology in Tokyo that uses sensors to identify how a person sits). Rather than focus on how datafication will affect any particular field, Mayer-Schönberger and Cukier consider how it will affect the fundamentals of human thought and decision-making, and how that in turn might affect approaches to privacy and individual liberty.

The book outlines three major “shifts of mindset”:

  • Analysts are now more likely to be able to collect and analyze an entire data set (or a significant portion of it) instead of seeking out statistically representative portions of a data set.
  • Analysts can now tolerate more errors in the data. Because the size of data sets has increased so much, a few errors are far less likely to skew and undermine the entire analysis. Precision may be sacrificed in order to identify general trends.
  • Analysts and decision-makers may now care less about causation than they do about correlation. Rather than generating and testing a hypothesis by examining the relevant facts, analysts can use big data to answer questions with brute force, plowing through hundreds of millions of mathematical models until a correlation is discovered.

If the last point is true, it means the Scientific Method has been be flipped on its head. It means considering fundamental changes to how humans answer questions and solve problems. “Society will face a great temptation to… shift to managing risks, that is, to basing decisions about people on assessments of possibilities and likelihoods of potential outcomes,” write Mayer-Schönberger and Cukier, “Big data presents a strong invitation to predict which people are likely to commit crimes and subject them to special treatment, scrutinizing them over and over in the name of risk reduction.”

Furthermore, they worry that as projections become more prevalent and accurate, society may punish people for predicted behavior, negating free will. “The future must remain something that we can shape to our own design,” they write. “If it does not, big data will have perverted the very essence of humanity: rational thought and free choice.”

It’s tempting to dismiss Minority Report-style prophecies as exaggeration, but we’re already seeing shifts in how individuals think and act as a result of vast quantities of readily available data. Just a decade ago, knowledge of youthful indiscretions such as drinking too much at a college party would have been confined to a small circle of friends and perhaps a couple of blurry photos. Today, such activities may be broadcast to hundreds or even thousands of individuals well beyond the college campus, and future employers may make hiring decisions based on this behavior. The digital age has made more aspects of our lives instantly accessible and virtually unforgettable. Consequently, individuals must dramatically adjust their risk calculations before they put that lampshade on their heads at a party, or—more soberly—attend a protest rally in support of a controversial cause.

But agonizing about this particular dystopian future devoid of free will misses the simpler point: big data analysis can have a profound effect on our basic interaction with the world. Sometimes the effects are positive—for example, eliminating certain cognitive biases, or correcting our tendency to see patterns or causalities where none exist. In other cases, these changes can be negative—sapping our curiosity to understand why something occurs, or stifling innovation and inspiration that might “go beyond the data.” It’s the understanding of these analytical shifts as ultimately helpful or harmful that will define the ongoing data revolution.

Companies will need to be ahead of these changes as they design and build new technologies. Palantir has always put a great deal of stock in the role of humans in data analytics. We design capabilities that enhance the ability of a human analyst to understand data and derive actionable intelligence from it; we never displace the human as the ultimate data processor. Even so, design itself affects users—intentionally or not—and influences behaviors based on how the user interface is built and data is presented. Any responsible player in the big data game must wrestle with the question: are we enhancing some human ingenuity at the cost of stifling other unique aspects of our intelligence?

Making this determination is not easy. We can point to enormous success in facilitating criminal investigations for our law enforcement customers. Police officers using Palantir can assess swaths of information deftly, allowing them to identify new suspects or lines of inquiry that might have never surfaced in the past. But it’s harder to know how this powerful tool is changing how officers think in their day-to-day work. Are officers ignoring their instincts—sometimes honed by years of police work in the field—in favor of what the data is telling them? Would those instincts have led them up an investigative blind alley or might they have resulted in the sort of intuitive mental leap that can crack a case wide open—and that no computer could ever make?

These questions are why Palantir engineers spend so much time in the field working directly with customers and analysts to understand how they did their jobs before Palantir. We want to preserve what worked while addressing friction points that hampered success. It’s why the development of new features in Palantir is largely user driven rather than dictated by the assumptions of engineers. Mayer-Schönberg and Cukier’s book reinforces the urgency of our mission: to provide technology that preserves the essential role of human judgment and individual responsibility in big data analytics, and to do so through design and engineering practices that augment sound decision making rather than artificially shifting or displacing that which already works.