Theory-Driven Big Data (TDBD) Approaches: Final Report

Author(s):

Stephen R. Mitroff, Dwight J. Kravitz, Chloe Callahan-Flintoft, Sean M. Fitzhugh, and Kelvin S. Oie

Abstract:

Traditional theory-based methods are often reductive, struggling to detect smaller but critical effects, and their predictions frequently fail outside controlled laboratory settings. Conversely, while the statistical power of “Big Data” is immense, purely data-driven approaches can result in poorly generalizing “black box” models that limit explanatory interpretation and practical understanding. This project explored a novel perspective on human behavior—the Theory-Driven Big Data approach—by leveraging the complementary strengths of traditional theory-driven and contemporary data-driven research. Project efforts focused on addressing fundamental questions about human behavior to inform performance improvements in demanding situations, such as those faced by U.S. Soldiers. We applied novel, theory-driven analyses to a unique dataset of nearly 4 billion trials from over 15 million individuals playing the mobile game Airport Scanner, which provided an unprecedented opportunity to quantify the effects of established theoretical mechanisms, such as memory decay and task interference, and to examine the impact of standard experimental design practices like counterbalancing and randomization in a real-world context.