How do Humans Interact with Algorithms? Experimental Evidence from Health Insurance
Algorithms are increasingly available to help consumers make purchasing decisions. How does algorithmic advice affect human decisions and what types of consumers are likely to use such advice? We use data from a randomized controlled trial of algorithmic advice in the context of prescription drug insurance to examine these questions. We propose that algorithmic recommendations can affect decision-making by influencing consumer beliefs about either product features (learning) or how to value those features (interpretation). We use data from the trial to estimate the importance of each mechanism. We find evidence that algorithms influence choices through both channels. Further, we document substantial selection into the use of algorithmic expert advice. Consumers who we predict would have responded more to algorithmic advice were less likely to demand it. Our results raise concerns regarding the ability of algorithmic advice to alter consumer preferences as well as the distributional implications of greater access to algorithmic advice.
We thank Palo Alto Medical Foundation (PAMF) patient stakeholders and trial participants, as well as numerous Palo Alto Medical Foundation Research Institute (PAMFRI) team members for making the trial possible. We are grateful to Liran Einav, Aureo de Paula, Jonathan Kolstad, Amanda Kowalski, Jennifer Logg, Matthew Notowidigdo, Bobby Pakzad-Hurson, Stephen Ryan, Frank Schilbach, Jesse Shapiro, Justin Sydnor, Kevin Volpp, StefanWager and seminar participants at McGill University, University of Pennsylvania, CESifo Digitization, NBER Summer Institute, NBER Economics of AI Conference, ASHEcon, Chicago Booth Junior Health Economics Summit, Stanford University, Indiana University, Temple University, Boston University, Brown University, University of California Berkeley, AHEC 2019, 2019 Conference on Health IT and Analytics, University of North Carolina-Chapel Hill, the Electronic Health Economics Conference, University of Illinois, Johns Hopkins University, Duke University, and Claremont Graduate College for their comments and suggestions. We also thank Sayeh Fattahi, Roman Klimke, Vinni Bhatia, and Sarah Boegl for outstanding research assistance. Research reported in this paper was funded through a Patient-Centered Outcomes Research Institute (PCORI) Award (CDR-1306-03598). The statements in this presentation are solely the responsibility of the authors and do not necessarily represent the views of the Patient-Centered Outcomes Research Institute (PCORI), its Board of Governors or Methodology Committee. The project also received financial support from Stanford Innovation Funds. Polyakova further gratefully acknowledges support from the National Institutes on Aging (K01AG059843). The experiment reported in this study was pre-registered in the ClinicalTrials.gov Registry (NCT02895295) and can also be found in the AEA RCT Registry under AEARCTR-0004465. The views expressed herein are those of the authors and do not necessarily reflect the views of the National Bureau of Economic Research.
M. Kate Bundorf
In the last three years, I have received consulting, expert witness or advisory fees from St. Joseph Health System, Quinn Emanuel Urquhart and Sullivan, LLP on behalf of Health Republic Insurance Company, Premera Blue Cross, United Healthcare, and Fidelity Investments. I have received research support from Blue Shield of California, the Agency for Healthcare Research and Quality, the National Institute of Health Care Management, the Patient Centered Outcomes Research Institute, and the Robert Wood Johnson Foundation.