How do Humans Interact with Algorithms? Experimental Evidence from Health Insurance
Algorithms are increasingly available to help consumers make purchasing decisions. How does algorithmic advice affect human decisions and what types of consumers are likely to use such advice? We conducted a randomized, controlled trial comparing the effects of offering personalized information, either with or without algorithmic expert recommendations, relative to offering no personalized information for consumers choosing prescription drug insurance plans. Treated consumers were more likely to switch plans and to choose a plan that lowered their total spending on drugs. The behavioral response was more pronounced when information was combined with an algorithmic expert recommendation. We develop an empirical model of consumer choice to examine the mechanisms by which expert recommendations affect choices. Our experimental data are consistent with a model in which consumers have noisy beliefs not only about product features, but also about the parameters of their utility function. Expert advice, in turn, changes how consumers value product features by changing their beliefs about their utility function parameters. We further document substantial selection into who demands expert advice. Consumers who we predict would have responded more to algorithmic advice were less likely to demand it.
We thank Palo Alto Medical Foundation (PAMF) patient stakeholders and trial participants, as well as numerous Palo Alto Medical Foundation Research Institute (PAMFRI) team members for making the trial possible. We are grateful to Liran Einav, Aureo de Paula, Jonathan Kolstad, Amanda Kowalski, Jennifer Logg, Matthew Notowidigdo, Bobby Pakzad-Hurson, Stephen Ryan, Frank Schilbach, Jesse Shapiro, Justin Sydnor, Kevin Volpp, Stefan Wager and seminar participants at McGill University, University of Pennsylvania, CESifo Digitization, NBER Summer Institute, ASHEcon, Chicago Booth Junior Health Economics Summit, Stanford University, Indiana University, Boston University, Brown University, University of California Berkeley, AHEC 2019, and 2019 Conference on Health IT and Analytics for their comments and suggestions. We also thank Sayeh Fattahi, Roman Klimke, and Vinni Bhatia for outstanding research assistance. Research reported in this paper was funded through a Patient-Centered Outcomes Research Institute (PCORI) Award (CDR-1306-03598). The statements in this presentation are solely the responsibility of the authors and do not necessarily represent the views of the Patient-Centered Outcomes Research Institute (PCORI), its Board of Governors or Methodology Committee. The project also received financial support from Stanford Innovation Funds. Polyakova further gratefully acknowledges support from the National Institutes on Aging (K01AG059843). The experiment reported in this study was pre-registered in the ClinicalTrials.gov Registry (NCT02895295) and can also be found in the AEA RCT Registry under AEARCTR-0004465. The views expressed herein are those of the authors and do not necessarily reflect the views of the National Bureau of Economic Research.
In the last three years, I have received consulting, expert witness or advisory fees from St. Joseph Health System, Quinn Emanuel Urquhart and Sullivan, LLP on behalf of Health Republic Insurance Company, Premera Blue Cross, United Healthcare, and Fidelity Investments. I have received research support from Blue Shield of California, the Agency for Healthcare Research and Quality, the National Institute of Health Care Management, the Patient Centered Outcomes Research Institute, and the Robert Wood Johnson Foundation.