Sampling Bias in Entrepreneurial Experiments
Using data from a prominent online platform for launching new digital products, we document that ‘sampling bias’—defined as the difference between a startup’s target customer base and the actual sample on which early ‘beta tests’ are conducted—has a systematic and persistent impact on the venture’s success. Specifically, we show that products with a female-focused target market launching on a typical day, when nine in ten users on this platform are men, experience 45% less growth a year after launch than those for whom the target market is more male-focused. By isolating exogenous variation in the composition of beta testers unrelated to the characteristics of launched products on that day, we find that on days when there are unexpectedly more women beta testers on the platform—reducing the amount of sampling bias for female-focused products—the gender-performance gap shrinks towards zero. Our results highlight how sampling bias can lead to fewer successfully commercialized innovations for consumers who are underrepresented among early users.
-
-
Copy CitationRuiqing Cao, Rembrand M. Koning, and Ramana Nanda, "Sampling Bias in Entrepreneurial Experiments," NBER Working Paper 28882 (2021), https://doi.org/10.3386/w28882.Download Citation
-
Published Versions
Ruiqing Cao & Rembrand Koning & Ramana Nanda, 2024. "Sampling Bias in Entrepreneurial Experiments," Management Science, INFORMS, vol. 70(10), pages 7283-7307, October, DOI: 10.1287/mnsc.2021.01740. citation courtesy of 
Ruiqing Cao & Rembrand Koning & Ramana Nanda, 2024. "Sampling Bias in Entrepreneurial Experiments," Management Science, vol 70(10), pages 7283-7307.
More from the NBER
In addition to working papers, the NBER disseminates affiliates’ latest findings through a range of free periodicals—the NBER Reporter, the NBER Digest, the Bulletin on Health, the Bulletin on the Economics of Alzheimer's, and the Bulletin on Entrepreneurship—as well as research spotlights, lectures, and panel videos.
