Biased Sampling of Early Users and the Direction of Startup Innovation
Using data from a prominent online platform for launching new digital products, we document that the composition of the platform's ‘beta testers’ on the day a new product is launched has a systematic and persistent impact on the venture's success. Specifically, we use word embedding methods to classify products on this platform as more or less focused on the needs of female customers. We then show that female-focused products launched on a typical day – when nine in ten users on the platform are men – experience 45% less growth in the year after launch. By isolating exogenous variation in the composition of beta testers unrelated to the characteristics of launched products on that day, we find that on days when there are unexpectedly more women, this gender-product gap shrinks towards zero. Further, consistent with a sampling bias mechanism, we find that the composition of beta testers appears to impact VC decision making and the entrepreneur's subsequent product development efforts. Overall, our findings suggest that the composition of early users can induce systematic biases in the signals of startup potential, with consequential effects – including a shortage of innovations aimed at consumers who are underrepresented among early users.
Author order is alphabetical and all authors contributed equally. We are extremely grateful to Kevin Laws and Ryan Hoover for providing us access to data, and to participants at Berkeley Haas, Columbia Business School, FDIC-Duke Fintech Conference, HBS, SIE Workshop, Chalmers University, Strategy Science Conference, London Business School, Duke's Strategy PhD Seminar, NBER Productivity Lunch, Max Plank Institute for Innovation and Competition, Rice University, Rotman's Workshop on Gender, Race and Entrepreneurship, and Wharton School of the University of Pennsylvania for helpful comments. Koning and Nanda thank the Division of Research and Faculty Development at HBS for financial support. Koning thanks the Kauffman foundation for financial support. During the academic years 2019-2021, Nanda is a Visiting Professor at Imperial College, London. The views expressed herein are those of the authors and do not necessarily reflect the views of the National Bureau of Economic Research.