Understanding scientific and technical progress requires measures of both the rate and direction of resources directed to innovative efforts and the outputs produced. This project leverages advances in computational power to map the evolution of research fields based on researchers' project portfolios in idea spaces. The research takes advantages of shocks, such as changes in policies and research costs, to use empirical research techniques to understand the factors that affect the directions into which science and technology evolve. This work has important implications for understanding the rate and direction of technological change.
Specifically, the project uses techniques based on machine learning and natural language processing that measure the incidence and configuration of keywords in published research to quantify the similarity of groups of such articles to define idea space and to subsequently measure the ways in which idea space evolve in response to shocks in research costs and public policies. The research applies these techniques to three contexts: (a) how changes in the costs of research materials affect research trajectories in motion-sensing technology, (b) how researchers in quantum computing change their project portfolios in response to a controversial approach that differs from an established research paradigm; and (c) how pharmaceutical firm research trajectories change in response to news about rivals' drug discovery projects.