The Effect of Public Funding on Research Output: the New Zealand Marsden Fund
We estimate the impact of participating in the NZ Marsden Fund on research output trajectories, by comparing the subsequent performance of funded researchers to those who submitted proposals but were not funded. We control for selection bias using the evaluations of the proposals generated by the grant selection process. We carry out the analysis in two data frames. First we consider the researcher teams behind 1263 second-round proposals submitted 2003-2008, and look at the post-proposal publication and citation performance of the team as a whole, as a function of pre-proposal performance, the ranking of the proposal by the panel, and the funding. This estimation does not deal with individual researchers’ multiple proposals and funding over time. To disentangle these effects, we consider the 1500 New Zealand researchers who appeared on any of these proposals, and estimate a model predicting annual individual performance as a function of previous performance, recent proposal activity, ranking of any recent proposals, and funding received through recent proposals. Overall, we find that funding is associated with a 6-15% increase in publications and a 22-26% increase in citation-weighted papers for research teams. For individuals, funding is associated with a 3-5% increase in annual publications, and a 5-8% increase in citation-weighted papers for 5 years after grant; however, the lag structure and persistence of this effect post-grant is difficult to pin down. Surprisingly, we find no systematic evidence that the evaluation of proposals by the Marsden system is predictive of subsequent success. We conclude that the Marsden Fund is modestly successful in increasing scientific performance, but that the selection process does not appear to be effective in discriminating among second-round proposals in terms of their likely success.
This project grew out of a suggestion made by Dean Peterson, formerly of the Royal Society of New Zealand. We are grateful to Ryan Burnell for compiling the publications information and to Dave Maré and Shaun Hendy for advice. Useful comments were provided by Ariel Dora Stern, Bronwyn Hall and seminar participants at University of Melbourne, University of Waikato, the New Zealand Economics Association, NBER Summer Institute and Otago University. Financial support from the Motu Research and Education Foundation, Queensland University of Technology and the Ministry of Business Innovation and Employment is gratefully acknowledged. All errors and opinions belong solely to the authors. The views expressed herein are those of the authors and do not necessarily reflect the views of the National Bureau of Economic Research.
I am employed by the administrator of the Fund being studied, i.e., the Royal Society of New Zealand, as their "Programme Manager - Insights and Evaluation".
Jason Gush, Adam Jaffe, Victoria Larsen & Athene Laws (2017) The effect of public funding on research output: the New Zealand Marsden Fund, New Zealand Economic Papers, DOI: 10.1080/00779954.2017.1325921 citation courtesy of