NBER Reporter: Research Summary 2008 Number 4
Using Field Experiments in the Economics of Charity
John A. List*
The experimental approach in scientific inquiry is commonly traced to Galileo Galilei, who pioneered the use of quantitative experiments to test his theories of falling bodies.1 Extrapolating his experimental results to the heavenly bodies, he pronounced that the services of angels were not necessary to keep the planets moving, enraging the Church and disciples of Aristotle alike. For his efforts, Galileo is now viewed as the Father of Modern Science. Since the Renaissance, fundamental advances making use of the experimental method in the physical and biological sciences have been fast and furious.2 Within economics, the use of controlled experiments has steadily increased, fueled by the exploration of important economic phenomena in the laboratory more than one half century ago.
Although laboratory experiments have dominated the experimental landscape in economics, the past decade has witnessed a significant surge in studies that gather data via field experiments. In economics, field experiments occupy an important middle ground between laboratory experiments and studies that use naturally occurring field data.3 This is convenient because, on the one hand, economic theory is inspired by behavior in the field, so we would like to know if results from the laboratory domain are transferable to field environments. Alternatively, because it is sometimes necessary to invoke strict assumptions to achieve identification using naturally occurring data, we wonder whether similar causal effects can be found in studies that have different identification assumptions.
Field experiments can play an important role in the discovery process by allowing us to tackle questions that are quite difficult to answer without use of randomization in a field setting. They also can serve an important complementary role -- similar to the spirit in which astronomy draws on the insights from particle physics and classical mechanics to make sharper insights, field experiments can supplement insights gained from lab and naturally occurring data. To date, field experiments have shed insights on areas as diverse as tests of auction theory, tests of the theory of private provision of public goods, tests that pit neoclassical theory and prospect theory, tests that explore issues in cost/benefit analysis and preference elicitation, tests that explore competitive market theory in the field, tests of alternative incentive schemes in developing nations, and tests of information assimilation among professional financial traders.4
In the remainder of this research summary, I will summarize field experiments within the realm of the economics of charity, with an emphasis on my work, completed with several colleagues.
The charitable marketplace represents an interesting set of actors, which might be usefully parsed into three distinct types. 5 First, is the Government, which decides on tax treatment of contributions and the level of grants to charities. This insightful literature includes studies that explore crowding out, and studies that measure responsiveness of giving to price changes.6 Second are the donors, who provide the resources to produce public goods. The final set of actors is the charitable organizations, which develop strategies to attract resources to produce public goods. The economic interplay of these three actor types represents a vibrant area of research.
My own research has focused on studying the relationship between individual donors and charities. In the United States alone, annual individual giving now exceeds 2 percent of GDP, with roughly 90 percent of people giving money to at least one cause annually. Further, there is at least one capital campaign under way in virtually every major population center in North America that has an objective of raising between $25-$100 million. Smaller capital campaigns are even more numerous.7 Increased individual wealth, an aging population, and recent devolutionary trends across governments worldwide combine to set the stage for continued rapid growth in the sector.
Even though the stakes are clearly high, many economic facts concerning the interrelationship between solicitors and solicitees remain unknown. My own interest in this area was shaped in the late 1990s, when I was an Assistant Professor at the University of Central Florida and my Dean asked me to help him raise money for a proposed Center in the Economics Department. He provided me with $5,000 as "seed money." After a significant amount of discussion, the administration agreed to allow me to run a charitable fundraising field experiment with the $5,000.
This first study, completed with David Reiley8, was an effort to test economic theory, to learn how best to use the seed funds, and to actually multiply the $5,000 seed funds. We split the full capital campaign into several smaller capital campaigns to fund one of six computers the Center needed, each of which served as a separate experimental treatment. For example, one household received a solicitation that noted we had already secured $1000 of the goal, and we were asking solicitees to make up the shortfall. We solicited contributions from 3000 Central Florida residents, randomly assigned to six different groups of 500, with each group asked to fund a separate computer for use at the Center. Most importantly, we found that seed money increased the average gifts of donors: more seeds led to more money.9
In a similar spirit, Daniel Rondeau and I used a natural field experiment, dividing 3000 direct mail solicitations to Sierra Club supporters into four treatments and asking solicitees to support the expansion of a K-12 environmental education program. We find that announcement of seed money increases the participation rate of potential donors by 23 percent and total dollar contributions by 18 percent, compared to an identical campaign in which no announcement of leadership gift is made. Other scholars also have shown the importance of seed money.
Dean Karlan and I explore a different use of upfront monies: we extend this line of inquiry by soliciting contributions using a conditional, rather than an unconditional, match.10 Soliciting contributions from more than 50,000 supporters of a liberal organization, we randomize households into several different matching rate treatments to explore whether presence of a match, and the match rate, influence giving. We find that simply announcing that a match is available considerably increases the revenue per solicitation - by 19 percent. In addition, the offer of a match significantly increases the probability that an individual donates - by 22 percent. Yet, while the match treatments relative to a control group increase the probability of donating, larger match ratios -- for example a $3 match for every $1 donated, or $2 for every $1 donated -relative to smaller match ratios (such as one to one) have no additional impact.
Other studies also have shed light on the value of using a match. For example, Eckel and Grossman11 use lab experiments to compare matching to an equivalent rebate of one's contributions in the context of a dictator game; they find that matching contributions lead to significantly larger contributions than the rebate mechanism. Rondeau and I also report evidence consonant with the positive effects of having a match available.
In a study that explores whether upfront money can be used more effectively by purchasing lottery prizes for donors, several colleagues and I investigated the effects of using lotteries to induce giving. 12 Our efforts were to support the Center for Natural Hazards Mitigation Research at East Carolina University. Federal gambling laws prohibited us from executing a mail solicitation, a phone-athon, or an email drive. We therefore turned to a door-to-door fundraising drive, wherein we approached nearly 5000 households, randomly assigned to standard voluntary contributions mechanism treatments (VCM - in other words, merely knocking on the door and asking for a contribution) or lottery treatments, where every dollar given provides one chance at a lottery prize.
Some insights emerged from that experiment. For example, the lottery treatments raised roughly 50 percent more in gross proceeds than our VCM treatments. This result was largely driven by greater participation rates in the lotteries: lotteries increase participation rates by roughly 100 percent. This finding highlights an attractive feature of lotteries: they provide fund-raisers with a tool to generate "warm lists," or a larger pool of active donors to draw from in future fund-raising drives. Our experimental design also permits an exploration of whether, and to what extent, individual solicitor characteristics influence fund-raising success. In this regard, we find that a single-standard deviation increase in physical attractiveness of women solicitors increased the average gift by approximately 35 to 72 percent. This result highlights the power of social factors in effecting fundraising.13
Whether, and to what extent, these factors influence long-run giving patterns is an open question. To move the exploration from one of measuring short-run substitution effects to one that measures dynamic effects, we use a second door-to-door study. 14 In this study, we use detailed information on the households that were previously approached and how they were approached. A little more than one year after the first drive, we randomly allocated previous givers ("warm" list agents) and those who have never given ("cold"list agents) into one of three treatments: a VCM, and two gift treatments - asking solicitees for money, but giving a large or a small gift to the potential donors. We also randomized solicitor types across households - in some cases removing the "beauty incentive" that was present in the first drive, for others adding the "beauty incentive," and for others keeping the "beauty incentive" the same.
In that experiment, we find that the warm-list enhances fund-raising success: donors who contributed in the first fund-raising drive were twice as likely to give, and provide average gifts that were roughly twice as large, as households that had never given. We find, however, a difference in the efficacy of how warm-list givers were initially attracted: removing the lottery incentive had no discernable impact on the behavior of warm-list households, whereas removing the "beauty incentive" leads to a significant reduction in average contribution levels.
Finally, concerning the effect of the gifts in this natural field experiment, we find that donor gifts can have important effects on the intensive and extensive margins. Importantly, whether the gift is given unconditionally (solicitee receives the gift regardless of whether she gives) or conditionally (solicitee receives the gift if she gives a pre-specified dollar amount) influences the probability and size of the monetary contribution.
A first lesson that I take from this body of research is that what we do not know dwarfs what we know. So, what have we learned from this body of work? One feature of this line of research is that it has been able to lend insights into the types of models that accurately predict giving behavior. In this sense, simple models that treat individual contributions as if they are identical to purchases of private goods should be reconsidered in light of the findings from this literature. Additionally, work in this area that measures key parameters has provided economists with useful information and lent guidance into policymaking discussions. Equally as important, these studies have collected enough facts to help us construct new economic theories of giving.
For practitioners, understanding what motivates people to give, how to use upfront monies efficiently to generate the greatest level of gifts, and learning about appropriate ask strategies for the present and future are invaluable. In this regard, the data point to the fact that upfront money is important, and that it can be used effectively as announced seed money, a matching grant, to purchase donor gifts, or to purchase lottery gifts. Critically, long-run fundraising success depends on incentives used to attract first-time donors. More specifically, some types of incentives might crowd out reasons for giving later. The literature also pinpoints that non-price incentives can have important effects on givers; in some cases surprisingly large effects. For example, giving small donor gifts to new solicitees can substitute for a warm-list.
I suspect that this line of research will continue to be a strong growth area. As fundraisers continue to recognize the value of experimentation, economists will increasingly be called upon to lend their services. Likewise, as economists continue to recognize the value of using naturally occurring settings as laboratories, such domains increasingly will be used to generate new datasets.
* List is a Research Associate in the NBER's Program on Public Economics and a Professor in Economics and the College and Co-Director of Graduate Admissions at the University of Chicago.
2. For a more complete discussion see J.A. List and D. Reiley, "Field Experiments" in The New Palgrave Dictionary of Economics, S. N. Durlauf and L. E. Blume, eds. Palgrave Macmillan Publishing.
5. See the valuable work of J. Andreoni, "Philanthropy," In Handbook of Giving, Reciprocity and Altruism, S-C. Kolm and J. Mercier Ythier, eds. Amsterdam: North Holland (2006), pp. 1201-69.
6. See the important work of Martin Feldstein and Charles T. Clotfelter. For example, M. Feldstein, "The Income Tax and Charitable Contributions: Part II- The Impact on Religious, Educational and Other Organizations," National Tax Journal 1975, 28(2): pp. 209-26; C.T. Clotfelter, Federal Tax Policy and Charitable Giving, University of Chicago Press, (1985), and more recently G. Auten, H. Sieg, and C.T. Clotfelter, "Charitable Giving, Income and Taxes: An Analysis of Panel Data," American Economic Review, 2002, 92: pp. 371-82.
7. See D. Rondeau and J.A. List, "Matching and Challenge Gifts to Charity: Evidence from Laboratory and Natural Field Experiments, NBER Working Paper No.13728, January 2008, and "Special Issue on Field Experiments in Charity," Experimental Economics, 11(3), pp. 253-67.
8. J.A. List and D. Lucking-Reiley, "The Effects of Seed Money and Refunds on Charitable Giving: Experimental Evidence from a University Capital Campaign," Journal of Political Economy (2002), 110(1), pp. 215-33.
9. One unique recent example is S. Huck and I. Rasul, "Comparing Charitable Fundraising Schemes: Evidence from a Natural Field Experiment," working paper, UCL.
10. D. Karlan and J.A. List. "Does Price Matter in Charitable Giving? Evidence from a Large-Scale Natural Field Experiment," NBER Working Paper No.12338, June 2006, and American Economic Review, (2007), 97(5), pp. 1774-93.
11. C.C. Eckel, and P.J. Grossman. "Rebate versus Matching: Does How We Subsidize Charitable Contributions Matter?" Journal of Public Economics, 87, 2003, pp. 681-701. See also B. Frey and S. Meier, " Social Comparisons and Pro-social Behavior: Testing 'Conditional Cooperation' in a Field Experiment," American Economic Review 94(5), 2005: pp.1717-22.
12. C.A. Landry, A. Lange, J.A. List, M.K. Price, and N. Rupp, "Toward an Understanding of the Economics of Charity: Evidence from a Field Experiment," NBER Working Paper No. 11611, September 2005, and Quarterly Journal of Economics, 121 (2): pp.747-82.
13. For more in the economics literature on the effects of social factors, I direct the interested reader to the clever work of Frey and Meier (2005; cf. footnote 11) and R. Croson and J. Shang, "The impact of downward social information on contribution decisions," "Special Issue on Field Experiments in Charity", Experimental Economics, 11(3), pp. 221-33.
14. C.A. Landry, A. Lange, J.A. List, M.K. Price, and N. Rupp, "Is a Donor in Hand Better than Two in the Bush? Evidence from a Natural Field Experiment," NBER Working Paper No. 14319, September 2008.