Program Evaluation as a Decision Problem

I argue for thinking of program evaluation as a decision problem. In the context of California's GAIN experiment (a randomized trial of a welfare-to-work alternative to AFDC), I show that GAIN first-order stochastically dominates AFDC when considering the choice between the treatment and control programs in terms of average earnings, even though the treatment effect is not statistically significant. I also argue for incorporating the post-evaluation assignment mechanism for the program under consideration into the evaluation process. I show that if policies, such as allowing a career counselor to choose which program individuals join, are included in the evaluation, then GAIN is superior to AFDC whereas the opposite ranking emerges from the standard treatment versus control comparison which ignores potential heterogeneity in the treatment impact.