Evaluating the Econometric Evaluations of Training Programs with Experimental Data

Abstract

By the late 1970's researchers had produced dozens of studies that evaluated the effect of federally sponsored manpower and training programs on the earnings of their participants. In any of these studies that used non-experimental data, a consistent estimate of the impact of training depended on the specification of the earnings and program par- ticipation equations. In practice satisfactory tests of these specifi- cations are difficult. The National Supported Work Program employed an experimental design that randomly assigned some participants into a treatment group, receiving training, and the rest into a control group, receiving no training. The simple difference between the post-training earnings of the two groups provided an unbiased estimate of the impact of the program. This paper evaluates the non-experimental methods of program evaluation by comparing different non—experimental estimates to the experimental estimate of the impact of the Supported Work Program. The results indicate that non—experimental methods generate a disconcer- tingly large range of training effects, but that these methods are better suited for the analysis of female rather than male training participants.