Another Look at the New York City School Voucher Experiment

Abstract

This paper reexamines data from the New York City school choice program, the largest and best implemented private school scholarship experiment yet conducted. In the experiment, low-income public school students in grades K-4 were eligible to participate in a series of lotteries for a private school scholarship in May 1997. Data were collected from students and their parents at baseline, and in the Spring of each of the next three years. Students with missing baseline test scores, which encompasses all those who were initially in Kindergarten and 11 percent of those initially in grades 1-4, were excluded from previous analyses of achievement, even though these students were tested in the follow-up years. In principle, random assignment would be expected to lead treatment status to be uncorrelated with all baseline characteristics. Including students with missing baseline test scores increases the sample size by 44 percent. For African American students, the only group to show a significant, positive effect of vouchers on achievement in past studies, the difference in average follow-up test scores between the treatment group (those offered a voucher) and control group (those not offered a voucher) becomes statistically insignificant at the .05 level and much smaller if the full sample is used. In addition, the effect of vouchers is found to be sensitive to the particular way race/ethnicity was defined. Previously, race was assigned according to the racial/ethnic category of the child's mother, and parents who marked “other” and wrote in Black/Hispanic were typically coded as non-Black and non-Hispanic. If children with a Black father are added to the sample of children with a Black mother, the effect of vouchers is small and statistically insignificant at conventional levels.