An Interview with Andrew J. Coulson: Director, Cato Institute Center for Educational Freedom

Jun 12, 2011 by

Michael F. Shaughnessy
Eastern New Mexico University
Portales, New Mexico

1) Andrew, you have just released a new study this past week. Tell  us about it.

The biggest problem facing American education isn’t a lack of great schools, but rather our inability to routinely replicate them. The highest-profile strategy for solving that problem over the past decade has been for philanthropists to fund the growth of what they believe to be the best charter school networks. By testing the correlation between charter network funding levels and academic performance, my study answers the question: how well is that working? I find that there is essentially no correlation between academic performance of charter networks and the amount of philanthropic funding they’ve received.

2) What exact statistics did you use and what was your sample size?

My main dataset is the California Standards Tests, the only source that includes scores for every traditional and charter public school in the entire state, because I wanted to be able to compare the results of all the charter networks to each other and to the statewide average. That precluded the use of a longitudinal (a.k.a. “value-added”) student-level model, so I fell back on the next best thing: a hierarchical linear model of classroom-level results grouped within charter school networks. Fortunately, the CST makes it possible to control for student Socio-Economic Status factors as well as school-level peer effects, and I spent a lot of time investigating whether selection bias could be responsible for the paper’s central findings (among other things, I conducted a Monte Carlo simulation and an extensive literature review).

I conclude that selection bias could not be playing a major role in the findings. My results are consistent with both the randomized and non-randomized studies that have been done to date on both particular cities/charter school networks and on large national datasets. The sample sizes range from tens of thousands to hundreds of thousands of classroom-level test results, across subjects and grades, and each of those results represents the average performance of multiple students—so it’s a much larger dataset than is normally used in this kind of study.

As a check on those results, I also looked at Advanced Placement test results, and found essentially the same result.

3) How did you measure “ academic performance “?

As noted above, I used the California Standards Tests, across subjects and grades, and the Advanced Placement tests.

4)   Do you actually have hard data as to how much money these charter schools are getting?

Yes, I used a combination of two searchable databases of philanthropic giving and a California political donation reporting website. There is likely to be some noise in the resulting data—due to data entry errors, for instance—but there’s no reason to think that any such errors would be so dramatic and systematic as to significantly affect the results one way or the other.

5) Let’s face it. Most parents want their children to go to charter schools. Is beginning motivation a big factor or variable?

The central question of the study is addressed by looking at the charter networks relative to one another, so any overarching “I’m smart so I like charters” effect would drop out. It is still possible that some charter schools might differentially attract high or low achieving students (“I’m smart so I like this particular charter network”), but I guarded against this by controlling for individual student SES, school-level peer effects, and exploring the possible role of selection bias through a Monte Carlo simulation and review of the literature. Differential self-selection into particular charter networks cannot begin to explain the results of the study.

In fact, the top performing charter networks outscore two of the most elite and overtly academically selective schools in the country, Lowell and Gretchen Whitney. Those two elite schools admit students selectively based on their test scores, whereas the charter schools accept all students or use a random lottery if they are oversubscribed—and even with the advantage of that active school-based selection of students, which charters cannot practice, they were beaten by the top charter school networks.

6) How would you  summarize the results?

Philanthropy + charter schooling does not appear to be a reliably mechanism for routinely scaling up the best education models. On the contrary, philanthropists appear to have been scaling up California charter school networks at random—the mediocre and even sometimes the bad as well as the good.

7) I am not up on what is happening in California. What seems to be the educational climate there?

California has more charter schools, and more networks of charter schools, than any other state—which was the reason I chose it for this study. Charter schools in that state alone have received about a quarter of a billion dollars from philanthropists in just the past eight years.

8) What do you see as the implications of this study?

Clearly, we do need a mechanism that will routinely identify and replicate the best education models, crowding out the inferior ones. Just as clearly, based on my findings, we need to look beyond the popular “philanthropy + charters” approach to fulfilling that goal. If we want to know how to solve this problem, a good place to start would be to look at parts of the world where excellent teachers and schools do routinely reach a mass audience. That’s actually the next project I’ll be working on.

9) For interested others, is there a way to access the report on line or get a copy?

The study is available on-line here:


Print Friendly, PDF & Email

Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.