Are Test Scores as Meaningless as “Height Effects”? No, but Technocratic Misuse of Scores Is

Dec 9, 2019 by

tra-nguyen-TVSRWmnW8Us-unsplash

Guest post by Greg Forster –

I’m already seeing this study being discussed as if it debunked all use of test scores. Four researchers used statistical methods usually associated with measuring teacher effect on year-to-year test score gains, and used them to measure teacher effect on student height. They found a substantial apparent teacher effect on year-to-year changes in height, which is obviously a false positive.

This definitely debunks one way of using test scores – the way commonly used by technocrats and central controllers of the Common Core type. If you use only one year’s worth of data (or, technically, use two years of data to track one year’s worth of change in the data) you are getting a lot of noise along with your signal. Multiple years of change must be tracked before you can sort out signal from noise to measure a teacher’s effectiveness.

But serious scholarship had already long since debunked the one-year way of using test scores. This particular way of showing that technocratic abuse of test scores is absurd gains points for cleverness. However, the finding itself isn’t new. People who really care about measuring effective teaching have been complaining for years about technocratic abuse of test score data!

The technocrats and central controllers have done a lot to make the use of test scores look worthless and even counterproductive. If they don’t want look ridiculous in the way this study makes them look ridiculous, maybe they should start listening to serious scholars about the responsible use of data. Of course, if they did, they’d have to give up being technocrats entirely because technocracy always abuses data.

My thanks to Jay for helping me think this through before posting; thoughts here are my own.

Source: Are Test Scores as Meaningless as “Height Effects”? No, but Technocratic Misuse of Scores Is | Jay P. Greene’s Blog

Print Friendly, PDF & Email

Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.