An Interview with Neal McCluskey: Data, Data, and Who Has it and What to do with it?

Mar 19, 2013 by

Neal McCluskey

Michael F. Shaughnessy –

1) Neal, there seems to be this preoccupation with gathering data in the U.S. Government. When the heck did this start?

I don’t know the exact date, but the federal government has been gathering data in education for well over a century. Indeed, that was the goal of the original U.S. Department of Education, started in 1867 and quickly reduced to a bureau. I think data collection ballooned after “A Nation At Risk” was published in 1983, setting off the modern “standards and accountability” movement, central to which is measuring outcomes. Data collection seems to have hit fever pitch, though, with national curriculum standards — the “Common Core” — and its promise to make every student, everywhere, comparable.

2) We all like facts and figures and data- but the devil is in the details of interpretation- or am I off on this?

Facts and figures are useful, but there is a danger that simply putting a number on something gives the impression that you’ve measured it rigorously and scientifically. But lots of things — say, critical thinking — cannot be meaningfully reduced to a number or two. Worse, numbers can easily be presented out of context, painting a picture that is at best incomplete. For example, some in Congress have been berating for-profit colleges for their high loan-default and low completion rates. But they ignore the even worse completion rates for community colleges, and the fact that for-profit schools have higher default rates because they don’t get subsidies upfront like public institutions. And then there are a host of serious privacy issues: Who gets access to the data and what do they get to do with it?

3) Often the data is counter-intuitive- but does that mean a call for more data?

When data doesn’t give you what you want, it often doesn’t mean you need more data. It means you need to reconsider what you’re after.

4) I was once at the Grand Canyon and these tourists were taking hundreds of pictures. I wondered what they were going to do with all those pictures. It seems similar in the government- we collect data, but I am not sure what we do with it—any insights?

I’d guess a lot of government data collection is done largely to employ data collectors, but I also think many people calling for more data have specific goals for it, such as showing which colleges are best for the money; how students stack up against their peers around the world; etc. The problem is that data collection opens up numerous, unintended pitfalls, dangers which I think far outweigh the value of the collection. And the fact of the matter is that a lot of data people say they want Washington to collect, such as college completion rates, are already pretty easy to find.

5) You and I probably remember that good old book ” How to Lie with Statistics “….should we do a book on “How to Lie with Data “?

I’d guess they’d be almost identical. The data are just what get manipulated to do that lying with statistics.

6) Actually, I have found some data- that is very dangerous, particularly when they do not disaggregate it—for our audience- could you discuss the dissaggregation of data- and the resultant problems?

Well, “disaggregation” as it is usually used just means breaking the overall data into groups, such as race or English-language status. President George W. Bush used to chuckle about how he learned the meaning of that fancy-sounding word as a result of No Child Left Behind, which called for tracking test scores by numerous groupings. And that might be OK if you know what the scores are actually telling you and, often more important, not telling you. But that then means knowing what is on all the tests, the conditions under which the tests were taken, etc.

7) And now, for the coup de grace—when they DO NOT dissaggreate the data- and the resultant problems.

Not disaggregating means you could be getting a big picture with lots of smaller pictures that look very different. Again, the really big problem is not knowing what the data are not showing you.

8) What have I neglected to ask?

I think you got it all.

Print Friendly, PDF & Email

Related Posts


Share This

Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.