An Interview with Mark Bauerlein: NAEP and Reading

Oct 18, 2020 by

Mark Bauerlein - Wikipedia
Mark Bauerlein

Michael F. Shaughnessy –

  1. Mark, you recently co-authored a piece in the City Journal about the NAEP and their new approach to reading. First, who was your co-author and what was the main point you were trying to make?

David Steiner is the Director of the Institute for Education Policy at Johns Hopkins University. We worked together at the National Endowment for the Arts in the mid-00s when he was the Director of Education and I was the Head of the Office of Policy and Research.  We both serve on the Board of the Core Knowledge Foundation and we have collaborated before. (https://www.insidehighered.com/views/2019/04/02/important-ways-revitalize-humanities-opinion)

Our motivation for writing the piece was a proposal from the National Assessment Governing Board. The Proposal laid out a new framework for the NAEP reading exam that redefined reading along the lines of cultural diversity.

While the existing framework defines reading in straight forward cognitive terms as “understanding written text” and “developing and interpreting meaning,” the proposed framework adds a fresh factor: Reading is now “expanded to a sociocultural model that positions the reader, the texts, and the activities in a sociocultural context.”

We regard this change as a disaster.

  1. NAEP has ” expanded to a sociocultural model that positions the reader, the texts, and the activities in a socio-cultural context.” In your mind, what does this mean? Why is it so bad?

The problem is that it introduces into the reading exam a dimension of “cultural sensitivity” that doesn’t belong there. If reading depends on sociocultural context, how do you assess students who come from different contexts and thus read differently?

The proposal anticipates the problem by adding elements to the test instrument that will assist test-takers with passages and questions that ( apparently) come from a context of which they are unfamiliar.

This includes “scaffolds that optimize comprehension performance for every reader,” videos relative to a passage subject that test-takers can watch if the subject is remote from their experience, and digital “avatars.” In other words, the test will do its best to make up for differences in cultural background.

The goal will be to “optimize” student performance, the designers say. But is that what NAEP is designed to do? Is that the purpose of this test? Remember it is not a high-stakes test, not for the test taker. It is diagnostic and what it diagnoses is how well schools and districts and states are educating the young. It should not try to optimize performance. It should aim for an accurate appraisal of reading ability.

3) E.D. Hirsch, whose work is well known, has talked about cultural literacy, background knowledge and general information as important factors in the reading process. How does his thinking fit in here?

It gets to the fundamental issue of cultural background.  Hirsch understands that every reading test is to a great extent a test of cultural knowledge. If you know something about the subject of the passages on the exam, your score will go up significantly, not because your skill suddenly improves, but because there is no such thing as abstract reading skill. Reading comprehension is a function of basic decoding skills plus the implementation of prior knowledge.

This is because every communication has implicit meanings, those aspects of the subject matter that are left unsaid because assumed to be known by the reader. A passage about a baseball game doesn’t bother to explain the rules of the game, only how this game turned out. A reader who knows a lot about baseball picks up all the implied meanings, while a knowledge poor reader doesn’t. Their respective comprehension of the passage corresponds to their familiarity with the material.

The designers of the new framework seem to understand this well, but they have the wrong solution. The NAEP exam does, indeed, test background knowledge relative to the content of the exam and the designers wish to minimize that factor by providing assistance that makes the test-takers’ ignorance less harmful. This is, in effect, to supply to test-takers in advance part of what should be tested.

What educators should do instead is take the results of the existing exam and use them to locate gaps in background knowledge that prove harmful to students, especially when they proceed to the next level. That information should be passed along to teachers who can address those knowledge gaps with their classroom instruction. Unfortunately, what the new framework tries to do is neutralize the cultural content aspect of the reading exam, and avoid the cultural bias allegation, a turn that will hurt students later on (see below).

4) In this day and age of video games and movies and cell phones- some people are simply not reading. How will this impact the big picture in the assessment of reading?

Well, we certainly see a strong correlation between the arrival and spread of digital tools and falling reading scores.  Look at SAT scores from 2005 to the present or ACT college readiness rates in reading for the last ten years. It’s a downward slope. I remember all the hype in 2005 about how digital tools are going to make the Millennials the most informed and adept generation in human history. That prediction sounds laughable at this point. One thing particular fact to be noted here is that the NAEP long-term assessment shows is that leisure reading is much a determinant of reading ability as is homework reading. As screens have steadily replaced books, the deterioration has continued. Leisure reading is falling or, rather, it’s gotten so low that it can’t get any lower. 

According to the Bureau of Labor Statistics, young adults spend barely six minutes per day doing any leisure reading. (https://www.bls.gov/charts/american-time-use/activity-by-age.htm) For most of them, books are a disappointing diversion. The kids are on games and social media, instead. It’s a big reason why reading scores are so perpetually disappointing in spite of the billions of dollars spent. Until they pull away from screens and get back to print, we shouldn’t expect scores to rise.

5) Quite often, in Informal Reading Inventories, we see a ” background knowledge ” check or prompt or assessment. How important is it for us or the NAEP to assess student background knowledge? 

It is very important for the reasons noted above, but there is a danger. What if it turns out that for the reasons noted above, that the background knowledge students lack falls into areas that don’t meet diversity criteria?

Let’s expand the question: What if the background knowledge that enables youths to survive the first year of college proves to be too white-male and Eurocentric? What will the multiculturalists say? They cannot deny that when students take general education courses in college, those courses will favor the kind of core knowledge that Hirsch has advocated for decades—and that the educators have resisted. A student who read a lot of classics in high school is going to have an advantage over a student who did mostly contemporary literature when it comes to college courses in history, literature, philosophy, politics, . . . anything with a historical dimension.

6) And if this background knowledge is weak or missing- what should the educational system be doing?

Well, they should be teaching it!

7) I really do not play golf, but this NAEP thing sounds like handicapping to me, or am I off on this?

Yup, handicapping sounds about right.

8) The NAEP test score are not punitive- but reflect a states’ proficiency. But can we really compare and contrast Hawaii and Alaska? Or even Maine and Florida?

I think we have to compare and contrast them, but make allowance for population differences such as income level. Most importantly, we should use those comparisons constructively, for instance, comparing different curricula state by state to see which ones produce improvements.

9) This “socio-cultural model” kind of assumes that people from different races read different materials. Yet when Harry Potter came out, it seemed everyone was reading that stuff. Or is NAEP hypothesizing that no one reads the McDonald’s menu? or the same primers in schools?

It goes farther than saying kids read different things. It says that the actual act of reading varies by socio-cultural context. That is, a student in the city of Chicago reads differently than does a student in rural California. Even if all students read Harry Potter and McDonalds menus, their cognitive activity differs. If that is true of course, then a national test is impossible.

10) What have I neglected to ask about this NAEP proposal? ( or will it be implemented? )

If this proposal goes through, it will raise the scores of low achievers. We will, indeed, see results for African American and Hispanic/Latino students go up. But this will be a false elevation. The improvement will be due not to improved reading ability but to greater assistance in the instrument. It will give educators the wrong impression about student progress, and they will not identify areas in which students need help. This will send more students out of high school less prepared than they should be, though the problems won’t surface until they reach college and face reading tasks without those supports the new framework provides. Their grades will suffer.

In sum, this is a feel good exercise that pleases the educators who hate the racial achievement gaps that currently exists. It is, unfortunately, not a genuine advance and it will hurt the very populations the proposal aims to help.

Print Friendly, PDF & Email

Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.