WURMAN: MISLEADING REPORT BY WILLIAM SCHMIDT OF COMMON CORE MATH STANDARDS

May 6, 2012 by

Ze’ev Wurman

I asked Ze’ev Wurman to post his response after studying Dr. William Schmidt’s report entitled “Chiefs for Change” which is a “puff piece” sent out to convince the public that the Common Core Math Standards are superior.  (An introduction to Schmidt’s report is posted at the bottom of this page, and it contains the link to the report.)

Here are Ze’ev’s comments (zwurman@gmail.com ).  I think you will find them very enlightening.  – Donna Garner

===================
5.5.12

Last week Bill Schmidt, of Michigan State University, rolled out in a highly publicized national press event the “key conclusions” from his recent research. We can’t see any of the underlying research, as Schmidt did not publish it; its supposed findings, however, already got so much uncritical exposure and praise from the usual suspects that it is important to put Schmidt’s words in their proper context. And that context seems more problematic than organizations like Achieve, or Chiefs for Change, who sponsored this research, would like us to believe. I have reviewed Schmidt’s presentation and these are some of my observations.

1) First, we should note how carefully Schmidt hedges his bets. His first (and last) slide says that the Common Core Standards for Mathematics “[c]an potentially elevate the academic performance of America’s student,” with the emphasis on the “potentially” in the original. It is hard to imagine a more sweeping disclaimer—almost anything can “potentially” elevate academic performance. More money; more professional development; more unionization; more school choice; more selectivity in choosing teachers; better textbooks; better parent education via public campaigns; better movies from Hollywood that will improve character education and discipline of youth; and so on.

2) Schmidt repeats in multiple slides that parents and teachers support them and claim to be familiar with them. A large fraction of teachers even believes it is prepared to teach them. I can believe that teachers heard about them, but I doubt many have any real basis for liking them, or for claiming to be prepared to teach them. Other surveys found most teachers and parents don’t really know or understand the actual content of the standards and the implications of teaching them. After teachers actually try teaching them in the classroom and we see the assessments, maybe we could put more trust in these surveys.

3) On slide #2 Schmidt, gives the old picture (from TIMSS ’95 and ’99) of topics progressions based on the so-called “A+” countries (TIMSS very high achievers). In the next slide (#3) he shows state “averages” generally following this pattern (due to his own influence a decade ago, to some degree). But then, on slide #4, he shows the Common Core mapping. It looks, at first glance, similar to the previous overall shape. Upon closer examination, however, we see that the order of the topics has changed, and a few new ones show up!

This sleight of hand is doubly troublesome because Schmidt, during the press conference, repeatedly referred to “looking at the pattern” that emerges and the “coherence” it implies. He never mentioned anything about rearranging old topics or adding new topics for the Common Core categorization. Clearly, the “coherent pattern” that is supposed to emerge is completely different if one re-arranges the sequence and the nature of the topics.

To observe how misleading—and, frequently, simply wrong—Schmidt is, one should closely compare slides #2 and #5. For example, the “3D Geometry” topic is taught in the A+ countries in grade 7 and 8. In contrast, Common Core teaches this topic starting in grade 1(!) and until grade 8, with a two-year break in grades 3 and 4. To hide this, Schmidt moved this topic from the bottom in slide #2 to close to the top in slide #5. With respect to “Measurement Estimation & Errors” and “Number Theory,” not only were these topics significantly moved between slides #2 and #5 to hide the fact that Common Core starts teaching them in grade 2 (while the A+ countries teach them only in grades 7 or 8), but also slide #5 wrongly (and misleadingly) marks the A+ counties as if they teach them starting already in grades 4-5, further minimizing the visual difference between them.

To convince yourself simply check those same topics on slide #2. Another example is “Functions” that are handled by the A+ countries in grade 8, while Common Core topic (called “Patterns, Relations, & Functions” in slide #5) teaches them starting in grade 4 for a full five years, up to grade 8. There are many more examples of such mislabeling between slides #2 and #5, such as “Properties of Whole Number Operations,”  “Fractions,” “Percentages,” and more. All this does not speak kindly of the focus and coherence of the Common Core, and does not show a progression closely similar to that of the A+ countries.

Specifically, here is what Schmidt says (minute 14 of the press conference):

Using the very same TIMSS methodology of coding we took the documents from the Common Core and coded those. And I just want to make a one really important point, all of which is saying it is not somebody’s opinion whether the Common Core’s good or bad or indifferent. They really simply were graduate students given the job of coding and finding out what topics were taught at what grade levels. … Here is the graph that represents the CCSSM and you can see a very similar sort of shape to this to what you saw with the earlier slide with the A+ or the top achieving countries so this alone suggests a great deal of conformity between the two sets of standards.

Note his misleading use of “very same TIMSS methodology.” This means that the methodology of assigning standards to topics is the same, but not the order of the topics, or the topics themselves. Yet it is clearly his implication that they are identical when he urges us to “see a very similar sort of shape to this what you saw with the earlier slide with the A+ countries.”

What Schmidt is doing here borders on the dishonest. He switches the underlying topics and their order and then urges us to watch “a very similar” pattern, never mentioning that the pattern represents completely different underlying topics on the different charts—just as a magician makes sure the audience watches his face and not his hands.

If Schmidt’s misleading way reorganizing topics and mislabeling what A+ countries do was not enough, one cannot even trust his underlying categorization of the standards. For example, he marks “Constructions Using Straightedge and Compass” as taught in grade 7 by the Common Core. Yet the Common Core clearly does not teach it before high school geometry. In grade seven the Common Core only expects students to draw shapes with ruler and protractor, not with straightedge and compass. This major difference between informal and formal geometrical constructions somehow escaped those “graduate students” who coded the standards “using the very same TIMSS methodology.”

4) In slide #5 we see that Schmidt, even with the tabulating errors, finds about 15% (18 of ~130) of the A+ standards in different grades than Common Core, sometime two grades apart; some of the Common Core topics are not in A+ countries and vice versa.  But not all standards are made equal—not every topic is as important as the next, as any mathematician will easily tell you. Except that Bill Schmidt, a statistician, does not go there—he’s after simplistic statistical correlation of badly classified standards.

5) An even bigger issue with the new list of categories is that we don’t know how these categories were put in place, and we still have no idea of the relative importance of topics.  Was the categorization created long ago as the result of Schmidt’s research on curriculum over the last decade, or was it custom made for the Common Core? If it’s the latter, the whole comparison is meaningless and serves effectively only as an advocacy research. It is easy to “tune” the categories so that they will show excellent alignment between standards of high-achieving countries and Common Core, or show a pseudo-coherent progression pattern by rearranging the topics. I don’t know the answer to that, but at his talk Schmidt did not mention what caused him to make the changes, or when.

6) Slide 6 shows number of topic table and other than observing general reduction among the states between 1995 and now. There is little to say — it doesn’t show that Common Core is significantly “slimmer” than the state average, and certainly doesn’t show that the Common Core is slimmer than any given state. All this even if we were sure that reducing number of topics is of cardinal importance — which we are not![1][1] One can easily accept that having a huge number of topics per grade is a problem, but at the same time once you get to below 20-25 it is quite unclear that 21 is better than 25 and worse than 17. Could be just the opposite! This slide is yet another example of Schmidt’s statistical leanings rather than his understanding of content — it pretends to say something meaningful yet throws up mostly meaningless data in the air.

7) Slide #14 is probably the most indicative of the overall weakness of this whole story. Schmidt’s research is supposed to show “alignment” between Common Core and standards of high-achieving A+ nations, and hence “conclude” that Common Core will lead us to higher achievement because of that alignment. Yet here we see California at the top of the scale being Common Core-like, and Massachusetts being somewhere in the middle of the pack. One does not need a PhD in statistics, however, to realize that Massachusetts had made extraordinary progress with their “mediocre” (by his measure) standards, while California made only a mediocre progress with its “exceptionally-aligned” standards. This, yet again, brings up Tom Loveless’s recent argument that the correlations (standards=>achievement) are small and the causality argument is highly problematic.

Schmidt tries to address this visible problem in his story by developing a new measure of congruence that adds the rigor of state’s cut-scores, arguing that, in some mysterious way, cut-scores are reflective of the quality of the standards themselves. With this correction, Massachusetts suddenly moves up on his scale of congruency, and this is supposed to explain why it achieves so well. Yet adding cut-scores to a measure of the quality of the standards is unsupported by logic or reason. Cut-scores may indicate the seriousness that the state attaches to student achievement, its level of expectations from them, and possibly its quality of curriculum implementation. But cut-scores have little to do with the quality of the standards themselves. This seems yet another example of creating a fancy statistical “measure of congruence” to “prove” a statistician’s point, while hiding the fact this measure has neither educational nor policy logic behind it.

(As an aside, it is interesting that Schmidt studiously avoids mentioning cut-scores of the Common Core; we don’t know how high—or low—they will be set, and this is potentially a huge area of contention; it’s hard to see how Massachusetts and Mississippi will agree on common cut scores.)

To conclude, Bill Schmidt centers his argument around two themes: that the Common Core standards are similar to those of the A+ countries; and that states with standards more congruent to the A+ countries show bigger progress on the NAEP. To make the last claim work Schmidt redefines “congruency” to include cut-scores for no logical reason. Both claims are unsupported by his own data and, in addition, his own data is riddled with errors. Yet the Chiefs for Change already tout that “Dr. Schmidt’s research shows that state leaders are on the right track.  Common Core State Standards have the potential to raise student learning and performance across America.  Most importantly, they are competitive with the standards found in the highest achieving countries.” What Dr. Schmidt presented is just another piece of misleading advocacy research, brought to you and paid for by the Bill & Melinda Gates Foundation and channeled through the friendly services of Achieve (which received a recent $375K grant for advocacy from the Gates Foundation), the Foundation for Excellence in Education (which received a recent $1M grant for advocacy from the Gates Foundation), CCSSO (which received $9.5M last year from the Gates Foundation to promote the Common Core), and Chiefs for Change (funded by the Foundation for Excellence in Education).

Ze’ev Wurman

===========================
FOR IMMEDIATE RELEASE

Contact: Jaryn Emhof

May 3, 2012

(850) 391-4090

jaryn@excelined.org

Statement by Chiefs for Change on New Research Linking Common Core Math Standards to Higher Achievement

Washington, DC – Today, Chiefs for Change, an organization of chief state education officers committed to putting children first through visionary education reform, joined Achieve and the Foundation for Excellence in Education to co-host an event releasing research by William Schmidt of Michigan State University. At the event, Dr. Schmidt revealed key findings on the Common Core State Standards for mathematics.

“Students’ opportunity for success will be determined by the quality of their education. Academic standards in the classroom must be aligned to the knowledge and skills required in today’s global economy. Recognizing the need for higher, more focused state standards, state leaders across the nation developed and adopted the Common Core State Standards.”

“Dr. Schmidt’s research shows that state leaders are on the right track. Common Core State Standards have the potential to raise student learning and performance across America. Most importantly, they are competitive with standards found in the highest achieving countries. This is critical to ensuring every student can meet the expectations of 21st century colleges and careers.”

According to Dr. Schmidt’s research, 33% of all eighth grade students in the United States are performing below proficiency in mathematics compared to only 13% of the world’s top five performing countries. Dr. Schmidt’s work confirms that Common Core State Standards strongly resemble the standards of the highest-achieving nations, and his research finds that states with mathematic standards more aligned with Common Core standards have higher NAEP scores. Click here to learn more about Dr. Schmidt’s research.

 

###



[1][1] See, for example, http://mathematicallycorrect.com/interp.htm

Print Friendly, PDF & Email

Related Posts

Tags

Share This

Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.