Dr. Sandra Stotsky’s June 2015 Testimony at Bridgewater State University – Public Hearing
Why Massachusetts Should Abandon the PARCC tests and the 2011 Coleman et al English Language Arts Standards on which the MCAS Tests are Based
June 10, 2015
Acknowledgments: I want to thank Chairman Paul Sagan of the Board of Elementary and Secondary Education for his invitation to testify at the public hearing at Bridgewater State University on whether the Board should abandon the MCAS tests and adopt the PARCC tests.
Overview of my Testimony: I first describe my qualifications, as well as the lack of relevant qualifications in Common Core’s standards writers and in most of the members of Common Core’s Validation Committee, on which I served in 2009-2010. I then detail some of the many problems in the 2011 Massachusetts English language arts (ELA) standards, written by David Coleman, Susan Pimentel, James Patterson, and Susan Wheltle (so the document indicates), in the tests based on Common Core’s standards (PARCC), and in the two external reports—one issued in February 2015, the other yet to be completed—comparing the PARCC tests with MCAS tests. I offer several recommendations for parents who want civically sound and academically rigorous standards and tests written and reviewed by English teachers and who want a form of accountability that doesn’t penalize their children’s teachers for results of tests based on either the Coleman et al standards or Common Core’s standards.
I. My Qualifications: I am professor emerita at the University of Arkansas, where I held the 21st Century Chair in Teacher Quality until retiring in 2012. I was Senior Associate Commissioner in the Massachusetts Department of Elementary and Secondary Education (DESE) from 1999-2003, in charge of developing or revising the state’s K-12 standards, teacher licensure tests, and teacher and administrator licensure regulations. I served on the Massachusetts Board of Elementary and Secondary Education (BESE) from 2006-2010, on the National Mathematics Advisory Panel from 2006-2008, and on the Common Core Validation Committee from 2009-2010. I was one of the five members of the Validation Committee who did not sign off on the standards as being rigorous, internationally competitive, or research-based.
I was also editor of the premier research journal, Research in the Teaching of English, published by the National Council of Teachers of English, from 1991 to 1997. I have published extensively in professional journals and written several books. In recent years, I have testified before many state legislative committees and boards on the flaws in Common Core’s standards.
II. Lack of Relevant Qualifications in Common Core’s Standards Writers
The absence of relevant professional credentials in the two standards-writing teams helps to explain the flaws in Common Core’s standards. The two “lead” writers for the ELA standards, David Coleman and Susan Pimentel, have never taught reading or English in K-12 or at the college level. Neither has a doctorate in English or reading. Neither has ever published serious work on K-12 curriculum and instruction. Neither has a reputation for literary scholarship or research in education. At the time they were appointed, they were virtually unknown to English and reading educators and the public at large. They now earn large fees for Student Achievement Partners (their business) consulting to school systems trying to implement their ELA standards.
The three lead standards writers in mathematics were as unknown to K-12 educators as were the lead ELA standards writers. None of the three mathematics standards writers (Phil Daro, William McCallum, and Jason Zimba) had ever developed K-12 mathematics standards that had been used—or used effectively. The only member of this three-person standards-writing team with K-12 teaching experience had majored in English as an undergraduate (although Phil Daro had taught mathematics at the middle school level for two years).
Who recommended these people as standards writers and why, we still do not know. No one in the media commented on their lack of credentials for the task they had been assigned. Indeed, no one in the media showed the slightest interest in their qualifications for standards writing.
III. Lack of Academic Qualifications in Most Members of the Validation Committee
The federal government did not fund an independent group of experts to evaluate the rigor of the standards, even though it expected the states to adopt them. Instead, the private organizations in charge of the project created their own Validation Committee (VC) in 2009. The VC contained almost no academic experts in any area; most were education professors or associated with testing companies, from here and abroad. There was only one mathematician on the VC—R. James Milgram—although there were many people with graduate degrees in mathematics education or with appointments in an education school, and/or who worked chiefly in teacher education. I was the only nationally recognized expert on English language arts standards by virtue of my work in Massachusetts and for Achieve, Inc.’s American Diploma Project.
Professor Milgram and I did not sign off on the standards because they were not internationally competitive, rigorous, or research-based. Despite our repeated requests, we did not get the names of high-achieving countries whose standards could be compared with Common Core’s standards. (We received no “cross-walks.”) Nor did the standards writers themselves offer any research evidence or rationale to defend their omission of the high school mathematics standards needed for STEM careers, their emphasis on writing not reading, their experimental approach to teaching Euclidean geometry, their deferral of the completion of Algebra I to grade 9 or 10, or their claim that informational reading instruction in the English class leads to college readiness. They also did not offer evidence that Common Core’s standards meet entrance requirements for most colleges and universities in this country or elsewhere.
IV. Flaws in the 2011 Massachusetts ELA Standards (the document lists David Coleman, Susan Pimentel, James Patterson, and Susan Wheltle as the four lead writers)
A. Most Coleman et al standards are content-free skills, not “content” standards. They do not address specific literary knowledge, specific literary history, or specific reading levels, i.e., they omit significant literary/historical content. E.g., there is no standard on the history of the English language, on British authors or texts, or on authors or texts from the ancient or classical world.
Examples of Coleman et al literature standards in grades 11/12:
- Cite strong and thorough textual evidence to support analysis of what the text says explicitly as well as inferences drawn from the text, including determining where the text leaves matters uncertain.
- Determine two or more themes or central ideas of a text and analyze their development over the course of the text, including how they interact and build on one another to produce a complex account; provide an objective summary of the text.
Examples of authentic ELA literature standards
*In California’s pre-2010 standards for 11/12:
3.7 Analyze recognized works of world literature from a variety of authors:
a. Contrast the major literary forms, techniques, and characteristics of the major literary periods (e.g., Homeric Greece, medieval, romantic, neoclassic, modern).
b. Relate literary works and authors to the major themes and issues of their eras.
*In Massaschusetts’ pre-2010 standards for grades 9/10:
16.11: Analyze the characters, structure, and themes of classical Greek drama and epic poetry.
B. The 2011 Coleman et al standards expect English teachers to spend at least half of their reading instructional time at every grade level on informational texts. They contain 10 reading standards for informational texts and 9 for literary texts at every grade level, reducing literary study in the English class to about 50%. Pre-2011 Massachusetts English classes spent about 20% of reading instructional time on nonfiction (which included informational material). No research studies support increasing the study of nonfiction in English classes to improve college readiness.
C. The 2011 Coleman et al standards reduce opportunities for students to develop analytical thinking. Analytical thinking is developed when teachers teach students how to read between the lines of complex works. As noted in a 2006 ACT report titled Reading Between the Lines: “complexity is laden with literary features.” According to ACT, it involves “literary devices,” “tone,” “ambiguity,” “elaborate” structure, “intricate language,” and unclear intentions. Thus, reducing complex literary study in the English class in order to increase informational reading, in effect, retards college readiness.
D. The 2011 Coleman et al standards discourage “critical” thinking. Critical thinking is based on independent thinking. Independent thinking comes from a range of observations, experiences, and undirected reading. The Coleman et al document contains no standards for writing a research paper like those spelled out in the pre-2011 Massachusetts ELA standards.
V. Why the MBAE and Fordham Studies Cannot Tell Us Much
As noted by the Commissioner of Education in his announcement of the five public hearings on MCAS vs. PARCC, the Board would review studies conducted by “outside organizations.”
The first outside study, commissioned by the Massachusetts Business Alliance for Education (MBAE), was released in February 2015. It recommended abandoning MCAS, yet it did not indicate that current MCAS tests are based on the Coleman et al standards, while the PARCC tests are based on Common Core’s. Do the contents of the test items differ? We don’t know. Nor do we know what test items were examined in this study. Nor does the study give us a single clue to the contents of the test items in either set of tests at any grade level.
A second outside study is being undertaken by the Thomas B. Fordham Institute. The MBAE study had earlier indicated that “the Thomas B. Fordham Institute and the Human Resources Research Organization will conduct a full-scale evaluation of how well aligned PARCC, MCAS, and other national assessments are to the Common Core State Standards and the extent to which they meet the criteria for high-quality assessments established by the Council of Chief State School Officers.” It is not clear why CCSSO is qualified to establish criteria for high-quality assessments. All we know at present is that the Fordham Institute decided to use a portion of the Bill and Melinda Gates Foundation funds it regularly receives to compare MCAS and PARCC test items and to let BESE know what it would recommend as an organization dedicated to the Common Core project. Its report will be issued in time for BESE’s official vote to adopt PARCC in fall, 2015.
Nevertheless, we face the same problems in learning anything from the Fordham report that we face with the MBAE report. The test items for both the 2015 MCAS ELA tests and the 2015 PARCC ELA tests are test-secure and can’t be discussed in a public report. I have twice asked directors of both assessments for permission to examine under secure conditions all their ELA test items for 2015 but have not been given permission to do so. I have also asked DESE for a copy of all proposals or requests to DESE for permission to examine non-released MCAS test items, but DESE has not sent me a copy of this public information.
The public CAN examine “sample” and “practice” test items that PARCC has made available online (which I have done). The public CAN examine all released test items for all MCAS tests from 1998 to 2007 (which I have done—see Appendix A for URLs to these test items). And parents and teachers CAN testify about what students say about the test items they have responded to on their computers or in their test booklets. But researchers cannot present either an evaluation of the grade appropriateness of PARCC test items or a comparison of the contents of MCAS and PARCC test items, two of the sub-topics that testifiers were asked to address at the Bridgewater hearing, because they are not allowed to say anything about the actual contents of the test items if indeed they examined them.
Until all the test items used by PARCC and MCAS in ELA in 2015 are available to BESE and all parents, legislators, and other citizens for inspection under secure conditions, BESE has no legitimate information on which to base an official decision. In fact, the entire process leading to a decision on which set of tests to use appears to be a sham, beginning with the fact that the Commissioner of Education chairs the Governing Board of PARCC, yet is to make the final recommendation to BESE, and ending with the fact that all local superintendents were told in 2014 that the decision had already been made (according to a letter from Superintendent William Lupini to the Brookline School Committee in June 2014, in Appendix B). The public, including the media, have been abused by a fake process. Only a post hoc, pro forma vote for PARCC remains to be taken.
Yet there are significant differences between PARCC and MCAS for ELA tests that can be brought to public attention. These differences have their source in the criteria established by English teachers in Massachusetts in 1997, as explained above, and in other sources.
VI. Problems with PARCC in 2014-15, based on the examples/test items given
* The overall reading level of PARCC sample test items in most grades seems to be lower than the overall reading level of test items in MCAS ELA tests based on the pre-Coleman et al standards—sometimes by more than one reading grade level. E.g., an excerpt from The Red Badge of Courage is an example in the 2015 grades 10 and 11 PARCC. But an excerpt from this novel was assessed in a pre-2011 grade 8 MCAS. E.g., an excerpt from Joseph Conrad’s Heart of Darkness is an example in the 2015 grade 11 PARCC but appears in a 2010 grade 10 MCAS.
* PARCC doesn’t tell us who determines the cut (pass/fail) score, where it will be, and who changes it, and when. Cut scores on MCAS tests are set by Massachusetts citizens.
* PARCC test specifications do not indicate from what authors or kinds of text the literary passages are to be drawn, and how they are to be balanced. English teachers in Massachusetts have had higher expectations for MCAS than do test-developers at PARCC, it seems.
* PARCC 2015 grade 11 test samples are not aligned with Common Core’s standards; there are no passages from founding political documents.
* PARCC offers too many tests at each grade and across grades.
* PARCC requires extensive keyboarding skills and too much time for test preparation.
* PARCC plans to provide only a few released test items for teachers to use, it seems.
* The change to a grade 11 PARCC for fulfilling the requirement for a high school diploma hurts low-achieving students, who often need two years for remediation and retests before graduation.
* The PARCC tests are very long (see the chart in Superintendent Lupini’s June 2014 letter to the Brookline School Committee), even though they have been recently shortened.
* The writing prompts in PARCC in 2015 do not elicit “deeper thinking” because students are not given a provocative question about a reading assignment and encouraged to make and justify their own interpretation of an author’s ideas based on a range of sources, some self-chosen. They are almost always given the sources to use, beginning in grade 3: e.g., “Write an essay comparing and contrasting the key details presented in the two articles about how endangered animals can be helped. Use specific details and examples from both articles to support your ideas.”
* The two-part multiple-choice format in PARCC (and in SBAC) often requires students to engage in a textual scavenger hunt for the specific words, phrases, or sentences that led to their own thinking when answering the previous question. This two-part multiple-choice format is especially taxing and problematic in the early grades. E.g., in grade 3: “Part B: Which sentence from the story supports the answer to Part A?” “Which detail supports the answer to Part A?” “Which detail from X shows another example of the answer to Part A?” “Which detail from paragraph 14 best supports the answer to Part A?” “What phrase from paragraph 14 helps the reader to understand the meaning of thriving?” “Which section in X introduces how the scientists made wolves feel comfortable in the park?” In sum, the questions are poorly worded, confusing, tedious, unfriendly to children, and cumbersome.
VII. Criteria for MCAS ELA Selections Developed in 1997 by the State’s English Teachers
- About 60% of the selections should be literary.
- At least half of the literary selections should come from authors in a list of suggested authors or works reflecting our common literary and cultural heritage
- About half of the literary selections could come from authors in a second list of suggested contemporary authors from the United States, as well as past and present authors from other countries and cultures.
These criteria were enforced in two ways for MCAS ELA tests: by the Guiding Principle on literary study in the introduction to the ELA standards and by the use of texts by authors in the two lists. The Guiding Principle itself (“An effective English language arts curriculum draws on literature from many genres, time periods, and cultures, featuring works that reflect our common literary heritage.”) indicated that a “comprehensive literature curriculum contains works from both [lists].” The two lists of recommended authors served as guides to choosing MCAS passages at all grades. MCAS ELA tests from 1998 on were dominated by literary selections because of these criteria, the Guiding Principle on literary study, and the two lists.
BESE voted to add the Guiding Principles and the two lists in the 2001 Massachusetts ELA curriculum framework to the Common Core standards adopted in 2011. But DESE altered the wording of the Guiding Principle on literary study to read “An effective English language arts and literacy curriculum draws on literature in order to develop students’ understanding of their literary heritage” so that it no longer expected the school curriculum or literary passages on MCAS to feature works reflecting “our common literary heritage.”
VIII. Recommendations for Massachusetts:
- Fewer grades tested (just 4, 8, and 10), as in the 1993 MERA and 1994 authorization of ESEA
- Paper and pencil tests; no computer-based tests
- All or most test items released every year, as MERA requires
- Retention of grade 10 competency determination for a high school diploma, required in MERA, for the benefit of low-achieving students
- Tests requiring less time for preparing for and teaching to the tests
- Test passages and questions chosen and reviewed by Massachusetts English teachers
- A Massachusetts-determined cut score
Appendix A. URLs for locating all MCAS ELA test items from 1998 to 2007, plus some URLs for later items
http://www.edbenchmarks.org/schoolimprovement/stuach.htm On MEAP 1992-1999
https://www.brocktonpublicschools.com/uploaded/TeachingLearning/MathResourcesK-8/MCAs-Questions/MCAS-2004.pdf (Grade 10 ELA includes an excerpt from Tartuffe)
https://www.brocktonpublicschools.com/uploaded/TeachingLearning/MathResourcesK-8/MCAs-Questions/MCAS-2005.pdf (grade 10 ELA includes excerpts from Macbeth and Pride and Prejudice; and Theodore Roethke poem)
https://www.brocktonpublicschools.com/uploaded/DistrictDepartments/Assessment/mcas_2007.pdf (grade 10 ELA includes excerpt from Oliver Twist)
http://www.doe.mass.edu/mcas/2010/release/g10ela.pdf (grade 10 ELA includes excerpts from Heart of Darkness and Love in a Time of Cholera; Shakespeare’s Sonnet #73)
http://www.doe.mass.edu/mcas/testitems.html?yr=14 (Selected items from 2010 to 2014 available here.)
Appendix B: Letter from Superintendent William Lupini to the Brookline School Committee in June 2014
THE PUBLIC SCHOOLS OF BROOKLINE
333 WASHINGTON STREET BROOKLINE, MASSACHUSETTS 02445
Office of the Superintendent of Schools
William H. Lupini, Ed.D.
June 3, 2014
To: Members of the Brookline School Committee
From: William H. Lupini, Ed. D. Superintendent of Schools
Re: State Assessment for 2015
On May 22, 2014, I recommended that the Public Schools of Brookline administer the PARCC Assessment for grades 3-9 and 11 for the 2014-2015 school year. This recommendation was based on the following considerations:
• Our experience with the recent PARCC field test allowed our team to gain a deep understanding of all that is required to administer this assessment to support students’ success. Our learning was detailed in my presentation to the School Committee at our last meeting.
• The Department of Elementary and Secondary Education (DESE) will “hold harmless” the accountability status of Districts choosing to administer PARCC in 2015. Specifically, a school’s level will either stay the same or improve but cannot decline due to PARCC test results.
• MCAS will be phased out in favor of either PARCC or another new “next generation” assessment after the 2015 test administration.
• Administering PARCC in 2015 will allow all students tested the opportunity to get comfortable with the new expectations and testing environment, and will give us the opportunity to fine-tune its administration, which may reduce the risk of disruption in future years.
• The high school did not participate in the 2014 pilot. Administering PARCC in grades 9 and 11 in 2015 offers BHS a year to pilot the new assessment. Also, a score of 4 or 5 on the PARCC Assessment would allow an 11th grader to skip remedial courses at Massachusetts state colleges. MCAS will still be administered to all 10th grade students through the class of 2018 for competency determinations.
• In addition to being “held harmless,” DESE has mitigated other risks for districts that choose to administer PARCC in 2015, including:
-Pencil and paper tests will be an option for a number of years in order to allow districts to adequately prepare their technology to meet the needs of the online test; and,
-Student Growth Percentiles (SGP) will be calculated continuously; therefore, there will be no interruption in utilizing SGP in the educator evaluation system.
The purpose of this Memorandum is to provide you with additional information about PARCC testing, our revised recommendation for your consideration during the June 5th Public Hearing and your June 19th vote, and the reasoning for these revisions to our thinking. Additional Information One of the main areas of discussion during our May 22nd presentation involved the number of PARCC testing sessions at each grade level.
Following is a chart detailing the grade-by-grade and subject area testing sessions for both PARCC and MCAS (grades 3-8):
Grade Level PARCC & Science MCAS Difference
3rd 9 5 +4
(5 ELA; 4 Math) (3 ELA; 2 Math)
4th 9 7 +2
(5 ELA; 4 Math) (5 ELA; 2 Math)
5th 11 7 +4
(5 ELA; 4 Math) (3 ELA; 2 Math)
(2 MCAS Science) (2 MCAS Science)
6th 9 5 +4
(5 ELA; 4 Math) (3 ELA; 2 Math)
7th 9 7 +2
(5 ELA; 4 Math) (5 ELA; 2 Math)
8th 11 7 +4
(5 ELA; 4 Math) (3 ELA; 2 Math)
(2 MCAS Science) (2 MCAS Science)
These differences are somewhat governed by the addition of end-of- year (EOY) testing in PARCC, along with the inclusion of a writing composition component for grades beyond the fourth and seventh grade currently tested in MCAS.
The amount of time to be spent in testing is a much more complicated analysis. Students are permitted 50% additional time beyond what is recommended in PARCC, while MCAS is an untimed assessment. Below is a comparison of the “expected” times for both grade 3-8 scenarios described above:
Grade Level PARCC & Science MCAS Difference
3rd 490 minutes (8.2 hours) 270 minutes (4.5 hours) +220 minutes (+3.7 hours)
4th 530 minutes (8.8 hours) 360 minutes (6.0 hours) +210 minutes (+3.5 hours)
5th 620 minutes (10.3 hours) 360 minutes (6.0 hours) +260 minutes (+4.3 hours)
6th 570 minutes (9.5 hours) 270 minutes (4.5 hours) +300 minutes (+5.0 hours)
7th 570 minutes (9.5 hours) 370 minutes (6.2 hours) +300 minutes (+5.0 hours)
8th 660 minutes (11.0 hours) 370 minutes (6.2 hours) +290 minutes (+4.8 hours)
These numbers are somewhat misleading in that the PARCC timing is probably much closer to actual for most students, given the “timed” nature of the assessment. Furthermore, given that factor, it would be possible to schedule multiple testing sessions in one day with PARCC, while this is not possible in our current MCAS assessment configuration.
The high school analysis is even more difficult, given the following factors:
• As noted earlier, current MCAS assessment occurs only in 9th grade with a Science test, 10th grade with the English Language Arts and Mathematics exams, and again beyond 10th grade for those students who did not initially meet the competency determination standards.
• The PARCC assessment system is designed to provide 11th grade students who score of 4 or 5 on the PARCC Assessment to skip remedial courses at Massachusetts state colleges.
• MCAS will still be administered to all 10th grade students through the class of 2018 for competency determinations.
• PARCC high school math assessments are based on courses aligned to the Common Core State Standards, not grade levels.
Assessments are available for Algebra I, Geometry, Mathematics I, Mathematics II, Algebra II and Mathematics III.
Given these factors, it is more difficult to provide a comparison of numbers of testing sessions and total time devoted to assessment for PARCC v. MCAS. However, it is very safe to conclude that students would experience a greater volume of testing under the PARCC plan than is currently the case.
After considering input from the Headmaster and her administrative team, as well as issues raised by School Committee members at our May 22nd meeting, we are now recommending that the Public Schools of Brookline participate in the PARCC operational test for grades 3-8 only during the 2014- 2015 school year.
High School testing would be limited to those MCAS tests required for the competency determination in 9th and 10th grades.
We do not come to any of these recommendations lightly. This new assessment will consume more valuable teaching time than the current program. The timed nature of the assessment for students who do not have an IEP is not in the best interest of any of our students and represents a significant change in beliefs for the Commonwealth. The PARCC assessment is still in development and, as such, will continue to represent a learning opportunity for all of us, even while students are receiving scores for their performance on the exams. Finally, we are not at present prepared to move to an on-line testing environment as a school system, meaning that some of our students will participate in a paper and pencil assessment and, therefore, we will have students being tested on somewhat different competencies and skills across our schools.
However, much of our rationale for this recommendation is, in our view, compelling and remains the same as discussed in May. We cannot recommend staying with MCAS for another year if this assessment is to be phased out in favor of either PARCC or another new “next generation” assessment. We believe that students should be given the opportunity to experience “next generation” expectations and testing environments, and that we need the chance to work with the administration of these assessments. Finally, we need to take advantage of having school accountability status held “harmless” while we work to support student, teacher and school success within this new testing situation.
While this same logic exists with respect to high school testing, we simply do not believe that it outweighs the issues for our students. As was discussed on May 22nd, eleventh grade students would be taking a PARCC assessment after most of them had already met the competency determination in their sophomore year, without the benefit of knowing up front that this was to be the case. Ninth grade students would be participating in a “next generation” pilot program, only to revert to MCAS as a competency determination exam. Therefore, we do not believe that the benefits of PARCC testing outweigh these concerns for our high school students in 2014-2015.
I am looking forward to continuing our discussion of this recommendation with you at our meeting on Thursday, June 5, 2014.