Google Find us on Google+

Alan Singer: Florida, Pearson and Rooting for Julie and Daryl.

Jun 22, 2017 by

An Interview with Alan Singer: Florida, Pearson and Rooting for Julie and Daryl.

Michael F. Shaughnessy –

1) Alan, first of all, you just published an article about concerns about the teacher certification exam in Florida. What do you see as the main issues here?

I regularly blog for Huffington Post on educational issues. A veteran Florida teacher contacted me who had read some of my blogs about Pearson Education, I call the company Pearson (Mis) Education. Pearson is a global giant involved in high-stakes testing and textbook publishing in the United States and in low-cost but third-rate schools they market in the Third World.

This teacher, the Julie in the title, failed the essay portion of the Florida Educational Leadership Examination (FELE) test four times. Each time she received the exact same score, one point below passing. The FELE test was created by the Florida Department of Education but is administered and graded by Pearson. Although Julie has outstanding evaluations and is an excellent student herself, she is being denied a salary increase and professional advancement because she failed the test. She challenged the legitimacy of the test in a Florida administrative court hearing and is waiting for a judicial decision. I spoke with her legal team and made some recommendations for their presentation. Daryl is another Florida teacher and test-taker who took the Florida Department of Education and Pearson to court.

There are two main issues from my perspective and these issues relate to all high-stakes testing. Is the test valid and is the scoring reliable? Validity means the test measures what it claims to measure, in this case teaching excellence. Reliability means the grading system accurately measures the test-takers performance. In my view this test, and many high-stakes tests, fail by both criteria.

The case in Florida centers on the reliability of the grading. At the day long administrative hearing FDOE produced five “expert witnesses” to defend the testing process and Pearson sent its lawyers to observe. A representative of FDOE maintained that Pearson’s grading system is extremely detailed and thorough. FDOE’s attorney said, “the idea of human error is beyond belief.” While one of the FDOE “expert witnesses” was a Florida school administrator, he is also a paid Pearson employee. During the past two years he reviewed 20-25 failing FELE essays and acknowledged he never reversed a score. One hundred and sixty failing FELE test-takers challenged their scores last year, and none were reversed by Pearson.

This must be the only time in test assessment history that grading is 100% reliable. I found an article on a Pearson website where they bragged that their Versant Technology when reading essays had an inter-rater reliability of 0.89, which was HIGHER than human inter-rater reliability, and is considered very high. But it still means that about 10% of the test grades were not consistent.

But there is another reason the FDOE expert witness’ assessments are unreliable. If the test scorer only reviewed failing exams that were being appealed, he already knew these test-takers had failed. Essentially he was being asked to confirm what FDOE and Pearson wanted confirmed. In a fair review, without bias, these tests would have been mixed in with ungraded exams and the reviewer would not know that any of them had already received a failing grade

2) Now, how in God’s name is any test able to predict whether a teacher will be competent in the first grade, third grade, or twelfth grade?

The FELE essay test, and similar Pearson teacher and administrative certification tests, provide test-takers with a school-based scenario imbedded in one or multiple “prompts” and asks them to address a classroom or school “situation.” When I help candidates prepare for one of these essays, the first thing I tell them is that Pearson and the scorers are not interested in your opinions about teaching, children, or about schools. The answer they are looking for, the answer they want, is in the prompt. Find the answer and give it to them. This is about getting certified, not showing how good you are as an educator, because the test does not measure that. Second, Pearson does not want to see creative ideas or creative writing. It is a very structured writing assignment. Follow their outline and answer all the parts. People who follow my advice generally pass the tests without difficulty. This tells us nothing about their qualities as a teacher or administrator, only that they followed the directions and are good at giving the reader what they want to hear.

3) Taking this one-step further, how can any test predict how well a teacher will do with a group of 10, 20, 30 or even forty kids?

One of the most troubling Pearson product is the edTPA portfolio developed by a group at Stanford University and administered and graded by Pearson. The Pearson/SCALE edTPA is used to evaluate student teachers by over 700 teacher education programs in forty states and is required for certification in sixteen states. It is a roughly sixty-page portfolio plus video that is subject to arbitrary grading practices, arbitrary practices that Pearson denies. I have seen very similar submissions get very different evaluations and the better student teachers often perform less well than weaker candidates on edTPA because they concentrate on developing as teachers rather than on an exhausting and time consuming portfolio that takes them away from mastering their craft. I always wonder if the test evaluators sitting on their computers have any experience working with the student populations that the student teachers are teaching.  So the answer to your question is that tests like these have virtually no predictive validity.

4) And one more step; how can any test predict how well a teacher will do with a class of students with attention deficit disorder, or hyperactivity, or autism, or brain damage, or a second language learner?

New York State has an Educating All Students multiple choice and essay test that claims to do exactly that. One set of prompts focuses on diversity issues, another on students with special needs, and the third on English language learners. Once again the key to passing the test is finding the answer the state wants in the prompts that are always similar. I recommend that test-takers review sample questions before taking the test, outline their answers in advance, and then modify their outline based on the specifics of the prompt. The New York State tests provide test-takers with sample highly evaluated essays so you know exactly what they want. A problem for Florida test-takers is that there are no sample correct answers in the guidebook so you have to guess what the scorers are looking for.

5) Somebody somewhere is making money from all this testing and the pencils seem to point to Pearson. How much does the average teacher spend taking these tests and how much do they charge if a teacher should fail a test?

The union representing teachers working in the State University of New York estimates that students could spend up to $400 to complete the battery of state certification exams. Pearson profits each time someone fails one of these exams. Prior to 2009, the Florida Department of Education subsidized test takers. Candidates paid $25 to take each part of the multi-part tests and did not pay to retake a section that they failed. Pearson now charges test-takers up to $200 per section, an increase of 800%, and an additional $20 to retake a section. Test-takers can appeal failing scores, but they have to pay $75 for a reevaluation.

6) I would say that trying to predict a teachers behavior from some multiple choice test, is like trying to predict my performance on the basketball court with the Dallas Mavericks – I might be able to do well on the test, but once you put me on the court with Dirk Nowitzki, I am pretty well finished. Appropriate analogy?

I am not sure about Dirk, he is entering the tail end of his career, but otherwise the analogy holds. I like smart, but academic knowledge and performance ability do not always correlate. The best teachers may not be your best test-takers. Getting back to sports, superstars do not make the best coaches or managers because their own talent and drive often makes it difficult for them to understand the nitty-gritty of the game and the struggles of other players. Joe Girardi of the Yankees was a major league catcher, but never a star. Greg Popovich of the San Antonio Spurs played basketball in college before becoming a coach, but did not play pro-ball.

7) Asking someone to reassess a test that they already know the person has failed is somewhat setting that person up for failure? Or am I off on this?

I agree 100%, but of course, nothing is ever 100%, except maybe for Pearson test grader reliability (sarcasm). The deck was stacked against these test-takers. I found an article on a Pearson website where the company bragged that their Versant Technology when reading essays had an inter-rater reliability of 0.89, which was HIGHER than human inter-rater reliability, and is considered very high. But it still means that about 10% of the test grades were not consistent from one grader to the next.

8) Do you have any data on disaggregate numbers? Are Hispanics doing better on these tests than White Anglo Saxon Protestants and are African Americans doing better than Asians? (Or am I so politically incorrect, that I am not supposed to ask these questions?)

This was a big issue in New York State where a federal judge threw out teacher certification tests she considered both discriminatory and unrelated to job performance. Judge Kimba Wood ruled that exams developed by a Pearson sub-division and used by New York State to evaluate teaching candidates was racially discriminatory because the pass rate for African-American and Latino candidates was as low as half the pass rate for White candidates. According to Judge Wood, once this was established, the State Education Department had to demonstrate that the exams actually measured the skills required to be a teacher, which the state education department and Pearson did not do. Note Kimba Wood is far from a radical left-wing judicial activist. President Ronald Reagan nominated her to the federal bench in 1988 on the recommendation of New York State Republican Senator Alfonse D’Amato.

9) Now, guess what, there is this thing called a teacher shortage. Will having all these tests that teachers have to take do anything to help or hinder the shortage?

I believe the teacher shortage is being artificially created as part of an effort to deprofessionalize teaching, eliminate Schools of Education, break the teachers’ unions, and turn teaching into a temporary job where classroom monitors follow scripts and leave before accruing seniority, pensions, or salary increments and learning the craft of teaching. If you deprofessionalize teaching it opens the door for for-profit edu-entrepreneurs, hedge funds, and charter companies to make big bucks at the expense of children and the public. To quote a prominent American tweeter: “SAD!”

10) Alan, I have a great idea. Why don’t you and I devise some test that will make all the children HIGHLY MOTIVATED to learn, and thus make the teachers all excel? Perhaps we could eliminate poverty while we are at it. What do you think?

Great idea. We can start a for-profit, market the program, make billions, sell it to Pearson, and get out of the business before anyone discovers it doesn’t work. We can also use the company to brainwash millions of children and turn them into liberals. I found a brainwashing tutorial on Youtube by anonymous Teacher X we can use.

Print Friendly, PDF & Email

Leave a Reply

Your email address will not be published. Required fields are marked *