MCAS 2001 – Hindsight is 220

Last week, with relatively little fanfare, the Massachusetts Board of Elementary and Secondary Education voted to “raise MCAS graduation requirements” beginning with the Class of 2026 (i.e., students beginning high school this fall).

Although the approved requirements do have the effect of “raising” the test-based graduation requirements as reported, what they actually do is align those requirements with the state’s current content standards, state tests, and corresponding achievement standards established in 2017 for grades 3-8 and in 2019 for grade 10.

This change, delayed most recently by the pandemic, is long overdue. Due to a series of unfortunate events that even Lemony Snicket would find implausible, the current graduation requirements had become the last vestige of the original 1998 MCAS Achievement Standards – an invisible string, frayed, knotted in several places, and spliced in others that refused to let go.

Although there is plenty to discuss and debate with regard to the specifics of these requirements or whether there should be test-based graduation requirements at all, what caught my eye in the article reporting on the Board vote was the following sentence describing a letter that a group of nearly 100 state legislators opposed to raising the requirements had sent to the Commissioner in June:

“The letter also detailed how more than 52,000 students have reached the end of high school without passing the MCAS exams since certain test scores became a graduation requirement in 2003. These students have been labeled as “dropouts,” the lawmakers wrote, regardless of whether or not they fulfilled all their other graduation requirements.”

I don’t know the source of the 52,000 figure but I will take it as face value.

More than 52,000 students.  Two thoughts ran through my mind as I read about those 52,000 students.

My first thought was, “Damn! Are you telling me that after 20 years there are still people who think that you have to pass the MCAS test in order to graduate.” That has never been true. We made sure of that beginning with the Class of 2003 – the first class subject to the graduation requirements.

My second thought was that if you came to me in the summer of 2001, as we were hammering out these graduation requirements, and said, “Charlie, if you implement these requirements now, in 20 years there will have been more than 52,000 students who “passed” all of their courses, but not “passed” the test,” I would have breathed a huge sigh of relief.

That’s an outcome I would have taken in a heartbeat, a New York minute, the blink of an eye; in other words, without the slightest bit of hesitation or regret.

Getting Our Ducks in a Row

It would be difficult to overstate how much attention was being paid to the work we were doing at the Department leading up to the first high-stakes administration of the grade 10 MCAS tests in 2001 – for students graduating in the Class of 2003.

  • There were FOIA requests and legal challenges from local media to access item-level data virtually as soon as we had it.
  • There were protests on top of protests and a media blitz from the largest teachers’ union in the state.
  • Our “friends” over at Boston College were lobbing grenades on a regular basis.
  • The AERA Position Statement on High-Stakes Testing in Pre-K – 12 Education was adopted in July 2000.
  • There was the specter of the lawsuit in Texas and the lawsuit(s) that we knew were being prepared in Massachusetts.
  • The Deputy Commissioner showed up at my cubicle one day with the education reporter from the Boston Globe in tow and directed me to give them my home phone number.

Oh, people were paying attention, Ray. People were paying attention.

We worked side-by-side with the Department legal team until the legal folks at the State House took over.

We knew that there was a trade-off between student accountability and school accountability, so we set the test-based bar for students at the lowest proficiency threshold – an MCAS score of 220 – while making schools responsible and accountable for getting kids to the Proficient level (a policy still in place in the newly approved requirements).

We knew that we had to offer students multiple opportunities to pass the test between the spring of 10th grade in 2001 and their scheduled graduation in June 2003, so we developed a robust retest schedule.

The creative juices were flowing when we concocted and sold the idea of focused retests – test forms designed specifically to measure whether students had met the graduation standard and not the full performance continuum – an idea that worked better on paper than in practice.

On top of that we knew that we had to offer some sort of appeals process.

More than anything else, I wanted Commissioner Driscoll to be able to stand at a podium and say truthfully that no student who had achieved the knowledge and skills required by the standards would ever be denied a diploma simply because of their MCAS score– while at the same time protecting the test and maintaining the teeth in the graduation requirement.

Student portfolios were still quite popular in the field and a possible approach to performance appeals, but they were unwieldy and the technical challenges of implementing a portfolio program were well documented and still fresh in our minds.

Then one Saturday morning with my head in the clouds (literally, we were on a flight back from a meeting with the assessment contractor in San Antonio) the framework for an appeals process seemed to materialize out of the thin air.

A student’s performance in standards-aligned coursework in grades 10-12 would be their portfolio. If other students with similar performance in those courses had a high probability of passing the test that would be sufficient evidence that this individual student had indeed achieved the required level of proficiency on the state content standards – even if they were unable to demonstrate that on the MCAS test after repeated attempts.

I would love to say that I sketched it out on the back of a napkin, but I’m pretty sure I had a notepad with me.

I shared the idea with Jeff Nellhaus, assessment director, as we switched planes. We fleshed out the framework and shared it with the rest of the assessment team, legal, and the Commissioner on Monday. The Commissioner convened a “blue-ribbon panel” of representatives from advocacy groups, the harshest critics of the program, and a couple of measurement people from across the state. After sufficient bureaucratic language was added, a performance appeals process that places responsibility and accountability for student proficiency at the local level was born.

So no, the test should not have been the wall separating those 52,000 kids from a diploma.

Get Everyone Over The Wall ™

More than 52,000 students since 2003 reached the end of high school, ostensibly met all other graduation requirements, and didn’t receive a diploma because they couldn’t demonstrate the required level of proficiency. That’s a lot of kids.

More than 52,000 students since 2003 is approximately 3% of the more than 1.6 million students enrolled in tenth grade since the first high-stakes MCAS test was administered in 2001.

In the years leading up to 2001 and the graduation requirement, more than 3% of students who entered twelfth grade each year in Massachusetts did not receive a diploma at the end of the year. That, too, is a lot of kids.

More than 52,000 students since 2003 is a lot of kids.

In spring 2001, as tenth grade students in the Class of 2003 were sitting down to take the first high-stakes MCAS test, there were more than 60,000 students currently enrolled in grades 11 and 12 who had not passed the MCAS Mathematics test. That number grows a bit if you include students who failed the English language arts test. Mathematics was always the gatekeeper, but there were a small number of students who passed the Math test, but not ELA.

When the results of the 2001 MCAS tests were released, there were more than 20,000 students who did not pass the ELA and Math tests on their first try.

IMG_3313Those individual students, now names not numbers, became the focus. George Bush may have had No Child Left Behind, but Massachusetts had Barney Brawer and the Program for Educational Change Agents at Tufts University. Barney trademarked the phrase Get Everyone Over the Wall (GEOTW) and was committed to ensuring that each and every one of those students received the instruction and supports they needed to succeed.

Likewise, the state was committed to addressing the “specific needs” of those students who had not yet met the graduation requirement.  At MCAS Reporting Workshops across the state, we rallied the troops while sporting spiffy “Failure Is Not An Option” hats (purchased online from the NASA gift shop).

Were some of the resulting efforts misguided, placing too much emphasis on “passing the test” and too little on students acquiring necessary knowledge and skills? Of course. But others weren’t. As each year passed more students had the skills needed to pass the tenth-grade tests on their first try and to continue to grow, not simply catch up, in eleventh and twelfth grade.

In an ideal world, all students would acquire the knowledge and skills necessary to meet tenth grade standards by the end of the tenth grade – at the latest. But this is not an ideal world. There are kids who reach the end of twelfth grade without the “necessary” knowledge and skills.

Some of those kids are students with disabilities who now receive instructional services and supports until they are 21 rather than being handed a diploma and sent on their way by the public school system at 18 – a number that has grown in Massachusetts from 224 students in 2004 to over 1,500 students per year in recent years.

There always will be, and always should be, spirited debate about what knowledge and skills should be required to earn a diploma. Sadly, there likely will always be a need for debate about whether students are harmed more by receiving a diploma without being able to demonstrate the required knowledge and skills or by being denied a diploma. Those are issues worthy of debate. But they have nothing to do with MCAS or any other state test.

It was never about the test.

It rarely is.

Image by Claudia from Pixabay

Published by Charlie DePascale

Charlie DePascale is an educational consultant specializing in the area of large-scale educational assessment. When absolutely necessary, he is a psychometrician. The ideas expressed in these posts are his (at least at the time they were written), and are not intended to reflect the views of any organizations with which he is affiliated personally or professionally..

%d bloggers like this: