Taking Stock

Two years ago, I wrote that Fourth of July was a time to take stock.

It’s a time midway through the calendar year to review those goals and resolutions we set for ourselves back in January.

On our nation’s birthday, we can pause to take stock on the state of our country, what it has been through, where it is now and how it got here, and most importantly, where it is going, where we want it to go, and what we can do to help it get there. But I strongly recommend staying away from cable news and off of Twitter (X) or any other social media platforms for at least a week before engaging in that activity.

This July 4^th, I find myself taking stock of state testing and large-scale testing, in general.  Where do we stand a quarter century into the NCLB era of state testing, a decade beyond the long-awaited shift to computer-based and computer adaptive testing, and five years since being excoriated as the new Jim Crow.

Taking stock back in 2010, I thought that state testing was at a crossroads. Around 2015, it felt like we had reached a tipping point – state tests simply were too much and there was too much testing. By the end of 2020, we were lost, rudderless, questioning everything about out past and present and unsure of our future.

My sense today is that as a field we are stuck. We are desperately trying to figure out our place in the education world, to determine the role that we can play in supporting instruction and improving student learning. As part of that process, we find ourselves trying to be all things to all people, to please everyone, or at least trying to stay on the good side of those shouting the loudest.

As I was writing this paragraph, that sense was reinforced by an email that arrived from nwea with the subject line: “MAP Growth is 50,000 items strong and more.” It was a personalized e-mail (“Dear Charles,”) telling me

No two students are the same. That’s why you need an assessment solution flexible enough to allow all kids to show what they know and are ready to learn next. MAP Growth can help your teachers reach every single student, whether they’re above grade level and ready for a challenge or Spanish speakers needing extra support while they learn English.

I don’t mean to single out nwea because the message in that email reflects the talking points heard wherever two or more people are gathered in the name of state testing; not to mention the thoughts rattling around in the head of one crusty old state testing person on an afternoon stroll. (The “50,000 items strong” might be unique to nwea. Impressive. Kudos. All written by people? Never mind; we’ll save that question for another post.)

Over the years, my colleagues and I have engaged in several passionate discussions about the level of personalization and alignment with instruction required for large-scale testing. As opposed to instruction, which should be grounded in a great deal of personalization, I’ve always been of the mind that large-scale testing is more about generalization than personalization. I’m not really interested in the path that you took to get to proficiency; you should be able to demonstrate that proficiency on the test.

I’ve also always been skeptical of the argument that there are a host of students out there who are proficient in reading, mathematics, or science, but just cannot demonstrate it on the test. Most of that skepticism is based on the fact that there has been sufficient cause and ample opportunities for schools, parents, etc., to bring those cases and students forward, but it just never happened – even when a high school diploma was on the line.

A single case or outlier here or there that must be considered outside of the large-scale system, sure, but not huge subsets or subgroups of students. And please, don’t bring me the student who is the mechanical, musical, or mystical prodigy, but is just not good at reading or mathematics. Or that poor kid with disabilities who cannot get a job because they couldn’t pass the mathematics test. Those are policy questions to resolve. These kids cannot read or do mathematics, but… And I know you would never dare to mention the student who is getting As and Bs in all of their classes, but just cannot pass the test. Seriously. Those kids are the primary argument for large-scale state testing.

Take proper care in building the test, sure, but large-scale testing only works and makes sense if it is, in fact, large-scale testing. It’s a limitation, but it’s also a feature.

A test, like a man (or a woman) has got to know its limitations.  Large-scale state testing has limitations, but that’s OK. As several of my colleagues have pointed out, state testing had one job and has done it pretty darn well – estimating the percentage of students in a school who have met the state standards.

We should be happy with that.

So many recent failed attemps at assessment innovation can be traced back to well-intentioned efforts to get the state test to do more than it ever possibly can do. Advances in technology have enabled us to make state testing more efficient and will continue to do so. And those advances may have opened up other avenues for collecting the information provided by state tests.

But no amount of innovation or inspiration is going to turn a state test into anything more than it is. It is what it is. Large-scale state testing is all that it can be and ever will be. If you’re happy working on large-scale testing, fine. There will likely be a place for you to prosper for years to come.

To fulfill its promise, assessment innovation must be focused on innovation in assessment, which is different than innovation in testing, and different still from innovation in measurement. (That ever-awkward relationship among testing, assessment, and measurement is rarely recognized or acknowledged, but once again will be front and center as the field works to revise the Standards for educational testing. Again, a topic for another day and post.)

As some of my large-scale testing sisters and brothers have already done, if it is assessment innovation that interests you, it’s time to let large-scale testing be and focus on assessment – assessment that lives in schools with students and teachers. Assessment that includes testing but is so much more than testing. Assessment of students on standards rather than assessment of standards.

The world is wide enough for both large-scale state testing and innovative assessment, but those of us in the field, particularly those just starting out, need to take stock of our own situations, our own interests, our own strengths and limitations, and make a choice. In what part of the testing, assessment, or measurement worlds do we want to reside? And I haven’t even mentioned the data and secondary analysis folks. They are an important part of our world, too. It’s an exciting time, but one test cannot do it all and one person cannot do it all.

 

Image by Bernhard Falkinger from Pixabay