The Future of Assessment is Assessment

Educational assessment, defined largely by large-scale educational assessment, in general, and state testing, in particular, is at a crossroads.

If you don’t feel that “at a crossroads” is the right image to depict the current situation, feel free to insert your own alternative such as at a pivotal moment, a turning point, or an inflection point.

Personally, I don’t favor the use of the all-too-obvious inflection point to describe the current situation.  Although the mathematical (i.e., technical) definition of the term tells us that an inflection point is “a point of a curve at which a change in the direction of curvature occurs,” and the colloquial use of the term suggests that an inflection point is “a time of significant change,”  our monotonically-conditioned eyes tell us that nothing much changes after the inflection point.

Our inflection points, rather than a crossroads, fork, or turning point, are usually nothing more than a speed bump after which the curve continues on in the same direction as it was headed before the inflection point. Even worse, the image most likely to pop into our heads is that of a curve flattening out after the inflection point which conveys a sense of slowing down, inertia, and frankly, little change.

Maintaining the status quo is certainly not the path we want to follow at this point in the history of state testing. It’s clearly time to do something. But what?

There are those who would have us believe that current state testing policy is a dumpster fire threatening to burn down the House and that the only way out is to sound the alarm.  Unfortunately, behind their door #2 is an ESEA reauthorization that while rhetorically radical manages only to re-form rather than reform existing testing policy and uses the latest buzzwords to repackage the same old school improvement wish list that has been in place for decades while offering no reasons why things will work better this time around.

More extreme are those who feel that current testing policy, test instruments, and test use are so entangled in and strangled by weeds, thorns, and invasive vines that fire is the only answer. Burn the whole thing to the ground and start again. Burn, baby! Burn!

That’s an option, but let’s leave the pin in that grenade for a moment.

Before putting a match to state testing, and hoping against hope that better, more equitable tests and testing practices rise from the ashes, let’s try to extricate ourselves from the weeds and thorns, take a few steps back, and consider the big picture of where assessment is, what it is, and where it’s going.

Educational Assessment – What it is and what is isn’t

We have an annoying habit in educational assessment of using the same word to mean different things (e.g., standards) and using words interchangeably that mean very different things. It’s a neat trick that comes in handy if you want everyone to leave a meeting thinking that they are on the same page, but we’re going to need to clean that up if we want to move educational assessment forward.

At the heart of the issue today are three words that we love to use interchangeably: measurement, testing, and assessment.  Throw the word “educational” in front of them and pick your favorite. Unfortunately, they don’t mean the same thing.

Fortunately for us, in two recent books that everyone should read, two leaders in our field clarify the distinctions among them.

Derek Briggs, in the opening section of his 2022 book, Historical and Conceptual Foundations of Measurement in the Human Sciences, states emphatically that testing is not measurement.

Testing and measurement are two distinct activities. This assertion is so important that it bears repeating: Testing and measurement are two distinct activities.

As he explains the differences between the two, my conclusion is that our efforts to pick and choose which aspects of measurement to apply to our primary activity, which is testing, have not served us well.

Greg Cizek, early in his 2020 book, Validity: An Integrated Approach to Test Score Meaning and Use, tackled the differences between testing and assessment.

[Test] In the social science, a straightforward and broadly generalizable definition is that a test is a sample of information about some intended characteristic of persons that is gathered under specified, systematic conditions.

[Assessment] Prior to its use in education contexts, the term assessment was correctly applied in many other fields, including finance, medicine, and psychology. For example, when a retirement advisor conducts a financial assessment for a client, he or she examines a diverse set of variables…to provide an overall summary and plan for the client.

Thus, an assessment is best defined as the collection of many samples of information – that is, many tests – toward a specific purpose…In every case, assessment involves collecting and summarizing information in order to develop a course of action uniquely tailored to an individual’s needs.

There you have it, definitive words from two of the best that our field has to offer, both past presidents of NCME, both of whom assorted bios and generative AI platforms will tell you are respected leaders in educational assessment, one of whom I am comfortable calling a measurement expert and the other an expert in large-scale testing.

Cizek ends his discussion with the conclusion that “it is also likely that the terms are now so pervasively used as synonyms that fussing about the distinction is fruitless.” Nearly two decades of monthly staff meetings tell me that Cizek is probably right with regard to the use of the terms. What the distinctions do, however, is to allow us to see and understand the past and present of the endeavor that we call educational assessment, and in doing so increase hope for its future.

Educational Assessment: What it was, is, and can be

There is no shortage of recent books, articles, blog posts, and screeds that trace the origins of educational assessment, large-scale testing, and state testing back to its measurement past.

You may view that measurement past as educational assessment’s original sin that stains all that we do today, or as the means by which tests and testing policy were “more easily appropriated as vehicles for social injustice, even when this may well have been the opposite of the intent of the test designer” (to borrow from Briggs), or you may ascribe no particular importance to our measurement roots. For our current purpose, that is fine. To each their own.

What should be clear to all, however, is that tests and testing replaced measurement as the focus of and driving force behind educational assessment long ago. The Measurement Era was replaced by the Testing Era when psychometricians replaced psychologists (educational psychologists and real psychologists). When we stopped worrying about defining things like constructs in favor of the more practical task of rank ordering students based on their performance on amorphous blobs of content. When we expanded the concept of validity beyond anything that our educational measurement forefathers and foremothers, or our colleagues engaged in physical measurement might recognize. Not that there’s anything wrong with that.

Testing replaced Measurement and it was good. For a while. Well, maybe.

Where things started to go off the rails in the Testing Era is when we became hyper-focused on the test rather than on the purpose of testing and the needs of those being tested. When all of our attention was devoted to building a better test. A test that included all students (for policy not measurement reasons). A test that was “aligned” with that now better-defined, but still fairly amorphous, blob of content. A test which was more authentic and less expensive and more expansive and less time-consuming. A test which gave precise information about a student’s location on an underlying scale which, oops, we didn’t actually create when we left the Measurement Era behind.

So, it should have come as no surprise that the Testing Era, on the cusp of a next generation of assessments that it couldn’t support, collapsed under its own weight and the weight of expectations it could never meet.

And that is where we find ourselves now, witnessing the decline and fall of the Testing Era.

The end of the Testing Era, however, does not portend the end of educational assessment.

Having run through measurement and testing, what is left is assessment.

Assessment (i.e., collecting and summarizing information in order to develop a course of action uniquely tailored to an individual’s needs) is ironically, the one thing that has never been the primary focus of educational assessment (with apologies to my formative assessment friends).

And after 125 years, give or take, we may finally have found our calling.

Unlike measurement and tests, it is much easier to view assessment as an integral and integrated component of instruction, curriculum, and even program evaluation.

Measurement allowed us to over-emphasize what we were measuring, and testing allowed us to over-emphasize how we were testing. Assessment forces us to get our heads out of our, umm, tests, and focus on whom we are assessing, for what purpose (beyond measuring achievement of the standards), and on how we are going to use the results.

In simpler terms:

Measurement focused on the What.
Testing focused on the Who, Where, When, and How.
Assessment forces us to focus on all of that plus the Why, which we have always said is the most important.

As Laura Hamilton described in a recent interview “reimagining everything from what we measure to how we use data from those measures to inform decision-making…is a major shift in the field.”  Major shifts are never easy.

A major shift in the field gives rise to new areas in which expertise is necessary and thus creates a demand for new experts. As suggested above, we have measurement experts and we have testing experts. We don’t really have any assessment experts yet because we’ve never really done assessment. Inevitably, there will be salespeople, charlatans, and other self-proclaimed assessment experts, some with fancy degrees, peddling their magic assessment elixirs. But they are nothing new. We must remain vigilant, and we must work on building expertise.

Also, to be fair, this conception of assessment as a focus of state testing is not totally new. There was a brief period of time prior to NCLB when state assessment programs routinely included one feature which Hamilton cites as focused on assessing not just outcomes, but the learning opportunities that contribute to those outcomes: “a survey component, asking questions about which topics students received exposure to in their classes, or what kind of professional development their teachers could access.” It would behoove us to understand, and not simply make assumptions about, why those opportunity indicators faded from the testing scene.

Some might find it advantageous to view the recent focus on building better accountability systems, as a bridge between the Testing Era and the Assessment Era. Others may prefer to view the Assessment Era as emerging from advances in formative assessment.  That’s fine. Whatever works for you.

What’s important is to recognize that the Assessment Era is here, and it might be our last and best chance to do something right with educational assessment. Are you ready for it?…

Testing is dead. Long Live Educational Assessment.

Image by Gerd Altmann from Pixabay