As I took in the post-decision analysis in the wake of the affirmative action ruling something jumped out at me. Among those lamenting the Supreme Court’s decision, two very different arguments were being made in support of the importance of colleges and universities being able to use race as a factor in their admissions process.
On the one hand, there were the traditional reparative arguments about affirmative action as an equity issue, arguments which focused very much on the individual students of color who would now be denied the opportunity to attend these selective institutions of higher education.
On the other hand, the argument being made on behalf of those institutions focused very little on equity and individual students of color. Rather, that argument was all about the importance of a diverse student body to all students, the institution, and society as a whole. Not surprising, of course, because this is the primary line of reasoning that has been offered since affirmative action in admissions was first challenged in the courts decades ago.
Let me make clear up front that I am not suggesting in any way, shape, or form that the two arguments are necessarily contradictory, or mutually exclusive, or that one is less important or valid than the other.
I did note, however, that the two arguments reflect the expectation that this single policy, race-conscious admissions (for lack of a better term), will accomplish two distinct purposes – one purpose focused on the individual student and one purpose focused on a group.
And as I listened to the experts and advocates making these arguments talk past each other on various panels, it occurred to me that there was a parallel between the admissions case and the dual expectations that we have placed on state assessment to provide information to support the instruction of individual students and to inform educational policy.
It’s a neat trick if you can create a single policy, or test, to accomplish multiple, distinct purposes, but it’s not very easy to do, and doing so often involves tradeoffs. As I discussed in a recent post, with regard to state testing that tradeoff might mean that your dual-purpose test is accomplishing neither purpose (supporting individual students or informing policy) as well as a test dedicated solely to one purpose or the other would have.
As a field, we opened the dual-purpose door just a crack in the early 1990s with the hope that reporting individual student results on state tests would improve student motivation to take the tests seriously. Then we watched as the federal government pushed the door wide open with IASA in 1994 and tore it off of its hinges with NCLB in 2001.
Fine. We could live with providing individual student scale scores and achievement levels. In fact, doing so even allowed us to flex our IRT muscles, as long as you were not all that big on transparency and had undying faith in underlying scales.
Reporting individual scores inevitably led to an increased demand for accommodations, which sharpened the focus even more on individuals and chipped away a bit at standardization, one of our pillars for reporting group scores. But accommodations allowed us to include more students in the assessment program, which made the group scores we were reporting more reflective of the performance of the actual group. A net positive.
There were tradeoffs, but the focus was still on the school and group scores – our primary purpose. Maybe we could pull off this dual-purpose thing after all.
Actionable
But then the word actionable started to show up in conversations, proposals, and laws. They wanted “actionable information” from state tests.
We’ll ignore for the moment that the first definition of actionable is “subject to or affording ground for an action or suit at law.” Hell, if that’s what they wanted, our tests, test scores, and test uses could provide that in spades. High school graduation tests. The SAT. Teacher evaluation policies. In fact, in recent years some of the leading experts and organizations in our field have led the way in actions pointing out the inequity in test-based admissions policies at the secondary school level, directly or indirectly making the case for affirmative action, or race-conscious admissions policies.
No, they meant actionable in the sense of “information that could be acted on” and were quick to point out that a test given at the end of the year, with results reported over the summer, could not provide actionable information to inform the already completed instruction of individual students.
My educated guess is that vacuous truth about end-of-year summative state tests would have gained little traction if not for the unstated argument underlying it:
Now that you have broken our implicit social contract by developing state tests that take an inordinate amount of time and you are attempting to ham-handedly use the results of those tests to evaluate our teaching, we no longer consent to blindly administer tests that do not provide actionable information that we can use to improve instruction and student learning (or at least produce better test scores).
For the last decade or so we have been struggling to figure out ways to squeeze actionable information from tests not designed to provide that kind of information. We have tried to work around the issue by describing the different types of actionable information that state tests are designed to provide. But much like the lingering effects of NCLB, once the “actionable information” door has been blown from its hinges and the door frame reduced to a pile of splinters it not easy to put the genie back in the bottle. (There’s nothing like inserting a mixed metaphor in the latter section of a post to keep you from drifting off.)
Affirming
Then in the aftermath of the dual cataclysmic events of 2020, came the call for assessment, and education as a whole, to be more affirming.
We don’t really know quite yet what student-affirming assessment looks like.
We have been advised to take more care in the labels that we assign to achievement levels. We are encouraged to rethink what we are asking students to do and to report those things that they can do, the knowledge, skills, and competencies they possess, rather than emphasizing what they cannot do. Some leaders in our field have gone so far as to argue that it is unethical to test students on material that they have not been taught, taught well, or been provided an adequate opportunity to learn.
I’m not sure that any of those things hit the mark. Nevertheless, there are grains of truth, of course, in each of those three calls for affirming or affirmative assessment – perhaps enough grains to bake a robust loaf of home-made, whole-grain bread in the first diminishing exponentially to only the starchy endosperm found in a single slice of toasted white bread in the third, but grains of truth, nonetheless.
As always, however, what we ask of assessment, and the best that we can hope for, is that it work together with and follow the lead of standards, curriculum and instruction. If there is one thing that we have learned about education reform it’s that assessment alone cannot change education – at least not for the better.
Assessment
So, where do we go from here? How do we proceed?
There is a need for assessment that provides actionable information to inform the instruction of individual students. I’ll even go way out on a limb and say that there is a desperate need for such assessment to be an integral part of, and an integrated component of, instruction. You all know what that’s called.
There are compelling reasons for providing teacher and students, in particular, with more student-affirming experiences and information on a regular basis, both from assessment and throughout their educational experience, in general.
But it is also important to provide information that helps them understand what they need to do better and what they need to do next in order to reach their goals (emphasis intentional).
And there is also a need for state testing to do what it was intended and designed to do, and what it does well.
Which brings me back to the affirmative action discussion referenced at the top of this post.
We can continue to talk past each other, smiling as though we agree with each other, pretending that we believe that it is possible to develop a single test that can make everyone happy.
We can continue to devote so much of our time, effort, and other resources into trying to build a state test that is all things to all people.
Or we can be more honest in discussing what state tests and test scores can and cannot do and commit to building better assessment programs with components designed to meet specific needs.
The latter approach may be practically and politically more difficult, but it’s really not all that challenging from a technical perspective.
Much less challenging than the problems that we are trying to solve with affirmative action.
Image by Gerd Altmann from Pixabay
Embrace the Absurd 