Sometime late last week I recalled a pearl of wisdom Rich Hill shared at lunch one day after returning from a meeting. To paraphrase:
On any given day, managing a state assessment program involves dealing with immediate, short-term, and long-term issues, challenges, or outright problems. And you cannot have the same person or team of people working on more than one of those three at the same time.
I believe that he attributed the statement to Tom Fisher, but it might have been Bill Schafer. [Update: Scott Marion informs me it was Mark Moody from Maryland.]
In any event, the mindset (and perhaps skill set) needed to focus on and solve one type of issue does not translate to the other two. That type of multi-tasking is rarely effective and is often a recipe for failure, at best. (see IADA)
That insight shaped how I viewed my work and shaped my interactions with clients and TACs for years afterward. And I think that it remains as true and relevant today as state assessment directors and staff still struggle to rebound from the pandemic (immediate), integrate tools like automated scoring while reading the tea leaves smeared between the lines of the latest Dear Colleague letter (short-term) and plan for a future that will involve artificial intelligence, personalization, and a new set of joint Standards (long-term).
As I sat in a TAC meeting last week, I saw state folks focused on the quality of their solution to last summer’s minor hiccup and their proposed fix for next year. We discussed issues that they would have to deal with in 2026 or perhaps 2027 but no one was really anticipating those that might pop up in 2028 or 2029. I don’t fault them for that, not at all.
In all my experience with TACs over the years, it was rare when we were able to anticipate and address a problem more than one meeting in advance. Of course, the one rare time we did have multiple meetings to discuss an issue prior to implementation, the NECAP TAC was unable to arrive at an elegant solution – how do you include a stand-alone science inquiry task in equating? Or any performance event for that matter.
The current state assessment staff knows that they are in a forest. They may even have a good sense of the scope of the forest and where they are in it, but they are focused on the trees surrounding them.
It all felt right to me.
Then as an issue resurfaced that the TAC and state had resolved a few years back. It happens. But why had the state reverted to their old practice?
I remembered that the state’s entire technical staff and contractor had turned over in the interim. In some ways, the current team is like that group of clueless teens in far too many horror films who have been blindfolded, driven into a forest, and left on their own with minimal provisions and the clothes on their backs. They have limited information about their current location, less about how they got there, and even less history to help them predict what dangers might lie ahead. That frightening scenario raises focusing on the immediate to a whole different level.
Up In The Air
Decompressing from the TAC meeting, I listened to a discussion of air traffic control on an episode of Freakonomics Radio on NPR. In it, a controller described three layers of air traffic control:
- The Tower: responsible for activities occurring in the immediate vicinity (i.e., 7-8 miles of the airport (e.g., takeoffs, landings)
- Approach Control: responsible for planes in between takeoff/landing and approximately 20,000 ft. About 80-100 miles surrounding and enveloping an airport
- Air Control Center: responsible for planes during the en route portion of their flights, from 20,000 ft. to their cruising altitudes and the rest of their trip.
“Three different types of controllers, working in three different centers.”
Different roles. Different people. Different skill sets.
Stringing Pearls of Wisdom
If it works for challenges and air traffic, why not assessment?
It occurred to me that we can swap out immediate, short-term, and long-term issues or the tower, approach control, and air control center for formative, interim, and summative assessment and apply it to the search for the elusive balanced assessment system.
I suggested in a recent post, Seeking Balance, that we cannot find (and should stop searching for) balanced assessment systems because “system implies an interrelationship and sense of interconnectedness that simply does not and should not exist.” In this analogy formative, interim, and summative assessment are distinct tools that serve distinct purposes. It makes little sense to imagine one assessment serving all three purposes (at least not well) or even to imagine the same people having expertise in all three areas.
But there must be cohesion, alignment, something that connects the individual components. The string that turns the individual pearls into a necklace.
In a subsequent blog post, my colleague and friend Juan D’Brot reimagined balanced assessment systems as a relay race with Curriculum, Instruction, and Assessment serving as three legs of the relay, each with a distinct role, and a win produced only “when the baton moves cleanly between them.” One “person” cannot run two legs of the same race, but all are connected by the single baton that unifies their individual efforts.
The Future of State Assessment
My vision of the future of state assessment is clear. It should be because it’s been stirring in the back of my mind for nearly forty years, since that spring morning when I ran into Rich Hill in the parking lot at Advanced Systems and asked what I had missed while away for a week at AERA/NCME. By the time we reached his office, Rich had outlined a vision of state-supported, school-based assessment embedded in the curriculum and occurring naturally as part of instruction.
We fell short of realizing that vision in the 1990s for a number of reasons which I will discuss in future posts. Yes, we were trying to fly the plane as we designed and built it. Yes, we were well out over our skis. But we also failed to recognize the scope and density of the forest we were entering as we attempted to revolutionize state-supported assessment. We didn’t realize the distinction between immediate, short-term, and long-term as we struggled to make it from one tree to the next as the forest rose up and closed in on us.
Then we, along with the rest of the field, were consumed by the immediate task of implementing state testing at grades 3 through 8 and high school and the challenge of making adequate yearly progress toward the relatively short-term goal of 100% of students proficient in reading and mathematics.
Large-scale testing and state assessment crept along in its petty pace from day to day.
Because we focused on the immediate task of producing an operational test by 2015 and not the long-term goal of transitioning curriculum, instruction, and assessment to college-ready standards and student assessment (as opposed to testing), the promise of life beyond the bubble test was an illusion, a walking shadow, a tale told by an idiot, full of sound and fury, signifying nothing, and then heard no more.
But we have the capacity to learn from our missteps, escape our fate, and not be condemned to repeat our past. We have tomorrow.
Tomorrow, and tomorrow, and tomorrow.
Image by Nghĩa Đặng from Pixabay
Embrace the Absurd 