Why is the future so much clearer than the present or the past?
With respect to the present, at least, the answer to that question is as obvious as the nose on our face. We often find ourselves so deep in the weeds with our heads down putting out brushfires that we are incapable of looking beyond that aforementioned nose to see the forest for the trees.
The cloudiness of the past, however, is a bit more of a surprise. We’ve always believed that hindsight is 20/20, that once removed from the chaos and fog of the present we will see the big picture, what really happened and why, and understand how the series of small decisions, actions, or inactions, led us down one path or another.
Recent “history” and events, however, have taught us that the only thing clear about the past is that there is nothing simple, straightforward, or black-and-white about interpreting it. As the wonderfully Wicked wizard explained to Elphaba,
We believe all sorts of things that aren’t true
We call it “history”
A man’s called a traitor or liberator
A rich man’s a thief or philanthropist
Is one a crusader or ruthless invader?
It’s all in which label is able to persist.
We are learning that the past is one of those objects that looks different from different perspectives and in different lights.
We also like to believe that the journey toward truth and clarity is inexorable and monotonic – we will be more enlightened and see things more clearly tomorrow than today. History, however, has taught us that is not necessarily the case.
Which leaves us with the future.
A Blank Slate
The future, still unwritten, is ours to see. It is a blank slate upon which we can draw the picture that we want to see, the future that we should see – the promised land.
This is not to say, of course, that simply because our vision of the future is clear that we will make it to the promised land.
We don’t have the flying cars that we were promised by the Jetsons, the four-day work week and improved family life envisioned by Richard Nixon in the 1960s, or the free tickets to professional sporting events that would be the result of the portion of total revenue generated by ticket sales becoming so small (i.e., negligible) that teams would simply give the tickets away.
In large-scale assessment, we don’t have the authentic assessments that we saw so clearly in the 1980s, or even national consortia administering next generation state assessment programs that took us beyond the bubble test. And we certainly don’t have balanced assessment systems – or even a solid concept of what they are and why we want them.
Still, it is our job to keep looking toward the future, because that’s all that we have. We can try to better understand the past, but we can do nothing to change it. We can really only make marginal changes to the present for the reasons listed above.
So, what will we write on our blank slate about the future of assessment? When we get to the mountaintop, what will that promised land of assessment laid out before us look like?
Before we start writing, however, there are a couple of things that we must do.
The first thing that we have to do is to recognize the difference between the present and the future.
One of the critical lessons from the past is that a big reason previous present assessment reform efforts and initiatives like the Race to the Top Assessment Program and the Innovative Assessment Demonstration Authority failed (or at least didn’t fully succeed) is that they conflated the present and the future.
The second thing that we have to do is to come down from the mountaintop.
Coming Down From the Mountaintop
When I have highlighted the importance of seeing the big picture and warned of the dangers of being caught in the weeds, it may seem contradictory to suggest that we need to come down from the mountaintop when describing the future. What I mean by coming down from the mountaintop, however, is that we have to stop relying exclusively on 30,000-foot view words and ideas that are general, vague, or generally vague, to describe our vision of the future of assessment and assessment systems.
Let’s start with the holy trinity of measurement and assessment. It’s a given that future assessment systems must be valid, reliable, and fair. I’m less confident that future assessment developers and users will make better use of the three concepts than we do. As a starting point, you need to convince me you understand each of those terms as they apply to particular situations, can communicate their meaning to a variety of audiences, and explain to them how you will determine whether an assessment or assessment system is sufficiently, valid, reliable, and fair. Some evidence that you intend to do a better job with validation and evaluating the use of assessment results than we have done would also go a long way toward giving me confidence.
The tricky part is that as you describe your criteria for validity, reliability, and fairness, you will be tempted to use terms and phrases that sound nice but are too amorphous to guide yourself or others in the future. In the latter portion of the 20th century, examples of such terms and phrases included the following: authentic, higher-level and high-quality, comprehensive and criterion-referenced, and of course, “21st century skills.” More recently, we added terms such as actionable, accessible, and balanced to the mix. Current favorites include terms such as culturally-relevant, antiracist, student-centered, inclusive, justice-oriented, engaging, and competency-based.
Like validity, reliability, and fairness, each of these phrases might describe a desirable, or even critical component or characteristic of an assessment system. In some cases, the people using the terms may even know exactly what they mean by them. Assessment history has proven that’s not enough.
History has taught us that there will always be people using each term who are intentionally vague or who are appropriating the phrase for their own purposes.
In describing our desired future, we need to be clear, direct, and avoid use of terms that can be easily misunderstood or misappropriated. . What do I mean by that?
Fortuitously, while cleaning out an old desk last week, stuck between hanging folders at the bottom of a drawer, I found a perfect example to illustrate my point in the form of A Letter From The Future featured in the Fall 1997 edition of The CRESST Line (A hard copy newsletter of The National Center for Research on Evaluation, Standards, and Student Testing that was delivered to my house via U.S. mail three times per year. How quaint.)
A Letter From The Future
In 1997, Robert Glaser and Gail Baxter penned a letter from the future describing to folks preparing for the annual CRESST conference some of the key features and characteristics of assessment in the 21st century. A few excerpts to share their vision of the future of assessment:
On Standards, assessment curriculum, and instruction
I was struck by the title of your next-to-last session, “Making Standards and Assessment Work Through Curriculum and Instruction.” This is an oxymoron. It appears ludicrous to us if curriculum and instruction do not obviously and implicitly imply “standards” and “assessment.” These two buzzwords of the 20th century are not just part of the job of a reputable profession in the 21st century. What takes on importance is how expert the practitioners are in carrying out their task, because in our schools, standards and assessment are routinely introduced as integral parts of a curriculum and program of instruction.
On the importance of validating standards and assessments
There is significant social and legal responsibility now attached to the development of standards and related assessments. Standards are legal entities that are required by law to show means for the clear presentation of anticipated goals, and for the provision of relevant learning opportunities. The conditions and effectiveness of these opportunities are documented through educational trials, like your current FDA clinical trials.
On communicating the meaning of achievement levels, proficiency, competency
Standards are described quite directly through media displays of criterion performance in domain knowledge and skill. We have abandoned the use of relative cutoff scores to describe performance as basic and advanced. A language of competence in subject matter domains is used to communicate in various ways to those who understand a domain and to those who may not.
On test development
I must further say that in looking at your procedures for test development, we think it odd that the psychometric technology of what you call “test items” is designed to occur after the items are constructed. As we understand it, the analysis of item difficulty, discrimination index, and scaling take place after the assessment situations are written. In significant contrast to this, we give sustained prior attention to the characteristics of assessment situations and the theory behind them before and during design. This allows us to anticipate the kinds of performances we are likely to observe from students with differing levels of subject matter competences, and also to provide information to them that is most useful for learning.
Characteristics common across assessments
[Across domains,] there is a similarity of assessment that reflects the general characteristics of developing competence and achievement. Let me give you three examples; they are managing information, integrating basic skills with advanced performance, and changing the agency for learning. [Followed by an explanation of each of the three.]
On the quality of data
I must point out another significant feature in the design of our assessment situations. This is that major concentration is devoted not just to content and process frameworks for a domain, but primarily to providing high-quality data and information on student performance. This is a reaction on our part to the practices of your century where the tactics and complexities of data analysis were of a much higher quality than the nature of the data collected. Your power of analysis was much greater than the quality of data and information on student performance, and how it was reported.
On the purpose of assessment
Assessment concentrates on helping the learner and the teacher recognize different stages of knowing.
Their Future, Our Present, Our Children’s Future
I could live with and in that future.
I encourage you to read the entire letter and to think not only about how we might envision and describe assessment in our future, but also to reflect on where our present stands relative to Glaser and Baxter’s vision of assessment in the 21stcentury.
The good news is that the one thing they were not clear and direct about is exactly when in the 21st century they were living. With three-fourths of the century remaining, it may not be too late for us to move 21st century assessment closer to their vision.
Embrace the Absurd 