Pondering the role that educational assessment and measurement (and their practitioners) can play in improving instruction and student learning, I have concluded that we need to adopt the mindset and repeat the mantra
Mind over Matter over Measurement.
In the vernacular of social media: Mind > Matter > Measurement.
For those so inclined: (Mind Ç Matter) > Measurement.
Expressed in simple terms, we get two principles that should sound quite familiar:
- Decisions about why we are teaching students should precede decisions about what we are teaching to students.
- Decisions about what we are teaching students should precede decisions about how to assess those students on what they have been taught.
Of course, I am using the word “precede” as a thinly-veiled euphemism for “are more important than.”
I will stop short of making the claim that “bad” measurement of the right thing is better than good measurement of the wrong thing, but not too far short. I am neither dismissing nor being dismissive of technical concerns.
I am not saying that validity is more important than reliability because it’s really difficult to claim that you have validity without reliability. It is quite easy and common, however, to design assessments with reliability but little to no validity.
Similarly, fairness with regard to accessibility and fairness in the use of assessment results are of paramount in assessment design, development, and use.
Measuring the wrong thing, however, simply because we can measure it well or measure it efficiently is bad in its own right. Decisions about why and what we are teaching should never be unduly influenced by concerns about whether students can be assessed efficiently and/or what can be measured effectively.
Mind: The Really Big Picture and Really Big Question
Who do we want students, the children who are becoming young adults under our watch, to be when they leave school?
I struggled with the phrasing of the question above. My first inclination was to write “What do we want students to know and be able to do?” but that sentence is much too focused on content and specific skills. It’s too focused on the here and now.
Whenever I ponder this question, I return to the description of an educated child that was offered in two of the landmark court cases that kickstarted the modern era of education reform:
[a]n educated child must possess ‘at least the seven following capabilities: (i) sufficient oral and written communication skills to enable students to function in a complex and rapidly changing civilization; (ii) sufficient knowledge of economic, social, and political systems to enable students to make informed choices; (iii) sufficient understanding of governmental processes to enable the student to understand the issues that affect his or her community, state, and nation; (iv) sufficient self-knowledge and knowledge of his or her mental and physical wellness; (v) sufficient grounding in the arts to enable each student to appreciate his or her cultural and historical heritage; (vi) sufficient training or preparation for advanced training in either academic or vocational fields so as to enable each child to choose and pursue life work intelligently; and (vii) sufficient level of academic or vocational skills to enable public school students to compete favorably with their counterparts in surrounding states, in academics or in the job market.’
McDuffy v. Sec’y of the Executive Office of Educ., 415 Mass. at 618-19, 615 N.E.2d at 554 (quoting Rose v. Council for Better Educ., Inc., 790 S.W.2d 186, 212 (Ky. 1989)).
You can certainly find specific content knowledge and skills embedded across those seven capabilities (a really nice word), but they contain so much more. We want students to be happy, healthy, productive, and informed citizens, able to make appropriate choices for themselves and society.
To get there, as my former NEERO colleague, Gary Ackerman, wrote in a recent post titled, Passing, Practicing, Progressing, we need to be thinking beyond the immediate knowledge and skills needed to passcourses, and even beyond the short term practices they will need to employ to attain traditional indicators of postsecondary success. We need to consider their ability to be able to continue to progress (i.e., learn), acting as their own educational agents.
The terms student-centered education and student agency come to mind, but I think that I’ll stick with the more well-worn and battle-tested term, lifelong learners.
That’s a lot to ask of four courses in English, three in mathematics, a couple of lab sciences, and a survey course in US History. [sarcasm, cynicism, or facetiousness, take your pick]
The key to producing lifelong learners may lie somewhere within that collection of courses, but you will have to craft a compelling case to convince me of that.
We have to do a much better job of demanding a good answer when we ask ourselves why we are asking students to take particular sequences of courses – perhaps asking why multiple times until we hear an answer, or series of answers, related to and leading toward the ultimate goal – producing lifelong learners.
Matter: Whatever you do, do it well.
Knowledge is awareness, and to it there are many paths, not all of them paved with logic. – Louis L’Amour
I believe that there are many paths to the same place, and that is a belief that there is a higher power, a belief that we are connected as a people. – Barak Obama
If we focus on the big picture and ask the “why” question enough times we will inevitably conclude that there is no single path that all students should follow from PK through twelfth grade. PK-12 education is already moving in that direction. States are planning and implementing personalized pathways for secondary school students – some of which may find students moving on to postsecondary experiences prior to the end of twelfth grade. Student choice and student agency (however, ill-defined or misunderstood those terms might be at this time) are trending in discussions of elementary and middle school curriculum across the country.
But don’t be lulled into thinking that matter doesn’t matter. Not all choices and paths are created equal.
Some paths will prove to be dead ends – not that there’s necessarily anything wrong with that. There are some dead ends that are worthwhile destinations in and of themselves. Others where the destination might be disappointing, but the scenery viewed and lessons learned along the way make the little side trip worthwhile. There are still other paths not leading anywhere that you have to follow for a while just to get them out of your system, and those detours, too, can be valuable.
Other personalized pathways, however, will prove to be dangerous – slippery slopes that turn into tracks that lead students off of a cliff. As I write this, I’m picturing a chilling combination of the steep grade/ice scene from The Polar Express, the climactic scene from Back to The Future III, and the end of the chase scene in Groundhog Day.
We must move toward personalization, choice, student agency, and pathways along with whatever other changes will enhance engagement, learning, and achievement, but we must do so carefully and do so well. Whatever you do, do it well.
There is still a common core of knowledge and skills that all students need. That core includes traditional literacy and numeracy skills, the critical 21st century skills documented nearly 50 years ago, and the new crop of 21st century skills identified more recently. That core must be efficiently, effectively, and elegantly integrated into all available pathways.
The last thing that we want to do is to set students on their pathway naked, without provisions, and generally ill-equipped to deal with the challenges they will encounter along the way.
Measurement: What gets measured gets done
If you cannot measure it, you cannot improve it. – Lord Kelvin
The two statements above roll off the tongue of anyone who has been involved in PK-12 education in the 21stcentury; that is, the NCLB era. Their messages have been applied, often passionately, sometimes accurately, to arguments for and arguments against state tests and test-based accountability.
What concerns me, however, and the reason for this post, is the extent to which they have been interpreted by the measurement and assessment community as a birthright rather than a sacred duty to support the mission of public education to the best of our abilities.
If we cannot measure it then it cannot be that important.
I’ll be the first to admit that it’s been easy to sit back for the past quarter century and do what we do best – efficiently and effectively measure relatively low-level basic skills in English language arts and mathematics. And make no mistake, we do that very, very well – validly, reliably, and fairly.
When the education world demanded a little more from us, however, we bent their needs to fit our models. Compositions and integrated reading/writing tasks – we can make those fit, at least until we have to equate tests from year to year. Reading, Speaking, Listening, and Writing on English Proficiency exams – no, you see, they’re not really separate dimensions. Trust me, I’m a doctor. We were willing to adapt, but not to change. That attitude won’t cut it any longer. To paraphrase that respected scholar Dean Wormer, fat, drunk, and unidimensional is no way to go through life, son.
PK-12 education needs our support, and we need to step up. To make a positive contribution, however, we first have to come to grips with our role in educational society. Like the police, we are here to protect and to serve. Like physicians, we must first do no harm.
Then we have to figure out how to measure all of those things that need to be measured, because if you cannot measure it, you cannot improve it; and what gets measured, gets done.
And if that strains our minds, it doesn’t matter.
Mind over matter over measurement.
Embrace the Absurd 