Know Thyself

To know thyself is the beginning of wisdom – Socrates

Identity. That’s the first word that comes to mind when I think about the state of educational measurement and assessment in 2021. On so many levels, our field, and the individuals within it, are in search of an identity.

Derek Briggs, our nominal and effective leader, noted during a recent episode of the Quantitude podcast that the educational measurement community, including NCME, does not have a shared definition of educational measurement.  He drew a contrast between NCME and the Psychometric Society website which features a selection of responses from noted psychometricians to the question, “What is Psychometrics?”

That’s nice for them, but as I noted in the post that kicked off this blog in April 2015, their responses focus quite a bit on statistics and don’t bring those of us who think of ourselves as psychometricians any closer to understanding our identity within educational measurement and assessment.

Later that same year, Gene Glass, published a blog post decrying what the field had become to explain why he no longer considered himself a measurement specialist. He raised some valid points, but the most eye-opening aspect of his post to me was that I had been in the field for 25 years at that point, was well aware of his work in meta-analysis and education policy, but it had never once occurred to me that he was a measurement specialist.

Who are we?  What do we do?

In the opening paragraph I mentioned educational measurement and assessment, but when I think about it, I am not really certain of the relationship between the two. What would a Venn diagram describing that relationship look like? Are they separate but overlapping fields? Is one a tool or a component of the other.  I’m pretty certain they are not the same thing, although most of us do exchange the terms quite freely when describing our work.

“Don’t let others define you. Don’t let the past confine you. Take charge of your life with confidence and determination and there are no limits on what you can do or be.” – Michael Josephson

There is no need for me to spend time discussing how events of the past two decades, culminating in those of the past two years, have brought the field as a whole to this critical point – whether you view it as a tipping point, an inflection point, a point of no return, or vanishing point (either definition).

Where we go from here and how we get there, however, remains an open question. Nature abhors a vacuum – almost as much as it abhors standardized testing. If we don’t define ourselves, someone else will. Given that we are almost always interested in measuring something for some purpose determined by someone else, it would be easy to fall into the trap of allowing others to define the profession.

It is a dangerous trap that has been set but falling into it is not inevitable. The decisions we make about our work are our own and are on us, particularly as a field. Automatic responses such as “but, federal law requires this” or “there is not enough money for that” or “we’ve always done it that way” have never been acceptable. 

We may be different from the rest
Who decides the test
Of what is really best?

Identity and a seat at the educational measurement table is also a concern for groups of practitioners within the field – groups defined both by the work that they do and who they are. Two prominent examples of this from the past five years are the formation of the Classroom Assessment Task Force and the organization Women in Measurement, Inc.

The NCME Classroom Assessment Task Force is the result of an initiative by 2016-17 President Mark Wilson to support and facilitate the integration of classroom assessment into the NCME consciousness and scholarship in hopes of broadening from the historical focus on large-scale, summative assessment.  Although we may need another set of Venn diagrams to work out how the classroom assessment and the “broader” educational measurement community fit together, the goal is to benefit all concerned, including the users and subjects of educational assessment.

Women in Measurement, Inc., led by Susan Lyons, Jenn Dunn, and Fiona Hinds, describes themselves as connectors and disruptors, champions for women in measurement. Founded in 2020, their mission is to advance gender and racial equity in educational measurement leadership and amplify the diverse voices of all women and provide structures of support for career advancement.

Clearly, it’s critical that all not only have the opportunity to participate in educational measurement and assessment but also that all are part of shaping who we are and what we do.

 

Sometimes I’m clueless and I’m clumsy
But I’ve got friends who love me
And they know just where I stand
It’s all a part of me
And that’s who I am

Let’s face it friends, John Oliver is hilarious and on point until he decides to take on your profession – especially when you agree with many of his main points.

Identity and a sense of self has long been a concern at the individual level for those of us working in high-stakes large-scale assessment. Working as the lone psychometrician at a state department of education, hypothetically, one might find oneself wondering what sequence of seriously bad decisions leads one from receiving standing ovations at Boston’s Symphony Hall as a supporting member in an elite group of high school musicians to being heckled and jeered at by 450 rabid educators as the psychometrician on the Massachusetts state assessment management team. [Note: Not hyperbole, foaming at the mouth was clearly visible on a number of occasions]

It’s not easy to accept that at family gatherings the three topics that can never be discussed are religion, politics, and what Charlie does.

And as a member of a profession noted for our lack of interpersonal skills, how does a major component of your leadership position become reassuring staff members on a regular basis that they are not evil, vicious people out to hurt children, or on a good day, simply incompetent and ignorant, unknowingly destroying public education. Well, when that’s what they have been told all day (every day) via angry phone calls and e-mails or what they read on Twitter when they go home at night, that has to be a major part of your job.

(It wasn’t all bad, of course. There were moments.)

Who am I?  Why am I doing this?

So, when I make big mistake
When I fall flat on my face
I know I’ll be alright

Finally, there is what is likely to be the biggest challenge of all to educational measurement and assessment. To a greater extent than ever before, we are going to have consider the identity of the people who are using our measurement instruments and information and the identify of those whose knowledge, skills, etc. are being measured or assessed. We need to be aware of how those identities may (or may not) affect what we measure, how we measure it, and the inferences we can make from those measurements.

Understand that this will require more than a detached cognitive acknowledgment of, and attempts to statistically account for, the many socio-cognitive factors that influence student performance.

This will require direct contact, regular interaction, collaboration, and even empathy with the people involved in the “education” part of educational measurement and assessment. No, not humanity!

Yes, it’s frightening.

Yes, it promises to be quite awkward.

Yes, it will take time to get it right.

Yes, we will make big mistakes and fall flat on our face along the way. (But we kind of do that now anyway, right.)

Ultimately, I believe that we’ll be alright individually and collectively if we remember why we chose education and live by these words written by (or at least attributed to) every high school student’s favorite dead, white male:

This above all: to thine own self be true,
And it must follow, as the night the day,
Thou canst not then be false to any man.

Image by Lucas Wendt from Pixabay