Having observed last week’s NCME conference from afar via LinkedIn, my biggest takeaway was not about AI, IRT, or any other combination or collection of letters. Rather, it was about the combination and collection of people that make up our hybrid field of educational measurement, assessment, and testing. The diverse group of people filling the even more diverse set of roles necessary for a field to survive and advance.
In a post earlier this month, I singled out leaders and the nominal face of assessment, but as critical as they are, those folks are the proverbial tip of the iceberg. Below the surface, the field is teeming with big fish, plankton (carried along by the tides and currents, not strong enough to swim against these forces), colorful coral, barnacles (not quite parasites, but able to create a drag on the system), and of course, schools. We cannot forget the schools.
Let’s move on from the aquatic analogy and return to the surface before I run out of air in my tanks.
On display at NCME last week were innovators and disruptors laser-focused on the future of the field. As they say, all disruptors are innovators, but not all innovators are, or want to be, disruptors. The innovators are using AI to help us do what we currently do better, faster, and cheaper – sometimes in conjunction with existing tools and methodologies, sometimes introducing new approaches. The disruptors, of course, might soon render much of the innovators’ work obsolete, but such is the circle of life.
There were also a fair share of managers and facilitators of various stripes doing what managers and facilitators do.
There were mentors, both from industry and academia, helping their mentees find their footing in the field and/or navigate their career pathway.
And, of course, there were the graduate students – bright, shiny, idealistic – ready to redeem freshly minted degrees for a seat at the table. The vast majority of them will assume the crucial role of worker bees, dedicating their careers to performing the functions necessary to keep our hive functioning smoothly.
Across presentations, breaks, receptions, formal and informal gatherings, all of those people came together to create a pretty powerful, positive, experience. Sunday and Monday, my LinkedIn feed was filled with posts describing the feeling of being physically tired but mentally refreshed, renewed, and reinvigorated. As was always the case for me, people are returning home from the conference filled with new ideas and a new sense of resolve.
Today, however, I want to focus on the people who create a slightly different feeling and experience. They may come from any one of the roles already described above and they come in a variety of flavors. They are the people who challenge us, make us feel a bit uncomfortable. It is important to note that I am specifically referring to colleagues within the field and not external critics whom you might have encounter upon wandering inadvertently into an AERA-sponsored session.
Rising To Challenge
At the top of the hierarchy of “challengers” are measurement and assessment elite, those special few whom I will call the conscience of the community. They keep us mindful of the big picture, the moral high ground, our higher calling. Merely through their presence and force of will they keep us on the straight and narrow, making sure that we’re doing what we purport to do to support and improve student learning. The people in this category are held in the highest regard.
Next comes the group that I label as the constructive criticizers. They will inevitably question our methodological choices, sometimes our motives, but never our parentage. They may hold our feet to the fire, but at the end of the day we’ll join them for a nice dinner where a lively debate is likely to continue well into the night over a beverage of choice.
Then there is the final group whom I’ll call the burr in the saddle gang. Alternatively, I could have gone with thorn in the side or my favorite term of endearment for those in this group, pain in the ass.
The Burr In The Saddle Gang
You know who I’m talking about. The first thing you do when you walk into a presentation room is to check whether they are there. And they’re always there, front and center.
When it comes time for questions, you usually have two choices:
- Just call on them first and get it over with and out of the way.
- Pretend that you don’t see them with their hand raised.
The latter has a certain appeal, but it can be stressful and is not really fair to them or to your audience.
Because although the questions (i.e., comments) from the burr in the saddle gang may not be directly related to the details of your presentation, they are usually in the ballpark. More often than not, their concern is general enough to “fit” with most presentations and has to do with some real or generally perceived shortcoming related to one of the big three of validity, reliability, or fairness, with alignment and utility rounding out the top five. And as we all know it’s important to address real and perceived shortcomings.
Some are one trick ponies striking the same chord over and over again, while others take a more scattershot approach. They are all passionate, and I may be getting soft in my old age, but I believe that they are all sincere in their concerns. The alternative is that they are just trying to carve out a spot for themselves within the field and this is a really bad way to go about that. So, I’ll take them at their word, all their many words.
Therefore, as we celebrate the emergence of spring and all that comes with it, some beautiful, some more of a nuisance, let’s take at least a brief moment to give these people their due. They’re part of the family.
Embrace the Absurd 