My biggest takeaway from the NCME conference in Philadelphia was that this is an organization and a field in search of an identify. And given the location, I couldn’t help but make the connection to the classic 1940 film, The Philadelphia Story. Like the main character, a young socialite played by Katherine Hepburn, I saw an organization trying to find itself – trying to resolve its past decisions, deal with the present situation it finds itself in, and make the right choices for its future. A quick plot summary, adapted from the film’s Wikipedia entry:
Tracy Lord (no relation to Frederick) is the elder daughter of a wealthy Philadelphia Main Line socialite family. She was married to C.K. Dexter Haven, a yacht designer and member of her social set, but divorced him two years prior because, according to her father, he does not meet the standards that she sets for all her friends and family. She is soon to marry the more socially acceptable George Kittredge. As the wedding nears, she finds herself torn among George, Dexter, and Mike – a reporter sent to cover the wedding and perhaps disrupt her family.
A plot summary of our Philadelphia Story is pretty comparable:
A couple of years ago, after our field was called out by AERA and then thrust into the tumult of 2020, NCME dramatically and publicly divorced itself from its immediate and distant past for failing to live up to the Standards it set along with friends and family from AERA and APA. NCME seems poised to marry a more socially and political acceptable view of measurement and testing, one that will be reflected in a revised set of Standards. As the wedding nears, however, NCME finds itself torn among the good parts of its past, its progressive future, and a third group – those pesky classroom assessment folks who clearly must be an integral part of any student-centered view of education and educational measurement.
What a movie that would make! And I didn’t even mention any of the secondary characters, the twists and turns, the subplots and subscores, the embedded standard setters (that just sounds conspiratorial, doesn’t it), the bombastic bureaucrats, AI, and the evil testing serpent.
If you are not familiar with The Philadelphia Story, you may know the parallel form of the film, the 1956 musical remake, High Society, starring Grace Kelly as the socialite. The Personally, I prefer a musical, particularly one that includes Louis Armstrong and his band and the Newport Jazz Festival, not to mention that True Love, written by Cole Porter and sung by Bing Crosby and Grace Kelly in the film was song my wife and I chose for the first dance at our wedding. The 1993 version of True Love by Elton John and Kiki Dee is good, too, although nowhere near as successful for the duo as Don’t Go Breaking My Heart. But I digress.
Obviously, it is much easier for me to make the connection between NCME and Philadelphia than it ever will be to equate what we do with High Society.
Measurement in Education – A Complex Construct
As complex as life was for Tracy Lord, it is exponentially more complex for NCME and those of us trying to make sense of educational measurement or the use of measurement in education. Who are we, what do we do, and why are we here?
Are we psychometricians? Psychometrics self-identifies as amorphous umbrella which appears to cover some combination of measurement, data analysis, and statistics, as I noted in the 2015 NCME presentation that kicked off this blog, Psychometrician Do No Harm.
Are we about tests, testing, and assessment? NCME past president Derek Briggs (21-22) in his book, Historical and Conceptual Foundations, made the assertion that “Testing and measurement are two distinct activities.” [emphasis included]. Greg Cizek (12-13), in his book, Validity, further distinguishes between testing and assessment. And as has been reflected in the change in title and focus of the Standards over the years, there is a distinction between testing and tests.
Even within the distinct categories of measurement, assessment, and testing, there is variation among NCME presidents. Derek Briggs is not the same type of measurement expert as Mark Wilson (16-17). Ye Tong (20-21), a testing practitioner, is different from Randy Bennett (17-18), an assessment researcher, and Richard Patz (15-16), current NCME executive director, perhaps falls somewhere between the two of them on this nominal name scale that I am creating.
And with all due respect, I would probably place newly-minted NCME president Andrew Ho (24-25) in a different class altogether – one deeply grounded in statistics and data analysis with a keen intuition for the application of those skills to finding pragmatic solutions to technical and policy issues in large-scale testing and accountability. Not that there’s anything wrong with that. It’s a good thing.
Who are we, what do we do, and why are we here?
The recent NCME task force on Foundational Competencies in Educational Measurement identified a wide ranging set of knowledge, skills, and abilities (i.e., foundational competencies) “that we would expect any newly arriving member to the educational measurement profession to eventually know and be able to do.” But even that comprehensive framework was challenged last week along “four substantive dimensions” related to the rise of AI, the aforementioned issues related to classroom assessment, the need for a greater emphasis on community engagement, and with respect to issues related to fairness and context.
Who are we, what do we do, and why are we here?
And then there is the fact that we continue to find ourselves in the position of being the little sibling to or red-headed stepchild of AERA. Strangely, that second class position felt even more acute this year, when we were housed in the same building as AERA, than it did trying to trudge through the streets of San Francisco between sessions (uphill both ways) or to navigate the multi-level grid of downtown Chicago in a blizzard. Walking blocks through those cavernous convention center hallways, wearing our different-colored lanyard, seeing those “Jim Crow”-esque signs on the doors (AERA Must Enter at Market St.) in some ways left a more profound sense of isolation.
Who are we, what do we do, and why are we here?
Accepting Ourselves for Who We Are
Despite my repeated questioning above, each of us knows who we are, what we do, and why we are here. We know that we are coming at the use of educational measurement from different places and different perspectives, and we know that’s a good thing. We know that we have shortcomings and that our field is flawed – we have more ways of classifying error than Baskin Robbins has flavors of ice cream.
We know that education is constantly changing, measurement is constantly changing, measurement in education is constantly changing, and that we must change with it. This conference was not same as the NCME conferences I wrote about in a post last spring. NCME 2024 was not the same as NCME 2018, the last NCME conference I attended in person and helped organize with president Randy Bennett as program co-chair with April Zenisky. It was certainly not the same NCME conferences I attended in the early 2000s. This is not my father’s NCME – or perhaps it is more his now given that he was a classroom teacher.
We also know, however, that, like Tracy Lord, we must stay true to ourselves, and we cannot simply abandon our past and its people. I cannot count the ways that the people and the past of NCME has supported me throughout my career.
Just considering one small, select, sample, over the course of my career, I have had the honor to interact professionally with two dozen NCME presidents, starting with Jack Merwin (71-72) who taught the first assessment course I took at the start of my doctoral program at the University of Minnesota and continuing through to the two whom I will join at a TAC meeting next month. Some of those interactions and intersections were brief – a single project or committee. Others were more extensive. A half dozen or so of those relationships continued for decades. Three, I regarded as mentors.
Only one of those twenty-four individuals, however, was in the same position as me; that is, directly responsible for the operation of a K-12 state assessment program. Nevertheless, they all contributed to making me, with all of my flaws, a better psychometrician, assessment specialist, data analyst, test developer, and user of measurement in education.
That’s who we are, that’s what we do, that’s why we are here.
Image by Gerd Altmann from Pixabay
Embrace the Absurd 