Look What You Made Me Do

A 2018 Blog Year in Review

Charlie DePascale

We have reached the end of 2018 and another year of posts on Embrace the Absurd. When I look back at the ten essays posted this year, I think that the phrase that best sums up this year of posts is look what you made me do – and not simply for the obligatory Taylor Swift reference.

A primary theme that ran across my posts this year is that we, as a field, may be just a tiny bit out of control; reactive rather than proactive; allowing ourselves to be defined by others; or perhaps overwhelmed by the moment.

I began 2018 with the post, Implausible Values, discussing the stress and strain being put on the field and our equating infrastructure by demands for shorter tests, alternate tests and adaptive forms, less standardization and more flexibility, more accuracy and precision, and immediate results.  I also wrote of the paradox of taking at least six months to produce results for a few NAEP tests and no more than six days to complete equating for a dozen state assessments.

NAEP returned as a topic in April with, If I Did It, a satirical treatment of the efforts to control mode effect and preserve the trend line in the reporting of the 2017 NAEP Reading and Mathematics state results; an effort which could serve as the poster child for our 2018 theme. I have little doubt that 2017 NAEP results will serve as a cautionary tale in educational policy and measurement courses for generations to come.

Across the year, a trio of posts addressed the broad issues of time, validity, and the essence of educational measurement.  In It’s About Time we address not only the lack of time mentioned above, but also the extent to which our measurements and interpretations are dependent upon and bound by time, and the growing need to incorporate time into our measurement models.  Bring Back Valid Tests addresses our ongoing struggle to develop an operational definition of validity.

In my 2016 NERA presidential address, Living in a Post-Validity World: Cleaning Up Our Messick, I argue that in the nearly 30 years since Messick’s 1989 chapter, we have wandered the desert searching for the Promised Land of a unified theory of validity.  As with many of the constructs that we attempt to measure, we still lack a clear understanding of validity; yet one of our guiding principles is that you have to understand and clearly define something before you can measure it.

This leads to my call for Rebranding Educational Measurement with the argument that the field will be better served both by not only acknowledging, but also embracing the uncertainty in what we do; and included this reminder from the 1951 first edition of Educational Measurement, “[t]he primary concern of measurement, however, should be for an understanding of the entire field of knowledge rather than with statistical or mathematical manipulations upon observations.”

 A Year of Professional and Personal Journeys

2018 was also a year of personal and professional journeys. In Ten Years of Taylor I describe literal and figurative journeys with my daughter across ten years of Taylor Swift concerts from 2008 – 2018.  In My Miss Brooks I describe the 4^th and 5^th grade class that set me on this assessment/measurement journey nearly fifty years ago; and with the benefit of hindsight reflect on the high-stakes test that awaited at the end of those two years that may not have been as high-stakes as we thought at the time.

And throughout 2018, there were other journeys not noted directly in the blog, including the Red Sox eight-month, 119-win journey from Opening Day to their fourth World Series championship since 2004.  And now we have solid empirical evidence (n=2) that the Red Sox own the first 18 years of the century.

After 25 years of organizing regional conferences in small venues in places like Rocky Hill (CT), Springfield (MA), Buffalo, and Pittsburgh, in April 2018 I finally made it to the big time – a national conference on Broadway. Serving as 2018 NCME co-chair with long-time friend and colleague, April Zenisky, we were able to bring together past, present, and future leaders of our field to reflect on the past, present, and future of the field.

Outside of the conference, New York City brought feelings of awe when standing in the middle of Times Square at night or earlier that day sitting in front of a Renoir painting at the Metropolitan Museum of Art. That feeling was matched, if not surpassed, a month later driving through the mountains of Northern Utah and Southern Idaho on a Sunday afternoon with Marren Morris’ My Church on repeat on iTunes. And then on my first trip outside of North America, there was the incomparable and simply indescribable feeling standing in the middle of Anne Frank’s room in Amsterdam.

A New Year and New Beginnings

Today we look forward to a new year with new journeys, and new beginnings.  For the second time in my career it feels like we are on the cusp of a new era in K-12 assessment and educational measurement. Technology, personalized learning, big data, more complex and higher-order content standards, and a renewed interest in assessment in the classroom have created a perfect storm of challenges and opportunities for assessment and educational measurement. NCME has begun work on the fifth edition of Educational Measurement, which brings with it the opportunity to take the time needed to reflect on where the field is now, how it got here, and the directions it might, could, and dare I suggest, should go in the future.

So, as we begin 2019, let’s renew our commitment to keep the faith, fight the good fight, and as always, embrace the absurd.