In August 2019, Taylor Swift released the song Cruel Summer as the second track on her long-awaited seventh studio album, Lover. Despite initial commercial success, critical acclaim, and the sickest bridge you’ve ever heard, the song was never released as a single. Blame it on the pandemic.
A few weeks earlier in the summer of 2019, I published a post titled “Gold Standard?” about NAEP. Despite the topic, a couple of retweets, and the clever combination of my cynicism about NAEP, some of my best pre-retirement sarcasm, and just the right amount of optimism mixed in, the post did just OK. Somewhere around the 50th percentile of all of my posts in terms of views. Blame it on the writing.
Fast forward four years.
Thanks in part to TikTok, Instagram, a little concert tour, and that bridge whose destiny was to be sung by 60,000 shrieking Swifties (a shrieking Swiftie is a happy Swiftie), Cruel Summer is surging to the top of the charts. It is being released as a single, and there will probably be a video, fingers crossed, a concert video.
On its own scale, “Gold Standard?” is experiencing a similar revival. For no apparent reason that I can discern, last week it became one of my top five most viewed posts of all time. More people have viewed the post in 2023 than when it was released in the summer of 2019, and I have no idea why.
It’s not so much the not knowing why “Gold Standard?” has moved from the middle to the top of the pack that bothers me.
Trust me.
One doesn’t spend three decades in large-scale assessment, in general, and state testing, in particular, without being very comfortable not understanding why things happen the way they do:
What makes this item more difficult than that one?
Why did the results go up, go down, stay the same?
Or not being able to offer much to support interpretation and use on topics like:
What a student with a particular test score actually knows and is able to do.
What a year’s worth of growth looks like.
How to use a score on an end-of-year test to inform instruction.
Part of the psychometric induction process in graduate school was learning to accept not knowing or fully understanding why things happen as they do.
Do you think that it’s an accident that there are so many parallels between psychometrics and organized religion? Seriously? Do you think that it’s by chance that we have turned validity into a false god – a never-ending process that we cannot quantify or explain to the laity – or that we continually try to describe things in terms of three mysterious concepts that are distinct but cannot be separated (construct/content/criterion; validity/reliability/fairness; curriculum/instruction/assessment)?
Run a significance test on any of those things occurring by chance and then get back to me.
No, we are fine with not understanding “why” and are willing to accept outcomes on blind faith as long as everything behaves as expected. Our entire field of study, our way of life, our raison d’etre, our own validation, if you will, is built upon the sandy foundation of things behaving as expected, as predicted, as they always have.
And that’s exactly what’s got me puzzled.
Why is “Gold Standard?” behaving so differently from all of the other 237 posts that I have published since 2018?
Tell Me Why.
A Peek Behind the Blog Curtain
In theory (i.e., in my mind), my posts should have a long shelf life. I am writing about perennial topics, issues, and problems in large-scale testing and educational measurement, and all of my published posts are neatly stored on this virtual shelf. In practice, however, my posts are the very definition of ephemeral.
Even the most radioactive of my posts have had a very short half-life.
Whether a post is wildly popular, mildly popular, appeals to a very unique niche audience, or I am the only one who looks up grinning like a devil after reading it, the pattern is the same:
- a burst of views on the first day or two,
- a big drop off by day three,
- a trickle over the next 10 days to two weeks, and
- everything wraps up within a month.
After that first month, a post may pick up a view here or there as someone scrolls through my site or inadvertently enters just the right combination of words in a Google search.
Even when a popular post includes a link to a previous post, rarely do more than a handful of people click on it.
But “Gold Standard?” has behaved so very differently.
As seen in the graph below, the post now has more views in 2023 than it did when it was published in 2019.
A single jump due to a bulk viewing in 2023 might suggest that one of my friends in academia had assigned the blog post as reading for a class discussion or perhaps that the post had become fodder for some nefarious AI bot. I gave those two possibilities equal weight before looking more closely at the data.
The data from 2020 through 2022 shows that interest has been building over time.
Drilling down, you can see the steady buildup over the past 12 months.
So, that’s the data I have been watching compile for the past year, wondering what’s going on with this post.
And I’m like, I mean, this is exhausting, OK?
Take the Words for What They Are
Then I was at the point that every psychometrician dreads – the point where we have to leave the comfort of data and take a close look at the content.
I even went back and read the post again, something that I rarely do.
Perhaps it was the title of the post that was grabbing people’s attention: “Gold Standard?”
I often use titles without keywords (e.g., NAEP or large-scale testing in this case), so maybe people just happened upon my post while searching for information about the actual gold standard After all, inflation is high, crypto has had some scandals, and with the debt ceiling “crisis” looming our economic standing depended on our political leaders.
Turns out, I cannot rule out “chance” or “measurement error” as a possible explanation.
Although my post and I don’t show up in searches for “gold standard” alone, add the question mark, “Gold Standard?”, to your Google search and you’ll find my post in the top 5 returns – right up there with an article from The Atlantic and a report from the Cato Institute, slightly above an academic treatise listed on JSTOR.
But I would like to think that there is more to the resurgence than an ambiguous title.
If not an economic accident, perhaps it was the topic, NAEP, that has made given this post legs.
Since 2018, I have written seven posts about NAEP.
Among those posts is my most viewed post of all time, Midnights (NAEP Edition), in which I commemorated the historic four days last fall which featured the releases of Taylor Swift’s tenth studio album and the most important NAEP results released, like ever.
Also included among the seven is one of my personal favorites from April 2018, If I Did It, which addressed the release of the long-delayed 2017 NAEP results. Inspiration for that cynical, satirical, sarcastic post came from a week in New York city, with days immersed in the beauty of the Metropolitan Museum of Art, nights spent at Broadway shows, and even a few hours attending sessions of the NCME conference I was co-chairing. Hey, don’t blame me for what you made me do.
“Gold Standard?” was not quite as cynical as my previous posts about NAEP and did represent a turning point, an inflection point if you prefer. It was a post in which I enumerated the many positive aspects of NAEP that are distinct advantages over what we try to do annually with state testing.
Because, each year it is a cruel summer for psychometricians working on state testing programs.
They roll the dice, trying to equate things that just don’t want to be equated (i.e., writing tasks, which we knew were trouble when they walked in) – cue the goat. They roll their eyes trying to place results on vertical scales which are arbitrary by definition, “scales” which on their best days strive to be capricious rather than pernicious. And those psychometricians do so knowing that the show must go on, results will be reported, come hell or high water.
We say, “I’m fine.” But it isn’t true. It’s a cruel summer.
And while I was writing this post, “Gold Standard?” received enough additional views to move into the #4 spot on my all-time list, and I still don’t know why.
It’s a cruel summer.
Header image by Rosy from Pixabay
Embrace the Absurd 
You must be logged in to post a comment.