Gold Standard?

By any metric, 2017 was, and continues to be, a very bad year for NAEP.  Troubles began in April 2018 with the utter fiasco that was the long-delayed release of the 2017 Reading and Mathematics results.  Seldom in the course of human history have so many good statistics been sacrificed in the name of preserving an illusory trend line.  Then late last week came the announcement that no amount of statistical sleight of hand could save the results from the 2017 Writing assessment.  (And by announcement, I mean burying information deep on a website on the first Friday of summer that no results were forthcoming and that a more detailed report would be available in the spring of 2020.)

Perhaps, however, the worst news for NAEP was when they announced that they were not releasing results from a major assessment, few people noticed and fewer people cared.

Where does all of this leave NAEP as we await the results from the 2019 NAEP Reading and Mathematics assessments?

As I started to write this post, I will admit that I was feeling a bit cynical toward NAEP and my original title was ‘Gold Standard, My Ass!’

But then I thought, who among us hasn’t wanted just 3 more days or even 3 more hours to figure out what the hell was going on as we tried to equate writing results across years. If NAEP can lead the way and establish 3 years as an acceptable time frame, more power to them.

Plus, there have been plenty of times in the last 30 years when I have had to help state leaders make the hard decision between making necessary changes to their Reading and Mathematics tests or preserving their reporting scale and trend line.  If NAEP can lead the way on having your cake and eating it, too, pass me another slice.

And why should a state go through all of those hoops to convince USED that they really did administer a test and hold all students accountable this year if it is possible to just decide not to report results.  Be that shining light in the darkness, NAEP! We will follow!

Still not totally sure which direction I should go with this post, I thought a little bit more about the term ‘gold standard’ and what it represents.

There are several things about NAEP that do make it a symbol of the ideal in large-scale assessment:

• Testing periodically rather than every year.
• Testing intermittently across grade levels rather than at every grade level
• Testing samples of students rather than all students
• Using matrix sampling to improve the sampling of content on each assessment
• Separate scaling of domain areas so that subscores might actually be useful
• Demonstrating a total disdain for deadlines in the name of getting it right
• The willingness to serve as an example of how difficult it is to set meaningful performance standards on a large-scale assessment.

Those things should be more than enough to offset the lack of transparency and no individual student scores and establish NAEP as an ideal; that is, a gold standard.

The final thing that turned me around on NAEP as a gold standard, however, was remembering that the gold standard is an antiquated monetary concept that was abandoned by virtually all nations decades ago; it is an anachronism that simply no longer works in the real world.

So NAEP, I owe you an apology.  Feel free to hold firm as the old white male of assessments in the changing world of 2019.  In so many ways, you are and will always be the gold standard.