Give NAEP A Chance

Followers of this blog over the past few years may recall that I have been just a tad critical of NAEP, on occasion, for their interminable process of analysis and reporting, and their obsessive clinging to a trendline that serves as both their raison d’etre and a noose around their necks.   In this through-the-looking-glass year, however, I am looking to NAEP as a model for states to learn from as they plan for some type of spring 2021 administration of their summative assessments.  Yes, there are many critical differences between NAEP and state assessments. However, even with those differences and the above-mentioned peccadillos there are also some valuable lessons to be learned from the NAEP experience. Below are five areas in which I think that we should give NAEP a chance to inform the planning, administration, and reporting of results from spring 2021 state assessments. [Note for the purposes of this post, rather than trying to distinguish between NAEP (the assessment) and the roles of NAGB staff, NAGB board members, NCES, USED, assessment contractors, etc. I will simply refer to everything as NAEP.]

• Low-stakes testing – States are used to having relatively high-stakes consequences and the power of state law, in many cases, to compel participation in state testing. Many of those consequences are likely to be off the table in spring 2021. NAEP has experience with communicating about the importance of participation in a low-stakes testing program. The opt-out incidents of 2015 that accompanied the introduction of the common core assessments demonstrated the importance of messaging. States do not want to lose the messaging campaign this year.

• Centralized test administration – Despite all of their administration protocols and security measures, the administration of state tests, like elections, is largely the responsibility of local authorities. NAEP, in contrast, uses a highly centralized approach to test administration. The spring 2021 administration, with the real possibility of remote testing, might require a greater level of centralized control of the administration process than states are used to managing.

• Sampling – Since NCLB, state have tested all of the students in their state on a common test or a blueprint-aligned CAT. This year, however, some states may be considering a sampling approach – sampling students, distributing items across students, or both. NAEP obviously has extensive experience in sampling students and matrix-sampling items that might be useful to states.  Perhaps even more important for spring 2021, however, is the experience that NAEP has in deciding how, or even whether, to report results when the expected samples of students are not tested for one reason or another.

• The importance of trends – Yes, the obsession with their trendline is one of my primary criticisms of NAEP, but trends are important and they do play a key role in use of educational assessments to inform state education policy. NAEP should be able to offer states advice on communicating the importance of maintaining trends to key stakeholders and also offer advice on maintaining trends under the less-than-ideal circumstances that will exist in spring 2021.

• Contextualizing Reporting – Also related to one of my primary criticism of NAEP, one of the reasons that NAEP takes such a long time to report test results is that they put such effort into how those results are reported. They craft a message, develop a variety of resources, and carefully plan for the release of results. They also prepare states on how to convey those test results to key stakeholders such as policymakers, educators, and the public on their own and via the media. States often do not have the capacity to do the same with the reporting of results from their state tests, nor are they afforded the luxury of the time needed to do so. However, this year, more than ever, it will be critical for states to carefully decide what to report with regard to test results and then to communicate those results effectively.   It goes without saying that this is not a normal school year and these will not be normal test results.

In closing, I urge state assessment directors to consult with their state NAEP coordinators (if that’s still a thing) and to draw on the expertise of Lesley Muldoon and her staff to help make the spring 2021 state assessments as useful and informative as they possibly can be under these circumstances.  And in the spirit of extending an olive branch to NAEP for past blog posts, I offer this spring 2021 assessment version of John Lennon’s 1969 Give Peace a Chance (which coincidentally came out at about the same time as the beginning of the NAEP trendline.)

Give NAEP a Chance

Everybody’s talking about

Equity, validity, sanity, history, CBT, IRT
Don’t test, remote test, test, test, test

All we are saying, is give NAEP a chance
All we are saying, is give NAEP a chance

(Come on)
Everybody’s talking about

Administer, Sinister, Standard Error, Too much pressure
Waivers, Saviors, DeVos, COVID loss, bye, bye, bye, bye, byes

All we are saying, is give NAEP a chance
All we are saying, is give NAEP a chance

(Let me tell you now)
Everybody’s talking about

Revolution, evolution, innovation, validation, regulation
Administrations, divided nation, accommodation, standardization

All we are saying, is give NAEP a chance
All we are saying, is give NAEP a chance

Image by Gordon Johnson from Pixabay