When it seems the magic slipped away

A blog year in review

If there was ever a year in which it seems the magic of large-scale, K-12 testing had slipped away, 2020 was that year. Our field  found itself under attack for its racist past and a present in which tests produce outcomes that have a disproportionately negative impact on non-white and economically disadvantaged students. State accountability systems were under fire for being overly dependent on end-of-year tests state assessments which themselves were criticized for relying too heavily on selected-response items, failing to provide actionable information to inform and improve instruction, being too costly, and at best consuming, or at worst wasting, precious instructional time.

And then came the pandemic in March and Minneapolis in May.

Lest we forget, before the two defining events of this year (and very likely the first quarter of the 21st century), 2020 was already shaping up to be a year like no other for large-scale assessment.  Yes, standardized testing has always been a convenient target of critics, but nevertheless it has persisted, and has more often than not emerged even stronger, but as I discussed in Why is this time different?, such a positive outcome seemed much less certain this time around.  

When 2020 decided to “cry ‘Havoc!’ and let slip the dogs of war,” “the die was cast.”  Even as William Shakespeare lined up to receive the vaccine signaling the end of the beginning of the pandemic, it was clear that there will not be a return to normal for large-scale K-12 assessment and accountability.

A new normal for assessment and accountability, however, is not necessarily a bad thing. It offers an opportunity to revisit and renew unfulfilled promises.  In this end-of-year post, I return to six posts from the past year to reflect on 2020 and look ahead to an unknown future for educational assessment and measurement – a future that I am confident will be better and brighter.

We were dreamers not so long ago

As we sit here at the end of 2020, the PARCC consortium is already a fading memory and CCSS (Common Core State Standards) is still a four-letter word in many places;  but it was not so long ago that it was widely believed that we were on the cusp of a new era of large-scale assessment. As my colleague, Chris Domaleski, wrote, “It’s hard to describe the enthusiasm and high expectations for assessment in 2010. At the time, all the talk was about moving away from superficial “bubble tests” and measuring what really matters.”  

Two decades prior to the CCSS, PARCC, and Smarter Balanced, a similar sense of hope and enthusiasm greeted programs such as the California Assessment Program, the Kentucky Instructional Results Information System, the Vermont Portfolio Assessment Program, the Maryland State Performance Assessment Program, and the New Standards Project (prior to its operationalization through Harcourt), and others. Those programs, responding to a demand for increased validity and fairness, attempted to produce a seismic shift in large-scale assessment through the introduction of direct writing assessments and open-response items (which are now somewhat routine) along with portfolios and performance assessments (which unfortunately, are quite rare).  

The CCSS-era assessments and the innovative assessment programs of the 1990s had limited success in achieving their goal of changing the nature of large-scale assessment, but they demonstrate the capacity of our field to dream – to envision a world in which assessment is different and better.  

Imagining a future that is different is a critical first step.

When it seems that we have lost our way

Somewhere between the 1990s and the CCSS assessments, it seems that we lost our way; that is, we lost sight of the problem that we were trying to solve.

As I discussed in If a Tree Falls, is Johnny Proficient?, although we tend to think of  large-scale assessment in terms of measurement, state assessment has always been first and foremost a data collection problem. Students are proficient (or not) regardless of whether there is a state assessment. Our task is to find the most valid, reliable, and fair method for collecting data on student proficiency from schools. For most of the 20thcentury, an external test was determined to be the most efficient approach to collecting the necessary data; but that was always a hypothesis to be examined and never a given.

The innovative assessment programs of the 1990s were designed to shift the locus of control of state assessment from external (something the state was doing to measure their definition of academic achievement ) to internal (an opportunity for local educators and students to demonstrate their academic achievement).

The CCSS-era assessments programs, in contrast, were designed to serve a fundamentally different purpose – to build better external tests.

Our current situation suggests that it is time to refocus on the problem of the 1990s – figuring out the best way to collect valid, reliable, and fair data from schools and students.

Understanding the problem that you are trying to solve is a critical second step.

Believe in what your heart is saying

When you do something as expansive as what we did, you have to have a belief you can make almost anything happen.” (Lauren Resnick, co-director of the New Standards Project, EdWeek 2001)

The New Standards Project, and each of the other assessment programs I mentioned above, were driven by people who believed strongly that changes to assessment, in conjunction with changes to curriculum and instruction, could make a significant difference in improving student learning. In Hamilton & the Future of Assessment, I outline my beliefs about the direction assessment needs to take, but I am sure that there are others with different beliefs.

The point is that as we look to improve assessment and educational measurement, it will not be sufficient simply to point out what’s broken or to explain why something won’t work.  Those steps are necessary, true; and the people who devote their careers to them serve an essential function; but we need more.  We need innovative programs driven by research-based beliefs and supported by a well-designed, -supported, and -implemented theory of action and logic models

There’s so much to celebrate (as well as so much to critically examine)

There is no question that this is a time for serious reflection and introspection as we, as individuals, and the field, as a whole, attempt to better understand the ways in which we have contributed to, or even exacerbated, the inequities in public education through our actions, inactions, policies, practices, and scientific traditions. However, I believe that if we hope to contribute to anti-racism and a more equitable system of education we need to begin with a much deeper and more nuanced understanding or our field than eugenics and Jim Crow (old and new).  

We must identify the aspects of our work and advances in our science that should be celebrated along with areas that must be improved  As I write in  I Can See The Writing on the Wall, there is a fine line between constructive self-flagellation and self-immolation.  In that lengthy, very special post, I offer six pieces of advice to a field trying to find itself and its way into the future organized around the themes:

  • Know Thyself
  • Because Science
  • It’s Supposed to Be Hard
  • Dream Things That Never Were
  • Know Your Limitations
  • Remember the Past

Give your dreams the wings to fly

In The All-Decade Team – State Assessment Version, I identified 10 assessment programs which defined the 2010s. My most important takeaway from the list and the decade was that improvement is messy. “Innovation in assessment needs space, time, and resources for research and for things not to work correctly (never mind perfectly) the first time around – or even the second time around.”  

When the well-documented flaws of the innovative performance-based large-scale assessment programs of the 1990s were revealed, the field basically shut down that line of research for the next 25 years. 

We cannot afford to do that again. There’s no time to waste.

You Have Everything You Need – Just believe

In closing this year-end reflection, I return to my post Don’t Go There Charlie and leave you with two quotes from the Ronald Edmonds 1979 article “Effective Schools for the Urban Poor”:

There has never been a time in the life of the American public school when we have not known all we needed to in order to teach all those whom we chose to teach. 

It seems to me, therefore, that what is left of this discussion are three declarative statements: (a) We can, whenever and wherever we choose, successfully teach all children whose schooling is of interest to us; (b) We already know more than we need to do that; and (c) Whether or not we do it must finally depend on how we feel about the fact that we haven’t so far.

Image by 🎄Merry Christmas 🎄 from Pixabay 

Published by Charlie DePascale

Charlie DePascale is an educational consultant specializing in the area of large-scale educational assessment. When absolutely necessary, he is a psychometrician. The ideas expressed in these posts are his (at least at the time they were written), and are not intended to reflect the views of any organizations with which he is affiliated personally or professionally..

%d bloggers like this: