There’s a way to do it better. Find it. – Thomas Edison
The quote above, attributed to Thomas Edison, certainly applies to state testing. I think that it’s safe to say that few would disagree with the sentiment that there is better way to “do state testing better” and that it is incumbent upon us to find it.
Finding agreement on what it means to do state testing better, well, that might prove a bit more difficult.
And when you do manage to convene a group of like-minded people, figuring out how to actually implement their vision of doing state testing better, now there’s a challenge has kept smarter and more accomplished people than me up at night.
But that’s why we’re here. That’s what makes us tick.
Before jumping in with some thoughts on making things better, I want to take a moment to comment on the current state of state testing.
It’s good.
Good is Good But Could Be Better
It’s true that things didn’t work out quite as we expected, or at least hoped, back in 2010 with nearly universal adoption making the Common Core State Standards national (not federal) standards and the federal government ponying up a boatload of money to fund the development of next generation of state assessments that would take us beyond the bubble test.
No, we did not end up with a single, national “state test” that allowed cross-state comparisons of school effectiveness and student college-and-career-readiness – or worst case, two tests based on the same content and achievement standards whose results could easily be interpreted via a concordance table.
But all states did manage to complete the shift from paper-and-pencil to computer-based testing. The transition was not always smooth or pretty and results not as comparable as we hoped or claimed, but it was done. Technology issues seem to be lessening. There are some drawbacks to computer-based testing, but most would agree that CBT is a net positive over paper-based testing.
Despite all the teeth gnashing and hand wringing recently, there are plenty of good, decent, and adequate state assessment programs and state tests out there. Reports are being issued on time, accessibility is high, and mistakes are fewer and “far-er” between than they used to be. And the “mistakes” that do receive the most attention these days tend to be of a different caliber. Miskeyed items, printing errors, computational mistakes, and misplaced students have been replaced by misplaced outrage over talking (but not running) pineapples and the Underground Railroad (both fine passages and items sets – FWIW).
Test use has become more of an issue than the tests themselves.
However, despite the shift to computers, we are using largely the same approach to state testing today that we used a half century ago, or even a century ago, if we want to expand our scope to large-scale K-12 testing, in general: single, on-demand tests, administered to students at (or near) the end of the school year.
We have to ask ourselves whether there is a better way to obtain the two things that we need to get from state testing:
- To know whether students are able to do what we expect them to be able to do at a particular point in time.
- To be able to tell parents, teachers, administrators, perhaps policymakers, and students themselves whether students have made sufficient progress – again, during a specified period of time.
To that end, I offer five pieces of advice.
Work the Right Problem
For the past several decades, those of us in state testing have focused all of our efforts on building a better on-demand summative test – whether that test is administered once at the end of the year or three to four times per year as part of an interim assessment system.
- Test that are better aligned and universally designed.
- Tests that are free of bias and sensitive topics.
- Tests written at appropriate Lexiles, that support growth percentiles, and withstand the projectiles hurled at them.
We have not yet begun to seriously address the question of whether there are other ways (i.e., better ways) to get the data to provide necessary to provide the information needed.
To be fair and balanced, as I always attempt to be in my blog, there are some outside of state testing who have been asking that question, or rather, shouting through megaphones that there is a better way. Those folks, however, have tended to frame the issue in an either/or, us v. them, right v. wrong, good v. evil, think about the children manner which only causes policymakers to dig in their heels and psychometricians, the awkward introverts by nature that we are, to turtle.
Think Outside of The Box We Have Built Ourselves Into
As discussed in a previous post in this series, the feds and their regulations are a big reason why we have been so laser-focused on tests; but feds are made to be handled (gently, never overworked, of course, like other balls of dough) and regs are made to be challenged. That’s why we’re here.
Also, it’s hard to ask box makers to think outside of the box or test makers to think outside of the test.
Plus, we can’t discount the field’s residual skittishness lingering from early 1990s, the last time that we tried to open the lid and step outside of our box – only to see Pandora stamped on its side in big red letters and a Cheshire Cat (or was it Dan Koretz) standing there grinning at us. Getting our ears soundly boxed was enough to keep generations of assessment specialists tucked safely inside our testing box for the next quarter century.
But the times have changed since the 1990s and many of the barriers that existed then have been removed. Advances in technology have created the infrastructure and tools to make portfolio assessment, performance tasks, and other school-based, embedded forms of state-supported authentic assessment that we envisioned in the 1990s not only feasible and practical, but technically defensible – assuming that we take the time and steps necessary to do it the right way this time around.
Stop Looking Inward (so much)
Trapped inside of our box, it was easy to forget that everything that we are interested in assessing, all of the data that we are looking for, exists outside of the box, and outside of the test.
Students, like those noisy trees falling in the forest when nobody is listening, are proficient or not proficient, college-ready or not college-ready, regardless of whether we test them.
There are vast amounts of pre-existing data sitting out there in schools that we now have the capability and capacity to incorporate into state assessment programs.
We knew in 1990 that we could not get all of the information that we needed from a single, on-demand, selected-response test. We needed to get information directly from schools. We just didn’t have the means to do it well at that time.
Play to States’ Strengths
States are really good at collecting data. They are not as good at measuring things – and thankfully, particularly not as good at measuring individuals.
Fortunately, gathering data about student proficiency, validating it, and using it to produce information about student progress has always been a data collection problem and not a measurement problem.
We got confused, lost, and wandered around the testing desert for the last 40 years, because for the longest time, a short, selected-response test was the most effective and most efficient methods (i.e., the best way) to collect the data that we needed.
- But then the data that we needed became more complex.
- But then we needed to include all students.
- But then data were everywhere.
Now there are better ways.
Do The Right Thing the Right Way
So, state testing is not perfect, but pieces are finally in place to make it a whole lot better.
We will be able to collect much of the data that is needed for state assessment directly from schools, unobtrusively and efficiently.
But we won’t’ be able to do that tomorrow, or the next day, or the day after that.
We’re not ready yet. The schools are not ready yet. The policymakers are not ready yet. We haven’t thought about how to do it yet.
Last time, we just jumped off the cliff and hoped for the best. This time, we need take the time needed to do it the right way.
That means that a right-sized, on-demand summative test should continue to be part of the state testing picture for the foreseeable future – until we understand clearly whether it is needed, the role that it can play, and/or have something in place to replace it. That summative test may be a stand-alone end-of-year test or part of an interim, or through-year, system of assessments. Either way works for me.
Sooner rather than later, however, we should begin to think of that test as a gross indicator of student proficiency rather than a precise measure. We’ll all sleep better that way – especially the alignment folks.
Should that test continue to be administered to all students at every grade level? Probably so. At least students in grades 3 through 8.
It’s not that we need annual testing of all students for school accountability. We don’t.
But assessment and testing have been moving inevitably and inexorably in the direction of individual students. There is no doubt in my mind that our ultimate state testing solution will be focused at the individual student level. It would be foolhardy to walk away from longitudinal data at the individual student level at this time if that’s the place where we want to end up in the future.
Perhaps more importantly, the reasons that state testing was put in place in 1965, renewed in 1994, fortified in 2001, and maintained in 2015 when so many people were calling for its demise, still very much exist today. There are inequities in educational opportunities and inequities in other aspects of life that result in real gaps in achievement.
Finally, there is a common core of knowledge and skills that every American student needs to acquire by the end of the eighth grade whether that student and their family are old to the country or brand new, be they a shit-kicker or a city slicker, and regardless of what color of the rainbow they identify with.
Annual state testing, flawed and imperfect as it is, is still our best tool and our best hope for collecting the longitudinal data we need to monitor educational opportunities for individual students.
Someday soon we’ll get it all together and figure out a way to do it better – if the fates and feds allow. Until then, we’ll have to muddle through somehow.
Image by Gerd Altmann from Pixabay
Embrace the Absurd 