Clearing Our Heads on School Accountability

It’s the middle of the summer. It’s time to take a break, to refresh and reset. Maybe take in a movie or two. We need to clear our heads and start fresh on the topic of school accountability.

Because after reading the recent paper, The Path Forward on School Accountability, written by several of the fine folks at the Center for Assessment, The Accountability Conundrum by Checker Finn, and following the discussions at recent conferences,  I have reached a couple of conclusions regarding school accountability:

1. We’re in over our heads.
2. The Feds are in our heads (and not in a good way).

We’re in over our heads

That “we’re in over our heads,” will not be a revelation, or even a mild surprise, to anyone who has been involved, even peripherally, in school accountability for the past two decades.

We all know the score: We’ve brought a large-scale test to a gunfight. (And I employ that idiom, knowing full well that it’s roots may trace back to an anti-Italian American slur and stereotype. I’ve taken control of that narrative. I’ll take the cannoli and move on.)

There’s nobody involved in state testing or school accountability (two separate things, by the way) who ever believed for a moment that student scores on a Reading and Mathematics test were a sufficient measure of school effectiveness – even if you process those scores to include growth. We never should have allowed that to become an argument.

There were some people who believed initially that student scores on a Reading and Mathematics test were at least a sufficient measure of how effective schools were at teaching reading and mathematics. Jim Popham quickly disabused us of that notion. Nevertheless, that perception persists.

Even knowing that a school does so much more than teach reading and mathematics, and that we cannot blindly assume that a school (or its teachers) are effective simply because students are proficient in reading and mathematics, however, some of us bought into and clung to the idea that it is important to know whether students in a school (or subgroups of students in a school) are habitually not proficient in reading and mathematics; and it’s important for students and their parents to know whether they are proficient. Not to label them or to blame them, but so that something could be done about it.

Implicit in, or flowing from, that idea is the belief that students’ achieving and maintaining a minimum level of proficiency in reading and mathematics (and achieving proficiency in those areas at the right time in their personal and academic development) is

1. a) a prerequisite for many of the other important things that a school does and
2. b) important enough in its own right to feature in school accountability.

The mission assigned to those of us in assessment was to ensure that student scores on a Reading and Mathematics test were a sufficient measure of how well students knew reading and mathematics.  And we chose to accept it.

We did our best to make that assurance. We worked tirelessly on alignment to standards, while others were working tirelessly on improving those standards. We have been sincere and diligent in our efforts to ensure that our tests were accessible and fair to all and that our assessment programs inclusive of all. Within the constraints of an on-demand, large-scale test, we pushed the boundaries of measuring reading and mathematics as far as they could be pushed. We knew that we had not achieved perfection, but we were confident that our tests were good.

But now people are questioning all of it.

Are our tests, test scores, and content standards good? And not simply good as in good enough for government work; but good as opposed to evil, unethical, immoral, and perpetuating the work of a systemically flawed government.

We cannot “test” our way to understanding or making a judgment on the quality or effectiveness of a school. And now, it’s not clear whether we should be using our tests to do the one thing we were confident that they could do.

We cannot “test” our way out of this situation.

We’re in over our heads.

The Feds are in our heads

It’s difficult to reflect on the current state of school accountability and not also conclude that we’re stuck on the feds and the federal requirements for accountability and assessment. The common refrain:

If only the federal requirements would change, everything would be so much better.

The Center’s accountability paper goes in a slightly different direction, but still leaves me with the feeling that it is federal assessment and accountability requirements that have been standing in the way of states improving school accountability.

After paying due homage to the flexibility that ESSA offers and acknowledging the common critiques of accountability systems designed to meet NCLB and ESSA requirements, the authors suggest five ways for states to improve accountability while working within or alongside of the constraints of ESSA:

• Developing a comprehensive theory of action
• Employing a principled design process
• Supporting customized approaches
• Measuring more of what matters
• Connecting information to support

Checker Finn notes of the Center’s paper that as far as it goes “[n]othing in it is objectionable, and much is worth taking seriously.” And Joanne Weiss states that the paper is an “[e]xcellent resource for policymakers.”  As has been the case much more often than not over the years, I agree wholeheartedly with Finn and Weiss.

And I am 100% on board with the idea of states doing their own thing alongside of ESSA. You know all too wellmy attitude toward the USED, NCLB, and ESSA: the law and regulations are more what you’d call “guidelines” than actual rules (to paraphrase the Pirates of the Caribbean and to borrow an old quip from an old friend).

Throughout my career, I advised states on how to render unto the Feds what is the Fed’s and then do whatever it is they wanted to do. Few things brought me greater pleasure than when a state director possessed the tools she needed to look the USED storm…, er staffers, in the eye and calmy state, these are the assessment and accountability systems you’re looking for.

So, when I read the Center’s set of recommendataions, I feel that something is missing.

I have to wonder why more states aren’t already doing this. Are any states doing this? It’s been more than 20 years since NCLB. What’s stopping them? The Center doesn’t address this question directly.

Is it something that the states perceive about the law itself or the effects of the law?

My best guess is that a big reason that states have not moved beyond the federal requirements is that in the more than half a century since Title I was put into place, it has been so very difficult for any state to clear the very first hurdle; that is, to find a way to ensure that all students attain the necessary level of proficiency in reading and mathematics.

When a state engages the appropriate stakeholders and sets out to build a new accountability system by developing a comprehensive theory of action, employing a principled design process, measuring more of what matters, etc. how can they conclude anything other than the motto of my adopted home state: You can’t get there from here.

How do we get there from here?

With apologies to Checker Finn, in my mind, the real accountability conundrum is that the premise on which current school accountability systems are based is patently false. It is not true that all students assigned to a particular grade level at the beginning of the year can meet the standards for that grade level by the end of the year – even allowing for the accepted exceptions to the meaning of all students.

That simple statement of fact is not news to anyone, except perhaps to some students and parents.

Attempts to continue to build school accountability systems centered on achievement outcomes without acknowledging that fact will inevitably continue to lead to conditioning in the name of fairness; and no amount of conditioning alone will ever lead to necessary school improvement. That’s not what conditioning is for.

On the other hand, attempts to build outcome-based school accountability systems without conditioning will inevitably lead to nearly 100% of schools being identified as failing, in need of support, or whatever the label of the day happens to be, and educators throwing up their hands, throwing in the towel, and/or devoting their precious energies to gaming the system; but it won’t lead to school improvement.

But we know that accountability systems need to include grade-level outcomes. History has taught us that accountability without outcomes is foolhardy and that attempts to build interim, or transitional, targets for outcomes is a fool’s errand.

With apologies to the Center, in my mind, the path forward on school accountability begins not with a developing a comprehensive theory of action for school improvement/accountability, but with acknowledging and addressing that conundrum.

That being said, if we don’t figure out a way develop the infrastructure or build the foundation from which it makes sense to hold schools accountable for students achieving grade-level outcomes, then I fear that we will continue to spin our wheels for another 50 years. Off the top of my head an enhanced sense of urgency surrounding the need for early childhood education and full funding of Title I programs would be a good place to start.

Image by Gerd Altmann from Pixabay