Things change. It’s inevitable.
Sometimes the changes are big, sudden, and can’t be missed. Sometimes they are much more subtle, occurring gradually over a long period of time.
Some changes are deliberately planned, years and “steps” in advance: first we’ll change A, then B, then C.
Some changes are the unintended consequence of another change: we did A and then, much to our surprise, B happened.
Other changes just seem to occur spontaneously on their own.
Oftentimes, you find yourself 20 years down the road, 3 presidents, 4 state commissioners of education, and 5 assessment and accountability directors removed from the beginning or your school accountability program sitting at a meeting asking, “Why do we do it this way?”
Unless there is an old DOE staffer counting the days until retirement sitting nearby, or perhaps there is a long-time assessment consultant (whom successive assessment directors have inherited), the sad fact is that you may never know how and why you got here. There may be nobody with the institutional memory to unravel the story behind this tangled up mess of indices, indicators, and incomplete data sets all held together by the current set of business rules that we call a school accountability system.
And that’s too bad because I’m sure it’s a helluva good story that includes lots of phone calls, e-mails, and faxes, angry editorials, and rowdy board meetings (no, those aren’t a 2021 invention). Throw in some subtext, subgroups, subscores, and subplots. Then wrap it up all in a bow with competing rationales, random rationalizations, and a few rational decisions. Voila! Your 2021 School Accountability System.
Before moving on to some thoughts about what school accountability might look like in the future, let’s take a quick look back at that story of how we arrived at 2021 and this nice mess we’ve gotten ourselves into. Let’s recall the conversations and decisions that took place around five key aspects of school accountability systems.
% Proficient to Proficiency Index to Scaled Scores to “Scaled” Scores
In the beginning there was “% Proficient” and nothing else really except for graduation rate. The goal was that all students (100%) would be Proficient, and the metric of interest was the percentage of students who were Proficient, as in the current “% Proficient” and the projected or target “% Proficient” over the next several years.
The ultimate goal was 100% Proficient. This year there were 60% Proficient. Next year’s target was 64% Proficient.
“% Proficient” was too gross an indicator they argued. It didn’t paint an accurate picture of “real improvement” that was occurring below the Proficient level. Shouldn’t a school get “credit” for moving students from the Below Basic to Basic level? So, the “Proficiency Index” was born.
|
Below Basic (0) |
Basic (50) | Proficient (100) | Advanced (100) | Proficiency Index | Percent Proficient | |
| Year 1 | 20% | 20% | 30% | 30% | 70 | 60% |
| Year 2 | 10% | 30% | 30% | 30% | 75 |
60% |
So, the target was still 100% Proficient and an index score of 100 would mean that 100% of students were Proficient. And we could show improvement below proficient.
What did index scores of 70 and 75 mean?
Something gained, something lost.
But proficiency (straight percentage or index) is not a good metric for measuring school improvement from one year to the next. It’s too “bumpy” and too dependent on the current distribution. Let’s switch to scaled scores.
Can we average scaled scores across grade levels the same way we could count the number of proficient kids across grade levels? Ask the psychometricians? (Hardly ever a good idea if you need a yes/no answer by the way, just sayin’.)
Well, you could scale the scaled scores so that they are on the same scale and then scale that scale to look like the score scale and then you can average the scaled scaled scores.
Never mind, we’ll just average the scaled scores. Close enough for government work.
OK, so how do I interpret an average scale score across grades 3-8. What does a 10-point gain in scale scores mean?
I was supposed to report the standard deviation, too?
What’s the ultimate goal now?
Something gained, something lost.
Growth
The Proficiency Index helped, but not enough. We needed a way to get more students into that numerator (i.e., # Proficient/# Students). By George W, I think I’ve got it. We’ll let schools include kids who are “on track” to be Proficient – in a reasonable amount of time, of course. And new metrics were born.
Note that we are not talking value-added growth models, Student Growth Percentiles, and Growth as a separate indicator quite yet. This “growth” was just being able to make a “reasonable claim” that some students might be Proficient within the next few years and tossing them into the numerator. All of those other fun things followed closely behind.
Fun fact: This was also the first time that many of us in K-12 testing ever heard the term “bright lines” (or bright-line rule or bright-line test). Before growth, I guess the term was used primarily in legal circles and law schools – you know, like critical race theory – but we soon made it our own, as is our wont.
Alternate Assessment – 1% Cap
Much like growth, the “1% cap” and accountability rules related to alternate assessments were the result of attempt to include more students in that darned numerator – federal law making it clear that students taking the alternate assessment would be included in the denominator.
To put it bluntly and succinctly, school and district leaders were seeking a waiver to count students taking the alternate assessment as Proficient in the accountability system. The USED said, OK, but if that’s more than 1% of your students, you have to provide additional evidence. Why 1%? Well, the national rate for students with disabilities was about 10%, and about 10% of those students were considered to be students with significant cognitive disabilities who would be taking the alternate assessment (10% of 10% = 1%) Ergo, a 1% soft cap.
OK, the school leaders were happy for a New York minute.
Then the special education folks (i.e., the folks taking the alternate assessment seriously) and the psychometricians (i.e., the folks trying to take standard setting seriously) got involved and the result was 90% of students taking the alternate assessment classified as “not proficient” against the alternate academic achievement standards. So much for the idea of a waiver.
Then the short-lived 2% test turned the 1% soft cap into a hard cap even though it wasn’t clear that the 2% was adjacent to, or anywhere in the same neighborhood as, the 1%. When the 2% went away, the hard cap remained, and then was made even harder.
Oh, what a tangled web we weave, when first we practice to relieve (pressure on schools trying meet school accountability requirements)!
Your Father’s “Proficient”
“I hope you can tell that education is dear to my heart. I care a lot about whether or not our children can learn to read, write, and add and subtract.”- George W. Bush
I think that we can all agree that when it comes to defining what Proficient means there is a whole lot of gray area between children “learning to read, write, and add and subtract” and college-and-career readiness, 21stcentury skills, and deeper learning.
When we turned the Paige on NCLB, apparently, we set states off on a Race to the Bottom. The writing was on the wall, and it was clear that it was Spellings different standards for different states. In response, we donned our white robes, stood on the shore, and prepared to Duncan ourselves fully in the waters of national (but not federal) standards, something not talked about openly since the days of George H.W. Bush. But as we emerged from the water, cleansed to the core, to race back to that summit, it became clear that the King had no clothes. We had no real consensus on what Proficient was or why it was important.
Alas, we are a long way from % Proficient, but are we any closer to all kids learning to read, write, and add and subtract.
Can it be that it was all so simple then? Or has time re-written every line?
If we had the chance to do it all again Tell me, would we? Could we?
School Accountability?
A final question to consider before we turn to the future of school accountability is how this became “School Accountability” in the first place.
As I recall, when all of this began there was a great deal of focus on the District and its responsibility. What happened there?
Teacher Accountability? We know what happened there.
Student Accountability? Hmm, they’ll get back to us on that one.
What about the state and federal government? What about Facebook? (Sure, let’s pile on Facebook)
What do we really mean by accountability? Who Is Accountable? To Whom? For What?
What does it mean to be accountable?
OK, let’s go.
Modified from Image by Eric Perlin from Pixabay
Header Image by Dimitris Christou from Pixabay
Embrace the Absurd 
You must be logged in to post a comment.