Build Back Better: School Accountability

Or Thirteen Thoughts on Making School Accountability Great Again

1. Stop turning to assessment specialists for advice about school accountability

Well, if you take my advice seriously this was a pretty short read. If you choose to read on, do so at your own risk.

On a good day assessment specialists have a pretty good handle on K-12 assessment and how it works. On a great day, we acknowledge and accept how little we truly understand about K-12 assessment.  And that is how we function in our area of expertise, our chosen field, the Satan to which we have sold our souls and sacrificed our precious work-life balance.

At the risk of saying something that will cause my validity friends to get their panties in a bunch or their knickers in a twist (a lot of validity folks have roots in the Commonwealth):

Assessment and Accountability are two distinct, and very different things, even when test results are the primary component of accountability systems.

There is no reason to expect people with expertise in assessment to also be specialists in the inner workings of accountability systems, in general, and school accountability systems, in particular.

The same advice applies to econometricians, data scientists, statisticians, and generic educational measurement specialists.

If you want help designing a new school accountability system (not simply implementing federal requirements), find people with expertise in school accountability or at least experience with program evaluation in school settings. (Some of them may also have expertise in assessment.)

2. Start talking, really talking, with large-scale assessment specialists about assessment.

Tell us what you want to measure. Tell us how you intend to use the results, the inferences and claims you need to make.  Tell us the constraints you are under. All of that information is a good starting point for a conversation that can lead to the design of a high-quality assessment program to support school improvement and accountability.

If you tell us you want to know whether students are Proficient, we can help you to define proficiency and to determine how accurate and precise the classification needs to be. Together we can determine whether you need to know whether individual students are Proficient or whether a solid estimate of the percentage of Proficient students in a school or subgroup, would be sufficient.

If you tell us you need to know whether students have mastered a standard or attained certain competencies, we can design tests to provide that information.

If you would like to determine whether students are on track to attaining proficiency, mastery, or competency, I hope that we would tell you that’s venturing outside of the type of information that can be provided by a test score.

If you would like to know exactly where each student falls on the proficiency continuum at any given point in time, I hope that we would help you appreciate the enormity of such a task.

If you would like to do all of the above with a single test that can be administered in no more than an hour and produces immediate results, I hope that we would wish you the best and walk away.

3. Be more protective of the concept of school accountability

We are much too loose with our use of language associated with school accountability. The term “failing schools” under NCLB is an obvious example, but perhaps not the most egregious. That dubious honor might fall to the use of the broad term school accountability to refer to performance on the small set of indicators contained in federally mandated accountability systems – even the expanded set of indicators under ESSA.

There are good reasons to measure the performance of students in grades 3-8 in English language arts and mathematics regularly. I could argue that it also makes sense to classify that performance as to whether it is satisfactory.

You could convince me that such information should be included in a school accountability system.

You might even convince me that test results along with school attendance and out-of-school suspension rates were sufficient indicators, as proxies for achievement and school climate, to trigger a deeper dive into how a school was functioning.

You could never convince me that such information is sufficient to determine school quality.

Measure school performance in English language arts, mathematics, and other content areas. Measure school climate. Measure SEL. Track inputs. Monitor processes.

Don’t refer to any of those individually as the whole of school accountability.

4. Bring accountability into school accountability

A school rating system is not accountability. A couple of consequences triggered by a performance index is not accountability. A policymaker sitting at a board meeting saying “the buck stops here” in a somber, but determined, tone – not accountability.

The next generation of school accountability, whatever it looks like, must include a true accountability component. That is, an accountability system must include a requirement for someone to be responsible for examining the results, explaining them, and describing – in detail – what is being done in an effort to make them better the next time.

And most importantly, all of that must be regarded as part of the accountability system.

5. Stop conflating school accountability and school support

Some schools need additional support from their district, the state, or other agencies.

Some schools need to be held accountable for subpar performance on the part of school leaders, particularly over an extended period of time (e.g., 3 years, 5 years).

Those may or may not be the same schools at some point in time.

Seeking and receiving additional support, however, should not automatically be regarded as a sign of weakness or as an indication of failure. That’s not a good educational improvement model.

6. Respect boundaries

The mix of regulations and responsibilities across federal, state, districts, and school levels blurs boundaries. To the extent possible, however, school accountability should function one layer at a time – the state is accountable to the federal government, the district is accountable to the state, the school is accountable to the district. All are accountable to the students.

That approach does not preclude the development and use of federal- or state-supported resources, materials, and funding.

That approach also does not preclude the use of statewide school accountability systems, although it should influence their design.

7. Be reasonable

One of the biggest problems with school accountability systems has been unreasonable (i.e., ludicrous) timelines.

Reporting school accountability results in the fall, testing again in the early spring, and expecting improvement to occur in between is just one example.

Not differentiating between a new version of state content and achievement standards that requires minor changes and adjustments to instruction and a version requiring new curriculum, instruction, professional development, and appropriate time for implementation is another example.

8. Collect more data

Statewide school and student information systems are now in place that make it feasible to collect much more data, more frequently, unobtrusively, and to control the quality of that data. We have had some false starts in collecting individual student data, and new privacy and data safety issues arise every day, but this is a challenge that can and must be met.

9. Focus more on information and less on data

We must direct much more of the school accountability effort to processing raw data, including test results, into useful information for local educators and policymakers.

We must devote more accountability resources to recognizing patterns and relationships in the data, and then presenting that information in ways that allow people to apply their knowledge to act upon it.

10. Reach real agreement on fundamental principles in K-12 education

It is difficult (pretty much impossible, actually) to have real school accountability without a shared understanding on issues as basic as what it means to be in the third grade, or what factors will influence how long it will take an English learner to be able to function in a classroom without additional support.

For example, is there really an expectation that all students enrolled in third grade on the first day of school, including students with disabilities, will meet grade-level standards by the end of the school year (or by the day the state test is administered)?

11. Understand the difference between cross-sectional and longitudinal improvement, and then choose wisely

When the current era of school accountability began under NCLB everything was based on cross-sectional improvement out of necessity. Monitoring longitudinal improvement is now feasible. Which aspects of school accountability should be based on cross-sectional data and which on tracking student performance over time?

12. Paint me a picture of a high functioning school

Describe a high functioning school (or a low functioning school). If you cannot do that before you begin to design your school accountability system, you cannot design a school accountability system.

Your description can include indicators that might end up in a school accountability system (e.g., solid attendance, moderate to high growth).

You cannot, however, rely on the school accountability system to define a high functioning school; just as you cannot rely on the state assessment to define proficiency in English language arts or mathematics.

13. Begin again

With some states on their fourth accountability system since 2015, it may seem unnecessary to say that we must not be afraid to begin again. Most changes to accountability systems have been tweaks to the current rules or additions to the current system – the equivalent of finishing rooms in the basement and attic or adding on a new garage or screened-in deck.

There are moments in time when we have to be willing to tear down the house and build something new.

I think this is one of those moments. Of course, I am an assessment specialist. (See point 1 and begin again.)

Image by danny moore from Pixabay