Do Your Job ™

Thinking again about the role of state assessment programs

Charlie DePascale

This week, most of the nation is focused on tournament brackets and one type of madness in March.  Some of us, however, are also celebrating our own special version March Insanity. Over the next twelve weeks, millions of students across the country will participate in their state assessment program.  Current high school seniors were in first grade when annual testing in grades 3-8 and high school began under NCLB.  This feels like an appropriate time, therefore, to pause a moment at the beginning of another season of K-12 State Assessment to ask ourselves a simple question: Why?

What is the purpose of state assessment programs?  Or perhaps,

What aspect of the state’s role in education is being fulfilled or supported by the annual administration of a state assessment?

For the purpose of this discussion, let’s eliminate any answers that simply are related to compliance with federal laws; that is, states administer assessment to meet the requirements of ESSA, NCLB, IASA, …, all the way back to the original 1965 ESEA.

Let’s also take a step back from rhetoric about the “Honesty Gap” and schools lying to parents and children about student achievement.  If we need state assessments to ensure that schools are not lying to parents about student achievement, and national assessments to ensure that states are not lying, and international assessments to ensure that nations are not lying….  I just don’t want to go there.

Compliance with federal laws and providing accurate information to parents about student achievement are reasons that answer the “why” question at a superficial level, but don’t really address the fundamental question about the role that state assessments play in helping a state department of education fulfill its mission.

What is the mission of a state department of education?

The United States Department of Education provides a clear description of its mission and the four strategies it uses to achieve that mission:

ED’s mission is to promote student achievement and preparation for global competitiveness by fostering educational excellence and ensuring equal access.  [ED’s employees and budget] are dedicated to:

• Establishing policies on federal financial aid for education, and distributing as well as monitoring those funds.
• Collecting data on America’s schools and disseminating research.
• Focusing national attention on key educational issues.
• Prohibiting discrimination and ensuring equal access to education.

 Many states have similar mission statements about promoting student achievement through a commitment to excellence and equity.  How did mission statements and strategies such as those listed by USED result in our current state assessment reality?

It is easy to imagine scenarios that begin with state policy makers identifying the need to collect data on how districts and schools are performing.  Various options for collecting achievement data are proposed. The lack of a common curriculum across schools and a lack of trust of in achievement information self-reported by teachers and administrators are mentioned as problems.  Someone suggest administering a common assessment statewide, and away we go.

I am sure that there was a time when administering a common assessment to students throughout the state was the most effective and efficient solution to the data collection problem policy makers were trying to solve.  Such assessments were quick, inexpensive, machine-scored, and would provide a common metric against which to compare schools.  Problem solved.  Until it wasn’t.

Due to a variety of well-documented factors, the assessment world started to unravel in the late 1980s.  Concerns about norm-referenced standardized tests and a renewed interest in excellence combined with changes in state content standards and performance expectations led to a call for more authentic, innovative standards-based assessment.  Traditional state assessments were not adequate to measure student achievement on new content and performance standards.  Attempts to implement new, performance-based assessments, however, did not meet standards for technical quality in educational measurement.   Houston, we have a measurement problem.  The solution to the measurement problem was to build better (i.e., more technically sound) assessments. And so we did.

But what happened to the original data collection problem that policy makers were trying to solve.  As state assessments continued to grow more complex, more time consuming, and much more expensive, why didn’t we return to the original problem and conclude that a state assessment might no longer be the most effective and efficient way to solve our data collection problem?  At some point, it may have been prudent to seek an alternative solution.

Remember, a quick-and-dirty assessment to measure the achievement of students statewide was the solution to the data collection problem.  The problem was never to find an efficient and effective way for the state to measure the achievement of all students statewide.  We have school districts and school building staffed with highly qualified educators (certified by the state) to do that job.

Do Your Job ™

A mantra of the 5-time Super Bowl ™ champion New England Patriots is that each player, coach, and team official must “Do Your Job” – do the job that you have been assigned and do it well. A state assessment will never be able to measure student achievement as well as a competent teacher. That is not its job.

Over time state assessments were also asked to perform a myriad of other jobs such as:

• signaling what should be taught in schools and how it should be assessed,
• providing actionable feedback to students and information to teachers to improve student learning
• evaluating local educators,
• indicating student readiness for college and careers, and the aforementioned
• serving as the primary tool to monitor the use of federal funds, and
• providing honest information to parents and other stakeholders about student achievement.

This increased dependence on state assessments, of course, is at least part of the reason why we did not attempt to find a better solution to the original data collection problem.  Alas, we know all too well what happens when a person – or a state assessment – takes on too many jobs.

A Glimmer of Hope

To end this post on a positive note, there is a glimmer of hope that states may be making a move away from measuring student achievement and back toward solving the data collection problem.  And because the assessment gods have a well-honed sense of irony, the vessels for this shift are college admissions exams – the ACT and SAT; yes, those symbolic Satans, the very exams that the next generation high school state assessments were designed to make obsolete.  Rather than developing their own state assessment to measure student achievement against state standards, states across the country are turning to the ACT and SAT as the most effective and efficient method to collect data on the extent to which districts and schools have prepared college-ready students.  (Career-ready may have to wait.)

Thinking a little more deeply about the purpose and function of college admissions exams such as the ACT and SAT, states’ decision to use those assessments makes sense.   In general, studies have found that the college admissions tests and high school grades function similarly well in predicting future success in college. The SAT and ACT, however, provide a common metric to allow comparisons of performance across schools. They provide a means for colleges to quickly and efficiently collect data needed to calibrate information on a student’s transcript.  This is particularly important to students attending high schools with which a college might not have enough of a history to evaluate a student’s transcript with confidence.

Like colleges, states are seeking and finding an appropriate tool to collect data about student achievement that is comparable across schools.  States are not seeking to replace a student’s transcript and nobody is making the claim that assessment results provide as complete a picture of student performance as a student’s transcript.    The college admissions tests are an efficient and effective way for states to collect one piece of important data about schools.  And that, after all, is their job.