assessment, accountability, and other important stuff

This Is My Fight Song

Arizona, Connecticut, and a fuchsia wristband

Charlie DePascale

 

Last weekend I attended a concert in Boston with my daughter; an opportunity that has become more rare and more appreciated since she left for college four years ago.  We arrived early and waited in line to hear Rachel Platten perform her breakout hit Fight Song in a pre-show soundcheck.  As we waited, a gentleman emerged from the nightclub and made his way down the line attaching fuchsia bands to the right wrists of everyone 21 years or older.  He proceeded past the group of high school girls ahead without a pause or word, walked up to me, and began attaching the wristband.   As he was leaving, my daughter, who looks much younger than her age of 22, held out her driver’s license.  He examined the ID intently for a good 30 seconds before returning it to her and attaching her wristband.

Why share this story?  How does it relate to Arizona, Connecticut, and the general subject of this blog?

At its core, this is a story about testing, and more specifically, testing for a particular purpose.

The man with the wristbands did not bother to test anyone who was too young (i.e., the high school girls ahead of us in line).  He also did not test anyone who clearly has been older than 21 for a very long time (i.e., me).  He did, however, spend a significant amount of time examining my daughter’s ID.  His test had a particular purpose – to identify people 21 years or older.  He also had clear directions on which type of error was more important.  He had little concern about not giving a wristband to someone who was actually 21 or older.  He was very concerned about giving a wristband to anyone under 21.

He did not check the ID of everyone standing in the line.  He did not give out different colored wristbands to those of us well above 21.  He did not ask those of us over 21 whether we intended to purchase alcohol during the show; he simply attached the wristband.  He also did not tell us anything that we did not already know about our own ages.  He was collecting data from us, not providing data to us.   However, he did not simply accept a self-report of how old we were; he required external verification (i.e., a state-issued photo ID).  When asked, he made it clear that he did not have any information about when the doors would open, when Rachel Platten would actually take the stage, or when the show would end.  He had a specific job to perform and he performed it efficiently.

Nobody in the line expected anything more or anything less from the man with the wristbands.  Why is it so difficult for us to view large-scale state assessment in the same way?

This is My Fight Song!

As I have addressed in several posts over the last year and in papers and presentations over the last decade, state assessments are best designed to serve a particular and limited purpose – to provide information to the state about the percentage of students meeting the state’s performance standards or perhaps to indicate whether an individual student has met those standards.  Well-designed assessments can perform those functions accurately and efficiently – in terms of time and cost.   State assessments begin to break down, however, when we ask them to do more than they are designed to do.  Complaints about tests that take too much time or tests that cost too much or tests that do not return any useful information to inform instruction and improve teaching begin when we ask state assessments to do more than what they are best suited to do.

For the last two decades, however, at every opportunity we have asked state assessments to do more.

  • To serve as a signal or model for good assessment and instruction in the classroom
  • To enable the measurement of student growth across years
  • To accurately and precisely measure the full range of student performance within a grade level
  • To measure the full depth and breadth of increasingly complex state content standards
  • To serve as an indicator of effective teaching

However, as Rachel said last Saturday while introducing Fight Song, “I hope that you understand, it is never too late to make something happen.”  So, this blog is my fight song.  It is my platform to continue to describe my vision of The Ideal Role of Large-Scale Testing in a Comprehensive Assessment System as I have done for the last fifteen years; because as the song says,

And I don’t really care if nobody else believes.

 ‘Cause I’ve still got a lot of fight left in me.

And perhaps there is a glimmer of hope.  Recently, Arizona and Connecticut announced changes to increase the efficiency of their state assessment programs, suggesting perhaps that there at least two states moving state assessment closer toward our image of the man with the fuchsia wristbands.  A closer look reveals, however, an important distinction between the two states’ views of the role and purpose of state assessment.

Arizona and Connecticut – so much the same, yet so different

In the last couple of weeks, state assessment programs in Connecticut and Arizona were in the news because of changes designed to reduce testing time, eliminate redundant or duplicative testing, and reduce costs.  In Connecticut, the Governor Malloy announced that the state would no longer administer the performance task portion of the Smarter Balanced assessment English Language Arts/Literacy assessment.  In Arizona, Governor Ducey signed legislation allowing school districts to replace the state assessment with another assessment selected from a menu of state-approved alternatives.  Both states cited a similar set of reasons for and benefits of the changes: reducing testing time, saving money, eliminating redundant or duplicative testing, and my personal favorite, allowing more instructional time for teachers and students.

On the surface, it may appear that Arizona has made the much more radical change.  Connecticut simply eliminated one portion of their English language arts assessment.  Arizona is allowing districts to completely abandon their state assessment in favor of other assessments.  Although it is certainly bold, the Arizona move to a menu of assessments may be much more in line with the fundamental purpose of state assessment than Connecticut’s decision to shorten its test.

In announcing the change in Connecticut, Governor Malloy emphasized that the eliminated essay portion of the test was duplicative with in-class work.  It did not provide teachers with information that they did not already have and therefore should be eliminated.  In this rationale, the focus of the state assessment is on providing information to schools and teachers to improve teaching quality.

In contrast, the change in Arizona requires any alternative test that is “adopted in Arizona will have to maintain or exceed the rigor in Arizona’s College and Career Ready Standards” and it appears that the alternatives will still have to be external tests.  There is discussion of high school students who have already clearly demonstrated the desired level of proficiency through their performance on recognized tests such as the ACT, PSAT, or the Cambridge exams. In this scenario, the focus of the state assessment remains on providing an external check, an audit, of the information that is generated within the school.

I would argue that the intent of the change to state assessment in Arizona is much more consistent with the ideal role of state assessment.  Yes, it will be a challenge to ensure that the alternative tests are aligned with the state’s content standards and maintain the rigor of the state’s performance standards.  Yes, it will be more complicated to compare performance across school districts with multiple assessments than it would be with a common state assessment.  Yes, there will be opportunities for inadvertent errors or intentional attempts to game the system with less rigorous assessments.  Addressing all of those challenges is worth the effort, however, because Arizona is attempting to find a more efficient way to provide the state and schools with the information that should be provided by the state assessment – a credible, external indicator of student achievement.

The move by Connecticut to eliminate the performance task from the Smarter Balanced English Language Arts/Literacy test presents fewer obvious challenges, but comes with much greater risk of perpetuating one of the biggest flaws of traditional standardized assessment programs – sacrificing validity in the name of reliability and increased efficiency.  It is not written in stone that changing the Smarter Balanced test blueprint will produce scores that are less valid, useful, or predictive of future performance.  It is not a foregone conclusion that eliminating the authentic writing portion of the assessment will have a negative impact on instruction in the classroom as teachers see what is valued and what is not valued on the state assessment.  A long history, however, tells us that simply making the high-stakes accountability test shorter and more efficient is not the answer.  I trust that policymakers in Connecticut intend to ensure that the full range of the state’s academic standards continue to be taught well in schools across the state.  Like their colleagues in Arizona, however, they will have to realize that announcing this change is just the first step in a process that is filled with risks and challenges to be overcome.

Like a small boat
On the ocean
Sending big waves
Into motion
Like how a single word
Can make a heart open
I might only have one match
But I can make an explosion

%d bloggers like this: