New Year’s Resolutions

Charlie DePascale

“If the body be feeble, the mind will not be strong. The sovereign invigorator of the body is exercise, and of all the exercises walking is best.” – Thomas Jefferson

The third week of 2017 has come to an end, and my New Year’s Resolution to improve my fitness level is off to a good start (#LeanIn17).  Of course, this is not the first time that my fitness resolution has looked good after three weeks in January.  I fondly recall the fitness program that I developed as my class project in Behavioral Psychology in 1983; the HealthRider and Covert Bailey books I bought in the 1990s; and of course, the pedometer that I have carried with me religiously since April 16, 2010.  Some of my fitness efforts over the years have been rather successful – in the short term.  Yet here I sit in January 2017, the same age that my father was when he had his first cardiac procedure, resolving once again to get fit before it’s too late (#2Big2Fail).

As I ponder my three decades of fitness programs, I cannot help but draw connections between my latest attempt to develop and implement a successful fitness program and states’ across the country feverishly working to develop test-based school accountability programs that meet the requirements of ESSA. Like my previous efforts at fitness, I fondly recall the cycles of accountability programs I have been involved in over the last 35 years:

• As my career in assessment began in the 1980s, comparison score bands were the rage – a regression-based approach in which a school’s performance was compared against schools with similar background characteristics (e.g., kind of community, poverty level, socioeconomic status).
• In the 1990s, the focus shifted to criterion-based performance levels. Relative comparisons against similar schools were replaced by absolute comparisons to standards for Basic, Proficient, and Advanced performance.
• The 2000s brought us NCLB and its nominal goal of all students ‘Proficient’ in reading and mathematics by 2014. The 2000s also brought us growth metrics and value-added models – which in many ways were a return to the relative comparisons against similar schools of the 1980s (but that’s our little secret).

And here we sit in January 2017, trying once again to rearrange a narrow set of available indicators to form a valid, reliable, and fair school accountability system that will produce actionable information that will lead to equity, improved student learning, college- and career-readiness, et cetera, et cetera, et cetera.

To this point, my attempts at fitness have fared no better than schools’ attempts to meet the 100% Proficient goal of NCLB.  However, as I considered the similarities between the two endeavors, I also noticed several critical areas in which the attempt to improve fitness have a distinct advantage over school accountability:

• There is a known intervention.
• There is a simple proxy measure.
• The relationship between proxy measures and desired behavior is well-established.
• There is a known desired outcome or goal.

There is a known intervention

At the most basic level, increasing activity level and eating right will inevitably result in improved fitness.  As Pres. Obama reminded us over the last few weeks, ‘progress is not a straight line,’ but if we stick with the program of increased activity and improved nutrition, it is inexorable.

Clearly, knowing the solution does not guarantee success – or I would be writing about something else today.  Knowing the solution, however, does free up time and mental energy to focus on important contributors to success, such as a) developing effective ways to implement the intervention, b) designing efficient ways to monitor implementation, and c) identifying and overcoming obstacles to implementing the intervention.

With regard to school improvement, the resources that we have to continue to devote to either determining  What Works or gaining consensus on what works necessarily detracts from our efforts to design and implement effective improvement programs on a large scale.

There is a simple proxy measure that is easy to monitor.

In fitness programs, weight can serve as an effective and easy to monitor proxy measure.  As I increase my activity level and improve my nutrition, I will lose weight.  As I will discuss in the next section, using weight as a proxy for fitness has its dangers, but under appropriate conditions, it works quite well. With nothing more than a bathroom scale, it is quite easy to monitor weight on a regular basis in the comfort of your own home.

To a large extent, bathroom scales defy the wisdom of the old saying, “We offer three kinds of services: good, fast, and cheap. Pick two.” You can pick up a pretty good scale at a modest price and spend no more than 30 seconds checking your weight (not counting the time preparing yourself emotionally to step on the scale or the time spent moving the scale around the bathroom to get a getter result).  No, your personal scale will probably not give you the same reading as those mean scales at your doctor’s office or the gym, but the results are close enough and the relationship is usually pretty constant.

In education, virtually all of our creative energy and an enormous amount of money over the last fifteen years have been spent trying to develop a better scale.  Really, it’s true.  School accountability systems and state assessments are simply the bathroom scale in the process of school improvement.  And it is not clear whether we have yet to achieve even one of the three options: good, fast, or cheap.

The relationship between proxy measures and desired behaviors is well-established

A proxy measure is defined as “an indirect measure of the desired outcome which itself is strongly correlated with the desired outcome.” As I mentioned above, there are some dangers in using proxy measures.  There are ways to lose weight that are not the result of increasing activity and eating right, and those methods generally do not lead to improved fitness.  The 60 billion dollar weight loss industry serves as Exhibit 1.

Nonetheless, the relationship between weight loss and behaviors associated with improved fitness and ness program and is well-established.  At the personal level, individuals should certainly focus on implementing effective behaviors and monitoring a wide variety of indicators of improved fitness.  As a quick check on the effectiveness or progress of an implemented program, however, weight serves a useful purpose.

In education, the relationship between the score on an assessment or an accountability rating and the desired behaviors that lead to improved school quality is not quite as clear or direct as with fitness and weight. Yes, there is a general tenet/belief/hope that a good curriculum and effective instruction aligned to clear and rigorous content standards will result in improved student learning; and improved student learning will be reflected in increased proficiency and growth on state assessments and better accountability ratings.  To this point in time, however, we have not succeeded in establishing that relationship.  Perhaps it is measurement issue – we need to build better systems.  Perhaps it is a communication issue – we need to do a better job explaining and demonstrating how desired behaviors will lead to improved ratings.  Perhaps it is simply an issue of stability and time – it is not uncommon for a state to have had three sets of content standards, three different state assessment programs, and made significant changes to its accountability system in the last 10 years.

Perhaps, however, part of the problem is that we have not defined clear goals

There is a known desired outcome or goal.

In fitness, there is at least enough agreement about what constitutes fitness to elicit general consensus about an individual’s level of fitness.  There is also a clear understanding that fitness is not necessarily an end in itself, but is a prerequisite for the long-term proper functioning of most of our organs and systems.

In education, however, there is still a lack of clarity over some of our most basic goals.

We would like all students to graduate from high school college- and career-ready.  But, …

• What do we mean by all students?
• What kinds of colleges and what kinds of careers are acceptable?
• Is it really college- and career-ready or is it college-ready OR career-ready?
• Are we OK with the outcome that some kids will be college-ready and that some kids will be career-ready when finishing high school?
• It’s probably not OK if one subgroup of students is career-ready, but not college-ready. Is it?

There seems to be agreement that literacy and numeracy are critical basic skills for all students. But, …

• What do we mean by literacy and numeracy?
• Do the Common Core State Standards or other state standards define the required literacy and numeracy skills?
• Is there a common level of literacy and numeracy that is needed by all students regardless of their intended career pathways?

And then there are the big questions –

• Do we know what it will take to ensure that all students have acquired sufficient literacy and numeracy skills to be college- and career-ready?
• Are we prepared to commit the necessary resources to achieve that outcome?

 

Looking ahead

At this point in time, I have stepped on the scale enough times to know that I am not fit; and we have administered enough tests to know that student achievement is not where we want it to be.  In both cases, it’s time to move on to the next step.

Looking ahead to January 2018, I hope to be leading a more active lifestyle, be eating better and to have made significant progress toward becoming fit – and to weigh much less than I do now.  I also hope that as states have settled on their basic school accountability measures (i.e., bathroom scales), we can begin to clarify the desired outcome and focus on the desired behaviors and interventions that lead to that outcome.