Category Archives: Measurement

Give NAEP A Chance

Followers of this blog over the past few years may recall that I have been just a tad critical of NAEP, on occasion, for their interminable process of analysis and reporting, and their obsessive clinging to a trendline that serves as both their raison d’etre and a noose around their necks.   In this through-the-looking-glass year,Continue reading “Give NAEP A Chance”

Measure Twice, Model Once

As states and their assessment contractors prepare for the possible resumption of state assessment programs in spring 2021, one critical question being considered is how, or even whether, to link results from the spring 2021 tests to the last-administered spring 2019 tests; or to put it more generally, what equating procedures should be used toContinue reading “Measure Twice, Model Once”

The Stakes are High, The Water’s Rough

In the wake of the events of 2020, the assessment and measurement community has made a commitment to do the right thing to ensure that tests are used appropriately.  The community has vowed to be more proactive in speaking out against policies and test uses that result in inequitable outcomes or consequences, while actively promotingContinue reading “The Stakes are High, The Water’s Rough”

Whose Job Is It, Anyway?

Whenever I come across the story Whose Job Is It, Anyway?, I cannot help but think of validity and K-12 large-scale assessment.  Who among us has not sat through countless TAC meetings where the mere mention of the lack of validity evidence in the technical report results in blank stares and handwringing; or in recent years, listenedContinue reading “Whose Job Is It, Anyway?”

The Butterfly Effect

This is the second in what unexpectedly became a series of posts on validity, validation, state tests, and state assessment programs. In these first two posts, I am focusing on the primary purpose of state tests and interpretation of student test scores as a measure of student performance on the state’s content standards; that is,Continue reading “The Butterfly Effect”

Even Validity Has Unintended Consequences

While the educational measurement community has devoted an uncomfortable amount of energy to debating the proper role of consequences in validity, I would argue that we have paid far too little attention to the unintended negative consequences of the Validity Standards on the validation of K-12 large-scale state assessments and state assessment programs.  We seemContinue reading “Even Validity Has Unintended Consequences”

I Can See The Writing on the Wall

Organizations, institutions, and individuals who have been called to educational measurement and/or educational assessment as their pathway through life have joined others this summer in an exercise in introspection and self-reflection. The goal of this exercise is to emerge with a better understanding of how we, through our life’s work,  have contributed to, endorsed, promulgated, perpetuated,Continue reading “I Can See The Writing on the Wall”

Too Much of a Good Thing

If asked to identify the biggest successes of the Education Reform movement over the past two decades I would have to put selling the importance of disaggregating data at or near the top of my list.  Acceptance and adoption of the practice of disaggregating data is well beyond what one might expect from mere complianceContinue reading “Too Much of a Good Thing”

Equating 2020 – Opportunity Lost

For a brief moment last spring the disruptions caused by COVID-19 appeared to offer a once-in-a-lifetime teachable moment in the form of an opportunity for real-time discussions with stakeholders about the concepts and assumptions that are at the heart of equating state assessments. Alas, much like people’s understanding of equating, that opportunity proved to beContinue reading “Equating 2020 – Opportunity Lost”