A cautionary tale

Charlie DePascale

Earlier this month I traveled to Lawrence, Kansas to attend the NCME special conference on the confluence of classroom assessment and large-scale psychometrics. In a panel discussion titled, “I’ve a feeling we’re not in Kansas anymore” Kristen Huff, Karen Barton, Paul Nichols, and I shared the perspective that when bringing together classroom assessment and large-scale psychometrics, co-existence might be a better goal than confluence.  Our goal  was to engage in a critical discussion about those differences between the purposes and uses of classroom and large-scale assessment that give each unique measurement and psychometric needs. To accompany that discussion, we offered the following look over the rainbow at what might result if we blindly attempt to apply large-scale psychometrics in the classroom.

 

Gillikin County Schools Embrace Classroom Psychometrics

Emerald City –  (September 13, 2017)

As a new school year opens, teachers and administrators in Gillikin County, just north of Emerald City, are clicking their heels with excitement over the impact of their use of a new process called Classroom Psychometrics. Some call it science, others magic, but all agree that it has changed the way that they look at teaching and at their students.

Superintendent Glinda Marvel describes how the district made the leap to classroom psychometrics

Our teachers have been hooked on data since we introduced TestWiz ™ back at the turn of the century.  Over time, however, they found that the data wizard just did not provide them with enough information – did not let them see what was going on behind the curtain.  Then last year, this new computerized adaptive test (CAT) system – TOTO is dropped in our laps.  Now that’s a horse of a different color.

At first, all of the thetas and sigmas were just Greek to me. And many of our teachers were fearful.  They didn’t understand how test questions that discriminate and just getting rid of the misfits could be a good thing.  But then we saw that with TOTO none of our little munchkins ever have to get a question wrong.  The little lights on their individual dashboards are always green.

S. C. Strawman, doctor of thinkology and president of Emerald Associates, explains that the beauty of classroom psychometrics is that it untangles the messy webs one often encounters with learning progressions and popular learning map models – “once teachers realize that the manifest monotonicity of classroom psychometrics is their students’ manifest destiny not even a horde of flying monkeys can pull them off course.”

Or as Castle High School teacher, Almira Gulch put it, “understanding that everything can be explained by just one dimension made our lives so much easier.”  And principal Frank Marvel added, “accepting that there will be some kids who just don’t fit the model no matter what we do, took so much pressure off of our teachers.”

Of course, not everyone is sold on TOTO and classroom psychometrics. Dorothy Gale, longtime school board member, summed it up this way

Well…I think that it’s just that … Sometimes we go looking over the rainbow for the answer when it was really right there in our back yard all along.  Classroom Psychometrics really isn’t telling us anything new about our classrooms and students.  We just needed a little help seeing what was already there right in front of our noses.