assessment, accountability, and other important stuff

Archive for August, 2018

Ten Years of Taylor

Ten Years of Taylor cropped

Charlie DePascale

Ten years ago, August 22, 2008, my daughter and I attended our first Taylor Swift concert in Hartford, Connecticut. The original plan was to attend the concert a few weeks later in Massachusetts – a bit closer to our home in Maine.  About to start high school, however, she was a bit worried (i.e., panicked): what if I have too much homework …  So, we made the late August trip to Hartford.  (The September concert was our second concert.)

This summer, my wife and I visited our daughter in Maryland.  On a very hot and humid night at FedEx field, the three of us attended my 30th Taylor Swift concert.

There are so many memories across 10 years and 30 shows – from that 8-song set opening for Rascal Flatts to the 2-hour reputation Stadium Tour show.

With my daughter –

  • Finding a public library with wifi close to my meeting in Rhode Island, waiting for Fearless tickets to go on sale and then sending her the two-word e-mail, We’re In!
    • Later that summer driving home from that concert in Connecticut through a tropical storm.
  • Eating lunch in the Wesleyan dining hall before driving home for Christmas and deciding, sure we can make the drive to Philadelphia for a concert during spring break; then buying tickets when they went on sale at noon.
  • Foxboro in the rain (more on that later)
  • Stopping in DC for a concert on the way to college visits in North Carolina and Virginia
  • Driving to Tanglewood on the rumor that Taylor would perform at the James Taylor concert. (she did)
  • Capping off a family trip to Colorado, Wyoming, and Mt. Rushmore with a concert in Denver
  • Walking around Georgetown last November taking pictures with ‘reputation’ UPS trucks.

And on my own –

  • Trips driving around North Carolina for back-to-back shows in the Raleigh – Greensboro – Charlotte triangle (extended to Charlottesville/UVA and back-to-back-to-back shows in 2013)
  • The 20-hour visit to NYC via Amtrak during Thanksgiving week for a concert at Madison Square Garden – with surprise guest James Taylor
  • Combining concerts in the Twin Cities with visits to old friends from the University of Minnesota

There are two memories, however, that will always stand out above the rest.

Taylor in Maine – aka Hello, Boats!  (August 27, 2010)

DSC09279

Hello, Boats!

Taylor was in Maine for the premiere of the video for Mine, to be broadcast that evening on CMT. The radio was full of Taylor Sightings and rumors about the location of the secret broadcast. I was spending one of the last Fridays of the summer with the family.  At the end of lunch, I decided to turn on my laptop to check my e-mail – no smartphone at that time.

There in my Inbox was an e-mail from Taylor Nation with the subject line “You’re Invited: Taylor’s CMT Taping in Maine”.  The e-mail contained instructions about how to dress/behave for a live television show, and directions to a school in Kennebunkport where a bus would take guests to/from the still unnamed location of the taping.

This had to be some sort of prank.  Sure, I lived in Maine, had attended a few concerts, and had already purchased way too much merchandise, but still…   But what if it’s real?  All we had to lose was the time for a short drive to Kennebunkport.

The e-mail didn’t mention anything about bringing a guest.  If it’s real, will it work for two people?  We made a plan and my daughter and I drove to the pickup location.

The good news, there was a bus and a small group of people.  The bad news, all of the other people seemed to have had some involvement in the taping of the video earlier that summer – as extras, volunteers, or staff in places Taylor visited.  There was a local person with a check-in list.  Of course, we weren’t on her list and she knew nothing about the e-mail invitation. Thankfully, after reading the invitation and looking at my daughter she told us to get on the bus!

 

DSC09240

 

While the bus was on its way, the secret location was announced on the radio.  When we arrived, there was a crowd lining the street and driveway leading up to the seaside property where the show would take place.  With orange wristbands secure, we were led through the crowd to our designated area on the lawn.  We were told what to expect and how to react during the run through without Taylor, the rehearsal with Taylor, the live show, and that there would be a small concert following the show.  We were told that some of George H.W. Bush’s grandchildren were there and that the former President would arrive soon.

I stayed off to the side and watched Taylor arrive, and then a bit later, President Bush.  My daughter, being better able to fit in small places worked her way up toward the front of the crowd.  It all went perfectly.  Taylor came out for the interview, turned and waved hello to the crowd of boats off shore, and the video premiered.  After a short break Taylor returned with her band returned and performed a short concert.  What a night!

Foxboro in the Rain (June 25, 2011)

DSC01404

I‘m not a fan of stadium shows, but I have attended five of Taylor’s ten concerts at Gillette Stadium (only half, clearly not obsessed). Each Gillette show has been a memorable experience, but none will ever match Foxboro in the Rain.

My daughter and I didn’t sit together.  I had a seat on the floor near the stage. She didn’t want to sit on the floor, so she had a seat near the top of the lower level – in what fortuitously was one of the few covered rows at Gillette Stadium.

For me, the two highlights of the night were Mean and the rain.  Coming into the show, I didn’t understand how strongly young fans related to Mean.  Many of Taylor’s songs were autobiographical, but Mean was the first that was not a generic experience. It was explicitly about her and the claim that she couldn’t sing – “drunk and grumbling on about how I can’t sing.”  I thought that the direct reference made the song less relatable. Apparently, however, the magic of Mean is that direct connection to Taylor’s experience.

Walking around before the show we saw a sea of ‘why you gotta be so mean’ t-shirts and small groups singing the song all around the stadium.  And then the concert; sitting in front of the stage and hearing 50,000 young voices singing out strong

But someday I’ll be living in a big old city
And all you’re ever gonna be is mean, yeah
Someday I’ll be big enough so you can’t hit me
And all you’re ever gonna be is mean
Why you gotta be so mean?

It was an overwhelming experience.

DSC01050

And then there was the rain.  It started out as a light shower over part of the stadium.  My daughter thought it was part of the show – Oh, they even have fake rain. There had already been fake snow earlier in the show (Back to December). It all fit together.  No, the rain was real.  It rained hard and it kept raining.  Taylor kept singing, dancing, and lying in puddles of water on the stage when the choreography called for it.  And fans kept singing, dancing, and taking pictures with their iPhones. This was so different than the time I was soaked to the skin in my wool band uniform at a Harvard-Cornell football game.

(I was amazed that the phones were not damaged by the rain.  Reading the online forums the next day, many of them were.)

As it turns out, it’s not a good idea to lie in puddles of water singing in the rain. Within a week, Taylor was sick and had to postpone shows.  I was there for her first show back in Montreal in mid-July, but that’s another story…

It Was The Start of a Decade…

When my daughter and I drove to Hartford in August 2008 I had no idea it was the start of a decade of 30 shows in eleven states plus Washington, DC plus Montreal.   The first time I listened to her first album, ‘Taylor Swift’, I knew these were great stories and this was a great storyteller. When I heard Fifteen those were words I wanted to tell my daughter.  And when I heard The Best Day those were words I hoped to hear from my daughter someday.  And when I heard All Too Well, well ….

So, next weekend I will sit in U.S. Bank Stadium in Minneapolis, sobbing through Long Live one more time.  And if fate steps in and there are no more Taylor Swift tours or concerts for me after this summer, I know that for as long as I live, and I hope as long as my daughter lives, these last ten years will be remembered.

 

 

 

 

 

Rebranding Educational Measurement

Charlie DePascale

When I think about educational measurement the first thing that comes to mind is a high-fructose corn syrup commercial from about 10 years ago.

 

 

On one side there is the man who holds, but cannot articulate, the widespread, but ill-defined, perception (misperception?) that high-fructose corn syrup is inherently bad.  On the other side is the woman with the tempting treat who provides a couple of carefully selected facts and makes the claim that high-fructose corn syrup is fine in moderation.  Man takes the treat from Woman and all is right in their world 30-second commercial world. It is truly an Adam and Eve moment – although that’s probably not the allusion the sponsors of the commercial were after.

In 2018, as a field and as an industry, educational measurement finds itself in much the same place as high fructose corn syrup.  We developed an appealing, inexpensive product (i.e., large-scale standardized tests), exulted in its success, and then could do little when we lost control of the product that defines us.   For much of this century, we have taken the role of the woman in the HFCS commercial.   We ensure people that there is nothing harmful or evil about educational measurement used properly and in moderation; all the while watching test use soar beyond anything that can be called moderation. Assuming that it will be impossible to produce a cute 30-second video on the benefits of educational measurement that will be as effective as the counterargument that John Oliver has already produced, where do we go from here?

I think that the only solution is to engage aggressively in rebranding. Educational Measurement is ripe for a makeover or perhaps even a complete do-over.  Now is the time to change not only the surface image of educational measurement, but to actually change what we mean when we talk about educational measurement.

Two assessment industry icons have already started this rebranding.  College Board began by changing the name of the SAT (much like KFC), conducted a major overhaul of their flagship instrument, and then created a new suite of products and services aimed at a new market.  ACT has gone even further as it redefines itself as a learning company rather than an assessment company.  As described in an EdSurge article earlier this year, ACT CEO Marten Roorda “wants the ACT to become more involved in the learning process, and provide more analytics solutions to teachers and students. “

Reforms to the ACT and SAT assessments, of course, are just the tip of the iceberg. Learning analytics, big data, personalized instruction, and adaptive learning are trending topics in education which are already impacting the measurement community.  At the ITC 2018 conference in July, John Hattie and Alina von Davier delivered keynote addresses on visible learning and computational psychometrics, respectively, which forced those listening to reconsider how we think about and do educational measurement.  As Kathleen Scalise explained at the 2018 NCME conference in April, it is not a question of if or when big data and learning analytics will impact educational measurement, they are already here and they already have.

Back to our roots

We must begin any attempt to rebrand, redefine, or refocus educational measurement by revisiting our roots.  And the best place to reconnect with those roots is the first edition of the so-called bible of our field, Educational Measurement, published in 1951.  We choose this as a starting point because as explained by E.F. Lindquist (editor), “prior to its publication … no book had yet been published that would even begin to fill an urgent need…for a comprehensive handbook and textbook on the theory and technique of educational measurement.”

It can also be argued that among the four editions of Educational Measurement (1951, 1971, 1989, 2006), the initial edition made the best attempt to ask and answer the Why? Question; that is, to define the purpose of educational measurement.  It is from understanding the purpose of educational measurement that we are able to glean the core values and guiding principles of our field which is the first step

In Part 1, The Functions of Measurement in Education, the 1951 edition begins with four chapters that address fundamental issues related to the primary functions of measurement in education at that time:

  • The Functions of Measurement in the Facilitation of Learning
  • The Functions of Measurement in Improving Instruction
  • The Functions of Measurement in Counseling
  • The Functions of Measurement in Educational Placement

Part 3, Measurement Theory, begins with a chapter on The Fundamental Nature of Measurement that ends with a section titled, Explanation as the End of Measurement, and the following admonition:

The primary concern of measurement, however, should be for an understanding of the entire field of knowledge rather than with statistical or mathematical manipulations upon observations.

Knowledge will be advanced by recognizing what the empirical methods of measurement ignore…The aim of measurement must ever be the explanation of, or the meaning for, observed phenomena.

A practical application of those statements is provided by Ralph Tyler in describing the organization of his chapter on the functions of measurement in improving instruction:

Since the purpose of this chapter is to outline the ways in which educational measurement, that is, achievement testing, can serve to improve instruction, we shall consider first what steps are involved in an effective program of instruction and then indicate the contributions that achievement testing can make to each of these steps.  In this connection it will be noted that educational measurement is conceived, not as a process quite apart from instruction, but rather as an integral part of it.

Tyler then goes on to describe four sequential phases of instruction

  1. To decide what ends to seek; that is what changes in student behavior to try to bring about
  2. To determine the content and learning experiences likely to attain those ends
  3. To determine an effective organization of those learning experiences to bring about the desired ends effectively and efficiently
  4. To appraise the effects of the learning experiences to determine whether they have brought about the desired ends or changes in student behavior.

He argued in 1951 that educational measurement as a field had become stuck on Step 4 (documenting the effects of instruction); not focusing enough on how educational measurement can and should inform and support the other three steps of instruction.

This problem has only been exacerbated in the last 60+ years as our field has become more technical, more specialized, and more separated from instruction.  While we may pay lip service to the notion that a key purpose of educational measurement is to facilitate learning and improve instruction, we do little to understand and support that function.

Educational measurement must find a way to support all aspects of the instructional process; toward the ultimate goal of improving student learning.  And having taken on that task, we must find a way to convey the message that measurement is more than a test of student outcomes.

Rebuilding and Rebranding

Obviously, rebuilding and rebranding educational measurement will not be simple.  It will require more than a quick fix like a 30-second commercial, a catchy new slogan, or a name change. However, although not sufficient, I do think that a name change is necessary.  The term ‘educational measurement’ is too closely associated with achievement testing to continue to serve a useful purpose.  Additionally, it does not accurately reflect either what we have been doing as a field for the last 60 years or the new directions in which the field is moving (e.g., with a focus on personalization and computational psychometrics).

My suggestion for a starting point is to replace the term ‘measurement’ with ‘modeling’ – Educational Modeling.  What is the case for modeling? Just for starters …

  1. With a few notable exceptions, modeling is a much more accurate description of what we do as a field than measurement. (Yes, I see you out there Rasch folks.)
  2. By its very nature, the term modeling conveys a sense of concern with an entire process or an entire system and the interactions among the components of that system.
  3. Measurement, not just educational measurement, is an outdated 20th century concept. The 21st century world is just much too complex to measure. Our field, and psychology in general, latched on to the term measurement last century because it was cool and gave the field credibility. Modeling is the new measurement.
  4. Finally, through the Common Core State Standards (and its offspring) we have invested nearly a decade in spreading the word to K-12 educators, students, and the general public of the importance of modeling and its central role in all that we do as intelligent human beings.  Let’s take advantage of that and create coherence between what we say and what we do in education.

The Common Core defines modeling as “the process of choosing and using appropriate mathematics and statistics to analyze empirical situations, to understand them better, and to improve decisions.”  That sounds like what we are doing (or should be doing) in educational measurement.  In further describing modeling, the Common Core further states “Real-world situations are not organized and labeled for analysis; formulating tractable models, representing such models, and analyzing them is appropriately a creative process. Like every such process, this depends on acquired expertise as well as creativity.”

Again, isn’t that what we are supposed to be doing in educational measurement?

Let the games begin

To get this ball rolling, I call on NCME, consistent with their vision to be the recognized authority in measurement in education, to take the first step by changing their name to the National Council on Modeling in Education.  They won’t even have to change their logo, URL, or Twitter handle,

The next step would be for NCME and co-editors Linda Cook and Mary Pitoniak to make the upcoming 5th edition of our bible, Educational Measurement, a New Testament for our field.  Educational Modeling has a nice ring to it as a title.

We have to start somewhere to restore the reputation of educational measurement.

Are you ready for it?