When you spend your career in large-scale testing, it’s easy to fall into an assessment-centric view of the world of public education. Every so often, I find in necessary to remind myself that that we play only a supporting role, and a minor one at that, in the vast enterprise that is public education.
To do so, I like to review some numbers that put things in perspective for me. An Education v. Assessment Tale of The Tape, if you will. In this post, I look at four categories: Professional Organizations (AERA v. NCME), People, Money, and Time.
Disclaimer: All numbers presented are approximate and were gathered during an exhaustive and exhausting 90 minutes of searching aided by Google and its AI Mode assistant.
Professional Organizations
There are countless professional organizations with members dedicated to some aspect of education, in general. There are also a fair number of organizations whose members focus on measurement and/or assessment such as the International Test Commission, Association of Test Publishers, and Psychometric Society to name a few.
For our purposes today, however, let’s just focus on two, AERA and NCME, as solid representatives of their fields.
| AERA | NCME | |
| Members | 25,000 | 1,800 |
| Conference Attendees | 14,500 | 1,200 |
| Conference Sessions | 2,500 | 150 |
People
For the sake of simplicity and to apply something of a “mercy rule” I decided to restrict the number of people in education to K-12 teachers. The number of teachers, apparently, are fairly equally distributed between elementary and secondary schools.
| Teachers | Assessment Professionals |
| 3.8 – 4.0 million | 30,016 |
Estimates of assessment professionals vary and have been quite fluid over the past couple of years (thanks ETS), but it appears that five largest testing organizations in the United States employ approximately 28,500 people. Include Smarter Balanced, TAC members, and a few other testing companies waxing, waning, or just hanging on and that number approaches 29,500. If we include independent consultants, a couple of think tanks, and a few econometricians who have made assessment their calling, we approach 30,000. Toss in the dedicated testing professionals at my former home, the Center for Assessment, and our estimate of the number tops out at 30,016.
Money
Ah, money the root of all evil and that which makes the world go around. Here’s where the rubber hits the road. It’s a fact that there is a lot of money spent each year on standardized testing in the US. It may also be true that we still don’t get the appropriate bang for each testing buck spent.
There may also even be some validity to the argument that some of that assessment money could be better spent elsewhere. What’s undisputed, however, is that a lot of money, an awful lot of money, is already spent elsewhere. Overall, the amount of money spent on large-scale testing in the United States each year is approximately 1/4 of 1% of the total amount spent annually on K-12 eduction.
| Education | Assessment | |
| K-12 | $1 trillion | $1.7 billion |
| Title 1 | $18 billion | $380 million |
| Race to the Top | $4.35 billion | $350 million |
I include Title 1 here because the narrative that we have allowed to thrive surrounding Title 1, state testing, and accountability has long been a pet peeve of mine. For years on end, we have let fester the argument that the federal theory of action under NCLB and ESSA is the following: if we hold them accountable for test scores, instruction and learning will improve. Somehow, we forget and/or conveniently ignore the other 98% of the $18 billion allocated annually to Title 1.
Is the total amount allocated for Title 1 inadequate to solve the problem it addresses? Perhaps. However, in recent years, apparently, nearly $2 billion of the allocated monies (i.e., 5 times that amount dedicated to testing) were unspent. From the earliest days of ESEA in the 1960s, the primary purpose of assessment associated with Title 1 was accountability for how Title 1 monies were being spent. Does anybody remember program evaluation?
I also include Race to the Top as it represents the last great attempt to reform K-12 assessment. Remember Arne Duncan and “Beyond The Bubble Test”? Coming in at 8% of the $4.35 billion allocated to the Race To The Top program, it may appear that a significant amount of money was devoted to assessment. Certainly more than the 2% of Title 1 or the 1/4 of 1% of overall spending. Let’s not forget, however, that the $4.35 billion for Race To The Top itself was only a fraction of the $100 billion devoted to education in Obama’s American Recovery and Reinvestment Act (ARRA). And the $100 billion for education was a fraction of the $833 billion ARRA total.
Time
Ah, time. That seemingly straightforward yet capricious and complex metric.
| Instruction | Large-Scale Testing |
| 180 days | Nobody Knows |
We know the school year is 180 days, but how much time on those days is devoted to instruction? I cringe at regulations requiring ‘x’ hours of instructional time per year because they inevitably lead in the wrong directions. Even more uncertain is how much time is “spent” on testing.
A decade or so ago, the Obama administration tried to put a 2% cap on time devoted to standardized testing. However, estimates of time spent on testing vary widely and nobody quite knows when to start and stop the clock.
And if we could reach agreement on what “time” to count toward testing time, how much is too much?
One metric that I’ve been playing with lately is the ratio of testing time to the number of days in your definition chronically absent. I’m thinking that it’s a red flag if that figure exceeds 0.50. If it approaches 1.0, you’re screwed. But like reliability, effect size, and alignment indices, the interpretation of my index is still a work in progress.
Trying Finding Our Place In This World
As I said at the outset, the purpose of this exercise is to remind ourselves of the role that those of us involved in assessment play in K-12 education. We cannot think of assessment as the straw that stirs the drink or the tail that wags the dog. But like the straw and tail, we can serve useful and important functions.
Like the police, we can protect and serve.
Like physicians, with the right kind of tests administered at the right time, we can diagnose.
With wealths of data on hand, we can advise and prescribe.
And before we jump headfirst and headlong into the next next generation of assessment, most importantly, we can listen and learn.
Image by Hanne Hasu from Pixabay
Embrace the Absurd 