Generalization (n): the act or process whereby a learned response is made to a stimulus similar to but not identical with the conditioned stimulus. (Merriam Webster)
From time immemorial, a primary purpose of educating children has been generalization. The long term goal has always been more focused on what students will be able to do with the knowledge, skills, habits, and dispositions that are acquired and/or cultivated over the course of 15 to 20 years of formal education. Primary schools prepare students for success in elementary schools, elementary schools for secondary schools, secondary schools for postsecondary success, or in today’s parlance ensure students’ readiness for college and career. (And some day, perhaps in the distant future, we’ll figure out what middle schools do.)
Over the past decade or so, however, I think that it’s fair to say that increasingly a major focus of those engaged in education has been personalization. Across the big three areas of curriculum, instruction, and assessment and all that they comprise individually and collectively it’s been nearly impossible to engage in any prolonged discussion related to education without encountering the term personalization.
I hesitate at this point to go beyond the word term to describe personalization. I’m not yet comfortable calling it a concept or philosophy or pillar because like so much of what we discuss in education, personalization is at best loosely defined. There are subtle but important nuances of meaning at play when we are considering personalization in the areas of curriculum, instruction, and assessment. And within each of those areas there are likely people agreeing on the importance of personalization with very different purposes and aims in mind.
This blog is not the appropriate space for a comprehensive discussion of all of the complexities in the use of the term personalization. In this post, however, I would like to consider, at least from my perspective as an assessment specialist, specifically a large-scale assessment specialist, the relationship between generalization and personalization.
Personalization and Standardization in Assessment
Before jumping into generalization, there is another relationship that I should probably address. With regard to large-scale assessment, personalization is often portrayed as the opposite of standardization.
- A custom fit versus one size fits all
- The individual versus the group
- Good versus evil
But while standardization is regarded by most as a fundamental principle of large-scale assessment, the facts are that for decades now we have been chopping away (yes chopping, not chipping) at each of the three legs of our standardization stool: content, administration, and scoring; but still the stool has not tipped over and continues to support our weight.
Content: With the shift away from off-the-shelf NRT, we quickly became comfortable with altering some or all of the content on a test from year to year and equating test forms. We are hesitant to use matrix sampling to generate individual student scores, but quick to adopt adaptive testing as soon as it became feasible. (We are a curious bunch.)
Administration: In the service of access and inclusion, we’ve accepted accommodations to timing, setting, presentation, and mode of response; and we’ve become fairly expansive in who has access to accommodations. Where we once adjusted scores based on the week of testing, we now accept months-long test administration windows out of practical necessity.
Scoring: In order to include essays and constructed-response items, we’ve been willing to accept a certain level of variability in scoring due to raters (think exact v. adjacent agreement).
It may be argued that some of these deviations from standardization, like accommodations, enhance personalization, but others serve different purposes. The point is that although there may be some lines remaining that we should not cross, we can probably sever concerns about standardization from the discussion of personalization and generalization.
Unfortunately, with our large-scale assessment stool now sitting so low to the ground, we are not able to see as much or as far as we once did or as we might like.
Generalization, we hardly knew ye
Given my time immemorial claim to open this post, one might assume that we have a solid understanding of the use of assessment to promote generalization, an understanding based on a foundation of academic research and practical applications. But one would be wrong.
Over the course of the standards-based movement, the focus of assessment design and development has shifted inward toward alignment to state content standards. And although many initial attempts at writing content standards produced statements that one might think supported inferences of generalization (e.g., at the end of the grade level students would be able to solve problems in a wide variety of listed settings and contexts), most of those types of standards were criticized as being too vague. Content standards became more precise, items were aligned to single standards, and supported inferences were trending more toward mastery more than generalization. The result, large-scale assessment now lives in that dreaded middle ground, not broad enough to support inferences about generalization, not specific enough to support inferences of mastery.
It’s true, of course, that standards don’t have to be written that way. The NGSS may be more focused on generalization than recent state science standards, although perhaps not as much as when the primary focus of science assessment was on inquiry and the scientific method. Likewise, competencies comprising inseparable combinations of complex knowledge and skills might prove to be more amenable to inferences of generalization.
If so, however, that brings us back to the bugaboo of aptitude-treatment interactions (ATI) that Shavelson and colleagues addressed so well the last time we tried to focus on generalization while assessing complex student performances in the 1990s. Around that same time, Snow tried to corral the host of potential ATI that Cronbach and Snow (1977) had previously identified into a workable set. Snow’s efforts however were well before ATI increased exponentially with the introduction of 21st century skills considered crucial today and the complex web of forces laid out by Mislevy in his 2018 book, Sociocognitive Foundations of Educational Measurement.
The pessimist walks away from Sociocognitive Foundations concluding that ATI are a wicked problem that cannot be solved. The pragmatist thinks, well we at least have to try. The optimist, however, comes away viewing ATI like those once fatal diseases that although they cannot yet be cured can be treated and managed successfully enough to offer people decades of quality life. There are treatments available.
Coincidentally, fortuitously, and inevitably, those treatments bring us to personalization. The assessment of complex student performances (individual and group) will require some level of personalization. Which brings us back to the question posed in the title of this post.
Are Personalization and Generalization Compatible?
The answer is unequivocally, it depends. It depends on the extent to which we use personalization as a tool or approach to help all students move forward toward a common goal versus the extent to which we use personalization as a tool or approach to help students self-actualize and proceed successfully down their own personal pathways.
Of course, the challenge is that we want personalization, and public education in general, to do both – to support students in finding themselves and their way in this world and ensuring that all students have the common core of knowledge, skills, and abilities that allow them to do so. It’s a delicate balance, but not an impossible one to achieve as long as we are aware of which purpose and goal we have in mind when we incorporate personalization into curriculum, instruction, and assessment. And particularly, when we are designing assessments for different and specific purposes.
The caveat is that we don’t want to allow personalization to cut off students from potential pathways by sending them too soon down one-way tracks with no opportunities to change direction. The easiest way to avoid that problem is to not get too absorbed in the incremental, the day-to-day, the here-and-now when making decisions about personalization. We must keep one eye looking forward, to see over the trees and beyond the forest, outward toward our students’ horizons.
Image by Vilius Kukanauskas from Pixabay
Embrace the Absurd 