Teaching Literacy

A Holistic Reframing of Assessment Literacy for Teachers

In this post, I summarize a presentation made at the 2021 annual meeting of the New England Educational Research Organization (NEERO). The presentation is based on my paper, Teaching Literacy – A Holistic Reframing of Teacher Assessment Literacy, in which I offer an alternative perspective on teacher assessment literacy – one in which the longstanding practice of regarding teaching as synonymous with instruction while separating instruction and assessment is replaced by the position that there is no teaching without assessment.

Under this holistic perspective of teaching and assessment, teacher assessment literacy can better be thought of from the perspective of the teacher as teaching literacy.

In the final sections of the paper, I discuss implications for  supporting teachers’ interpretation and use of  external, large-scale test results along with recommendations for reporting results from large-scale tests in a way that supports teaching literacy.

Assessment literacy for teachers is one area that appears to be nearly universal agreement on several key points:

• Assessment literacy is necessary for effective instruction.
• Teachers do not have an adequate level of assessment literacy.
• Efforts to enhance teacher assessment literacy have largely been inadequate and have had limited success.

As thinking around teaching and assessment has evolved over the past few decades, an increasing emphasis is being placed on formative assessment. The focus of assessment literacy for teachers, correspondingly, has shifted somewhat from large-scale assessment to the classroom. It is much more common now to consider teacher assessment literacy in the context of formative assessment practices in the classroom. It is also widely acknowledged that assessment literacy is context dependent and efforts to enhance teacher assessment literacy will be more effective when framed within the context of teachers’ use of assessment to inform their decision making in the classroom.

Even within the framework of formative assessment practices, however, assessment and assessment literacy is still often regarded as something distinct from teaching. That is, assessment is regarded as a tool that is used to support effective instruction (i.e., teaching) and assessment literacy is focused on how best to understand and make use of that tool.

A contributing factor to this separation of assessment from teaching is the perspective that assessment is an instrument (i.e., a test) rather than a process. The conflating of the process of assessment with the tools used to carry out the process (which may include a variety of tests and other assessment instruments) has been a focal point of recent attempts to define formative assessment as a process. The same principle, however, applies all aspects of assessment. As Cizek (2020) explains, assessment “involves aggregating all of the diverse sources of information [about a student], arriving at some tentative conclusion about what is happening for the student, and developing some tentative plans …” It “involves collecting and summarizing information in order to develop a course of action uniquely tailored to an individual’s needs.” (p. 7)

If one accepts the premise that it is, or should be, impossible to define teaching without including assessment as an integral part of the teaching process then I posit that teacher assessment literacy is better thought of as teaching literacy. 

Rather than asking teachers to embrace their identities as assessors as well as instructors, it seems more productive to simply to ask them to identify as teachers – a role which requires the knowledge, skills, and abilities necessary to impart relevant information (instruct) and collect relevant information (assess) among its many requirements.

Viewed from the perspective of the teacher (rather than the perspective of an assessment specialist), teaching literacy shifts the focus away from improving a teacher’s knowledge and skills in assessment toward improving teaching by enhancing a teacher’s knowledge and skills related to collecting and evaluating the evidence needed to inform decision making.

It is not sufficient to think of assessment literacy as context dependent or even as embedded, or situated, within a teaching context. Teachers’ assessment literacy, rather, is embedded within the teacher.

Consider the application of the “Data, Information, Knowledge, Wisdom” Pyramid depicted above to teachers, tests, and assessment literacy.  A test can be expected to provide data, the lowest level on the pyramid. In a best-case scenario, test results may be processed and reported in such a way that they provide useful and relevant information.

The knowledge and wisdom necessary to use that data and information effectively to support instructional decision making, however, is not an attribute of the test. A test, or any assessment instrument, alone cannot be the source of information on what a teacher should do next.  That knowledge and wisdom resides within the teacher. Ideally, the application of that knowledge and wisdom will be cyclical and iterative with the teacher’s knowledge and wisdom informing and being informed by evidence gathered through a continuous assessment process.

In other words, that knowledge or wisdom lies completely outside of the realm of assessment or measurement theory. Perhaps more importantly, assessment specialists and psychometricians lack the knowledge base needed to provide that type of information on the basis of a test score. Attempts, therefore, to frame teacher assessment literacy from an assessment/measurement perspective rather than a teaching perspective are destined to fail, or at best, have limited success.

From Horace Mann’s description of an aptness to teach in the mid-nineteenth century to Lee Shulman’s description of pedagogical content knowledge in the last quarter of the twentieth century, descriptions of the knowledge and wisdom necessary for effective teaching have been surprisingly consistent.

Since being introduced by Shulman in 1986, the concept of pedagogical knowledge has been applied to specific subject area contexts and expanded to include aspects of knowledge related to the student and the culture of the classroom. 

The shift in focus from assessment literacy to testing literacy, however, should not be regarded as a shift away from the fundamental measurement principles of validity, reliability, and fairness.

Although their instantiation may be different in the context of a teaching interaction, a classroom test, and a large-scale test, the same measurement principles apply in all three cases.

In the shift from norm-referenced to criterion-referenced testing we lost a sense of the fundamental purpose of reporting assessment results to teachers; that is, to convey information that will help those teachers interpret and use information from the test appropriately. 

For better or worse, norm-referenced “scores” were almost exclusively derived scores designed to provide useful information to teachers, administrators, and parents. In terms of the DIKW pyramid cited earlier in this paper, norm-referenced test scores might better be regarded as information (I) rather than data (D). 

In contrast, scales scores, along with the achievement levels based on them, have become the primary metric used for reporting the results of criterion-referenced large-scale tests such as state tests. Scale scores, however, are clearly data rather than information. They have no inherent meaning or practical meaning and force teachers toward statistics-based interpretations which are of limited usefulness and which teachers, and most other intended users of large-scale test results, are ill-equipped to make. 

Results reported from large-scale tests should be designed to best support their intended use; that is, to support the decisions that must be made by the intended users. The logical outcome of such an approach is that the results reported to teachers will likely differ significantly from the results reported to district and school administrators, and from the results reported to state policymakers.

Header image by Anand Kumar from Pixabay
Puzzle image by Thanks for your Like • donations welcome from Pixabay
Star apple image by Capri23auto from Pixabay
Measuring tape apple image by Shutterbug75 from Pixabay 
Bubble sheet image by Image by lecroitg from Pixabay