assessment, accountability, and other important stuff

Archive for January, 2019

IASA – Refreshing our Memory

Charlie DePascale

This year marks the 25th anniversary of the 1994 reauthorization of ESEA, known as the Improving America’s Schools Act (IASA).  Throughout the year, I will explore how various aspects of that law shaped my career, educational assessment and accountability, and K-12 education, in general. All of this will be done, of course, with an eye toward the next reauthorization of ESEA and the future of K-12 assessment and accountability.

As we begin the year, however, let’s just take a few minutes to refresh our memories on the thoughts about equity, excellence, and education that drove the 1994 law. Sometimes it’s not necessary to write anything new.  The words speak for themselves. I call particular attention to the middle section titled, What Has Been Learned Since 1988.

 

‘‘TITLE I—HELPING DISADVANTAGED

CHILDREN MEET HIGH STANDARDS

‘‘SEC. 1001. DECLARATION OF POLICY AND STATEMENT OF PURPOSE.

‘‘(a) STATEMENT OF POLICY.—

‘‘(1) IN GENERAL.—The Congress declares it to be the policy of the United States that a high-quality education for all individuals and a fair and equal opportunity to obtain that education are a societal good, are a moral imperative, and improve the life of every individual, because the quality of our individual lives ultimately depends on the quality of the lives of others.

‘‘(2) ADDITIONAL POLICY.—The Congress further declares it to be the policy of the United States to expand the program authorized by this title over the fiscal years 1996 through 1999 by increasing funding for this title by at least $750,000,000 over baseline each fiscal year and thereby increasing the percentage of eligible children served in each fiscal year with the intent of serving all eligible children by fiscal year 2004.

‘‘(b) RECOGNITION OF NEED.—The Congress recognizes that—

‘(1) although the achievement gap between disadvantaged children and other children has been reduced by half over the past two decades, a sizable gap remains, and many segments of our society lack the opportunity to become well educated;

‘‘(2) the most urgent need for educational improvement is in schools with high concentrations of children from low income families and achieving the National Education Goals will not be possible without substantial improvement in such schools;

‘‘(3) educational needs are particularly great for low-achieving children in our Nation’s highest-poverty schools, children with limited English proficiency, children of migrant workers, children with disabilities, Indian children, children who are neglected or delinquent, and young children and their parents who are in need of family-literacy services;

‘‘(4) while title I and other programs funded under this Act contribute to narrowing the achievement gap between children in high-poverty and low-poverty schools, such programs need to become even more effective in improving schools in order to enable all children to achieve high standards; and

‘‘(5) in order for all students to master challenging standards in core academic subjects as described in the third National Education Goal described in section 102(3) of the Goals 2000: Educate America Act, students and schools will need to maximize the time spent on teaching and learning the core academic subjects.

‘‘(c) WHAT HAS BEEN LEARNED SINCE 1988.—To enable schools to provide all children a high-quality education, this title builds upon the following learned information:

‘‘(1) All children can master challenging content and complex problem-solving skills. Research clearly shows that children, including low-achieving children, can succeed when expectations are high and all children are given the opportunity to learn challenging material.

‘‘(2) Conditions outside the classroom such as hunger, unsafe living conditions, homelessness, unemployment, violence, inadequate health care, child abuse, and drug and alcohol abuse can adversely affect children’s academic achievement and must be addressed through the coordination of services, such as health and social services, in order for the Nation to meet the National Education Goals.

‘‘(3) Use of low-level tests that are not aligned with schools’ curricula fails to provide adequate information about what children know and can do and encourages curricula and instruction that focus on the low-level skills measured by such tests.

‘‘(4) Resources are more effective when resources are used to ensure that children have full access to effective high-quality regular school programs and receive supplemental help through extended-time activities.

‘‘(5) Intensive and sustained professional development for teachers and other school staff, focused on teaching and learning and on helping children attain high standards, is too often not provided.

‘‘(6) Insufficient attention and resources are directed toward the effective use of technology in schools and the role technology can play in professional development and improved teaching and learning.

‘‘(7) All parents can contribute to their children’s success by helping at home and becoming partners with teachers so that children can achieve high standards.

‘‘(8) Decentralized decisionmaking is a key ingredient of systemic reform. Schools need the resources, flexibility, and authority to design and implement effective strategies for bringing their children to high levels of performance. ‘‘(9) Opportunities for students to achieve high standards can be enhanced through a variety of approaches such as public school choice and public charter schools.

‘‘(10) Attention to academics alone cannot ensure that all children will reach high standards. The health and other needs of children that affect learning are frequently unmet, particularly in high-poverty schools, thereby necessitating coordination of services to better meet children’s needs.

‘‘(11) Resources provided under this title can be better targeted on the highest-poverty local educational agencies and schools that have children most in need.

‘‘(12) Equitable and sufficient resources, particularly as such resources relate to the quality of the teaching force, have an integral relationship to high student achievement.

‘‘(d) STATEMENT OF PURPOSE.—The purpose of this title is to enable schools to provide opportunities for children served to acquire the knowledge and skills contained in the challenging State content standards and to meet the challenging State performance standards developed for all children. This purpose shall be accomplished by—

‘‘(1) ensuring high standards for all children and aligning the efforts of States, local educational agencies, and schools to help children served under this title to reach such standards;

‘‘(2) providing children an enriched and accelerated educational program, including, when appropriate, the use of the arts, through schoolwide programs or through additional services that increase the amount and quality of instructional time so that children served under this title receive at least the classroom instruction that other children receive;

‘‘(3) promoting schoolwide reform and ensuring access of children (from the earliest grades) to effective instructional strategies and challenging academic content that includes intensive complex thinking and problem-solving experiences;

‘‘(4) significantly upgrading the quality of instruction by providing staff in participating schools with substantial opportunities for professional development;

‘‘(5) coordinating services under all parts of this title with each other, with other educational services, and, to the extent feasible, with health and social service programs funded from other sources;

‘‘(6) affording parents meaningful opportunities to participate in the education of their children at home and at school;

‘‘(7) distributing resources, in amounts sufficient to make a difference, to areas and schools where needs are greatest;

‘‘(8) improving accountability, as well as teaching and learning, by using State assessment systems designed to measure how well children served under this title are achieving challenging State student performance standards expected of all children; and

‘‘(9) providing greater decisionmaking authority and flexibility to schools and teachers in exchange for greater responsibility for student performance.

 

 

 

 

 

Three Little Words

love_heart_svg.svg

Charlie DePascale

Life is full of three-word phrases.

Some tend to have profound and lasting consequences that extend far beyond what may have been intended when they were uttered.  Phrases such as I Love You, That Looks Safe, and for those among us wavering on new year’s resolutions, Just One Bite might fall into this category.

Other ubiquitous three-word phrases like While Supplies Last, Limited Time Offer, Exclusions May Apply, and Void Where Prohibited function exactly as intended; even if we are usually not happy to see them.  Often hidden in the fine print, their sole purpose is to put constraints on an offer or claim that is being made.

In the last couple of years, a three-word phrase has begun to make its way into the assessment lexicon – on this test.  At first glance, the phrase, or a close variation of it, seems neither new nor threatening when used to describe student or group performance. Charlie spelled 23 words correctly on this week’s spelling test. Karla met the college readiness benchmark on the SAT. In Vermont, 42% of grade 5 students performed at the Proficient level or higher on the Smarter Balanced mathematics test. Taken at face value, the phrase is used simply to identify the test that was taken.

Recent use of this common phrase, however, is intended to do much more than identify the source of performance.  Its purpose is to limit interpretation of student or school performance; to make it clear that the performance should be interpreted within the specific framework of the test or testing program.

Again, at first glance, we might regard this use of the phrase as innocuous or perhaps even a step forward in test use and interpretation.  Identifying the source of a test score seems quite consistent with many of our Standards for Educational and Psychological Testing, beginning with Standard 1.0:

Clear articulation of each intended test score interpretation for a specified use should be set forth, and appropriate validity evidence in support of each intended interpretation should be provided.

When considered as part of a larger effort to marginalize and vilify large-scale assessment, however, the connotation of the phrase on this test changes dramatically. It is the second punch in a one-two combination intended to knock out large-scale assessment.  The left jab that has weakened the credibility of large-scale assessment is the argument “Scores on large-scale assessments are ______” – fill in the blank with your favorite criticism: not valid, unfair, inaccurate, not representative, unstable, insufficient, not authentic, etc.  Now with the right cross of on this test, critics of large-scale assessment (or its uses) seek to nullify test scores by limiting their interpretation to that already weakened large-scale assessment.

Even the most well-designed assessment program can sustain only so many of these blows before collapsing in a heap to the canvas.

pow-1601674__180

My first encounters with the on this test crowd occurred while working with two states setting achievement standards on their new college-and-career-readiness tests.  A vocal minority in both states were adamant that the phrase on this test should be added to each achievement level description. Their stated intent was to convey that students’ performance on the state assessment was not representative of their overall level of achievement.

My most recent encounter came late last year in a Stephen Sawchuk post in Edweek about the decision to add the modifier NAEP in front of the achievement level classifications on the National Assessment of Educational Progress; as in NAEP Basic, NAEP Proficient, NAEP Advanced. As stated in the post, “[t]he rewording may seem awfully minor to the uninitiated. But there’s a deeper subtext behind the changes, and that’s why this is worth noting.”

For their part, the National Assessment Governing Board (NAGB) makes the argument that the addition of the NAEP modifier is intended to clarify that the NAEP Proficient level, “is not intended to reflect ‘grade level’ performance expectations, which are typically defined normatively and can vary widely by state and over time. NAEP Proficient may convey a different meaning from other uses of the term ‘proficient’ in common terminology or in reference to other assessments.” Forgoing for now a discussion of whether the NAEP achievement levels are defined any more or less normatively than any other achievement levels, nobody can deny that achievement standards can and do vary widely by state and over time. Consequently, there is confusion when the same label Proficient is used across states and assessments to describe those varying standards.  From that perspective, the label NAEP Proficient serves the purpose of clearly identifying the set of achievement standards against which student performance is being judged.

For long-time critics of the NAEP achievement standards, however, the modifier is another weapon in their fight to marginalize NAEP results. The achievement level results no longer represent what proficient fourth or eighth grade students across the United States should know and be able to do; rather, they simply reflect NAEP Proficient – a mythical concept that is not tied to any state’s grade level standards and expectations.

As assessment/measurement specialists, our professional values and standards have made us unwitting accomplices in the effort to undermine large-scale assessment.  We agree with and/or can be quoted making statements such as

  • test scores should not be considered in isolation,
  • a student’s score on a given day or test might not reflect her/his true performance,
  • multiple measures should be used to evaluate student achievement, or
  • a test score reflects student performance on this test.

In the past, we mastered the art of expanding on those statements via PowerPoint bullets and charts to defend large-scale assessment with winning arguments before policy makers and the courts.  In this era of soundbites, tweets, and memes, however, we may never get that far.

With hubris, we attach a great deal of importance to our work and our high-quality assessments.  Remember, however, that without the ability to generalize student or school performance beyond a particular test we have nothing.  The task before us is clear; and if we envision a future in which large-scale assessment makes a valuable contribution to improving student learning, we must not fail on this test.