HIGH-STAKES TESTING

Lessons from Chicago

Brian Jacob, Taubman Center Faculty Affiliate

In January 2002, President Bush signed the No Child Left Behind Act, ushering in a new era of accountability in education. The federal legislation is the latest development in the movement for test-based accountability in education, often referred to as high-stakes testing. Proponents of these efforts argue that linking student promotion as well as teacher salaries and employment to test scores will boost effort and performance. Opponents counter that linking incentives to performance on standardized tests may unfairly penalize certain students, and will turn educators away from teaching skills that the accountability exam does not test directly.

The Chicago public school system, which in 1996 became the nation’s first large urban school district to implement high-stakes testing, offers both hopeful and cautionary suggestions about what high-stakes testing is likely to accomplish. Students in the third, sixth, and eighth grades had to meet predetermined standards on the reading and math sections of the Iowa Test of Basic Skills. Students who did not meet the standards had to attend summer school and then retake the exam. Students who did not pass the exam at the end of the summer had to repeat the grade.

The Chicago school system also placed low-performing schools — those where fewer than 15 percent of students scored at or above national norms in reading — on academic probation. Those schools received additional resources and technical support, largely aimed at improving classroom instruction and school management. Probation schools that did not show enough improvement were subject to reconstitution.

Since the program began, test scores along with other data and surveys, indicate that these policies have had mixed impacts and suggest five lessons for those designing new high-stakes testing programs.

Lesson One: Students and teachers respond to incentives. Student achievement on the Iowa Test of Basic Skills rose sharply following the introduction of the accountability policy, with students scoring at levels suggesting they were three to five months ahead of where similar students in those classes had been before the onset of high-stakes testing.

Lesson Two: Students and teachers respond strategically to incentives. After the introduction of high-stakes testing, math and reading scores rose relative to scores on low-stakes exams such as science and social studies. The proportion of students excluded from testing because they were in special education or for other reasons increased, however. Also, the accountability policy had little effect on performance on the state-administered Illinois Goals Assessment Program, which was not linked, like the Iowa Test of Basic Skills, to serious consequences.

Lesson Three: Supplemental support can help at-risk students meet higher standards. A key feature of the Chicago policy was its attempt to support at-risk students. The summer program offered select teachers, relatively small classes, a highly structured curriculum, and four to five hours of instruction per day in math and reading. Students who attended the program made substantial gains, with many promoted to the next grade.

Lesson Four: Test-based accountability does not necessarily change core classroom practice. Observers have long noted that core instructional practices in schools are remarkably resistant to change, even in the face of powerful reform movements. Accountability is no exception. While one researcher documented several ways that teachers responded strategically to high-stakes testing in Chicago, she also found that the policy had little if any effect on the mix of didactic versus interactive instruction practiced in classrooms. Researchers also found that the resources, teacher training, and technical support associated with probation had no significant effect on either reading or math scores.

Lesson Five: Test security is important. As the consequences of test scores rise for students as well as teachers, so do the incentives to cheat. Documented cases in which teachers and administrators intentionally manipulated student exams have recently occurred in California, Massachusetts, New York, and Texas. In Chicago, unexpected test score fluctuations and suspicious patterns of answers from students in the same classroom strongly suggest that roughly 1 to 2 percent of teachers cheated during the years 1993 to 2000, and that instances of such cheating rose 40 to 50 percent following the introduction of the accountability policy. While the number of cheating classrooms is low, this trend is disturbing, especially considering the relatively minor incentives that teachers faced in Chicago.

These results suggest that test-based accountability must be approached with caution. On the one hand, survey and interview data indicate that many elementary students and teachers are working harder, particularly in the lowest-performing schools. Classroom pacing and opportunity to learn appear to have accelerated as well. Students, finally, report their parents are more involved in their schoolwork, and that their teachers are taking greater interest in their learning. On the other hand, the differential achievement trends on city and state tests suggest that test-day motivation, preparation, or cheating may have driven at least part of the gains on the former test, and therefore may not reflect greater student comprehension of mathematics and reading.

The Chicago experience illustrates the complexity of school reform, including the shortcomings of standardized measures of achievement, the difficulty of monitoring teacher and student behavior, and the importance of student and school capacity in responding to policy initiatives. Educators must carefully examine the incentives high-stakes accountability generates to ensure that they foster long-term student learning rather than simply raising short-term scores on specific tests.

The information from this paper is drawn from several articles available at www.ksg.harvard.edu/faculty/brian_jacob.