Study Identifies Ways to Prevent Test Score Manipulation

Contact: Doug Gavel
Phone: 617-495-1115
Date: November 17, 2003

Cambridge, MA – The pressures of high-stakes school testing may encourage teachers and administrators to doctor test results, according to a new study by researchers Brian Jacob of Harvard’s Kennedy School of Government and Steven Levitt of the University of Chicago.



Though cases of school personnel changing test results have attracted public attention in recent years, there has been a lack of hard evidence as to the extent of cheating by school personnel. The study of cheating in Chicago public schools published in the Nov. 17 edition of the journal Education Next indicates that the problem is significant enough to require attention, but not so widespread as to discredit the integrity of the nation’s educators.



Jacob and Levitt examined records from approximately 1,000 Chicago public school classrooms in grades 3-7 from the years 1993-2000. They found that on any given exam, school personnel appear to cheat in 4 to 5 percent of classrooms. The methods of cheating could include behaviors such as changing students’ answers, providing students with answers and quizzing students before the test with actual test questions.


Perhaps most importantly, Jacob and Levitt found evidence that the altering of test results rose when local policies attributed more weight to testing. “The incidence of teacher cheating has increased since high-stakes testing was implemented in Chicago schools,” said Jacob. “The challenge and the goal of the research is to identify predictors of this behavior so that it can be prevented in the future.”



The study results indicated that predictors for cheating in a particular classroom include:

Teachers administering tests to their own students;

Lower percentages of special education students (meaning that a higher proportion of student scores would “count” toward overall scores);

Younger, less-experienced teachers;

Classrooms in schools with lower achievement, higher poverty rates, and more African-American students.

The researchers posit that with better detection methods and deterrent techniques, schools can go a long way towards preventing cheating, which in turn will increase confidence in the results from high-stakes testing. For example, one strategy to deter cheating is to simply use external exam proctors as in exams such as the SAT and GRE. Another strategy is deterrence. The incidence of detected cheating in Chicago schools declined in the year after the results from this research were publicized, suggesting that teachers and administrators may have believed that they faced a greater chance of detection and were thus less likely to manipulate exam results.



“The good news is that cheating can only explain a fraction of test score gains in the Chicago public schools,” Jacob says. “That cheating occurs shouldn’t invalidate the progress these schools have made.”



From a broader public-policy perspective, the most important question that emerges from the research, according to Jacob, is how cheating responds to incentives. “If there are incentives in place, people will find a way to game the system,” he says. Tying the results of test scores to teacher bonuses, school “probation,” or student promotion leads predictably to an increase in the manipulation of test scores by teachers and administrators.



With the implementation of the federal No Child Left Behind act, which ties certain penalties to schools that consistently underachieve on test scores, the temptation to manipulate the results from high-stakes tests will likely grow, according to Jacob. He says the greatest lesson of the study for educators is that in the context of high-stakes testing they need always to be aware of the risk of cheating, especially as the stakes get higher, so that appropriate action can be taken.







Brian Jacob is an assistant professor of public policy at Harvard’s Kennedy School of Government.

Steven Levitt is Alvin H. Baum professor of economics at the University of Chicago.



Education Next is published by The Hoover Institution. It is co-sponsored by the Kennedy School’s Program on Education Policy and Governance (PEPG), the Thomas B. Fordham Foundation and the Manhattan Institute for Policy Research. It is available online: http://www.educationnext.org/

###

Print print | Email email