School Ranks Based On Tests Like MCAS Biased, Inaccurate: Study

CAMBRIDGE, MA — Schools that rank highly in ratings based on standardized tests, like the MCAS assessment in Massachusetts, are not necessarily the schools that helps students learn more, according to a study published this week by a Nobel Prize-winning professor at the Massachusetts Institute of Technology.

"If you just go by published accountability ratings, you will be guided to the schools that have the most white and Asian students," Josh Angrist, a professor at MIT who recently won the Nobel Prize in economics, told Chalkbeat. "You’re not actually being guided to the best schools."

While opponents of standardized testing and previous researchers have made similar claims in the past, the study by Angrist and three other economists is one of the first to take a close at the measures used by the popular third-party rating site GreatSchools. Sites like GreatSchools and U.S. News & World Report try to rank schools using a wide range of data, including test scores. But the Angrist study suggests those rankings — which are used by states, the media, school districts and parents to compare schools — are riddled with bias.

Angrist's study shows schools that were good at boosting test scores often still had low scores on GreatSchools. Put another way, school systems with a high percentage of minority students often do a better job of improving student scores on standardized tests, but their students still fall far from the top test scorers. Schools with higher percentages of white and Asian students score higher, but show smaller gains on standardized test, suggesting more actual learning and improvement is occurring in school systems where students score lower on standardized tests.

The study was instead based on data from middle schools in Denver and New York City and did not directly address Massachusetts school rankings. But opponents of standardized testing in Massachusetts public schools were quick to share the findings.

"So a rating system that relies on test scores...will often give high marks to mostly white schools even if the children at those schools climb slowly. It will also give low marks to high-quality schools that have mostly Black and brown children," Boston Parents Schoolyard News, an independent news site covering Boston Public Schools, said in a blog post about the study.

"Massachusetts uses a rating system that’s partly based on score levels and partly on a 'growth' measure similar to Angrist’s definition of quality," the post continued. "But the formula gives three times as much weight to score level as to growth."