ÐÜèÊÓƵ

For Immediate Release: May 22, 2019

Tony Pals, tpals@aera.net
(202) 238-3235, (202) 288-9333 (cell)

Collin Boylin, cboylin@aera.net
(202) 238-3233, (860) 490-8326 (cell)

Study Snapshot: States’ Performance on NAEP Mathematics and Reading Exams After the Implementation of School Letter Grades

Study: "States’ Performance on NAEP Mathematics and Reading Exams After the Implementation of School Letter Grades"
Authors: Audrey Amrein-Beardsley (Arizona State University), Tray Geiger (Arizona State University), Kevin Winn (Arizona State University)

This study was presented at the , April 5-9, Toronto, Canada. (Session: .) A copy of the paper, which is currently under review for publication in a peer-reviewed journal, is available for journalists upon request from the ÐÜèÊÓƵcommunications team.

Main Finding:

• Overall, the 13 states that have implemented policies for A-F school letter grade accountability have fared no better or worse than other states in terms of increasing student achievement on the National Assessment of Educational Progress (NAEP) post policy implementation. States’ adoption of A-F policies does not appear to increase student achievement as intended, but rather yields random results.  

Details:

• The authors examined the extent to which states’ NAEP scores increased or decreased after implementing a school A-F accountability system, in mathematics and reading in grades 4 and 8, as compared to the national average.
   
• Overall, the authors found mixed results as to whether there was a clear, causal relationship between implementation of an A-F accountability system and increased student achievement. The authors did not identify any consistent positive or negative relationship between policy implementation and NAEP scores on grade 4 and grade 8 mathematics and reading.
   
• For the purposes of this study, researchers defined A-F school letter grades as any state’s annual achievement profile required via policy or state statute to help the state define and then label school quality every year for every public school in the state.
   
• To date, 13 states use a school letter grade accountability system, with Florida being the first to implement a school letter grade policy in 1998. The other 12 states, and their years of implementation are Alabama (2013), Arkansas (2012), Arizona (2010), Indiana (2011), Mississippi (2012), New Mexico (2012), North Carolina (2013), Ohio (2014), Oklahoma (2011), Texas (2015), Utah (2013), and West Virginia (2015).
   
• Fundamentally, the school letter grades are meant to help parents and members of the public assess and compare schools’ performance and ultimately help states hold schools (and the educators within them) accountable for meeting higher standards to improve student achievement, graduation rates, and other indicators of school quality over time.   
   
• The authors analyzed trends in each state’s NAEP scores from the NAEP test that most immediately preceded its A-F system implementation to its 2017 scores. The authors then compared these states’ relative growth (or lack of growth) to the national average.
   
• For NAEP grade 4 mathematics exams, five of the 13 states (38.5 percent) had net score increases after their A-F systems were implemented; seven states (53.8 percent) had net score decreases after A-F implementation; and one state (7.7 percent) demonstrated no change.
   
• Compared to the national average on grade 4 mathematics scores, eight of the 13 states (61.5 percent) demonstrated growth over time greater than that of the national average; three (23.1 percent) demonstrated less growth; and two states (15.4 percent) had comparable growth.
   
• For grade 8 mathematics exams, five of the 13 states (38.5 percent) had net score increases after their A-F systems were implemented, yet eight states (61.5 percent) had net score decreases after A-F implementation.
   
• Grade 8 mathematics growth compared to the national average varied more than that of grade 4 mathematics. Six of the 13 states (46.2 percent) demonstrated greater growth over time compared to that of the national average; six other states (46.2 percent) demonstrated less growth; and one state (7.7 percent) had comparable growth.
   
• For grade 4 reading exams, eight of the 13 states (61.5 percent) had net score increases after A-F implementation; three states (23.1 percent) demonstrated net score decreases; and two states (15.4 percent) showed no change.
   
• Grade 4 reading evidenced a pattern similar to that of grade 4 mathematics in that eight of the 13 states (61.5 percent) had greater growth over time compared to the national average, while five of the 13 states (38.5 percent) had less growth.
   
• For grade 8 reading, eight states (61.5 percent) had net score increases after their A-F systems were implemented; two states (15.4 percent) had net score decreases; and three states (23.1 percent) showed no change.
   
• In grade 8 reading, states evidenced a pattern similar to that of grade 8 mathematics in that the majority of states demonstrated less growth compared to the nation’s average growth. Five of 13 states (38.5 percent) had greater growth over time compared to the national average, while six states (46.2 percent) had less growth, and two states (15.4 percent) exhibited comparable growth.
   
• In sum, the NAEP data slightly favored A-F states on grade 4 mathematics and grade 4 reading; half of the states increased and half of the states decreased in achievement post A-F implementation on grade 8 mathematics; and a plurality of states decreased in achievement post A-F implementation on grade 8 reading.
   
• The authors note that while some states did have consistent patterns across all four exams, either in favor or against the state’s NAEP performance, in comparison to the nation, the majority of states did not yield consistent results, and across all 13 states there was a lack of consistency across these states’ NAEP trends over time.
   
• “In reality, how these states performed post-implementation is not much different from random, or a flip of the coin,” said study co-author Audrey Amrein-Beardsley, a professor of education at Arizona State University.
   
• “These results should speak directly to other states already, or considering, investing human and financial resources in such state-level, test-based accountability policies,” said Amrein-Beardsley. “This is especially true given the concerns surrounding the use of letter grades, including that they may not provide complete pictures of schools and their students.”
   
• According to the authors, this is the first empirical study that they are aware of that offers evidence on the effectiveness of A-F accountability systems in improving student achievement.

To talk to the study authors, please contact ÐÜèÊÓƵCommunications: Tony Pals, Director of Communications, tpals@aera.net, (202) 238-3235, (202) 288-9333 (cell); Collin Boylin, Communications Associate, cboylin@aera.net, (202) 238-3233, (860) 490-8326 (cell).

# # # 

The ÐÜèÊÓƵ (AERA) is the largest national interdisciplinary research association devoted to the scientific study of education and learning. Founded in 1916, ÐÜèÊÓƵadvances knowledge about education, encourages scholarly inquiry related to education, and promotes the use of research to improve education and serve the public good. Find ÐÜèÊÓƵon , , and .