| Jie Li
Boston College
The effect of accommodations for special-needs students in NAEP
FINAL REPORT:
The National Assessment of Educational Progress (NAEP) is the only nationally representative, continuing assessment of what America's students know and can do in various subject areas. As one part of the student population, special-needs students, which refer to both students with disabilities (SDs) and students with limited English proficiencies (LEPs), are also included in NAEP assessments. Beginning in 1996, NAEP began offering test accommodations to special-needs students in order for them to participate meaningfully in the assessments. Although the use of accommodations increased participation rates in NAEP, the validity of results obtained under accommodated testing situations has not been fully established. Inferences drawn from test results are valid only if the analysis models fit not only the calibration sample as a whole but also the student subgroups within the calibration sample. The objective of this study is to examine whether the scaling models fit the data from LEP students with accommodations and SD students with accommodations. Data from the accommodations-permitted (AP) sample in NAEP 2000 Grade 8 mathematics state assessment were used. Fit statistics were calculated using estimated item parameters and student parameters (plausible values). Due to NAEP's complex survey design, the distribution of the fit statistics and the associated degrees of freedom cannot be exactly determined. Bootstrap resampling was used to obtain approximate sampling distributions of fit statistics and to conduct significance tests. The study examined overall model-data fit, item fit and person fit by mathematics subscales and by student groups. Results showed the LEP group had no significant subscale misfit, but the SD group had significant misfit in two of the five mathematics subscales. The SD group also had more misfit items across the five subscales than the LEP group had. One possible reason might be that the SD group is a more heterogeneous group than the LEP group. With students having various kinds of disabilities and varying degrees of disabilities, a few types of accommodations in NAEP assessment did not satisfy individual needs adequately for the SD group. Item fit analysis revealed that the dichotomization and collapsing of item categories for extended constructed-response items in scaling is problematic and may result in significant item misfit. Person fit analysis with respect to gender groups, racial groups and accommodation types shows significant results in some but not all subscales.
Back to Funded Dissertation Grants Page |