| Helena Yue Jia
Southern Methodist University
Diagnostics of using sampling weights in hierarchical models
FINAL REPORT
National Assessment of Educational Progress (NAEP) is collected via a complex multi-stage design. Ignoring sampling design in the NAEP data analysis might lead to biased estimation. Incorporating sampling weights is one way to reduce estimation bias due to unequal probabilities of selection. Several weighting methods have been proposed in the literature (Pfeffermann et al, 1998, Korn and Graubard, 2003) for estimating the parameters of hierarchical models, of which random effects models are a special case. In this dissertation, we consider estimation of parameters of random effects models assuming sampling designs which share many NAEP design features.
We first develop analytic bias expressions for the unweighted and weighted method of moments (MOM) estimators of the mean and variance components from a random effect model. The accuracy of the expressions is evaluated through Monte Carlo simulation. The derived analytic bias expressions explicitly display the relationship between the bias of the estimators and the sampling design features, including sample size and design informativeness. It is shown that the unweighted estimators are biased for informative designs. First-order weighting method effectively reduces bias for the estimators of the mean, and performs well to estimate the variance components when cluster sample size and ICC level are moderate. However, for the estimation of variance components, this weighting method should be used with caution for small cluster sizes (less than 20), particularly when ICC is less than 0.2.
In addition, 1.5-order weighted MOM estimates are proposed for the estimation of variance components. For the same sampling designs discussed in this dissertation, the 1.5-order weighted MOM estimators yield less bias than the first-order weighted ones if ICC is less than 0.5. Scaled 1.5-order weighted MOM estimators of variance components are also discussed, which result in much less bias for all the designs we have considered.
Back to Funded Dissertation Grants Page |