You can manage bookmarks using lists, please log in to your user account for this.
Media type:
E-Article
Title:
Effects of Design Properties on Parameter Estimation in Large-Scale Assessments
Contributor:
Hecht, Martin;
Weirich, Sebastian;
Siegle, Thilo;
Frey, Andreas
imprint:
SAGE Publications, 2015
Published in:Educational and Psychological Measurement
Language:
English
DOI:
10.1177/0013164415573311
ISSN:
0013-1644;
1552-3888
Origination:
Footnote:
Description:
<jats:p> The selection of an appropriate booklet design is an important element of large-scale assessments of student achievement. Two design properties that are typically optimized are the balance with respect to the positions the items are presented and with respect to the mutual occurrence of pairs of items in the same booklet. The purpose of this study is to investigate the effects of these two design properties on bias and root mean square error of item parameter estimates from the Rasch model. First, position effects are estimated using data from a large-scale assessment study measuring the competencies of 19,107 ninth graders in science. These results were then used for a simulation study with 1,540 booklet designs with systematically varied position balance and cluster pair balance. The simulation results showed a small effect of position balancing on bias and root mean square error of the item parameter estimates while the cluster pair balance was ignorable. This null effect is actually good news for test designers since it allows for deliberately reducing the degree of cluster pair balance without negative effects on item parameter estimates. However, it is recommended to try to achieve a high position balance when designing large-scale assessment studies. </jats:p>