Optimizing Implementation of Artificial‐Intelligence‐Based Automated Scoring: An Evidence Centered Design Approach for Designing Assessments for AI‐based Scoring
You can manage bookmarks using lists, please log in to your user account for this.
Media type:
E-Article
Title:
Optimizing Implementation of Artificial‐Intelligence‐Based Automated Scoring: An Evidence Centered Design Approach for Designing Assessments for AI‐based Scoring
Contributor:
Ercikan, Kadriye;
McCaffrey, Daniel F.
imprint:
Wiley, 2022
Published in:Journal of Educational Measurement
Language:
English
DOI:
10.1111/jedm.12332
ISSN:
0022-0655;
1745-3984
Origination:
Footnote:
Description:
<jats:title>Abstract</jats:title><jats:p>Artificial‐intelligence‐based automated scoring is often an afterthought and is considered after assessments have been developed, resulting in nonoptimal possibility of implementing automated scoring solutions. In this article, we provide a review of Artificial intelligence (AI)‐based methodologies for scoring in educational assessments. We then propose an evidence‐centered design framework for developing assessments to align conceptualization, scoring, and ultimate assessment interpretation and use with the advantages and limitations of AI‐based scoring in mind. We provide recommendations for defining construct, task, and evidence models to guide task and assessment design that optimize the development and implementation of AI‐based automated scoring of constructed response items and support the validity of inferences from and uses of scores.</jats:p>