• Medientyp: E-Book
  • Titel: Can AI solve the peer review crisis? : a large-scale experiment on LLM's performance and biases in evaluating economics papers
  • Beteiligte: Pataranutaporn, Pat [Verfasser:in] ; Powdthavee, Nattavudh [Verfasser:in] ; Maes, Pattie [Verfasser:in]
  • Erschienen: Bonn, Germany: IZA - Institute of Labor Economics, January 2025
  • Erschienen in: Forschungsinstitut zur Zukunft der Arbeit: Discussion paper series ; 17659
  • Umfang: 1 Online-Ressource (circa 73 Seiten); Illustrationen
  • Sprache: Englisch
  • Identifikator:
  • Schlagwörter: Artificial Intelligence ; peer review ; large language model (LLM) ; bias in academia ; economics publishing ; equity-efficiency tradeoff ; Graue Literatur
  • Entstehung:
  • Anmerkungen:
  • Beschreibung: We investigate whether artificial intelligence can address the peer review crisis in economics by analyzing 27,090 evaluations of 9,030 unique submissions using a large language model (LLM). The experiment systematically varies author characteristics (e.g., affiliation, reputation, gender) and publication quality (e.g., top-tier, mid-tier, low-tier, AI-generated papers). The results indicate that LLMs effectively distinguish paper quality but exhibit biases favoring prominent institutions, male authors, and renowned economists. Additionally, LLMs struggle to differentiate high-quality AI-generated papers from genuine top-tier submissions. While LLMs offer efficiency gains, their susceptibility to bias necessitates cautious integration and hybrid peer review models to balance equity and accuracy.
  • Zugangsstatus: Freier Zugang