Campbell, Douglas L.
[Author];
Brodeur, Abel
[Author];
Dreber, Anna
[Author];
Johannesson, Magnus
[Author];
Kopecky, Joseph
[Author];
Lusher, Lester
[Author];
Tsoy, Nikita
[Author]
The robustness reproducibility of the American Economic Review
Description:
We estimate the robustness reproducibility of key results from 17 non-experimental AER papers published in 2013 (8 papers) and 2022/23 (9 papers). We find that many of the results are not robust, with no improvement over time. The fraction of significant robustness tests (p<0.05) varies between 17% and 88% across the papers with a mean of 46%. The mean relative t/z-value of the robustness tests varies between 35% and 87% with a mean of 63%, suggesting selective reporting of analytical specifications that exaggerate statistical significance. A sample of economists (n=359) overestimates robustness reproducibility, but predictions are correlated with observed reproducibility.