• Medientyp: E-Book; Bericht
  • Titel: The anatomy of out-of-sample forecasting accuracy
  • Beteiligte: Borup, Daniel [VerfasserIn]; Goulet Coulombe, Philippe [VerfasserIn]; Rapach, David E. [VerfasserIn]; Montes Schütte, Erik Christian [VerfasserIn]; Schwenk-Nebbe, Sander [VerfasserIn]
  • Erschienen: Atlanta, GA: Federal Reserve Bank of Atlanta, 2022
  • Sprache: Englisch
  • DOI: https://doi.org/10.29338/wp2022-16
  • Schlagwörter: C53 ; machine learning ; C45 ; variable importance ; inflation ; Shapley value ; G17 ; out-of-sample performance ; C22 ; loss function ; E37
  • Entstehung:
  • Anmerkungen: Diese Datenquelle enthält auch Bestandsnachweise, die nicht zu einem Volltext führen.
  • Beschreibung: We develop metrics based on Shapley values for interpreting time-series forecasting models, including "black-box" models from machine learning. Our metrics are model agnostic, so that they are applicable to any model (linear or nonlinear, parametric or nonparametric). Two of the metrics, iShapley-VI and oShapley-VI, measure the importance of individual predictors in fitted models for explaining the in-sample and out-of-sample predicted target values, respectively. The third metric is the performance-based Shapley value (PBSV), our main methodological contribution. PBSV measures the contributions of individual predictors in fitted models to the out-of-sample loss and thereby anatomizes out-of-sample forecasting accuracy. In an empirical application forecasting US inflation, we find important discrepancies between individual predictor relevance according to the in-sample iShapley-VI and out-ofsample PBSV. We use simulations to analyze potential sources of the discrepancies, including overfitting, structural breaks, and evolving predictor volatilities.
  • Zugangsstatus: Freier Zugang