Anmerkungen:
Nach Informationen von SSRN wurde die ursprüngliche Fassung des Dokuments July 15, 2016 erstellt
Beschreibung:
One of the key assumptions of quantitative trading strategy evaluation is that Type II errors (missed discoveries) are preferable to Type I errors (false discoveries.) However, practitioners have known for long that the statistical properties of some genuine trading strategies are often indistinguishable from those of random trading strategies. Therefore, any adjustments of statistics to guard against p-hacking increase Type II error unless the power of the test is high. At the same time, the power of the test is limited by insufficient samples and changing market conditions. Furthermore, genuine strategies with statistical properties that are similar to those of random strategies may overfit due to favorable market conditions but fail when market conditions change. These facts severely limit the effectiveness of quantitative claims about trading strategy evaluation. Practitioners have instead resorted to Monte Carlo simulations and stochastic modeling in an effort to increase the chances of identifying robust trading strategies but these methods also have severe limitations due to changing market conditions, selection bias and data snooping. In this paper we present two examples that demonstrate the limitation of quantitative evaluation of trading strategies and we claim that the most effective way of guarding against overfitting and selection bias is by limiting the applications of backtesting to a class of strategies that employ similar but simple predictors of price. We claim that determining when market conditions change is in many cases fundamentally more important than any quantitative claims about trading strategy evaluation