The Probability of Backtest Overfitting (PBO), introduced by Bailey and López de Prado (2014), uses Combinatorial Symmetric Cross-Validation (CSCV) to estimate the probability that a strategy selected for maximum in-sample performance will underperform the median strategy out-of-sample.
The CSCV procedure
- Divide the track record into S equal sub-periods
- For each possible combination of S/2 sub-periods as the training set, evaluate all strategy variants on the training set and select the best performer
- Test whether the best training-set strategy also outperforms the median strategy on the complementary S/2 test sub-periods
- PBO = the fraction of all combinations where it does not
A PBO of 0.5 means the best IS strategy underperforms the median OOS half the time — i.e., the research process is producing no more signal than random selection. A PBO near 0 indicates genuine IS-to-OOS predictive stability.
PBO is complementary to the Deflated Sharpe Ratio: both address the inflation from multiple testing, from different methodological angles.