pbo-cscv-engine/README.md

Probability of Backtest Overfitting (PBO) — CSCV Engine for MQL5

Source code for the article "Measuring the Probability of Backtest Overfitting of Your MT5 Optimization (Part 1): The CSCV Engine in MQL5" (mql5.com article 24095), by Astralys LLC.

The engine implements the Probability of Backtest Overfitting of Bailey, Borwein, Lopez de Prado and Zhu (2015), computed through Combinatorially Symmetric Cross-Validation. You feed it a matrix of per-bar returns, one column per parameter combination, and it returns one number: how likely your optimization picked noise rather than signal. All 12,870 splits of 16 partitions run in about 0.2 seconds on 1,300 combinations and 3,610 bars.

Files

File	What it is
Include/PBO/CSCVEngine.mqh	The engine. Depends on nothing but the standard library.
Include/PBO/MACDp.mqh	MACD on price: distance of the close to its own SMA, optionally as a percentage.
Include/PBO/BearsPower.mqh	Bears Power: distance of the low to an EMA of the close, optionally as a percentage.
Scripts/PBO/RunPBO.mq5	Builds the 1,300-column returns matrix on the chart symbol, runs the engine, runs the control experiments, exports CSVs.
Scripts/PBO/TestCSCVEngine.mq5	Two known-answer validation cases: pure noise (PBO must be near 50%) and one planted winner (PBO must be 0, logit exactly ln(200)).
Scripts/PBO/ExportBars.mq5	Exports the bar history of the chart it is dropped on to CSV, used to draw the article figures.

Install

Copy Include/PBO/ into MQL5\Include\ and Scripts/PBO/ into MQL5\Scripts\ of your terminal data folder, then compile the scripts in MetaEditor (F7).

Minimal use on your own optimization

#include <PBO/CSCVEngine.mqh>

CCSCVEngine engine;
engine.SetPartitions(16);
engine.SetReturns(retMatrix, bars, sets);   // bars x sets, log returns, row-major
engine.Run();
PrintFormat("PBO = %.2f%%", engine.PBO());

Two constraints carry over from the article: the returns must be logarithmic, because the engine rests on them being additive, and the signal that decides the position on bar t must be read on bar t - 1.

Reference

Bailey, D., Borwein, J., Lopez de Prado, M., Zhu, Q. (2015), The Probability of Backtest Overfitting, Journal of Computational Finance.

The validation study behind the article: Bergerat, J. (2020), On the Relevance of Optimizing Technical Indicators on U.S. Stock Markets, University of Lausanne. SSRN abstract 5212650.