forked from Lerooy/pbo-cscv-engine
50 lines
2.4 KiB
Markdown
50 lines
2.4 KiB
Markdown
# Probability of Backtest Overfitting (PBO) — CSCV Engine for MQL5
|
|
|
|
Source code for the article **"Measuring the Probability of Backtest Overfitting of Your MT5
|
|
Optimization (Part 1): The CSCV Engine in MQL5"** (mql5.com article 24095), by Astralys LLC.
|
|
|
|
The engine implements the Probability of Backtest Overfitting of Bailey, Borwein, Lopez de Prado
|
|
and Zhu (2015), computed through Combinatorially Symmetric Cross-Validation. You feed it a matrix
|
|
of per-bar returns, one column per parameter combination, and it returns one number: how likely
|
|
your optimization picked noise rather than signal. All 12,870 splits of 16 partitions run in about
|
|
0.2 seconds on 1,300 combinations and 3,610 bars.
|
|
|
|
## Files
|
|
|
|
| File | What it is |
|
|
|---|---|
|
|
| `Include/PBO/CSCVEngine.mqh` | The engine. Depends on nothing but the standard library. |
|
|
| `Include/PBO/MACDp.mqh` | MACD on price: distance of the close to its own SMA, optionally as a percentage. |
|
|
| `Include/PBO/BearsPower.mqh` | Bears Power: distance of the low to an EMA of the close, optionally as a percentage. |
|
|
| `Scripts/PBO/RunPBO.mq5` | Builds the 1,300-column returns matrix on the chart symbol, runs the engine, runs the control experiments, exports CSVs. |
|
|
| `Scripts/PBO/TestCSCVEngine.mq5` | Two known-answer validation cases: pure noise (PBO must be near 50%) and one planted winner (PBO must be 0, logit exactly ln(200)). |
|
|
| `Scripts/PBO/ExportBars.mq5` | Exports the bar history of the chart it is dropped on to CSV, used to draw the article figures. |
|
|
|
|
## Install
|
|
|
|
Copy `Include/PBO/` into `MQL5\Include\` and `Scripts/PBO/` into `MQL5\Scripts\` of your terminal
|
|
data folder, then compile the scripts in MetaEditor (F7).
|
|
|
|
## Minimal use on your own optimization
|
|
|
|
```cpp
|
|
#include <PBO/CSCVEngine.mqh>
|
|
|
|
CCSCVEngine engine;
|
|
engine.SetPartitions(16);
|
|
engine.SetReturns(retMatrix, bars, sets); // bars x sets, log returns, row-major
|
|
engine.Run();
|
|
PrintFormat("PBO = %.2f%%", engine.PBO());
|
|
```
|
|
|
|
Two constraints carry over from the article: the returns must be logarithmic, because the engine
|
|
rests on them being additive, and the signal that decides the position on bar `t` must be read on
|
|
bar `t - 1`.
|
|
|
|
## Reference
|
|
|
|
Bailey, D., Borwein, J., Lopez de Prado, M., Zhu, Q. (2015), *The Probability of Backtest
|
|
Overfitting*, Journal of Computational Finance.
|
|
|
|
The validation study behind the article: Bergerat, J. (2020), *On the Relevance of Optimizing
|
|
Technical Indicators on U.S. Stock Markets*, University of Lausanne. SSRN abstract 5212650.
|