5a4b36bb61
- Implemented AFML part A for testing the dip-z book against search artifacts, including PBO, DSR, and CPCV metrics. - Developed AFML part B to generate time and tick bars from M1 broker data, including return distribution statistics. - Created AFML part C to build a pipeline for dip-z primary analysis, incorporating features and a random forest model for classification. - Added HCC history decoder to read and process broker M1 `.hcc` files, ensuring proper handling of data structure and integrity.
3,3 КиБ
AFML campaign - pre-registered plan (2026-09-27)
Lopez de Prado, Advances in Financial Machine Learning, applied to the validated book (4-index H4 vol-gated dip-z, long, risk 0.25%). Criteria written BEFORE any result was seen. Nothing below is tuned after the fact; any change is logged as a deviation with its reason.
A. Is the book a product of the search? (AFML ch.11-14)
Trial family = the neighbourhood the book was picked from, on real broker H4 (2021-01..2026-08), 4 indices, one equity curve per config: z threshold {-1.0,-1.25,-1.5,-1.75,-2.0} x z window {10,15,20,30,40} x max bars {5,10,15,20} x vol gate {none, >=.30, >=.50, >=.70} x stop ATR {2,3,4} = 1,200 configs.
- PBO by CSCV (Bailey, Borwein, Lopez de Prado, Zhu 2015), 16 monthly-block partitions, selection metric = Sharpe of monthly returns. PASS: PBO <= 0.20.
- Deflated Sharpe Ratio of the chosen config, with the variance of trial Sharpes measured on the 1,200-config family and N reported at 1,200 and at a project-wide 5,000 (style scan + FX families + state scans). PASS: DSR >= 0.95 at N = 1,200.
- CPCV (AFML ch.12): 10 time groups, 2 test groups, purge = 20 H4 bars, embargo = 1%; select best config on train by ret/DD, score on test; rebuild the 9 backtest paths. Report the distribution of path Sharpe and path maxDD at 0.25%. PASS: every path maxDD <= 5% and median path ret/DD >= 2.
B. Sampling outside the clock (AFML ch.2)
Broker M1 .hcc (2022-01..2026-08, the only M1 years for indices) aggregated into
(a) time H4 bars and (b) TICK bars (fixed count of broker tick-volume per bar),
threshold set per symbol so the average bar count equals the time-H4 count. Volume
and dollar bars are NOT possible: CFD feeds carry no traded volume. M1
granularity means a tick bar closes on the M1 bar where the count is crossed.
- Statistical claim: tick-bar log returns closer to normal (lower excess kurtosis, lower Jarque-Bera) and lower |lag-1 autocorr| of squared returns. Reported, not a gate - this is the book's claim, checked on our data.
- Trading claim: the identical dip-z rule (+ gate), same bar-count parameters, same constant cost (median broker spread), on tick bars vs time bars. PASS: tick bars improve portfolio ret/DD at >= 90% of time-bar cadence, in BOTH halves (2022-23, 2024-26).
C. The whole pipeline combined (ch.3, 4, 5, 7, 8)
On whichever bars won B (time bars if tick bars did not win): primary = dip-z signal; label = triple barrier (stop 3 ATR / exit at SMA / 10 bars) -> 1 if net R > 0; features = FFD log price at the minimum d passing ADF, FFD of the z-window SMA gap, vol percentile, bar duration (tick bars), recent return / range, cross-index agreement (how many of the 4 indices are in a dip); model = random forest, sample weights = average uniqueness (ch.4), purged k-fold
- embargo (ch.7), 5 folds, plus walk-forward refits. PASS: pooled OOS AUC >= 0.55 with bootstrap 95% CI excluding 0.50, AND the filtered book beats the unfiltered one on ret/DD in the walk-forward test.
Priors (logged so the result can be judged against them): C has failed three times in this project on bar-geometry features (AUC 0.50-0.56); the new inputs here are FFD memory and tick sampling. Expect A to show a real but search-inflated Sharpe; B is genuinely unknown.