# P2.1 — HTF BIAS FORENSIC TRACE (ROOT CAUSE) Status: **ROOT CAUSE FOUND + REPRODUCED (14849/14850 rows)** Date: 2026-08-21 Classification: **B = EA bug** (runtime indexing deviates from v4.4 and from the comment's intent) --- ## 1. Problem ```text f0 (D1 bias) mismatch py_full vs EA = 0.3609 f1 (H4 bias) mismatch = 0.4516 f2 (H1 bias) mismatch = 0.5130 f18_conf mismatch = 0.5189 (derived: follows f0-f2) ``` The Python replica `tf_bias_asof` (200 most-recent bars ending at the last closed bar) does not reproduce the EA `BTTFBias` output. --- ## 2. Method 1. Trace the call chain `ComputeAll -> g_bTF1/2/3 = BTTFBias(sD1/sH4/sH1)`. 2. Read the Engine 1 cache (`AF_Engine1_MTFData.mqh`): closed-bar lock, 250-bar capacity per HTF slot, series index `bars[0]=newest`. 3. Test window hypotheses against the parity mode-2 CSV (14850 rows): PY / CLOSED / ANCHOR / ANCHOR250 — all FAILED (mismatch 0.36-0.79). 4. Added debug mode 3 to the EA (full HTF cache dump at 5 picked timestamps + compact per-bar count/need/bias rows). Ran tester 2026.01.01-07.21. 5. Minimal EA isolate: VERBATIM copy of BTTFBias + pivot/break trace on the exact window from the tester cache. 6. Quantitative verification: `EA_hat = tf_bias_asof(E_ea - 50)` vs CSV. --- ## 3. Evidence ### 3.1 EA cache == training npz data (IDENTICAL INPUT STATE) Mode 3 FULLDUMP (5 timestamps) compared bar-by-bar with `Files\AlgoForge\Data\XAUUSD_{D1,H4,H1}.npz`: ```text D1/H4/H1 @ 2025.12.31 20:00, 2026.01.05 12:00, 2026.03.18 01:00, 2026.06.16 13:30, 2026.07.20 15:45 bars=250 matched=250 diff=0 missing=0 max|d| open=0.00000 high=0.00000 low=0.00000 close=0.00000 ``` Conclusion: **not an input-state mismatch** (P1 suspected the tester D1 cache differed from the training feed — NOT proven; caches are identical). ### 3.2 The tester provides ~1 year of pre-test history Tester log (mode 3): ```text XAUUSD,H1: history cache ... contains 5907 bars from 2025.01.02 01:00 to 2025.12.31 20:00 XAUUSD,H4: history begins from 2025.01.02 00:00 XAUUSD,Daily: history begins from 2025.01.02 00:00 ``` The full HTF cache (250 bars) is available FROM THE FIRST ROW -> `need = min(200, count) = 200` always, `count = 250` always (the tester pre-loads 1 year before FromDate). ### 3.3 EA BTTFBias window = 200 OLDEST bars of the 250-bar cache EA (`AlgoForge_Backtest_Baseline.mq5`): ```mql5 string BTTFBias(int slot) { int need=MathMin(200,e1.Count(slot)); // need=200, count=250 ... for(int i=0;i **EA adaptation bug**, not legacy semantics. ### 3.5 Quantitative reproduction `EA_hat = tf_bias_asof(E_ea - 50)` where `E_ea` = last closed HTF bar at `tc = t + 900` (M15 bar close time), lag = `count - need = 250 - 200 = 50`: ```text D1: mismatch = 0 / 14850 rate = 0.0000 H4: mismatch = 0 / 14850 rate = 0.0000 H1: mismatch = 1 / 14850 rate = 0.0001 ``` ### 3.6 The single H1 exception — cache staleness during a trading gap Row `2026-02-16 21:30`: EA H1 = -1, hat = +1. Cause: H1 data gap `21:00 -> 01:00` (daily break). The 20:00 H1 bar closes at 21:00, but `e1.Refresh()` only rebuilds when `Bars(symbol,tf)` changes; the next bar only appears at 01:00 -> the H1 cache stays ending at 19:00 during the gap. Effect: the window endpoint shifts by one extra bar. Only 1/14850 rows flip (bias robustness). --- ## 4. Root Cause (summary) ```text EA BTTFBias uses GetBar(count-1-i) -> window = the 200 OLDEST bars of the 250-bar cache = HTF bias lagged by 50 bars (D1: 50 days, H4: ~8.3 days, H1: ~2.1 days). Not an input-state mismatch (cache == npz). Not legacy semantics (v4.4 uses the 200 newest bars). Deterministic & reproducible (14849/14850). ``` Per protocol classification: **B = EA bug**. The secondary effect (gap) is an Engine 1 design characteristic (Refresh-on-Bars-change), documented as an edge case. --- ## 5. Source-of-Truth Decision ```text SOURCE_OF_TRUTH = CORRECTED SEMANTICS (not EA-as-deployed, not old py-full) Definition: f_htf(t) = BTTFBias over the 200 newest CLOSED HTF bars ending at E_ea(t) E_ea(t) = last closed HTF bar at tc = t + 900 = searchsorted(ht, tc - period, 'right') - 1 = tf_bias_asof(E_ea) Runtime (EA) : fix BTTFBias -> window = the 200 NEWEST bars of the cache (GetBar(slot, need-1-i) or equivalent), closed-bar lock kept. Training (py) : as-of M15 bar close mapping (E_ea), not as-of open (containing-1). Note on the 1-bar difference vs v4.4 (window ending at the inert forming bar): documented as a legacy nuance; the corrected contract uses the last closed bar (E_ea) — non-look-ahead, deterministic, one definition for both sides. RETRAINING IS PROHIBITED before full parity (P2.5) is complete. ``` --- ## 6. Evidence Artifacts ```text ml/parity/htf_forensic.py tests 4 window hypotheses vs CSV (PY/CLOSED/ANCHOR) ml/parity/compare_htf_dump.py EA FULLDUMP vs npz (identical) ml/parity/verify_bttf_dump.py bttf_window on the exact EA input ml/parity/verify_bttf_rootcause.py EA_hat = tf_bias_asof(E_ea-50) -> 0/0/1 mismatches ml/parity/probe_h4_window.py H4 window search (all -1 from npz) ml/parity/check_h1_gap.py H1 21:00-01:00 gap (cache staleness) Experts/AlgoForge_BTTF_Isolate.mq5 verbatim BTTFBias isolate + trace Experts/AlgoForge_Backtest_Baseline.mq5 (debug mode 3 added, additive) Profiles/Tester/AlgoForge_HTFDebug...ini / AlgoForge_BTTF_Isolate...ini Tester/Agent-127.0.0.1-3000/MQL5/Files/AlgoForge_htf_debug_XAUUSD_M15.csv (dump) ``` --- ## 7. P2.1 Status ```text Root cause : FOUND (B = EA bug, 50-bar lag) Source of truth : DECIDED (corrected semantics, tf_bias_asof(E_ea)) Runtime semantics : 200 newest bars ending at E_ea (BTTFBias fix) Training semantics : as-of close tc, tf_bias_asof(E_ea) Parity (current) : EA-as-deployed = tf_bias_asof(E_ea-50) -> 0/0/1 mismatches (runtime + training fix implemented in P2.5) ```