146 lines
7 KiB
Markdown
146 lines
7 KiB
Markdown
# P2 SOURCE-OF-TRUTH REPORT — SNIPERGOLD_ML
|
|
|
|
Date: 2026-08-21
|
|
Session: P2 SOURCE-OF-TRUTH ADJUDICATION
|
|
Forge HEAD (end): 0bdd58afc9bc16f693cdefa4af711edaea37d713 (unchanged — no commit without review)
|
|
Status: **P2 = COMPLETE** (all 6 success criteria met; corrected baseline P2.6 not yet run)
|
|
|
|
---
|
|
|
|
## A. HTF Bias (P2.1)
|
|
|
|
```text
|
|
Root cause : EA BTTFBias uses GetBar(count-1-i) -> window = the 200 OLDEST bars
|
|
of the 250-bar cache (bars[249..50]), not the 200 newest.
|
|
Effect: HTF bias lagged by 50 bars (D1 50 days, H4 ~8.3 days, H1 ~2.1 days).
|
|
Not an input-state mismatch (cache == npz, 250/250 bars identical);
|
|
not legacy (v4.4 uses CopyHigh(0,200) = 200 newest).
|
|
Source of truth : CORRECTED SEMANTICS (classification B = EA bug)
|
|
Runtime : BTTFBias fixed -> GetBar(need-1-i) = 200 newest bars
|
|
ending at E_ea (last closed HTF bar at tc=t+900)
|
|
Training : tf_bias_asof(E_ea), E_ea = searchsorted(ht, tc-period, 'right')-1
|
|
Parity : fixed EA vs training = f0 1/14850, f1 1/14850, f2 0/14850
|
|
(2 exceptions = cache staleness at daily-break boundary, 22:45 rows)
|
|
```
|
|
|
|
## B. EQH/EQL (P2.2)
|
|
|
|
```text
|
|
Chosen semantics : legacy runtime/v4.4 (provisional production source of truth)
|
|
- CONSECUTIVE pivot pairs in the swing list (g_sp[i-1], g_sp[i]), same type
|
|
- tol = EQ_TOL_ATR (0.10) x CURRENT row-bar ATR (not pivot ATR)
|
|
- pairs in window [t-649, t-50]; swept when high/low breaks the 2nd pivot price
|
|
Reason : runtime reproduction 0/0 mismatch (14850 rows); v4.4 == EA (faithful);
|
|
no redesign this session (recorded: non-monotonic flag due to row ATR)
|
|
Legacy status : v4.4 semantics = TRUSTED
|
|
Parity : f10 0/14850, f11 0/14850 (Python reconstruction)
|
|
```
|
|
|
|
## C. Context Window (P2.3)
|
|
|
|
```text
|
|
Minimum required : 700-bar M15 cache (600 analyzed + 100 warmup) for full parity;
|
|
HTF cache 250 (200 used). A LOOKBACK of 600 ALONE is technically
|
|
insufficient (rel pivots [50,99] = absolute [t-649,t-600] also
|
|
determine sw_high/sw_low & EQH/EQL pairs; pivot confirmation needs bars 0..99).
|
|
Production window: cache 700 / HTF 250 (runtime deployment window = SOURCE OF TRUTH)
|
|
Training window : 700 slice ending at the row bar, begin=100; HTF 200 newest ending at E_ea
|
|
Parity : f6/f14/f15/f17 = 0/14850 (windowed sw_high/sw_low [t-649,t-50])
|
|
```
|
|
|
|
## D. FEATURE_CONTRACT
|
|
|
|
```text
|
|
Status : COMPLETE (v1.0, 19 features x 14 fields, one definition)
|
|
Hash : C44CC6F2B740C32D06F776BD7C3E669DC5A8A6DE0484230544EBFFCF517D38DD
|
|
File : docs/FEATURE_CONTRACT.md (publish mirror, uncommitted)
|
|
Content : temporal anchor tc=t+900; closed-bar rule; M15 window 700/begin=100;
|
|
HTF E_ea; EQH/EQL ATR@row; per-feature formula; missing-data; parity eps.
|
|
```
|
|
|
|
## E. Full Feature Parity (P2.5)
|
|
|
|
```text
|
|
Timestamp match : 14850/14850 (missing 0); feed close max|d|=0.000000
|
|
Feature mismatch : 14/14850 exception rows (0.09%), 17 features exact:
|
|
f0=1 (22:45 D1 break), f1=1 (22:45 H4 break),
|
|
f13=11 (tester tick-level data vs broker, 07-02), f18=1 (derived f0)
|
|
Prediction mismatch: 2/14850 (LONG & SHORT, tol 6e-5 = mode-0 CSV rounding);
|
|
max|dp| LONG 6.1e-2 / SHORT 1.1e-1 only on the 2 feature-exception rows;
|
|
mean|dp| ~1e-5 (rounding); Python model == MQL5 for identical features
|
|
Evidence : docs/P2_5_PARITY.md; ml/parity/parity_p2.py, parity_prediction.py
|
|
```
|
|
|
|
## F. Corrected Baseline
|
|
|
|
```text
|
|
Status: NOT YET RUN (P2.6 handover)
|
|
Reason: retraining requires integrating build_features_p2 into train_model.py + full
|
|
training + calibration + evaluation + hash capture — a separate work package
|
|
that must not be rushed at session close.
|
|
Procedure (when run):
|
|
- integrate the corrected build_features into the training pipeline (single source of definition)
|
|
- same architecture (MLP 19->12->2), seed 42, label 24x0.75ATR, purge H_LABEL
|
|
- record: AUC_LONG/SHORT, VAL/TEST, calibration, dataset/feature/model/config hash, git SHA
|
|
- status name: POST-P2 PARITY-CORRECTED BASELINE (not yet deployment evidence)
|
|
```
|
|
|
|
## G. Evidence Classification
|
|
|
|
```text
|
|
TRUSTED : P2.1 root cause (50-bar lag, reproduction 14849/14850)
|
|
P2.2 EQH/EQL semantics (reproduction 0/0)
|
|
P2.3 700-bar window (reproduction 0/0)
|
|
P2.5 parity (17 features exact; exceptions documented)
|
|
prediction parity (Python model == MQL5)
|
|
corrected runtime dump (AlgoForge_bt_features_fixed_... 5F8AB5CB...)
|
|
INCONCLUSIVE: P1 baseline 0.6582/0.6582 (still TRUSTED for the old feed, but the
|
|
feed has moved to corrected semantics -> P2.6 rerun needed)
|
|
SUPERSEDED : old baseline 0.6270/0.6207; runtime cross-feed 0.5305/0.5487
|
|
(contaminated by the now-measured and fixed parity gap)
|
|
INVALID : no new verdict behind without evidence
|
|
```
|
|
|
|
## H. Remaining Risks
|
|
|
|
```text
|
|
1. 14 parity exception rows (0.09%) — runtime cache-staleness at break boundaries &
|
|
tester tick-level data; reproducible, tolerance documented.
|
|
2. Model freeze (SniperGold_ML.mqh) trained on OLD semantics — MUST NOT be used for
|
|
claims on the corrected feed until the P2.6 baseline.
|
|
3. Local EA modified (BTTFBias fix + debug mode 3) — NOT yet committed/reviewed.
|
|
4. build_features_p2 not yet integrated into train_model.py (still a parity module).
|
|
5. Non-monotonic EQH/EQL features (row ATR) — recorded as a future experiment.
|
|
6. Cache staleness (Refresh-on-Bars-change) is an Engine 1 characteristic — documented,
|
|
not changed in this session.
|
|
```
|
|
|
|
## I. Recommendation
|
|
|
|
```text
|
|
1. Proceed to P2.6 (corrected baseline) with procedure F above — retrain MLP
|
|
19->12->2 on the corrected feed, seed 42, purge H_LABEL.
|
|
2. Review & commit the P2 artifacts (docs + build_features_p2 + EA fix + parity dump)
|
|
through an explicit review process (not auto-commit).
|
|
3. After the P2.6 baseline, re-evaluate the runtime cross-feed AUC on the corrected feed.
|
|
4. Only then does SMC -> MTF -> Regime -> Temporal ML -> Meta-Label research have a
|
|
valid experimental foundation.
|
|
```
|
|
|
|
---
|
|
|
|
## Appendix — session artifacts (uncommitted, awaiting review)
|
|
|
|
```text
|
|
docs/FEATURE_CONTRACT.md, P2_1_HTF_FORENSIC.md, P2_2_EQH_EQL_FORENSIC.md,
|
|
P2_3_CONTEXT_WINDOW.md, P2_5_PARITY.md (publish mirror)
|
|
ml/parity/htf_forensic.py, compare_htf_dump.py, verify_bttf_dump.py,
|
|
verify_bttf_rootcause.py, probe_h4_window.py, check_h1_gap.py,
|
|
recon_eq.py, verify_window.py, verify_structure_window.py, verify_pivot_eq.py,
|
|
diag_*.py, build_features_p2.py, parity_p2.py, parity_prediction.py,
|
|
AlgoForge_bt_features_fixed_XAUUSD_M15.csv, AlgoForge_bt_prob_fixed_XAUUSD_M15.csv
|
|
Experts/AlgoForge_Backtest_Baseline.mq5 (BTTFBias fix + debug mode 3) (local)
|
|
Experts/AlgoForge_BTTF_Isolate.mq5 (verbatim isolate) (local)
|
|
Profiles/Tester/AlgoForge_HTFDebug*.ini, AlgoForge_BTTF_Isolate*.ini,
|
|
AlgoForge_Parity_Fixed*.ini, AlgoForge_Prob_Fixed*.ini (local)
|
|
```
|