chiki2bum2/SniperGold_ML
Watch
1
0
Fork
You've already forked SniperGold_ML
 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions
SniperGold_ML/docs/TAHAP4_PLAN.md

12 KiB
Raw Permalink Blame History

Algo Forge — Phase 4: Hybrid ML Plan (2-layer LSTM + Regime)

Status: SUB-SESSIONS 1–3 COMPLETE — GATE No-Go (2-layer LSTM LOSES) — 2026-08-21. This document is the result of the Phase-4 audit + execution plan. Binding for subsequent sub-sessions. Test protocol: FASE3_BATASAN_UJI.md (G1–G7). FINAL PHASE-4 DECISION (per protocol G7, one No-Go = done, no negotiation): the hybrid 2-layer LSTM + regime ML model DOES NOT pass the deploy gate → the baseline SniperGold_ML.mqh (MLP 19→12→2, 0.627/0.621) REMAINS the deployed model. The hybrid .mqh export is CANCELLED (no model worthy of export).

1. Executive Summary

  • ±20 years of data from the HFM feed (symbol XAUUSD, 2-digit, Gold Spot):
    • D1: 4.842 bars, 2007-11-29 → 2026-08-21 (18.73 years) — fully dense ✅
    • M15: 199.386 total bars, BUT dense only since 2018 (2007–2016 ≈ 1 bar/day)
    • Dense M15: 2018-01-01 → 2026-08-21 ≈ 8.6 years ≈ 186.500 bars (≈3x the 60k baseline) ✅
  • Honest decision: the "20-year M15" target is NOT available on the HFM feed. M15 uses the dense 2018+ window; D1 uses the full 18.7 years as HTF context. Dukascopy (different feed) is only an experimental fallback — not required.
  • Deploy gate: still G1–G7 vs the baseline long 0.6270 / short 0.6207 (SniperGold_ML.mqh, MLP 19→12→2, freeze). Comparison MUST be like-for-like: the new model is tested on the same baseline cache (XAUUSDc 60k, 75/25 split) before claiming a win. AUC on new data is reported separately.

2. Old Python Pipeline Audit (Shared Projects\SniperGold_ML)

File Content Phase-4 Relevance
train_model.py (28KB) 19 SMC features + 24×0.75ATR label + pure-numpy MLP + walk-forward + .mqh export Reused (features/label/harness)
train_lstm.py (19KB) 1-layer numpy LSTM (W=32, H=32) + BPTT + bootstrap CI Developed → 2 layers
train_regime.py (11.5KB) GARCH-t MLE + 2-state HMM + RV24 (causal) Reused (regime features)
train_regime_gate.py Regime as gate/context (Phase 2b, lost) Reference — do not repeat the gate pattern
train_ensemble.py 5-seed MLP ensemble Multi-seed pattern reused
retrain_deploy.py Deploy+rollback+audit log Export/audit pattern reused
download_data.py Old 60k-bar download (XAUUSDc) Replaced by download_bars.py
features_XAUUSDc.npz 60k-bar cache, 19 features, Phase-3 freeze Read-only (for the like-for-like gate)

2.1 Baseline features (19) — runtime–training consistency

htf1_bias, htf2_bias, htf3_bias, swing_trend, internal_trend, chart_bias, eq_pos_norm, sweep_dir, choch_dir, choch_confirms, eqh_swept, eql_swept, delta_sign, delta_mag, dist_high_atr, dist_low_atr, mom20_atr, range_atr, confluence → computed from M15 + H1/H4/D1 (HTF bias), ATR-normalized, train-only standardization.

2.2 Label & split (baseline)

  • Label: forward 24 bars × 0.75 ATR (long=+1, short=−1, else 0/dropped).
  • Split: walk-forward 75/25 (split_pos = 75% of labeled bars); validation = last 15% of train.
  • Anti-lookahead: features at bar t use only data ≤ t; standardization only from train.

2.3 AlgoForge Engine 2 (source of NEW features, runtime–training consistent)

  • Agents N/C/E/P produce structure/bias/confirmation (stateless, closed-bar lock).
  • Candidate new features for Phase 4 (backlog, tested later — NOT in the data sub-session): agent N/C/E/P scores & agreement, OB/FVG freshness, sweep distance, premium/discount magnitude, displacement strength. Used only if they raise validation AUC (not by assumption).

3. Data — Results & Decisions (SUB-SESSION 1 COMPLETE)

3.1 Downloads (complete)

  • ml\download_bars.py → Files\AlgoForge\Data\XAUUSD_{M15,H1,H4,D1}.{csv,npz}
  • Continuity verification: 0 duplicates, 0 anomalies (>4 days), longest gap 96 hours (long weekend).

3.2 Density (analysis ml\probe_density.py / probe_byyear.py)

Year M15 bars/year D1 bars/year Note
2007–2016 ~260 ~260 M15 ≈ 1 bar/day (midnight UTC) → unusable for an M15 model
2017 1.490 257 transition
2018–2026 ~23.500/yr ~258/yr dense M15 (~65 bars/day, hours 2–18 UTC)

3.3 Data decisions (BINDING)

  1. M15 model window: 2018-01-01 → now (~8.6 years, ~186.500 bars). This is the honest replacement for "20 years": the broker feed does not provide denser old M15.
  2. Full 18.7-year D1: used for HTF bias features (context) + a separate D1-model experiment (backlog, optional).
  3. Label: 24 bars × 0.75 ATR kept (baseline-consistent) — note: with the new M15 density of ~65 bars/day, 24 bars ≈ 6 session hours (not 6 full calendar hours). The effect is recorded and verified when features are built (sub-session 2).
  4. Baseline cache features_XAUUSDc.npz NOT touched (Phase-3 deploy freeze; used read-only for the like-for-like gate).
  5. Symbol: XAUUSD (XAUUSDc no longer exists in the terminal). ATR-normalized features → price scale (4532 vs 45320) does not change feature interpretation.

4. Target Architecture (Phase 4)

OHLCV XAUUSD (dense M15 2018+ | D1 18.7y)
   → 19 SMC features (train_model.py, runtime-consistent)
   → causal regime features (GARCH-t sigma/ATR, HMM state prob, RV24/ATR)   [train_regime.py]
   → [optional] Engine 2 features (N/C/E/P) — only if they win in validation
   → 2-layer LSTM (pure numpy, W=32, H1=H2=32, linear→sigmoid 2-head)
   → P(long), P(short) → Platt calibration (deploy) 
   → export AlgoForge_Hybrid.mqh + self-test parity
  • Anti-lookahead: regime fitted only on train; causal filters with fixed parameters; features at bar t only ≤ t; train-only standardization; closed-bar lock.
  • 2 heads: long & short (each BCE), same as the 2-head baseline.

5. Validation Protocol (G1–G7, BINDING)

  • Seeds: [42, 7, 123, 2024, 999] (5 seeds; all reported, no cherry-picking).
  • Paired bootstrap CI: 2000× (ΔAUC vs baseline on the same test).
  • Walk-forward: 75/25 primary + 70/30 & 80/20 (G7) — all on new data.
  • Like-for-like gate: the best model (chosen from validation) is tested once on the XAUUSDc baseline test (identical split) → compare with 0.6270/0.6207.
  • The test set is touched ONCE at the end; tuning decisions ONLY from validation.
Gate Condition
G1 unit tests B1–B5 (2-layer LSTM & regime: numerical vs reference)
G2 baseline reproduced (MLP 0.627/0.621) in the same run
G3 dAUC ≥ +0.005 vs baseline (like-for-like)
G4 95% bootstrap CI does NOT contain 0
G5 gap AUC_val − AUC_test < 0.05
G6 inter-seed std < 0.01 & ≥4/5 seeds > baseline
G7 gain holds at 70/30 & 80/20 splits

ONE No-Go = done (no negotiation). Deploy freeze: SniperGold_ML.mqh untouched; Phase-4 deploy output = a NEW file (AlgoForge_Hybrid.mqh + integration).

6. Sub-Session Plan (split; each sub-session = checkpoint + handoff)

Sub-session Content Output Status
1 Survey + audit + data download + continuity verification + plan TAHAP4_PLAN.md, Files\AlgoForge\Data\*.npz (19y D1, 8.6y M15) ✅ COMPLETE
2 (THIS) build_features.py: 19 SMC features + causal regime (GARCH-t + RV24 + regime score; HMM/MSM collapsed → replaced by proxy) + label + new npz cache + unit tests B1–B5 features_XAUUSD.npz + B1–B5 PASS ✅ COMPLETE
3 train_hybrid.py: 2-layer LSTM + 2-head + unit tests B1–B5 + gate G1–G7 like-for-like audit log + Go/No-Go decision ✅ COMPLETE — No-Go
4 Export best model → AlgoForge_Hybrid.mqh + parity — ❌ CANCELLED (no model passed the gate)
5 Integration + backtest + documentation + Forge push release (baseline stays) ⏳ (Phase 5 is for the baseline, not hybrid)

Sub-session 3 results (gate — FINAL decision)

  • ml\train_hybrid.py: 2-layer LSTM (W=32, H1=H2=32) + 2-target long/short head, pure numpy; unit tests B1–B5 PASS (2-layer forward == 2x 1-layer, BPTT gradcheck rel=2.7e-5, determinism, anti-lookahead, overfit AUC=1.0).
  • Like-for-like gate on the XAUUSDc cache (identical 75/25 baseline split), 5 seeds [42,7,123,2024,999], 2000× bootstrap CI (log ml\TAHAP4_HYBRID.log):
Hybrid 2-layer (ens) Baseline target dAUC G3 G4 G6
LONG 0.6074 (mean 0.5898±0.0066) 0.6270 −0.0196 ❌ ❌ (negative CI) 0/5 seeds ❌
SHORT 0.6073 (mean 0.5900±0.0056) 0.6207 −0.0134 ❌ ❌ (negative CI) 0/5 seeds ❌
  • Additional comparison: MLP reproduction on the same split = LONG 0.6795 / SHORT 0.6560 (consistent with Phase 3); hybrid−MLP dAUC = −0.0723 / −0.0487, 95% CI ALL NEGATIVE → hybrid is significantly WORSE.
  • CONCLUSION: 2-layer LSTM + regime does NOT beat the baseline; GATE No-Go → protocol complete, no negotiation. Consistent with old evidence (Phase 3: LSTM lost on 60k bars). Full walk-forward on new data NOT run (the gate was already decisive; §7 rule to stop when >2x estimate / No-Go).

Sub-session 2 results (detail)

  • ml\build_features.py (reuses train_model.py/train_regime.py; runtime–training consistent features):
    • 19 SMC features identical to baseline + 24×0.75ATR label, dense 2018+ window.
    • Causal regime: t-GARCH(1,1) fitted with classic bounds + 0.5% winsor → interior (alpha=0.0615, beta=0.9230, persistence=0.9844, nu=7.12); causal sigma (unconditional init); z standardization train-only; RV24 from raw returns.
    • 2-state Gaussian HMM (MSM) COLLAPSES on this feed (P(high)=1 on z, |z|, log(RV24), RV24 — evidence diag_hmm.py; consistent with the Phase-2 lesson). Replaced by a causal logistic regime score (RV24 vs train median/MAD): mean 0.585, std 0.229, test mean 0.701 (2024+ is indeed volatile). HONEST DECISION recorded; the final effect is still determined by the AUC gate (sub-session 3).
  • Files\AlgoForge\Data\features_XAUUSD.npz: F(197032,19), F2(197032,6), label, ATR, close, time, split_bar=148470, split_pos=115631, meta JSON. 0 NaN/Inf, 0 constant columns; bull 80.939 / bear 73.236 (ratio 1.105).
  • 24-bar forward = median 6.00 session hours (p10=6, p90=7; >8 hours 7%) — the label ≈ 6 session hours, per note §3.3.2.
  • Unit tests B1–B5 PASS (numerical ATR, streaming feature anti-lookahead, forward label, causal GARCH sigma, causal HMM filter + valid probs).

7. Risks & Honest Notes

  1. M15 ≠ 20 years (HFM feed) — decision: 8.6-year dense window + 18.7-year D1. This is HONESTLY recorded; do not claim "20-year M15".
  2. LSTM/regime lost before on 60k bars — not assumed to win; the like-for-like AUC gate decides. Regime is used as a feature/context, not a gate.
  3. XAUUSD vs XAUUSDc — different feeds; the like-for-like gate uses the same XAUUSDc cache for a fair comparison; the deploy model is trained on XAUUSD.
  4. Label density: 24 new M15 bars ≈ 6 session hours (median 6.00 hours; verified in sub-session 2). If the impact is large, a wall-clock label option is recorded as an alternative.
  5. Compute estimate: 186k bars × W=32 × H=32 2-layer LSTM × 5 seeds × 3 splits — pure numpy can take 1–4 hours/seed; limit epoch_max=30, patience=10 (protocol). If >2x the estimate → stop & report (Phase-3 rule §7).
  6. 2-state Gaussian HMM/MSM collapses on this feed (evidence diag_hmm.py): regime features use t-GARCH (valid) + a causal logistic RV24 regime score (proxy, not MSM). This is an honest deviation from "MSM" in the specification — recorded, not hidden. The gate still decides whether regime helps direction.
  7. GARCH MLE is fragile: classic bounds + 0.5% winsor are required for an interior fit (alpha 0.06/beta 0.92/nu 7); without winsor the MLE falls to an ARCH-heavy boundary solution (evidence diag_garch*.py). These parameters are EMBEDDED in the cache meta.