chiki2bum2/SniperGold_ML
Watch
1
0
Fork
You've already forked SniperGold_ML
 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions
SniperGold_ML/repaired/P1_REPAIR_REPORT.md

5.3 KiB
Raw Permalink Blame History

P1 REPAIR REPORT — SNIPERGOLD_ML (2026-08-21)

A. Changes Made

File Change Reason
Shared Projects\SniperGold_ML\train_model.py 1) train(): per-head validation AUC metric (val_auc_2head), early stopping using 2-head mean. 2) main(): purged temporal split (train | gap=H_LABEL | test) + explicit verification. 3) build_structure/build_features: optional begin parameter (additive, backward-compatible, for the parity harness). 4) selftest() (Test A+B) + --selftest. SB-01, SB-02, SB-06
MQL5\Experts\AlgoForge_Backtest_Baseline.mq5 Added InpMode=2 (dump 19 features/bar M15 -> CSV). The "mom fix" experiment (tc-21->tc-20) was CANCELLED — the harness proved the original formula is correct (tc param = tcv = total => close[tc-21] = 20-bar). SB-06 (parity harness)
Shared Projects\AlgoForge\ml\parity\parity_harness.py (NEW) Runtime-vs-training parity harness (timestamp join, per-feature statistics, A=window / B=formula classification). SB-06
New outputs repaired\pre_purge\*, repaired\post_purge\*, ml\parity\* Evidence (immutable originals preserved)

Baseline hashes: train_model.py 96BC258F->1B967C76; EA 3DE0E898->A19891E4; freeze SniperGold_ML.mqh UNTOUCHED (C0D5CAE6).

B. SB-01 Result

  • OLD: auc(yv.ravel(), Pv[:,0]) — 2-head labels raveled vs 1-head prediction -> validation AUC 0.5005, best_epoch=0 (early stopping selected a ~1-epoch model). Freeze 0.627/0.6207 = artifact of the broken metric.
  • NEW: per-head AUC_LONG=AUC(y_l,P_l), AUC_SHORT=AUC(y_s,P_s), mean for early stopping -> validation 0.6796/0.6796, best_epoch=55.
  • Test A: synthetic 2-head (per-head=1.0; legacy ravel=0.625 != 1.0) -> PASS.

C. SB-02 Result

  • OLD split: train 35652 / test 11885; max_train_outcome_end=44995 >= min_test_outcome_start=44973 -> OVERLAP.
  • NEW split: train 35652 / purge 13 / test 11872; split_bar=44972; 44995 < 44997 -> CLEAN.
  • Test B: synthetic case fails on the old split, PASS on the new one -> PASS.

D. SB-06 Result (RUNTIME_TRAINING_PARITY_REPORT.md)

  • Join: 14850/14850 timestamps matched; feed identical (close max|d|=0.0000).
  • Formula-parity CLEAN (rate=0.0000): f4_internal, f5_bias, f7_sweep, f12_dsign, f16_mom20 (after revert); nearly clean: f13 (0.0007), f3 (0.0017), f8 (0.0035), f9 (0.0025).
  • A-window (600-bar window vs full history): f6_eqpos (54.9%), f14/f15 dist (31%/30%), f17_range (54.9%) — LARGE in strong-trend regimes: range_atr 56.99 vs 10.44; eqpos -21 vs -0.86. The README claim "practically does not change results" is REFUTED by measurement.
  • B-formula: f10_eqh (98.5%), f11_eql (100%) — EQ semantics (Python same-type pairs + pivot-bar ATR vs EA/v4.4 consecutive-list pairs + current ATR) -> FLAG=BLOCKING DESIGN DECISION. f0-f2 HTF bias (36-51%) — EA output not reproduced by a faithful replica on the same data -> EA D1 cache content in the tester differs; root cause NOT FULLY TRACED -> FLAG=BLOCKING. f18_conf (52%) = derived from f0-f2/EQ.
  • Lesson: my initial "mom off-by-one" finding was WRONG (misread of the tc/tcv convention); the harness caught the mistaken fix and the revert restored perfect parity.

E. Regression Tests

  • Test A (AUC metric): PASS · Test B (purge): PASS · Test C (parity harness): report + clean-rate features + design flags · Test D: build_features --selftest B1-B5 PASS, train_hybrid --selftest B1-B5 PASS, EA compile 0/0.

F. New Baseline Evidence (POST-PURGE)

Cache features_XAUUSDc.npz (E60285F5), seed 42, MLP 19->12->2, H=24, purge=13:

  • TEST LONG 0.6582 / SHORT 0.6582 (PRE 0.6270/0.6207) | VAL 0.6796/0.6796 | calibration A=0.6913 B=0.1935 | best_epoch=55.
  • Interpretation: methodology correction (not optimization). The old freeze was a ~1-epoch model.

G. Evidence Classification

  • Freeze 0.6270/0.6207: SUPERSEDED (pre-repair: broken metric + overlap).
  • POST-PURGE 0.6582/0.6582: TRUSTED (corrected estimate on the training cache); not DEPLOY (not yet validated across feeds).
  • Parity report: TRUSTED. The "mom fix" experiment: INVALID (reverted).
  • Old backtest results (0.5305/0.5487; 308 trades PF 1.32): INCONCLUSIVE (contaminated by the now-measured parity gap).

H. Remaining Risks

  1. f0-f2 HTF bias (36-51%): root cause not yet fully traced (needs an EA BTTFBias input debug dump in the tester). BLOCKING.
  2. f10/f11 EQ: design decision (training vs live v4.4/EA) — do not silently align. BLOCKING.
  3. f6/f14/f15/f17 window (54.9%): runtime 600-bar != full-history training in strong-trend regimes — potential explanation for the runtime AUC drop 0.627->0.5305. Design decision: stateful runtime (stream from the start) vs retrain with the window. LARGEST ECONOMIC IMPACT.
  4. v4.4 live shares EQ semantics with the EA; v4.4 HTF bias not yet verified (only the EA was tested).

I. Recommendation for P2

Do NOT build LSTM/Informer/MS-GARCH/meta-gate. P2 = resolve the 3 design decisions (EQ, HTF bias, window) via: (a) EA debug dump to trace f0-f2; (b) user decision for the EQ & window source of truth; (c) align runtime<->training according to the decision; (d) re-measure parity and re-run the CSV mode-0 backtest on the parity-corrected basis. Only then evaluate H0/H1 (regime-aware ML).

STOP: SB-01 FIXED+TESTED, SB-02 FIXED+VERIFIED, SB-06 MEASURED+VERIFIED. No commit/publish.