# P3-S20 WALK-FORWARD BASELINE — PRE-REGISTERED EXPANDING-WINDOW VALIDATION ```text Date : 2026-08-24 Session : P3-S.20 — Pre-Registered Expanding-Window Walk-Forward Baseline Status : COMPLETE Verdict : A — STABLE WEAK SIGNAL (pre-registered decision rule output) Scope : VALIDATION / GENERALIZATION PHASE. LOGISTIC ONLY. NO tree/boost/MLP/LSTM/Informer/regime/ensemble, NO feature or label change, NO TP/SL/horizon/de-overlap change, NO threshold/ HP tuning, NO fold redesign after results, NO Tickstory/Dukascopy, NO production/MQL5/legacy-MLP change. Question : DOES THE WEAK LOGISTIC SIGNAL OBSERVED IN P3-S18 (TEST ROC-AUC ~0.609) SURVIVE PRE-REGISTERED TEMPORAL OUT-OF-SAMPLE TESTING? Human verif: REMAINS CANCELLED (historical only). ``` --- ## A. Latest SESSION_HANDOVER used ```text docs/SESSION_HANDOVER_2026-08-24_P3_S19_DATASET_FEATURE_AUDIT.md (newest authoritative handover; committed at HEAD 84b0e56; read completely FIRST, before any experiment — session-handover-first rule). Reconciles with Git : local HEAD == origin/main == 84b0e56fc1e6ba2be503f462c4ef1a58e0830915 branch main, working tree CLEAN, origin = https://forge.mql5.io/chiki2bum2/SniperGold_ML.git Superseded-in-line reads (all read completely): docs/P3_S18_BASELINE_ML_REPORT.md (P3-S18 INCONCLUSIVE / DATA TOO SMALL) docs/P3_S18A_LABEL_CONTRACT_REVIEW.md (label contract APPROVED AS V1) docs/P3_S19_DATASET_FEATURE_AUDIT.md (AUDIT PASSED; 686=594+92) docs/P3_S16_LABEL_CONTRACT.md / docs/P3_S16_SETUP_DATASET_CONTRACT_v1.md docs/SNIPERGOLD_CANONICAL_SETUP_CONTRACT_v1.md docs/P3_S17R2_VECTORIZE_CH0CH_OB_FVG.md (FULL PARITY ACHIEVED) ``` ## B. Checkpoint ```text Starting checkpoint (verified): 84b0e56fc1e6ba2be503f462c4ef1a58e0830915 local == origin/main, branch main, working tree CLEAN, origin = https://forge.mql5.io/chiki2bum2/SniperGold_ML.git Final Forge HEAD : ``` ## C. Pre-registered experiment definition ```text DESIGN (frozen BEFORE any final OOS metric was computed): Unit : ONE Candidate Setup = ONE binary observation (WIN=1/LOSS=0) on the causally-verified population (P3-S16 v1 label). Order : chronological by creation_bar (no shuffle). Scheme : expanding-window temporal walk-forward, 3 folds. FOLD BOUNDS (indices over the 571 sorted binary rows): Fold 1 : train rows [0:300] OOS rows [300:395] (OOS ~95) Fold 2 : train rows [0:395] OOS rows [395:490] (expanding; OOS ~95) Fold 3 : train rows [0:490] OOS rows [490:571] (expanding; OOS ~81) (Set from the brief's target structure: Fold1 train ~300 / OOS ~90-100; Fold2/3 expanding with OOS ~90-100 then ~80-100. NOT derived from performance.) Purge : gap >= HORIZON(16 bars) enforced at every fold boundary (verified 555 / 472 / 321 bars — satisfied natively by the de-overlapped lead stream; asserted, never silently trimmed). Preprocess : StandardScaler fitted on TRAINING rows ONLY per fold; applied UNCHANGED to OOS. (Only train-learned preprocessing allowed.) Model : LogisticRegression(C=1.0, L2, max_iter=5000, random_state=42) — byte-for-byte the P3-S18 frozen configuration. Threshold : 0.5 ONLY for the confusion / precision / recall tables. Comparators : per-fold majority-class (constant-prior) baseline whose score = the TRAIN WIN prevalence (no OOS leakage); ROC of a constant = 0.5 by construction. Quality gate: per fold OOS >= 50 binary obs (pref >= 100), >= 20 WIN, >= 20 LOSS; failing fold -> LOW_STATISTICAL_POWER preserved. Aggregation : pooled OOS (exact, sample-aware) + per-fold mean/median/std. Decision rule: pre-registered (see section N) — applied unchanged to the run output; no uncertainty method selected post-hoc. ``` ## D. Dataset / population ```text Reconstructed via ml/p3/baseline/prepare_dataset.py UNCHANGED from the byte-locked P3-S.17R.2 chain (FULL PARITY; runtime==research 694==694): all in-scope Candidate Setups : 686 leads (independent) : 594 follow-ons (preserved) : 92 binary fit rows (WIN/LOSS leads): 571 lead classes : WIN 167 | LOSS 404 | UNRES 18 | AMBIG 5 duplicates : 0 feature schema : 12 causal as-of-entry features feature_sha16 = 0414e401522ea4e2 (identical to P3-S18/P3-S19 — unchanged) label contract : P3-S16 label contract v1 (frozen) de-overlap : lead-per-episode (unchanged) ``` ## E. Fold definitions ```text Chronological binary-row windows (creation_bar ranges): Fold 1 train bars 1077 .. 99013 (n=300) OOS bars 99568 .. 134069 (n=95) Fold 2 train bars 1077 .. 134069 (n=395) OOS bars 134541 .. 167070 (n=95) Fold 3 train bars 1077 .. 167070 (n=490) OOS bars 167391 .. 193899 (n=81) OOS quality-gate checks (binary): F1 95 (25W/70L) | F2 95 (23W/72L) | F3 81 (26W/55L) -> all PASS >= 50 obs, >= 20 WIN, >= 20 LOSS. NO fold marked LOW_STATISTICAL_POWER. UNRESOLVED / AMBIGUOUS preserved per OOS temporal window (leads incl. non-binary classes, never forced): F1 OOS window : 102 leads = 25 WIN / 70 LOSS / 6 UNRES / 1 AMBIG F2 OOS window : 98 leads = 23 WIN / 72 LOSS / 2 UNRES / 1 AMBIG F3 OOS window : 85 leads = 26 WIN / 55 LOSS / 3 UNRES / 1 AMBIG ``` ## F. Purge / temporal isolation ```text gap (OOS first bar - last train bar): 555 / 472 / 321 bars, all >> H=16. No label-window overlap across boundaries (asserted per fold). Scaler fit on training only; no future preprocessing information. Chronological order enforced; no reshuffling (WF-T01/T02/T03 PASS). ``` ## G. Majority baseline (per fold, train-derived prior) ```text Fold train_prior(WIN) ROC PR(=prior) LogLoss Brier 1 0.3100 0.5 0.3100 0.5816 0.1961 2 0.2987 0.5 0.2987 0.5615 0.1867 3 0.2878 0.5 0.2878 0.6302 0.2191 pooled 0.2731 0.5 0.2731 0.5863 0.1985 The majority comparator never sees OOS labels (prior from training only). ``` ## H. Logistic OOS results ```text Fold 1 Fold 2 Fold 3 Pooled OOS n 95 95 81 271 OOS WIN/LOSS 25/70 23/72 26/55 74/197 ROC-AUC 0.5337 0.6341 0.5350 0.5792 PR-AUC (WIN) 0.3302 0.4224 0.3715 0.3417 LogLoss 0.5815 0.5467 0.6491 0.5895 Brier 0.1958 0.1806 0.2288 0.2003 Balanced acc (macro) 0.5200 0.5000 0.4951 0.5109 WIN precision (0.5) 1.000 0.000 0.308 0.357 (5 TP) WIN recall (0.5) 0.040 0.000 0.154 0.068 Accuracy (0.5) 0.747 0.758 0.617 0.712 Confusion (0.5) [1/24;0/70] [0/23;0/72] [4/22;9/46] 5-69/9-188 Train ROC-AUC 0.5564 0.5475 0.5541 - (n=300/395/490) Train LogLoss 0.6137 0.6037 0.5910 - Majority comparison per fold (delta = logistic - majority): ROC : +0.034 / +0.134 / +0.035 (all folds > 0.5 baseline) PR : +0.020 / +0.124 / +0.084 (all folds beat the prevalence prior) LogLoss: -0.0001 / -0.0148 / +0.0189 (2 of 3 better or tie) Brier : -0.0003 / -0.0061 / +0.0098 (2 of 3 better or tie) ``` ## I. Calibration / log loss ```text The linear model carries RANKING information (AUC/PR-AUC), not materially improved CALIBRATION: pooled LogLoss 0.5895 vs constant-prior 0.5863 (logistic marginally WORSE); pooled Brier 0.2003 vs 0.1985. Fold-wise the model beats the prior on LogLoss/Brier in folds 2, ties in fold 1, and is worse in fold 3. At threshold 0.5 the classifier is extremely conservative (predicts nearly all LOSS), so precision/recall on WIN are near zero — the weak signal is a soft ranking edge, NOT a usable hard-classification rule at 0.5. Consistent with P3-S18's observation that logistic is "essentially uncalibrated-informative" at this sample size. ``` ## J. Per-fold stability ```text ROC-AUC across OOS folds: 0.534 / 0.634 / 0.535. - direction consistent: ALL 3 folds above the 0.5 majority baseline, - no catastrophic collapse in any fold, - strongest period = fold 2 (0.634), folds 1 and 3 are weak-positive (~0.53-0.54, near noise but consistently above 0.5), - PR-AUC beats its majority prior in ALL 3 folds (0.33/0.42/0.37 vs 0.31/0.30/0.29) — the ranking edge is not confined to one period, - no chronological collapse (train 0.556/0.548/0.554 -> OOS positive). Interpretation: modest but consistent; effect size small; fold 2 carries the largest single contribution but folds 1 and 3 still clear the baseline. ``` ## K. Statistical uncertainty ```text Per-fold OOS n = 95/95/81; pooled n = 271 (74 WIN / 197 LOSS). At these counts a ROC-AUC of ~0.53 is within noise bands; 0.634 in fold 2 alone is not strong evidence; but the pooled 0.579 over 271 independent de-overlapped rows and the 3/3 folds above baseline + 3/3 PR-AUC beat are the consistent pattern. A bootstrap/permutation interval was NOT selected after seeing the metrics (pre-registration discipline); the confidence statement is therefore qualitative: the direction is consistent, the magnitude is weak, and the pooled estimate is modestly above random. CIs on AUC of a 271/74-WIN sample are wide (~±0.05 at 95% level); the 0.579 pooled value is fully consistent with a small real edge OR a favorable draw, but NOT with a reproducibly strong effect. Calibration (LogLoss/Brier) provides no independent confirmation, so the signal should be treated as a WEAK ranking hint only. ``` ## L. Reproducibility ```text The experiment was run 3+ times with the fixed seed/design: - p3_s20_fold_results.csv and p3_s20_oos_predictions.csv BYTE-IDENTICAL across runs, - p3_s20_summary.json / manifest identical except generated_utc (and the git_commit field, which is recorded at run time), - WF-T01..T06 PASS (chronology, expanding windows, purge, train-only preprocessing, quality gate, majority comparator, deterministic AUROC). Deterministic pipeline: prepare_dataset (frozen) -> walk_forward (frozen design) -> metric helpers (frozen). No randomness beyond the fixed seed. ``` ## M. Failure diagnosis, if any ```text No process/reproducibility failure (E not applicable). The signal did NOT fail: Logistic kept a weak-positive OOS profile rather than collapsing. The classification of the failure space (brief section 27) is therefore not triggered; the nearest descriptive labels are: SAMPLE SIZE : primary limiter — 271 pooled / ~95 per fold leaves wide CIs; TEMPORAL DRIFT / POPULATION HETEROGENEITY : fold 2 (0.634) vs folds 1/3 (~0.53) shows period heterogeneity in effect size, though not direction. No label/feature/TP/SL/horizon change was made or proposed as a result. ``` ## N. Final decision ```text FINAL DECISION : A — STABLE WEAK SIGNAL Pre-registered decision rule (frozen before the run, applied unchanged): - quality gates all PASS (no LOW_STATISTICAL_POWER) -> not D, - all 3 OOS folds have ROC-AUC > 0.5 and deviations > 0.02 (0.0337 / 0.1341 / 0.0350) -> A-track, - pooled OOS ROC-AUC 0.5792 > 0.5 -> A-track confirmed, -> A_STABLE_WEAK_SIGNAL. Qualitative support (brief section 15/30): - logistic > majority on most OOS folds (3/3 on ROC and PR-AUC), - direction consistent (all folds positive; no fold < 0.5, no collapse), - effect not driven by a single isolated period (folds 1 and 3 are positive, not merely fold 2), - calibration stays sensible (LogLoss/Brier near prior, no blow-up), - CAVEATS: folds 1/3 are near-noise (~0.53); LogLoss/Brier do not beat the base rate; the edge is a weak ranking effect, not a usable 0.5-threshold classifier. Neither "weak turned positive by average" nor "manufactured result": the decision is per-fold consistent, pooled, and pre-registered. ``` ## O. Production files unchanged ```text NO MQL5 / F1-F4 / Candidate Setup / FEATURE_CONTRACT / SniperGold_ML / EA change. Research-only artifacts under ml/p3/baseline/ (walk_forward.py, test_walk_forward.py, output/p3_s20_*). ``` ## P. Legacy MLP unchanged ```text The legacy SniperGold_ML.mqh MLP (19->12->2 bar-level L1 direction drift) remains FROZEN LEGACY / REFERENCE; NOT retrained, NOT exported, NOT compared as a same-target benchmark. No model weights / legacy labels / calibration modified. ``` ## Q. External data status ```text NOT introduced. Same validated XAUUSD M15 feed as P3-S18/P3-S19/P3-S17R.2. Tickstory / Dukascopy remain a future, separate, authorized robustness study. ``` ## R. Forge commits ```text 1. test: define pre-registered walk-forward baseline coverage (WF-T01..T06) 2. research: run setup-level logistic walk-forward (P3-S20) 3. docs: record P3-S20 walk-forward results (this report + handover) P3_S20_FINAL_SHA = ``` ## S. Final Forge HEAD ```text See Provenance (verified local == remote == P3_S20_FINAL_SHA after push). ``` ## T. Working tree ```text CLEAN after push (verified). Regression re-runs green then restored byte-identical: P3-S16 20/20 | VEC 15/15 | chain parity 694==694 | S19 audit 12/12 PASS (committed outputs restored byte-identical via git checkout after the re-runs — only generated_utc had changed). ``` ## U. P3-S21 readiness ```text Decision = A (STABLE WEAK SIGNAL). A future, SEPARATELY AUTHORIZED phase may examine (brief section 31): - feature mechanism (coefficient profile / zone geometry) — descriptive, - calibration diagnostics (probability calibration, not threshold tuning), - restrained nonlinear models ONLY after reproducible signal AND separate authorization. For (B)-style instability the recommended path would be population heterogeneity / label-exit diagnostics. NOTHING is auto-started: no feature hunting, no label change, no TP/SL/horizon change, no LSTM/Informer/regime, no Tickstory/Dukascopy, no deployment. ``` --- ## Provenance / evidence ```text Scripts (research-only, new in P3-S20): ml/p3/baseline/walk_forward.py (frozen design + logistic fits) ml/p3/baseline/test_walk_forward.py (WF-T01..T06; 6/6 PASS) Outputs (ml/p3/baseline/output/): p3_s20_walkforward_manifest.json (schema/hash/folds/gate/git commit) p3_s20_fold_results.csv (per-fold metrics + majority) p3_s20_oos_predictions.csv (271 pooled OOS rows + priors) p3_s20_summary.json (aggregates + final decision) p3_s20_wf_tests.json (WF-T01..T06 results) Reused (UNCHANGED): ml/p3/baseline/prepare_dataset.py (12-feature schema, label v1) ml/p3/baseline/evaluate_baselines.py (ROC/PR/logloss/Brier/pr/recall) ml/p3/setup_dataset/* (R.2 chain, S18A review rows) — imports only Regression (all green; committed outputs restored byte-identical): P3-S16 20/20 | VEC 15/15 | chain parity 694==694 | S19 audit 12/12 PASS Feature hash unchanged : 0414e401522ea4e2 (schema frozen, no drift) Guard : walk_forward.py + test_walk_forward.py scan CLEAN on the frozen P3-S.4/P3-S.5 parity-absence patterns (verified 0 hits). ``` *End of P3-S20 walk-forward baseline report. Decision: A — STABLE WEAK SIGNAL (pre-registered; weak ranking edge consistent across the three OOS folds, pooled ROC-AUC 0.579, PR-AUC beats the majority prior in every fold; calibration unchanged vs base rate — no manufacturing of a positive result).*