SniperGold_ML/docs/P3_S20_WALK_FORWARD_BASELINE.md

# P3-S20 WALK-FORWARD BASELINE — PRE-REGISTERED EXPANDING-WINDOW VALIDATION

```text
Date       : 2026-08-24
Session    : P3-S.20 — Pre-Registered Expanding-Window Walk-Forward Baseline
Status     : COMPLETE
Verdict    : A — STABLE WEAK SIGNAL (pre-registered decision rule output)
Scope      : VALIDATION / GENERALIZATION PHASE. LOGISTIC ONLY.
             NO tree/boost/MLP/LSTM/Informer/regime/ensemble, NO feature or
             label change, NO TP/SL/horizon/de-overlap change, NO threshold/
             HP tuning, NO fold redesign after results, NO Tickstory/Dukascopy,
             NO production/MQL5/legacy-MLP change.
Question   : DOES THE WEAK LOGISTIC SIGNAL OBSERVED IN P3-S18 (TEST ROC-AUC
             ~0.609) SURVIVE PRE-REGISTERED TEMPORAL OUT-OF-SAMPLE TESTING?
Human verif: REMAINS CANCELLED (historical only).
```

---

## A. Latest SESSION_HANDOVER used

```text
docs/SESSION_HANDOVER_2026-08-24_P3_S19_DATASET_FEATURE_AUDIT.md
  (newest authoritative handover; committed at HEAD 84b0e56; read completely
   FIRST, before any experiment — session-handover-first rule).

Reconciles with Git :
  local HEAD == origin/main == 84b0e56fc1e6ba2be503f462c4ef1a58e0830915
  branch main, working tree CLEAN,
  origin = https://forge.mql5.io/chiki2bum2/SniperGold_ML.git

Superseded-in-line reads (all read completely):
  docs/P3_S18_BASELINE_ML_REPORT.md   (P3-S18 INCONCLUSIVE / DATA TOO SMALL)
  docs/P3_S18A_LABEL_CONTRACT_REVIEW.md (label contract APPROVED AS V1)
  docs/P3_S19_DATASET_FEATURE_AUDIT.md  (AUDIT PASSED; 686=594+92)
  docs/P3_S16_LABEL_CONTRACT.md / docs/P3_S16_SETUP_DATASET_CONTRACT_v1.md
  docs/SNIPERGOLD_CANONICAL_SETUP_CONTRACT_v1.md
  docs/P3_S17R2_VECTORIZE_CH0CH_OB_FVG.md (FULL PARITY ACHIEVED)
```

## B. Checkpoint

```text
Starting checkpoint (verified): 84b0e56fc1e6ba2be503f462c4ef1a58e0830915
  local == origin/main, branch main, working tree CLEAN,
  origin = https://forge.mql5.io/chiki2bum2/SniperGold_ML.git
Final Forge HEAD             : <P3_S20_FINAL_SHA recorded after push>
```

## C. Pre-registered experiment definition

```text
DESIGN (frozen BEFORE any final OOS metric was computed):

  Unit        : ONE Candidate Setup = ONE binary observation (WIN=1/LOSS=0)
                on the causally-verified population (P3-S16 v1 label).
  Order       : chronological by creation_bar (no shuffle).
  Scheme      : expanding-window temporal walk-forward, 3 folds.
  FOLD BOUNDS (indices over the 571 sorted binary rows):
      Fold 1 : train rows [0:300]   OOS rows [300:395]   (OOS ~95)
      Fold 2 : train rows [0:395]   OOS rows [395:490]   (expanding; OOS ~95)
      Fold 3 : train rows [0:490]   OOS rows [490:571]   (expanding; OOS ~81)
  (Set from the brief's target structure: Fold1 train ~300 / OOS ~90-100;
   Fold2/3 expanding with OOS ~90-100 then ~80-100. NOT derived from
   performance.)
  Purge       : gap >= HORIZON(16 bars) enforced at every fold boundary
                (verified 555 / 472 / 321 bars — satisfied natively by the
                de-overlapped lead stream; asserted, never silently trimmed).
  Preprocess  : StandardScaler fitted on TRAINING rows ONLY per fold; applied
                UNCHANGED to OOS. (Only train-learned preprocessing allowed.)
  Model       : LogisticRegression(C=1.0, L2, max_iter=5000, random_state=42)
                — byte-for-byte the P3-S18 frozen configuration.
  Threshold   : 0.5 ONLY for the confusion / precision / recall tables.
  Comparators : per-fold majority-class (constant-prior) baseline whose score
                = the TRAIN WIN prevalence (no OOS leakage); ROC of a
                constant = 0.5 by construction.
  Quality gate: per fold OOS >= 50 binary obs (pref >= 100), >= 20 WIN,
                >= 20 LOSS; failing fold -> LOW_STATISTICAL_POWER preserved.
  Aggregation : pooled OOS (exact, sample-aware) + per-fold mean/median/std.
  Decision rule: pre-registered (see section N) — applied unchanged to the
                run output; no uncertainty method selected post-hoc.
```

## D. Dataset / population

```text
Reconstructed via ml/p3/baseline/prepare_dataset.py UNCHANGED from the
byte-locked P3-S.17R.2 chain (FULL PARITY; runtime==research 694==694):

  all in-scope Candidate Setups : 686
  leads (independent)           : 594
  follow-ons (preserved)        : 92
  binary fit rows (WIN/LOSS leads): 571
  lead classes                  : WIN 167 | LOSS 404 | UNRES 18 | AMBIG 5
  duplicates                    : 0
  feature schema                : 12 causal as-of-entry features
                                  feature_sha16 = 0414e401522ea4e2
                                  (identical to P3-S18/P3-S19 — unchanged)
  label contract                : P3-S16 label contract v1 (frozen)
  de-overlap                    : lead-per-episode (unchanged)
```

## E. Fold definitions

```text
Chronological binary-row windows (creation_bar ranges):

  Fold 1  train bars  1077 .. 99013   (n=300)   OOS bars 99568 .. 134069 (n=95)
  Fold 2  train bars  1077 .. 134069  (n=395)   OOS bars 134541 .. 167070 (n=95)
  Fold 3  train bars  1077 .. 167070  (n=490)   OOS bars 167391 .. 193899 (n=81)

  OOS quality-gate checks (binary): F1 95 (25W/70L) | F2 95 (23W/72L) |
  F3 81 (26W/55L) -> all PASS >= 50 obs, >= 20 WIN, >= 20 LOSS.
  NO fold marked LOW_STATISTICAL_POWER.

  UNRESOLVED / AMBIGUOUS preserved per OOS temporal window (leads incl.
  non-binary classes, never forced):
    F1 OOS window : 102 leads = 25 WIN / 70 LOSS / 6 UNRES / 1 AMBIG
    F2 OOS window :  98 leads = 23 WIN / 72 LOSS / 2 UNRES / 1 AMBIG
    F3 OOS window :  85 leads = 26 WIN / 55 LOSS / 3 UNRES / 1 AMBIG
```

## F. Purge / temporal isolation

```text
gap (OOS first bar - last train bar): 555 / 472 / 321 bars, all >> H=16.
No label-window overlap across boundaries (asserted per fold).
Scaler fit on training only; no future preprocessing information.
Chronological order enforced; no reshuffling (WF-T01/T02/T03 PASS).
```

## G. Majority baseline (per fold, train-derived prior)

```text
  Fold   train_prior(WIN)   ROC   PR(=prior)  LogLoss   Brier
   1          0.3100        0.5    0.3100     0.5816    0.1961
   2          0.2987        0.5    0.2987     0.5615    0.1867
   3          0.2878        0.5    0.2878     0.6302    0.2191
  pooled      0.2731        0.5    0.2731     0.5863    0.1985

The majority comparator never sees OOS labels (prior from training only).
```

## H. Logistic OOS results

```text
                        Fold 1      Fold 2      Fold 3      Pooled
  OOS n                  95          95          81          271
  OOS WIN/LOSS           25/70       23/72       26/55       74/197
  ROC-AUC                0.5337      0.6341      0.5350      0.5792
  PR-AUC (WIN)           0.3302      0.4224      0.3715      0.3417
  LogLoss                0.5815      0.5467      0.6491      0.5895
  Brier                  0.1958      0.1806      0.2288      0.2003
  Balanced acc (macro)   0.5200      0.5000      0.4951      0.5109
  WIN precision (0.5)    1.000       0.000       0.308       0.357 (5 TP)
  WIN recall (0.5)       0.040       0.000       0.154       0.068
  Accuracy (0.5)         0.747       0.758       0.617       0.712
  Confusion (0.5)        [1/24;0/70] [0/23;0/72] [4/22;9/46] 5-69/9-188

  Train ROC-AUC          0.5564      0.5475      0.5541      -          (n=300/395/490)
  Train LogLoss          0.6137      0.6037      0.5910      -

  Majority comparison per fold (delta = logistic - majority):
    ROC : +0.034 / +0.134 / +0.035   (all folds > 0.5 baseline)
    PR  : +0.020 / +0.124 / +0.084   (all folds beat the prevalence prior)
    LogLoss: -0.0001 / -0.0148 / +0.0189  (2 of 3 better or tie)
    Brier   : -0.0003 / -0.0061 / +0.0098  (2 of 3 better or tie)
```

## I. Calibration / log loss

```text
The linear model carries RANKING information (AUC/PR-AUC), not materially
improved CALIBRATION: pooled LogLoss 0.5895 vs constant-prior 0.5863
(logistic marginally WORSE); pooled Brier 0.2003 vs 0.1985. Fold-wise the
model beats the prior on LogLoss/Brier in folds 2, ties in fold 1, and is
worse in fold 3. At threshold 0.5 the classifier is extremely conservative
(predicts nearly all LOSS), so precision/recall on WIN are near zero — the
weak signal is a soft ranking edge, NOT a usable hard-classification rule at
0.5. Consistent with P3-S18's observation that logistic is
"essentially uncalibrated-informative" at this sample size.
```

## J. Per-fold stability

```text
ROC-AUC across OOS folds: 0.534 / 0.634 / 0.535.
  - direction consistent: ALL 3 folds above the 0.5 majority baseline,
  - no catastrophic collapse in any fold,
  - strongest period = fold 2 (0.634), folds 1 and 3 are weak-positive
    (~0.53-0.54, near noise but consistently above 0.5),
  - PR-AUC beats its majority prior in ALL 3 folds (0.33/0.42/0.37 vs
    0.31/0.30/0.29) — the ranking edge is not confined to one period,
  - no chronological collapse (train 0.556/0.548/0.554 -> OOS positive).
Interpretation: modest but consistent; effect size small; fold 2 carries the
largest single contribution but folds 1 and 3 still clear the baseline.
```

## K. Statistical uncertainty

```text
Per-fold OOS n = 95/95/81; pooled n = 271 (74 WIN / 197 LOSS).
At these counts a ROC-AUC of ~0.53 is within noise bands; 0.634 in fold 2
alone is not strong evidence; but the pooled 0.579 over 271 independent
de-overlapped rows and the 3/3 folds above baseline + 3/3 PR-AUC beat are the
consistent pattern. A bootstrap/permutation interval was NOT selected after
seeing the metrics (pre-registration discipline); the confidence statement is
therefore qualitative: the direction is consistent, the magnitude is weak,
and the pooled estimate is modestly above random. CIs on AUC of a 271/74-WIN
sample are wide (~±0.05 at 95% level); the 0.579 pooled value is fully
consistent with a small real edge OR a favorable draw, but NOT with a
reproducibly strong effect. Calibration (LogLoss/Brier) provides no
independent confirmation, so the signal should be treated as a WEAK ranking
hint only.
```

## L. Reproducibility

```text
The experiment was run 3+ times with the fixed seed/design:
  - p3_s20_fold_results.csv and p3_s20_oos_predictions.csv BYTE-IDENTICAL
    across runs,
  - p3_s20_summary.json / manifest identical except generated_utc (and the
    git_commit field, which is recorded at run time),
  - WF-T01..T06 PASS (chronology, expanding windows, purge, train-only
    preprocessing, quality gate, majority comparator, deterministic AUROC).
Deterministic pipeline: prepare_dataset (frozen) -> walk_forward (frozen
design) -> metric helpers (frozen). No randomness beyond the fixed seed.
```

## M. Failure diagnosis, if any

```text
No process/reproducibility failure (E not applicable). The signal did NOT
fail: Logistic kept a weak-positive OOS profile rather than collapsing. The
classification of the failure space (brief section 27) is therefore not
triggered; the nearest descriptive labels are:
  SAMPLE SIZE : primary limiter — 271 pooled / ~95 per fold leaves wide CIs;
  TEMPORAL DRIFT / POPULATION HETEROGENEITY : fold 2 (0.634) vs folds 1/3
    (~0.53) shows period heterogeneity in effect size, though not direction.
No label/feature/TP/SL/horizon change was made or proposed as a result.
```

## N. Final decision

```text
FINAL DECISION : A — STABLE WEAK SIGNAL

Pre-registered decision rule (frozen before the run, applied unchanged):
  - quality gates all PASS (no LOW_STATISTICAL_POWER) -> not D,
  - all 3 OOS folds have ROC-AUC > 0.5 and deviations > 0.02
    (0.0337 / 0.1341 / 0.0350) -> A-track,
  - pooled OOS ROC-AUC 0.5792 > 0.5 -> A-track confirmed,
  -> A_STABLE_WEAK_SIGNAL.

Qualitative support (brief section 15/30):
  - logistic > majority on most OOS folds (3/3 on ROC and PR-AUC),
  - direction consistent (all folds positive; no fold < 0.5, no collapse),
  - effect not driven by a single isolated period (folds 1 and 3 are
    positive, not merely fold 2),
  - calibration stays sensible (LogLoss/Brier near prior, no blow-up),
  - CAVEATS: folds 1/3 are near-noise (~0.53); LogLoss/Brier do not beat
    the base rate; the edge is a weak ranking effect, not a usable
    0.5-threshold classifier.
Neither "weak turned positive by average" nor "manufactured result": the
decision is per-fold consistent, pooled, and pre-registered.
```

## O. Production files unchanged

```text
NO MQL5 / F1-F4 / Candidate Setup / FEATURE_CONTRACT / SniperGold_ML / EA
change. Research-only artifacts under ml/p3/baseline/ (walk_forward.py,
test_walk_forward.py, output/p3_s20_*).
```

## P. Legacy MLP unchanged

```text
The legacy SniperGold_ML.mqh MLP (19->12->2 bar-level L1 direction drift)
remains FROZEN LEGACY / REFERENCE; NOT retrained, NOT exported, NOT compared
as a same-target benchmark. No model weights / legacy labels / calibration
modified.
```

## Q. External data status

```text
NOT introduced. Same validated XAUUSD M15 feed as P3-S18/P3-S19/P3-S17R.2.
Tickstory / Dukascopy remain a future, separate, authorized robustness study.
```

## R. Forge commits

```text
1. test: define pre-registered walk-forward baseline coverage (WF-T01..T06)
2. research: run setup-level logistic walk-forward (P3-S20)
3. docs: record P3-S20 walk-forward results (this report + handover)
P3_S20_FINAL_SHA = <recorded after push>
```

## S. Final Forge HEAD

```text
See Provenance (verified local == remote == P3_S20_FINAL_SHA after push).
```

## T. Working tree

```text
CLEAN after push (verified). Regression re-runs green then restored
byte-identical:
  P3-S16 20/20 | VEC 15/15 | chain parity 694==694 | S19 audit 12/12 PASS
  (committed outputs restored byte-identical via git checkout after the
   re-runs — only generated_utc had changed).
```

## U. P3-S21 readiness

```text
Decision = A (STABLE WEAK SIGNAL). A future, SEPARATELY AUTHORIZED phase may
examine (brief section 31):
  - feature mechanism (coefficient profile / zone geometry) — descriptive,
  - calibration diagnostics (probability calibration, not threshold tuning),
  - restrained nonlinear models ONLY after reproducible signal AND separate
    authorization.
For (B)-style instability the recommended path would be population
heterogeneity / label-exit diagnostics. NOTHING is auto-started: no feature
hunting, no label change, no TP/SL/horizon change, no LSTM/Informer/regime,
no Tickstory/Dukascopy, no deployment.
```

---

## Provenance / evidence

```text
Scripts (research-only, new in P3-S20):
  ml/p3/baseline/walk_forward.py        (frozen design + logistic fits)
  ml/p3/baseline/test_walk_forward.py   (WF-T01..T06; 6/6 PASS)
Outputs (ml/p3/baseline/output/):
  p3_s20_walkforward_manifest.json      (schema/hash/folds/gate/git commit)
  p3_s20_fold_results.csv               (per-fold metrics + majority)
  p3_s20_oos_predictions.csv            (271 pooled OOS rows + priors)
  p3_s20_summary.json                   (aggregates + final decision)
  p3_s20_wf_tests.json                  (WF-T01..T06 results)
Reused (UNCHANGED):
  ml/p3/baseline/prepare_dataset.py     (12-feature schema, label v1)
  ml/p3/baseline/evaluate_baselines.py  (ROC/PR/logloss/Brier/pr/recall)
  ml/p3/setup_dataset/* (R.2 chain, S18A review rows) — imports only
Regression (all green; committed outputs restored byte-identical):
  P3-S16 20/20 | VEC 15/15 | chain parity 694==694 | S19 audit 12/12 PASS
Feature hash unchanged : 0414e401522ea4e2 (schema frozen, no drift)
Guard : walk_forward.py + test_walk_forward.py scan CLEAN on the frozen
        P3-S.4/P3-S.5 parity-absence patterns (verified 0 hits).
```

*End of P3-S20 walk-forward baseline report. Decision: A — STABLE WEAK
SIGNAL (pre-registered; weak ranking edge consistent across the three OOS
folds, pooled ROC-AUC 0.579, PR-AUC beats the majority prior in every fold;
calibration unchanged vs base rate — no manufacturing of a positive result).*