5.4 KiB
P3-S23 — PRE-REGISTRATION
Date : 2026-08-26
Session : P3-S23 — CORRECTED-POPULATION PRE-REGISTERED ML BASELINE
Policy : docs/CODE_VERIFICATION_AND_SILENT_BUG_POLICY_v1.md
Comparator : P3-S20 walk-forward baseline (FROZEN; never modified)
Status : PRE-REGISTRATION — freezes the experiment BEFORE any final OOS
metric is computed. This file is committed before the run.
This document, together with
ml/p3/p3_s23_corrected_population_ml/s23_config.py and the coverage tests in
test_s23_walk_forward.py, is the frozen experiment definition. No value
below was chosen by looking at final OOS results.
1. Question
After correcting the research M30 construction and re-deriving the corrected Candidate Setup population, does the previously observed weak logistic ranking signal remain reproducible under a NEW pre-registered temporal out-of-sample experiment?
2. Frozen dataset (P3-S22.4-corrected)
corrected in-scope Candidate Setups : 707
corrected leads : 610
corrected follow-ons : 97 (preserved; excluded from binary fit)
corrected lead classes : WIN 175 / LOSS 404 / UNRESOLVED 25 / AMBIGUOUS 6
corrected binary : 579 (WIN 175 / LOSS 404)
UNRESOLVED and AMBIGUOUS remain present and are NEVER forced into WIN/LOSS. FOLLOW-ONS never enter the binary ML population.
3. Frozen feature contract
12-causal-feature schema from prepare_dataset.FEATURE_COLS. No feature added /
removed / redefined; no OOS-based selection. The population-specific feature
hash will differ from the P3-S20 artifact because the ROWS differ; the SCHEMA
is unchanged (schema_sha16 constant).
4. Frozen label contract
P3-S16 label contract v1 unchanged. No TP/SL/horizon/entry/precedence/ambiguous/ unresolved change; no label v2.
5. Frozen model
model : LogisticRegression(C=1.0, L2/lbfgs, max_iter=5000, random_state=42)
preprocess : StandardScaler fitted on TRAIN rows only per fold
seed : 42
NO tree/boost/MLP/LSTM/Transformer/Informer/regime. NO C tuning or other hyper- parameter tuning. Scaler never fit on OOS.
6. Frozen fold boundaries (chronological expanding window)
Over the binary-sorted 579 corrected rows (0-based indices):
Fold1 : train[0:300] OOS[300:395] (train 300, OOS 95)
Fold2 : train[0:395] OOS[395:490] (train 395, OOS 95) expanding
Fold3 : train[0:490] OOS[490:579] (train 490, OOS 89) expanding
Purge = HORIZON(16): OOS-first bar - last-train bar > 16 asserted at every fold (expected ≫ 16). No label-window overlap across the boundary. No shuffle; no random split. Fold indices are frozen; a population change (571→579) does NOT shift the pre-registered indices.
7. Frozen comparator
Constant majority/prior baseline per fold: score = TRAIN WIN prevalence (no OOS leakage). For each fold: prior probability, ROC-AUC (=0.5), PR-AUC, LogLoss, Brier. Pooled comparator likewise (train-derived aggregate).
8. Frozen metrics
Per fold + pooled: ROC-AUC, PR-AUC, LogLoss, Brier. Secondary (threshold 0.5 only, never for ranking claims): balanced accuracy, precision, recall, confusion. No threshold tuning.
9. Frozen statistical gate (per fold)
OOS n >= 50 ; OOS WIN >= 20 ; OOS LOSS >= 20
if not met -> LOW_STATISTICAL_POWER recorded, no redesign after results
10. Frozen uncertainty method (defined BEFORE results)
Bootstrap-permutation percentile intervals on the pooled OOS: 2000 stratified resamples (seed 42) -> 95% percentile interval for pooled ROC-AUC and PR-AUC. Descriptive only; no gating.
11. Frozen decision rule (applied unchanged)
| Code | Condition | Class |
|---|---|---|
| A | every fold & pooled ROC-AUC > 0.5, per-fold deviation > 0.02, no collapse | STABLE REPRODUCIBLE SIGNAL |
| B | >=2 folds meaningful deviation but not all-fertile | WEAK / INCONSISTENT SIGNAL |
| C | otherwise | NO REPRODUCIBLE SIGNAL |
| D | gates fail or no usable OOS AUC | LOW STATISTICAL POWER |
| E | leakage / alignment / reproducibility defect | PROCESS / DATA VAID FAILURE |
12. Comparison with P3-S20 (after OOS eval)
P3-S20 (frozen): pooled OOS ROC 0.5792 / PR 0.3417 / LogLoss 0.5895 / Brier 0.2003.
Explicit comparison text (required):
"Population changed from 571 to 686 binary-fit observations and the
Candidate Setup population changed from 686 to 707 in-scope setups."
Interpretation is always about the effect of the M30 correction; never "better than P3-S20 solely because AUC increased."
13. Pre-registered silent-bug / oracle design
- 8/8 mutation battery (future feature, global scaler, temporal shuffle, OOS leaking into training, row shift, boundary/train-OOS violation, label alignment shift, prediction/row alignment mismatch).
- independent metric oracle (manual trapezoid AUC, manual PR, manual log-loss, manual Brier) cross-checked against committed outputs.
- byte-level reproducibility across two runs of the substantive evidence files.
14. Auth statement
Production MQL5 : NO
P3-S18 / P3-S20 historical : NO (unchanged, read-only)
P3-22.3 / P3-22.4 artifacts : NO
Feature / feature contract : NO
Label contract : NO
TP/SL/horizon : NO
External data : NO
Calibration : NO
Nonlinear model : NO
Hyperparameter tuning : NO
Deployment / Trading : NO
P3-S24 : NOT STARTED
End of P3-S23 pre-registration.