forked from chiki2bum2/SniperGold_ML
151 lines
No EOL
5.4 KiB
Markdown
151 lines
No EOL
5.4 KiB
Markdown
# P3-S23 — PRE-REGISTRATION
|
|
|
|
```text
|
|
Date : 2026-08-26
|
|
Session : P3-S23 — CORRECTED-POPULATION PRE-REGISTERED ML BASELINE
|
|
Policy : docs/CODE_VERIFICATION_AND_SILENT_BUG_POLICY_v1.md
|
|
Comparator : P3-S20 walk-forward baseline (FROZEN; never modified)
|
|
Status : PRE-REGISTRATION — freezes the experiment BEFORE any final OOS
|
|
metric is computed. This file is committed before the run.
|
|
```
|
|
|
|
This document, together with
|
|
`ml/p3/p3_s23_corrected_population_ml/s23_config.py` and the coverage tests in
|
|
`test_s23_walk_forward.py`, is the **frozen experiment definition**. No value
|
|
below was chosen by looking at final OOS results.
|
|
|
|
## 1. Question
|
|
|
|
> After correcting the research M30 construction and re-deriving the corrected
|
|
> Candidate Setup population, does the previously observed weak logistic ranking
|
|
> signal remain reproducible under a NEW pre-registered temporal out-of-sample
|
|
> experiment?
|
|
|
|
## 2. Frozen dataset (P3-S22.4-corrected)
|
|
|
|
```text
|
|
corrected in-scope Candidate Setups : 707
|
|
corrected leads : 610
|
|
corrected follow-ons : 97 (preserved; excluded from binary fit)
|
|
corrected lead classes : WIN 175 / LOSS 404 / UNRESOLVED 25 / AMBIGUOUS 6
|
|
corrected binary : 579 (WIN 175 / LOSS 404)
|
|
```
|
|
|
|
UNRESOLVED and AMBIGUOUS remain present and are NEVER forced into WIN/LOSS.
|
|
FOLLOW-ONS never enter the binary ML population.
|
|
|
|
## 3. Frozen feature contract
|
|
|
|
12-causal-feature schema from `prepare_dataset.FEATURE_COLS`. No feature added /
|
|
removed / redefined; no OOS-based selection. The population-specific feature
|
|
hash will differ from the P3-S20 artifact because the ROWS differ; the SCHEMA
|
|
is unchanged (schema_sha16 constant).
|
|
|
|
## 4. Frozen label contract
|
|
|
|
P3-S16 label contract v1 unchanged. No TP/SL/horizon/entry/precedence/ambiguous/
|
|
unresolved change; no label v2.
|
|
|
|
## 5. Frozen model
|
|
|
|
```text
|
|
model : LogisticRegression(C=1.0, L2/lbfgs, max_iter=5000, random_state=42)
|
|
preprocess : StandardScaler fitted on TRAIN rows only per fold
|
|
seed : 42
|
|
```
|
|
|
|
NO tree/boost/MLP/LSTM/Transformer/Informer/regime. NO C tuning or other hyper-
|
|
parameter tuning. Scaler never fit on OOS.
|
|
|
|
## 6. Frozen fold boundaries (chronological expanding window)
|
|
|
|
Over the binary-sorted 579 corrected rows (0-based indices):
|
|
|
|
```text
|
|
Fold1 : train[0:300] OOS[300:395] (train 300, OOS 95)
|
|
Fold2 : train[0:395] OOS[395:490] (train 395, OOS 95) expanding
|
|
Fold3 : train[0:490] OOS[490:579] (train 490, OOS 89) expanding
|
|
```
|
|
|
|
Purge = HORIZON(16): OOS-first bar - last-train bar > 16 asserted at every fold
|
|
(expected ≫ 16). No label-window overlap across the boundary. No shuffle; no
|
|
random split. Fold indices are frozen; a population change (571→579) does NOT
|
|
shift the pre-registered indices.
|
|
|
|
## 7. Frozen comparator
|
|
|
|
Constant majority/prior baseline per fold: score = TRAIN WIN prevalence (no OOS
|
|
leakage). For each fold: prior probability, ROC-AUC (=0.5), PR-AUC, LogLoss,
|
|
Brier. Pooled comparator likewise (train-derived aggregate).
|
|
|
|
## 8. Frozen metrics
|
|
|
|
Per fold + pooled: ROC-AUC, PR-AUC, LogLoss, Brier. Secondary (threshold 0.5
|
|
only, never for ranking claims): balanced accuracy, precision, recall,
|
|
confusion. No threshold tuning.
|
|
|
|
## 9. Frozen statistical gate (per fold)
|
|
|
|
```text
|
|
OOS n >= 50 ; OOS WIN >= 20 ; OOS LOSS >= 20
|
|
if not met -> LOW_STATISTICAL_POWER recorded, no redesign after results
|
|
```
|
|
|
|
## 10. Frozen uncertainty method (defined BEFORE results)
|
|
|
|
Bootstrap-permutation percentile intervals on the pooled OOS: 2000 stratified
|
|
resamples (seed 42) -> 95% percentile interval for pooled ROC-AUC and PR-AUC.
|
|
Descriptive only; no gating.
|
|
|
|
## 11. Frozen decision rule (applied unchanged)
|
|
|
|
| Code | Condition | Class |
|
|
|------|-----------|-------|
|
|
| A | every fold & pooled ROC-AUC > 0.5, per-fold deviation > 0.02, no collapse | STABLE REPRODUCIBLE SIGNAL |
|
|
| B | >=2 folds meaningful deviation but not all-fertile | WEAK / INCONSISTENT SIGNAL |
|
|
| C | otherwise | NO REPRODUCIBLE SIGNAL |
|
|
| D | gates fail or no usable OOS AUC | LOW STATISTICAL POWER |
|
|
| E | leakage / alignment / reproducibility defect | PROCESS / DATA VAID FAILURE |
|
|
|
|
## 12. Comparison with P3-S20 (after OOS eval)
|
|
|
|
P3-S20 (frozen): pooled OOS ROC 0.5792 / PR 0.3417 / LogLoss 0.5895 / Brier
|
|
0.2003.
|
|
|
|
Explicit comparison text (required):
|
|
|
|
```text
|
|
"Population changed from 571 to 686 binary-fit observations and the
|
|
Candidate Setup population changed from 686 to 707 in-scope setups."
|
|
```
|
|
|
|
Interpretation is always about the effect of the M30 correction; never
|
|
"better than P3-S20 solely because AUC increased."
|
|
|
|
## 13. Pre-registered silent-bug / oracle design
|
|
|
|
- 8/8 mutation battery (future feature, global scaler, temporal shuffle, OOS
|
|
leaking into training, row shift, boundary/train-OOS violation, label
|
|
alignment shift, prediction/row alignment mismatch).
|
|
- independent metric oracle (manual trapezoid AUC, manual PR, manual log-loss,
|
|
manual Brier) cross-checked against committed outputs.
|
|
- byte-level reproducibility across two runs of the substantive evidence files.
|
|
|
|
## 14. Auth statement
|
|
|
|
```text
|
|
Production MQL5 : NO
|
|
P3-S18 / P3-S20 historical : NO (unchanged, read-only)
|
|
P3-22.3 / P3-22.4 artifacts : NO
|
|
Feature / feature contract : NO
|
|
Label contract : NO
|
|
TP/SL/horizon : NO
|
|
External data : NO
|
|
Calibration : NO
|
|
Nonlinear model : NO
|
|
Hyperparameter tuning : NO
|
|
Deployment / Trading : NO
|
|
P3-S24 : NOT STARTED
|
|
```
|
|
|
|
*End of P3-S23 pre-registration.* |