forked from chiki2bum2/SniperGold_ML
12 KiB
12 KiB
P3-S21.R — RETROSPECTIVE SILENT-BUG VERIFICATION REPORT
Date : 2026-08-25
Session : P3-S21.R — RETROSPECTIVE SILENT-BUG VERIFICATION
Status : COMPLETE (verification only)
Policy : docs/CODE_VERIFICATION_AND_SILENT_BUG_POLICY_v1.md
Result : NO SILENT BUG FOUND. P3-S20 REMAINS USABLE. P3-S21.2 REMAINS USABLE.
Scope : VERIFICATION ONLY. NO experiment, NO corrective phase started.
Hard stop : no P3-S21.3, no corrective experiment (see section V / final statement).
A. Latest SESSION_HANDOVER used
docs/SESSION_HANDOVER_2026-08-25_P3_S21_2_CALIBRATION_DIAGNOSTICS.md
(newest authoritative handover; git ancestry on main; read COMPLETELY first).
Predecessor chain read (continuity): S21.1 -> S20 -> S19 -> S18 -> S17R2 -> S16
(handovers/reports and relevant implementation/test files as required).
B. Starting Git SHA
da54c9b399eab92f696cbc38d4b4e5d52a764c13 (P3-S21.2 formal close)
verified local==origin==HEAD, branch main, working tree CLEAN at start,
no stash, no untracked files, origin=forge.mql5.io/chiki2bum2/SniperGold_ML.git.
C. Final Git SHA
428668d7718a8ac19cae2b8b51dd5de412f7d2de
D. Repository state
main, working tree CLEAN, local == origin/main at close.
Historical substantive artifacts (.py / .csv / .json / .md / MQL5) all byte-
untouched (git diff empty; only the new policy + this report + handover added).
E. Policy version used
docs/CODE_VERIFICATION_AND_SILENT_BUG_POLICY_v1.md (created first in this
session, per the mandate: policy before auditing). 15 sections covering
contract-first verification, independent oracles, boundary / temporal /
leakage / metamorphic / mutation testing, evidence grades, retrospective and
governance rules, and a prohibition on silently modifying historical artifacts.
F. Historical phases audited
Priority audit : P3-S20 (highest priority; downstream S21.1/S21.2 depend on it).
Second audit : P3-S21.2 (calibration).
Dependency only: P3-S19 / P3-S18 / P3-S16 (traced, NOT re-executed; see N).
P3-S17R2 (parity) : treated as high-confidence; NOT rebuilt; parity artifacts
preserved (see Phase 6 below).
G. Verification methods used
1. Independent dataset accounting (row-by-row recount of 686 / 594 / 92 / 571 /
167 / 404 / 18 / 5, duplicate setup_id and (bar,setup) checks).
2. Independent chronological + index-order reconstruction.
3. Independent purge-gap computation (OOS_first - train_last per fold) and a
synthetic boundary fixture for the gap>=H gate.
4. Independent model reconstruction per fold with train-only StandardScaler
and the frozen LogisticRegression config.
5. Independent metric oracles (manual rank/tie AUC, manual PR-AUC, manual
log-loss, manual Brier) on committed predictions and on fresh refits.
6. Structural code inspection of walk_forward.py / prepare_dataset.py /
p3_s21_2_calibration.py for scaler / model / calibration / OOS isolation.
7. Mutation tests on disposable copies (see I).
All implemented in a disposable scratch suite (never committed, never writing to the project output directory).
H. Independent oracle / reference methods
- manual_roc_auc : rank / Mann-Whitney-U, tie-averaged (independent of sklearn).
- manual_pr_auc : ranked precision-recall trapezoid.
- manual_log_loss : manual binary cross-entropy (clipped).
- manual_brier : manual mean squared error on probability.
- manual refit of the P3-S20 fold logistic and of the per-fold sigmoid (Platt)
calibrator, compared to committed parameters and probabilities.
I. Mutation tests performed (all DETECTED -> coverage sufficient)
| Fault injected (disposable copy) | Signal | Detected |
|---|---|---|
| purge boundary shifted by -1 (synthetic, H-1) | boundary=False | YES |
| one future observation into an earlier train set | preds differ 1.39e-02 | YES |
| scaler fit globally (train+OOS) | preds differ 1.06e-02 | YES |
| shuffle temporal order | chronological check fails | YES |
| OOS leaked into model fit (fold1 train+OOS) | preds differ 6.39e-02 | YES |
| feature rows shifted by one bar | preds differ 4.27e-01 | YES |
| calibrator fitted on OOS (S21.2) | probs differ 1.84e-01 | YES |
| row-order / probability-alignment mismatch | probs differ (perm) | YES |
No intentionally broken code was committed to the project.
J. Metamorphic / boundary tests performed
- structural redundancies reproduced (h4==m30==direction; dist==offset-0.5;
choch_latency==sweep_age-choch_age; zone_type constant) - consistent with
recorded S21.1 findings (not bugs).
- synthetic purge boundary fixture: gap==16 ACCEPTED, gap==15 REJECTED (the
purge gate assertion behaves as specified - ">= HORIZON(16)").
K. Leakage tests
- Preprocessing: scaler fit() verified to occur only on the fold TRAINING
partition (code inspection + numeric global-scaler mutation).
- Model fitting: LogisticRegression.fit() runs on the training rows only.
- OOS prediction isolation: OOS rows appear in no scaler.fit(), no fit(),
no calibration fit, no comparator prior fit.
- Calibration: sigmoid(Platt)+isotonic fitted on each fold's TRAINING rows
only; demonstrated via calibrator-OOS mutation detection.
- Comparator prior: per-fold majority prior from the fold train WIN prevalence.
L. Results for P3-S20 (VERIFIED)
Independently confirmed (exact / within tolerance):
dataset accounting : 686 = 594 leads + 92 follow-ons; binary fit = 571
(167 WIN / 404 LOSS); 18 UNRESOLVED + 5 AMBIGUOUS
leads retained; 0 duplicate setup_id / (bar,setup).
temporal order : chronologically non-decreasing creation_bar; every
train index < every OOS index per fold (expanding).
purge : gaps [555, 472, 321] == recorded, all > H(16); gate
boundary proven at gap==16 (acc) vs 15 (rej).
scaler/model/OOS : train-only fits; no leakage (mutation suite detects
global scaler, OOS-into-train, row-shift, shuffle).
independent oracle on committed P3-S20 OOS probs (n=271):
ROC-AUC 0.579160 (reported 0.5792) match <1e-6
PR-AUC 0.341710 (reported 0.34171) match
Log-Loss 0.589505 (reported 0.5895) match
Brier 0.200347 (reported 0.2003) match
per-fold independent refit ROC: 0.5337 / 0.6341 / 0.5350 - identical to
reported fold ROC-AUC 0.5337 / 0.6341 / 0.5350.
Conclusion: the P3-S20 decision "A - STABLE WEAK SIGNAL" is supported by
independent verification. No silent bug found.
M. Results for P3-S21.2 (VERIFIED)
- Baseline reproduction: uncalibrated OOS probabilities == committed
p3_s20_oos_predictions.csv EXACTLY (max_abs_diff = 0.0, 271/271 rows).
- Calibration fitting isolation: sigmoid/isotonic fit ONLY on each fold's
training rows (manual refit reproduces committed calibrator parameters in
all 3 folds); calibrator-OOS mutation is detected (coverage present).
- Fold isolation: fold k calibrator sees only earlier indices; all 3 folds
satisfy train < OOS.
- Independent metrics (pooled, n=271) match the committed summary:
raw sigmoid isotonic
logL 0.5895 0.5897 1.0108 (== committed)
Brier 0.2003 0.2005 0.2181 (== committed)
ROC 0.579160 (raw) / 0.578817 (sigmoid)
(pooled raw AUC etc. recomputed manually.)
- Decision (B - ranking exists, calibration does not improve) reproduces;
the calibration null and the constant-prior-best conclusion hold.
Conclusion: P3-S21.2 is independently VERIFIED; no silent bug found.
N. Escalation to P3-19 / P3-18 / P3-16
NOT triggered. Because P3-S20 dataset accounting, temporal ordering, purge,
preprocessing isolation, model-fitting isolation, and metrics were all
independently validated, no dependency concern warranted escalating backwards
into P3-19 / P3-18 / P3-16 / P3-17R2 per the "evidence-based escalation" rule.
The 571-row split (167/404) and 686==594+92 (WIN/LOSS) reconcile exactly, so
no S18 dataset transformation or S16 label contract is implicated.
P3-S16 label contract (v1) and P3-S17R2 parity are preserved.
O. Discovered potential or confirmed silent bugs
NONE.
- Potential silent bug : 0
- Confirmed silent bug : 0
- Independent contradicts historical : 0
P. Impact assessment
Pending the above, the P3-S20 result and its verdict A - STABLE WEAK SIGNAL,
and the P3-S21.2 result and its verdict B remain reliable. Downstream phases
(S21.1, S21.2) built on P3-S20 are not undermined. No invalidation, no
downgrade of any stored value.
Q. Evidence classification
| Phase | Classification | Basis |
|---|---|---|
| P3-16 (label) | (not re-audited; no escalation) | no dependency concern raised |
| P3-17R2 (parity) | preserved (not re-audited) | high-confidence parity evidence |
| P3-18 (baseline) | (not re-audited; no escalation) | |
| P3-19 (feature/dataset) | (not re-audited; no escalation) | S20 accounting validated |
| P3-20 | VERIFIED | independent accounting/order/purge/isolation/metrics; mutations caught |
| P3-21.2 (calibration) | VERIFIED | independent metrics, baseline repro, isolation; mutations caught |
| P3-21.1 (feature) | VERIFIED AS DEPENDENT REPRODUCTION | reproduces S20 exactly (its audit output not independently recomputed here) |
No phase classified less than the historical standard on the basis of a subject defect; no "PASS"-only classification was used.
R. Historical conclusions remain usable
P3-20 : YES - "A - STABLE WEAK SIGNAL" usable (independently supported).
P3-21.2 : YES - "B" usable (calibration does not improve; prior is best).
Historical artifacts remain byte-identical and auditable (governance
compliant).
S. Required owner decision (if any)
No corrective decision is required by this verification. Any future work
(including a P3-21 index advance) requires separate explicit owner
authorization. This document does NOT authorize any experiment, model,
threshold, label, or deployment change.
T. Explicit statement of what was NOT changed
- NO production code, MQL5 (F1-F4, Candidate Setup, FEATURE_CONTRACT), EA,
or legacy model was modified.
- NO frozen contract (label P3-16 v1, H=16, feature schema feature_sha16
0414e401523ea4e2) was modified.
- NO historical report/CSV/JSON/probability vector was modified.
- NO strategy classifier / threshold / fold definition / label / TP/SL/
horizon was altered.
- NO Trendstop / Dukascopy / external feed was introduced.
- NO LSTM / Informer / regime / ensemble / nonlinear model was run.
- NO new experiment, no P3-21.3, no corrective stage.
- The disposable verification suite and its broken copies live only outside
the repo (Common scratch) and are retained for audit trail but never counted.
F. Evidence-Graded Rollup (final statement)
FINAL : P3-S21.R COMPLETE. NO silent bug found.
INDEPENDENTLY VERIFIED : P3-20 (accounting/order/purge/isolation/metrics)
and P3-21.2 (baseline calibration/metrics/isolation).
HISTORICALLY REPRODUCED : P3-20/P3-21.2 raw and calibrated probability
vectors (identical numbers).
NOT RE-AUDITED (no escalation) : P3-16, P3-17.2, P3-18, P3-19 (no risk found
triggering a backward trace).
P3-20 USABLE : YES. P3-21.2 USABLE : YES.
HARD STOP : no automatic advance to any next phase (incl. P3-21.3).
End P3-S21.R report - explicit the policy, policy v1.