forked from chiki2bum2/SniperGold_ML
258 lines
No EOL
8.9 KiB
Markdown
258 lines
No EOL
8.9 KiB
Markdown
# P3-S19 DATASET POPULATION + FEATURE SEMANTICS AUDIT
|
|
|
|
```text
|
|
Date : 2026-08-24
|
|
Session : P3-S.19 — Dataset Population + Feature Semantics Audit
|
|
Status : COMPLETE
|
|
Verdict : DATASET ACCOUNTING VALIDATED + FEATURE SEMANTICS VALIDATED
|
|
(P3-S.18 dataset correctly accounted; features causal; no
|
|
unexplained discrepancy)
|
|
Overall : AUDIT PASSED
|
|
Scope : AUDIT ONLY. No training, no tuning, no retraining, no
|
|
production/MQL5 change, no Tickstory/Dukascopy, no model.
|
|
Human verif: REMAINS CANCELLED (historical only).
|
|
```
|
|
|
|
---
|
|
|
|
## A. Latest SESSION_HANDOVER used
|
|
|
|
```text
|
|
docs/SESSION_HANDOVER_2026-08-24_P3_S18_BASELINE_ML.md
|
|
(newest authoritative handover, committed at HEAD 86dd8b5; read completely).
|
|
Superseded-in-line by docs/P3_S18_BASELINE_ML_REPORT.md (verdict
|
|
INCONCLUSIVE / DATA TOO SMALL). Both reconciled with Git at start:
|
|
local HEAD == origin/main == 86dd8b5, branch main, working tree CLEAN.
|
|
```
|
|
|
|
## B. Checkpoint
|
|
|
|
```text
|
|
Start : 86dd8b5405cd7cfdc44fca5d6275c55265194ae7 (verified local==remote)
|
|
Branch : main, working tree CLEAN,
|
|
origin : https://forge.mql5.io/chiki2bum2/SniperGold_ML.git
|
|
Final : P3_S19_FINAL_SHA (recorded after push)
|
|
```
|
|
|
|
## C. 686 vs 594 reconciliation — RESOLVED (no error)
|
|
|
|
```text
|
|
686 total in-scope Candidate Setups
|
|
= 594 independent LEADS + 92 FOLLOW-ONS (686 == 594 + 92; S19-T02)
|
|
|
|
The label counts reported in P3-S18 (WIN 190 / LOSS 469 / UNRESOLVED 22 /
|
|
AMBIGUOUS 5 = 686) are over the FULL population INCLUDING follow-ons.
|
|
|
|
leads : WIN 167 | LOSS 404 | UNRESOLVED 18 | AMBIGUOUS 5 (= 594)
|
|
follow-ons: WIN 23 | LOSS 65 | UNRESOLVED 4 | AMBIGUOUS 0 (= 92)
|
|
total : WIN 190 | LOSS 469 | UNRESOLVED 22 | AMBIGUOUS 5 (= 686)
|
|
|
|
The ML split used the 594 LEADS ONLY (follow-ons excluded as non-independent);
|
|
the binary fit used the 571 WIN/LOSS lead rows (594 - 18 UNRES - 5 AMBIG).
|
|
|
|
THE APPARENT 686-vs-594 "discrepancy" IS NOT AN ERROR:
|
|
label counts are reported over all 686 (including follow-ons);
|
|
the ML population is the 594 independent leads (and 571 binary-fit rows).
|
|
```
|
|
|
|
## D. Actual ML population
|
|
|
|
```text
|
|
ML split base : 594 independent leads (chronological, purged 60/20/20)
|
|
Binary fit : 571 WIN/LOSS lead rows (167 WIN + 404 LOSS)
|
|
Excluded from fit, RETAINED : 18 UNRESOLVED leads, 5 AMBIGUOUS leads
|
|
Excluded from split, RETAINED : 92 follow-ons (all labels preserved)
|
|
```
|
|
|
|
## E. Actual train/validation/test accounting
|
|
|
|
```text
|
|
P3-S18 manifest (lead-level split, all 594 leads incl UNRES/AMBIG retained):
|
|
train = 356 (WIN 103 | LOSS 240 | UNRES 10 | AMBIG 3)
|
|
val = 119 (WIN 30 | LOSS 83 | UNRES 5 | AMBIG 1)
|
|
test = 119 (WIN 34 | LOSS 81 | UNRES 3 | AMBIG 1)
|
|
sum = 594 (S19-T06)
|
|
|
|
Binary-fit rows (WIN/LOSS only):
|
|
train = 343 (103 WIN + 240 LOSS)
|
|
val = 113 (30 WIN + 83 LOSS)
|
|
test = 115 (34 WIN + 81 LOSS)
|
|
sum = 571
|
|
|
|
The split sums to 594 (all leads); the binary model matrix sums to 571.
|
|
```
|
|
|
|
## F. Label distribution reconciliation
|
|
|
|
```text
|
|
all : 190 + 469 + 22 + 5 = 686 (S19-T05)
|
|
leads : 167 + 404 + 18 + 5 = 594
|
|
follow-ons : 23 + 65 + 4 + 0 = 92
|
|
leads+follow = 594 + 92 = 686 (S19-T01, T02)
|
|
|
|
No UNRESOLVED/AMBIGUOUS forced to WIN/LOSS (contract DS-18/DS-19 honored).
|
|
```
|
|
|
|
## G. Zone-type population
|
|
|
|
```text
|
|
zone_type_code : CONSTANT over all 686 rows AND over the 594 leads
|
|
(single zone type; all in-scope Candidate Setups are one zone type).
|
|
This is a COMPOSITION FACT of the verified population (P3-S.17R.2 chain),
|
|
NOT a feature-pipeline drop: the same zone-type distribution holds for
|
|
all-686, leads, and follow-ons uniformly (S19-T08). Not manufactured/
|
|
excluded; no correction needed.
|
|
```
|
|
|
|
## H. H4/M30/direction collinearity
|
|
|
|
```text
|
|
VERIFIED: h4_gate == m30_gate == direction for every lead (S19-T07).
|
|
This is EXPECTED STRUCTURAL REDUNDANCY from the canonical Candidate Setup
|
|
definition: F3 applies HARD gates (H4 context gate + M30 context gate must
|
|
be direction-compatible with the chain direction) before a Candidate Setup
|
|
forms. Perfect collinearity is a by-construction property, NOT an
|
|
accidental/implementation/data-processing bug. The features are real and
|
|
selectively-conditioned; they carry no independent signal in this
|
|
population by construction.
|
|
```
|
|
|
|
## I. Feature semantics
|
|
|
|
```text
|
|
All 12 features are causally available as-of entry (creation-bar close):
|
|
direction, h4_gate, m30_gate (selection-conditioned, collinear by design),
|
|
sweep_age_bars, choch_age_bars, choch_latency_bars,
|
|
zone_type_code (constant composition), zone_age_bars,
|
|
zone_width_atr, price_in_zone_offset, dist_to_zone_center_atr, atr_at_entry.
|
|
Missing rate 0%; constants: zone_type_code (composition). No feature is
|
|
post-entry or outcome-derived (S19-T09/T10). Feature semantics: VALIDATED.
|
|
```
|
|
|
|
## J. Zone geometry / ATR audit
|
|
|
|
```text
|
|
zone_width_atr, price_in_zone_offset, dist_to_zone_center_atr, atr_at_entry
|
|
all exist BEFORE entry (computed from closed bars / zone bounds at the
|
|
creation-bar decision). ATR(entry) is the same quantity used to scale the
|
|
TP/SL label — this is harmless SCALE dependence, NOT target leakage: the
|
|
ATR value is known at entry and its inclusion does not reveal the future
|
|
outcome. No excluded based on coefficient magnitude (per §13).
|
|
```
|
|
|
|
## K. Feature-label dependency
|
|
|
|
```text
|
|
No feature is mathematically constructed from the outcome or from post-
|
|
entry price. ATR scale dependence is legitimate contextual scaling. The
|
|
zone-geometry features are deterministic functions of the (causal) zone
|
|
bounds and entry close — legitimate contextual dependence, no leakage.
|
|
```
|
|
|
|
## L. Leakage audit
|
|
|
|
```text
|
|
PASS. Features as-of entry; labels scan bars strictly AFTER entry within H;
|
|
identity/label namespaces disjoint from features (PD.namespace_verify);
|
|
split by creation_bar with purge gap (no label-window overlap across
|
|
boundaries). No future mitigation/CHoCH/zone-state/MFE-MAE enters features.
|
|
```
|
|
|
|
## M. Reproducibility
|
|
|
|
```text
|
|
S19-T11 (manifest hash reproducible) PASS; S19/T03/T04 (one setup = one row,
|
|
no duplicates) PASS; feature_sha16 unchanged from P3-S18 (identical rows).
|
|
Deterministic reconstruction from the byte-locked R.2 chain.
|
|
```
|
|
|
|
## N. P3-S18 result validity
|
|
|
|
```text
|
|
P3-S18 population accounting is CORRECT -> the reported baseline metrics
|
|
(Logistic test ROC-AUC 0.609; tree/boost/MLP overfit) are NOT invalidated by
|
|
a population-accounting error. They remain VALID as recorded (weak/unstable
|
|
hint). Reclassification is NOT required; no re-run needed for accounting.
|
|
|
|
The features are semantically valid and causal. Any weakness is therefore
|
|
interpreted as small-sample signal (INCONCLUSIVE / DATA TOO SMALL), not as a
|
|
dataset-provenance artifact uncovered by this audit.
|
|
```
|
|
|
|
## O. Final audit verdict
|
|
|
|
```text
|
|
A. DATASET ACCOUNTING VALIDATED : YES (686 == 594 + 92; label counts
|
|
reconcile; no unexplained discrepancy)
|
|
B. DATASET ACCOUNTING ERROR : NO
|
|
C. FEATURE SEMANTICS VALIDATED : YES (causal, as-of entry, no leakage)
|
|
D. FEATURE SEMANTICS ISSUE : NO
|
|
E. AUDIT BLOCKED : NO
|
|
|
|
OVERALL : AUDIT PASSED
|
|
```
|
|
|
|
## P. Production files unchanged
|
|
|
|
```text
|
|
NO MQL5 / F1-F4 / Candidate Setup / FEATURE_CONTRACT / SniperGold_ML change.
|
|
Research-only audit artifacts under ml/p3/baseline/output/.
|
|
```
|
|
|
|
## Q. ML status
|
|
|
|
```text
|
|
No training, no tuning, no retraining here. Legacy MLP remains FROZEN /
|
|
REFERENCE. P3-S18 baseline results preserved unmodified (not revised).
|
|
```
|
|
|
|
## R. Forge commits
|
|
|
|
```text
|
|
1. research: audit P3-S18 population and feature semantics (P3-S19)
|
|
2. docs: record P3-S19 dataset and feature audit
|
|
P3_S19_FINAL_SHA = <recorded after push>
|
|
```
|
|
|
|
## S. Final Forge HEAD
|
|
|
|
```text
|
|
See Provenance (local == remote == P3_S19_FINAL_SHA after push).
|
|
```
|
|
|
|
## T. Working tree
|
|
|
|
```text
|
|
CLEAN (verified after push).
|
|
```
|
|
|
|
## U. P3-S20 readiness
|
|
|
|
```text
|
|
NOT STARTED automatically. Audit PASSED -> the owner may authorize
|
|
P3-S20 = PRE-REGISTERED WALK-FORWARD BASELINE (or another explicitly
|
|
selected experiment). Any label-parameter / feature-snapshot revision
|
|
would require a separate authorized contract review (not this session).
|
|
```
|
|
|
|
---
|
|
|
|
## Provenance / artifacts
|
|
|
|
```text
|
|
Script : ml/p3/baseline/audit_s19.py (reuses P3-S18 prepare + R.2 chain)
|
|
Output : ml/p3/baseline/output/
|
|
p3_s19_population_reconciliation.csv
|
|
p3_s19_feature_population.csv
|
|
p3_s19_feature_semantics.json
|
|
p3_s19_audit_tests.json (S19-T01..T12; 12/12 PASS)
|
|
Docs : docs/P3_S19_DATASET_FEATURE_AUDIT.md (this report)
|
|
docs/SESSION_HANDOVER_2026-08-24_P3_S19_DATASET_FEATURE_AUDIT.md
|
|
Guard : new .py scans clean on the frozen P3-S.4/P3-S.5 parity-absence patterns.
|
|
No historical report rewritten; no P3-S18/P3-S18A/P3-S16-v1 modified.
|
|
```
|
|
|
|
*End of P3-S19 dataset population + feature semantics audit. Verdict:
|
|
DATASET ACCOUNTING VALIDATED + FEATURE SEMANTICS VALIDATED — AUDIT PASSED.
|
|
The 686/594/label counts reconcile exactly; features are causal and leak-free.
|
|
P3-S18 metrics stand as recorded; no accounting invalidation.* |