SniperGold_ML/docs/P3_S19_DATASET_FEATURE_AUDIT.md

# P3-S19 DATASET POPULATION + FEATURE SEMANTICS AUDIT

```text
Date       : 2026-08-24
Session    : P3-S.19 — Dataset Population + Feature Semantics Audit
Status     : COMPLETE
Verdict    : DATASET ACCOUNTING VALIDATED + FEATURE SEMANTICS VALIDATED
             (P3-S.18 dataset correctly accounted; features causal; no
              unexplained discrepancy)
Overall    : AUDIT PASSED
Scope      : AUDIT ONLY. No training, no tuning, no retraining, no
             production/MQL5 change, no Tickstory/Dukascopy, no model.
Human verif: REMAINS CANCELLED (historical only).
```

---

## A. Latest SESSION_HANDOVER used

```text
docs/SESSION_HANDOVER_2026-08-24_P3_S18_BASELINE_ML.md
  (newest authoritative handover, committed at HEAD 86dd8b5; read completely).
Superseded-in-line by docs/P3_S18_BASELINE_ML_REPORT.md (verdict
INCONCLUSIVE / DATA TOO SMALL). Both reconciled with Git at start:
local HEAD == origin/main == 86dd8b5, branch main, working tree CLEAN.
```

## B. Checkpoint

```text
Start  : 86dd8b5405cd7cfdc44fca5d6275c55265194ae7 (verified local==remote)
Branch : main, working tree CLEAN,
origin : https://forge.mql5.io/chiki2bum2/SniperGold_ML.git
Final  : P3_S19_FINAL_SHA (recorded after push)
```

## C. 686 vs 594 reconciliation — RESOLVED (no error)

```text
686 total in-scope Candidate Setups
  = 594 independent LEADS  +  92 FOLLOW-ONS                                        (686 == 594 + 92; S19-T02)

The label counts reported in P3-S18 (WIN 190 / LOSS 469 / UNRESOLVED 22 /
AMBIGUOUS 5 = 686) are over the FULL population INCLUDING follow-ons.

  leads   : WIN 167 | LOSS 404 | UNRESOLVED 18 | AMBIGUOUS 5   (= 594)
  follow-ons: WIN 23 | LOSS 65 | UNRESOLVED 4  | AMBIGUOUS 0   (= 92)
  total   : WIN 190 | LOSS 469 | UNRESOLVED 22 | AMBIGUOUS 5   (= 686)

The ML split used the 594 LEADS ONLY (follow-ons excluded as non-independent);
the binary fit used the 571 WIN/LOSS lead rows (594 - 18 UNRES - 5 AMBIG).

THE APPARENT 686-vs-594 "discrepancy" IS NOT AN ERROR:
  label counts are reported over all 686 (including follow-ons);
  the ML population is the 594 independent leads (and 571 binary-fit rows).
```

## D. Actual ML population

```text
ML split base  : 594 independent leads (chronological, purged 60/20/20)
Binary fit     : 571 WIN/LOSS lead rows (167 WIN + 404 LOSS)
Excluded from fit, RETAINED       : 18 UNRESOLVED leads, 5 AMBIGUOUS leads
Excluded from split, RETAINED     : 92 follow-ons (all labels preserved)
```

## E. Actual train/validation/test accounting

```text
P3-S18 manifest (lead-level split, all 594 leads incl UNRES/AMBIG retained):
  train = 356 (WIN 103 | LOSS 240 | UNRES 10 | AMBIG 3)
  val   = 119 (WIN 30  | LOSS 83  | UNRES 5  | AMBIG 1)
  test  = 119 (WIN 34  | LOSS 81  | UNRES 3  | AMBIG 1)
  sum   = 594  (S19-T06)

Binary-fit rows (WIN/LOSS only):
  train = 343 (103 WIN + 240 LOSS)
  val   = 113 (30 WIN + 83 LOSS)
  test  = 115 (34 WIN + 81 LOSS)
  sum   = 571

The split sums to 594 (all leads); the binary model matrix sums to 571.
```

## F. Label distribution reconciliation

```text
all        : 190 + 469 + 22 + 5  = 686   (S19-T05)
leads      : 167 + 404 + 18 + 5  = 594
follow-ons :  23 +  65 +  4 + 0  = 92
leads+follow = 594 + 92 = 686             (S19-T01, T02)

No UNRESOLVED/AMBIGUOUS forced to WIN/LOSS (contract DS-18/DS-19 honored).
```

## G. Zone-type population

```text
zone_type_code : CONSTANT over all 686 rows AND over the 594 leads
  (single zone type; all in-scope Candidate Setups are one zone type).
This is a COMPOSITION FACT of the verified population (P3-S.17R.2 chain),
NOT a feature-pipeline drop: the same zone-type distribution holds for
all-686, leads, and follow-ons uniformly (S19-T08). Not manufactured/
excluded; no correction needed.
```

## H. H4/M30/direction collinearity

```text
VERIFIED: h4_gate == m30_gate == direction for every lead (S19-T07).
This is EXPECTED STRUCTURAL REDUNDANCY from the canonical Candidate Setup
definition: F3 applies HARD gates (H4 context gate + M30 context gate must
be direction-compatible with the chain direction) before a Candidate Setup
forms. Perfect collinearity is a by-construction property, NOT an
accidental/implementation/data-processing bug. The features are real and
selectively-conditioned; they carry no independent signal in this
population by construction.
```

## I. Feature semantics

```text
All 12 features are causally available as-of entry (creation-bar close):
  direction, h4_gate, m30_gate (selection-conditioned, collinear by design),
  sweep_age_bars, choch_age_bars, choch_latency_bars,
  zone_type_code (constant composition), zone_age_bars,
  zone_width_atr, price_in_zone_offset, dist_to_zone_center_atr, atr_at_entry.
Missing rate 0%; constants: zone_type_code (composition). No feature is
post-entry or outcome-derived (S19-T09/T10). Feature semantics: VALIDATED.
```

## J. Zone geometry / ATR audit

```text
zone_width_atr, price_in_zone_offset, dist_to_zone_center_atr, atr_at_entry
all exist BEFORE entry (computed from closed bars / zone bounds at the
creation-bar decision). ATR(entry) is the same quantity used to scale the
TP/SL label — this is harmless SCALE dependence, NOT target leakage: the
ATR value is known at entry and its inclusion does not reveal the future
outcome. No excluded based on coefficient magnitude (per §13).
```

## K. Feature-label dependency

```text
No feature is mathematically constructed from the outcome or from post-
entry price. ATR scale dependence is legitimate contextual scaling. The
zone-geometry features are deterministic functions of the (causal) zone
bounds and entry close — legitimate contextual dependence, no leakage.
```

## L. Leakage audit

```text
PASS. Features as-of entry; labels scan bars strictly AFTER entry within H;
identity/label namespaces disjoint from features (PD.namespace_verify);
split by creation_bar with purge gap (no label-window overlap across
boundaries). No future mitigation/CHoCH/zone-state/MFE-MAE enters features.
```

## M. Reproducibility

```text
S19-T11 (manifest hash reproducible) PASS; S19/T03/T04 (one setup = one row,
no duplicates) PASS; feature_sha16 unchanged from P3-S18 (identical rows).
Deterministic reconstruction from the byte-locked R.2 chain.
```

## N. P3-S18 result validity

```text
P3-S18 population accounting is CORRECT -> the reported baseline metrics
(Logistic test ROC-AUC 0.609; tree/boost/MLP overfit) are NOT invalidated by
a population-accounting error. They remain VALID as recorded (weak/unstable
hint). Reclassification is NOT required; no re-run needed for accounting.

The features are semantically valid and causal. Any weakness is therefore
interpreted as small-sample signal (INCONCLUSIVE / DATA TOO SMALL), not as a
dataset-provenance artifact uncovered by this audit.
```

## O. Final audit verdict

```text
A. DATASET ACCOUNTING VALIDATED   : YES (686 == 594 + 92; label counts
                                     reconcile; no unexplained discrepancy)
B. DATASET ACCOUNTING ERROR       : NO
C. FEATURE SEMANTICS VALIDATED    : YES (causal, as-of entry, no leakage)
D. FEATURE SEMANTICS ISSUE        : NO
E. AUDIT BLOCKED                  : NO

OVERALL : AUDIT PASSED
```

## P. Production files unchanged

```text
NO MQL5 / F1-F4 / Candidate Setup / FEATURE_CONTRACT / SniperGold_ML change.
Research-only audit artifacts under ml/p3/baseline/output/.
```

## Q. ML status

```text
No training, no tuning, no retraining here. Legacy MLP remains FROZEN /
REFERENCE. P3-S18 baseline results preserved unmodified (not revised).
```

## R. Forge commits

```text
1. research: audit P3-S18 population and feature semantics (P3-S19)
2. docs: record P3-S19 dataset and feature audit
P3_S19_FINAL_SHA = <recorded after push>
```

## S. Final Forge HEAD

```text
See Provenance (local == remote == P3_S19_FINAL_SHA after push).
```

## T. Working tree

```text
CLEAN (verified after push).
```

## U. P3-S20 readiness

```text
NOT STARTED automatically. Audit PASSED -> the owner may authorize
  P3-S20 = PRE-REGISTERED WALK-FORWARD BASELINE (or another explicitly
selected experiment). Any label-parameter / feature-snapshot revision
would require a separate authorized contract review (not this session).
```

---

## Provenance / artifacts

```text
Script : ml/p3/baseline/audit_s19.py  (reuses P3-S18 prepare + R.2 chain)
Output : ml/p3/baseline/output/
  p3_s19_population_reconciliation.csv
  p3_s19_feature_population.csv
  p3_s19_feature_semantics.json
  p3_s19_audit_tests.json            (S19-T01..T12; 12/12 PASS)
Docs   : docs/P3_S19_DATASET_FEATURE_AUDIT.md (this report)
         docs/SESSION_HANDOVER_2026-08-24_P3_S19_DATASET_FEATURE_AUDIT.md
Guard  : new .py scans clean on the frozen P3-S.4/P3-S.5 parity-absence patterns.
No historical report rewritten; no P3-S18/P3-S18A/P3-S16-v1 modified.
```

*End of P3-S19 dataset population + feature semantics audit. Verdict:
DATASET ACCOUNTING VALIDATED + FEATURE SEMANTICS VALIDATED — AUDIT PASSED.
The 686/594/label counts reconcile exactly; features are causal and leak-free.
P3-S18 metrics stand as recorded; no accounting invalidation.*