SniperGold_ML/docs/P3_2_LABEL_FORENSIC.md

# P3.2 LABEL CONTRACT FORENSIC REPORT — SNIPERGOLD_ML

```text
Date       : 2026-08-22
Session    : P3.2 (phase P3.2.1 — event extraction + MFE/MAE + time-to-outcome forensic)
Status     : RESEARCH / DIAGNOSTIC ONLY — no production changes
Source     : P2.6 corrected baseline + P3.1 (immutable checkpoints)
Artifacts  : ml/p3/label_event_audit.py ; ml/p3/output/p3_2_label_events.json ;
             ml/p3/output/p3_2_label_summary.json ; ml/p3/output/p3_2_deoverlap_check.json
```

---

## 0. PRODUCT QUESTION (defined before alternative labels — P3.2 §4)

```text
SMC candidate setup occurs
        -> would this setup be valid to trade?
        -> did the expected outcome occur before invalidation?

Three concepts are distinguished:
  directional prediction : future price direction (all bars)
  setup validation       : whether the EVENT setup produced the expected outcome
                           BEFORE invalidation          <- WHAT MUST BE MEASURED
  trade outcome          : exploitation result (TP/SL, costs, slippage)

The existing label (24-bar / 0.75 ATR) measures DIRECTIONAL PREDICTION
(point-to-point drift close[i+24]-close[i]), NOT setup validation.
```

---

## 1. CHECKPOINT (P3.2 §2)

```text
P2_SOURCE_SHA           : 8d330343af688e2de2b2a1d12ce19a582714709a   OK (HEAD == remote)
P2.6_SOURCE_SHA         : 8d330343... (train_p26=e510ef3b, build_features_p2=4680f905)
FEATURE_CONTRACT_SHA    : C44CC6F2... (canonical) | file 7b908b12... (discrepancy appendix, noted in P3.1)
P2.6_MODEL_SHA          : 06df8452a112290ecb9bae1a3dbff6df492e084f8b872885d808bb0303ec6a70  OK
P2.6_DATASET_SHA        : e85a08616191fb7ce4bbe79c1754228ddbcf54562034a322747501c7aabe124e  OK (recomputed)
P3.1 reproducible       : YES (deterministic script; baseline 0.5105/0.5063 still reproduced)
```

---

## 2. EVENT EXTRACTION (P3.2 §5 — D_EVENT)

Event families are built ONLY from existing semantics (FEATURE_CONTRACT v1.0).
No new setup rules; no production meta-label gate.

| family | definition (existing semantics) | direction | n |
|--------|-------------------------------|------|-----|
| E_BUY | f9==1 & f7>0 (CHoCH confirms bullish sweep) | long | 27096 |
| E_SELL | f9==1 & f7<0 (CHoCH confirms bearish sweep) | short | 25259 |
| E_BUY_STRICT | E_BUY & f18>=60 | long | 13143 |
| E_SELL_STRICT | E_SELL & f18>=60 | short | 9859 |
| E_EQH | f10==1 (equal-high swept) | short | 877 |
| E_EQL | f11==1 (equal-low swept) | long | 1617 |

Each event records: timestamp, direction, SMC state (from features), entry reference
(close of the decision bar), and the outcome path (bars i+1..i+H).

---

## 3. ENTRY SEMANTICS (P3.2 §6)

```text
Primary  : A = close of the closed decision bar
           (closed-bar causality; consistent with the existing label & runtime architecture)
Robustness (entry B = next-bar open):
           median |close[i]-open[i+1]|/ATR = 0.008 ATR ; p95 = 0.055 ATR
           -> the A vs B difference is NOT MATERIAL (< 0.06 ATR p95).
Decision: a single entry semantics (A) is used; B recorded as robustness.
```

---

## 4. MFE / MAE (P3.2 §7 L2)

Median excursions (ATR units), direction-aligned, per horizon:

| family | H=4 | H=8 | H=16 | H=24 |
|--------|-----|-----|------|------|
| E_BUY  MFE / MAE | 0.80 / 0.77 | 1.13 / 1.11 | 1.69 / 1.60 | 2.18 / 2.00 |
| E_SELL MFE / MAE | 0.78 / 0.78 | 1.12 / 1.15 | 1.65 / 1.68 | 2.10 / 2.09 |
| E_EQH  MFE / MAE | 0.77 / 0.68 | 1.15 / 0.94 | 1.65 / 1.32 | 2.02 / 1.64 |
| E_EQL  MFE / MAE | 0.72 / 0.78 | 1.03 / 1.06 | 1.48 / 1.49 | 1.79 / 1.73 |

```text
E_BUY / E_SELL : MFE ~= MAE at ALL horizons (symmetric).
                 -> no favorable payoff asymmetry on the f9-confirm setup.
E_EQH (all)    : MFE > MAE (1.65 vs 1.32 at H16) — THE ONLY asymmetry.
                 BUT: after de-overlap (see §8) the asymmetry DISAPPEARS (overlap artifact).
E_EQL          : symmetric up to MAE > MFE (slightly unfavorable).
```

---

## 5. TP-BEFORE-SL (P3.2 §7 L3 + §9 asymmetric)

L3 = WIN (TP before SL) / LOSS / UNRESOLVED / AMBIGUOUS. Semantic combos:
1.0/0.5, 1.5/0.75, 2.0/1.0 (R:R 2:1), 1.0/1.0 (symmetric).

| family | H | combo | P_WIN | P_LOSS | P_UNRES | P_AMB | med tTP | med tSL |
|--------|---|-------|-------|--------|---------|-------|---------|---------|
| E_BUY | 16 | 1.0/0.5 | 0.323 | **0.638** | 0.012 | 0.027 | 3 | 2 |
| E_BUY | 16 | 1.5/0.75 | 0.310 | **0.629** | 0.052 | 0.008 | 5 | 3 |
| E_BUY | 16 | 2.0/1.0 | 0.282 | **0.596** | 0.116 | 0.005 | 7 | 3 |
| E_BUY | 24 | 1.0/1.0 | 0.488 | 0.478 | 0.022 | 0.012 | 4 | 4 |
| E_SELL | 16 | 1.5/0.75 | 0.310 | **0.633** | 0.048 | 0.009 | 5 | 3 |
| E_SELL | 24 | 1.0/1.0 | 0.487 | 0.483 | 0.018 | 0.012 | 4 | 4 |
| E_EQH | 24 | 1.0/1.0 | 0.525 | 0.436 | 0.039 | 0.001 | 4 | 5 |
| E_EQH | 16 | 1.5/0.75 | 0.326 | 0.577 | 0.092 | 0.005 | 5 | 3 |
| E_EQL | 24 | 1.0/1.0 | 0.466 | 0.495 | 0.032 | 0.007 | 4 | 4 |

```text
1. E_BUY/E_SELL: LOSS > WIN for ALL R:R>1 combos and ALL horizons
   (consistent pattern: LOSS ~2x WIN for 1.5/0.75).
2. tSL < tTP consistently -> SL is touched FIRST:
   the f9-confirm setup is a LATE ENTRY that often hits a pullback before the target.
3. 1.0/1.0 H24 is near 50/50 (E_BUY 0.488/0.478) -> without distance asymmetry,
   there is no edge.
4. E_EQH all points to WIN>LOSS (0.525/0.436) — BUT it is an overlap artifact (§8).
5. Conclusion: NO event family has a TP-before-SL asymmetry that survives
   de-overlap. Asymmetric TP/SL does not help because the problem is in the
   timing/late-entry, not the distance.
```

---

## 6. TIME-TO-OUTCOME (P3.2 §7 L4 + §8)

```text
median time-to-TP : 2-8 bars (increases with TP distance & horizon)
median time-to-SL : 1-5 bars (always <= tTP)
median time-to-MFE: 2-12 bars ; time-to-MAE: 2-11 bars (H4->H24)

Setups empirically "live" very briefly:
  most outcomes (TP or SL) occur within 1-8 bars after the event.
  The 24-bar horizon FAR exceeds the setup lifetime; the 24-bar label captures
  long-term drift UNRELATED to the event.
```

---

## 7. HORIZON ANALYSIS (P3.2 §8)

| H | E_BUY 1.5/0.75 W/L/U | E_SELL 1.5/0.75 W/L/U | E_EQH 1.5/0.75 W/L/U | E_EQL 1.5/0.75 W/L/U |
|---|----------------------|-----------------------|----------------------|----------------------|
| 4 | 0.190/0.487/0.315 | 0.195/0.490/0.308 | 0.203/0.433/0.359 | 0.162/0.489/0.340 |
| 8 | 0.265/0.581/0.146 | 0.266/0.587/0.139 | 0.290/0.520/0.186 | 0.231/0.584/0.176 |
| 16 | 0.310/0.629/0.052 | 0.310/0.633/0.048 | 0.326/0.577/0.092 | 0.280/0.631/0.080 |
| 24 | 0.322/0.640/0.029 | 0.321/0.644/0.026 | 0.344/0.603/0.048 | 0.291/0.644/0.056 |

```text
- Censoring (UNRESOLVED) drops fast: H4 30-36% -> H8 14-19% -> H16 5-9% -> H24 2-6%.
- P_WIN is relatively stable after H8 (asymptote ~0.31-0.34); P_LOSS keeps rising.
- No horizon where the setup shows WIN > LOSS (de-overlapped).
- The 8-16 bar horizon captures most outcomes with low censoring;
  the 24-bar horizon adds nothing but additional SL.
```

---

## 8. EVENT OVERLAP AUDIT (P3.2 §12)

```text
E_BUY  : median gap 1 bar | P(next event within 16 bars) = 98.5% | cluster med 46, max 388
         P_isolated = 0.05% (14 of 27096!)
E_SELL : median gap 1 bar | P(next within 16) = 98.5% | cluster med 46, max 473
         P_isolated = 0.08%
E_EQH  : median gap 1 bar (bimodal; mean 202) | P(next within 16) = 98.1% | cluster 17-384
E_EQL  : median gap 1 bar | P(next within 16) = 98.9% | cluster 42, max 504
```

**MASSIVE overlap.** f9-confirm is a state living in long runs (median cluster
46 bars): consecutive f9 events share the outcome window -> repeated counting.
De-overlap (lead event per cluster, separation > 16 bars):

| family | all -> lead | 1.5/0.75 H16 W/L | 1.0/1.0 H24 W/L |
|--------|-------------|------------------|-----------------|
| E_BUY | 27096 -> 1822 | 0.321 / 0.621 | 0.499 / 0.465 |
| E_SELL | 25259 -> 1680 | 0.315 / 0.634 | 0.485 / 0.484 |
| E_EQH | 877 -> 62 | 0.258 / 0.661 | 0.435 / 0.500 |
| E_EQL | 1617 -> 109 | 0.294 / 0.633 | 0.486 / 0.468 |

```text
AFTER DE-OVERLAP:
  - The E_EQH asymmetry (0.525/0.436) DISAPPEARS -> 0.435/0.500. OVERLAP ARTIFACT.
  - All families: LOSS >= WIN (except E_BUY 1.0/1.0 H24 0.499/0.465 — 50/50).
  - MFE ~= MAE for all families.
CONCLUSION: the event-based formulation WITH existing semantics remains highly
serial-dependent; event-based labels MUST be de-overlapped (lead event per cluster)
before interpretation. And after de-overlap, there is no edge.
```

---

## 9. INVALIDATION SEMANTICS (P3.2 §10)

Uses existing semantics: opposite structure break = f8 (choch_dir) or
f5 (chart_bias) flipping sign against the event direction.

| family | H=16 P(opp break within H) | P(opp break BEFORE TP) |
|--------|------------------------|--------------------------|
| E_BUY | 0.538 | 0.474 |
| E_SELL | 0.594 | 0.526 |
| E_EQH | 0.881 | 0.828 |
| E_EQL | 0.933 | 0.887 |

```text
- f9-confirm BUY/SELL: the opposite structure occurs BEFORE TP on ~47-53% of
  events -> structure often cancels the setup.
- EQH/EQL: 83-89% (the EQH/EQL bias direction is often opposed by structure —
  consistent with E_EQH/E_EQL having no favorable asymmetry).
- Invalidation (opposite break) is a relevant semantic candidate for a
  "failed before target" label, BUT its frequency is very high for EQH/EQL.
```

---

## 10. CENSORING / AMBIGUITY (P3.2 §11)

```text
AMBIGUOUS (TP & SL on the same bar): 0.5% - 2.7% across all families/combos -> LOW.
  (no need to pick a favorable ordering; AMBIGUOUS excluded/recorded.)
Censoring (UNRESOLVED):
  H4  : 12-51% (high) -> short horizons need survival-style treatment.
  H8  : 14-19% (1.5/0.75)
  H16 : 5-9%  (low)  -> binary TP-before-SL is VIABLE.
  H24 : 2-6%  (minimal)
Conclusion: at H16+, binary WIN/LOSS/UNRESOLVED is viable; UNRESOLVED must NOT be
forced to LOSS (real censoring, not failure).
```

---

## 11. FEATURE INFORMATION UNDER CANDIDATE LABELS (P3.2 §13)

The frozen P2 features (19) are evaluated univariately against the L3 label
(WIN vs LOSS) on the event subset, WITHOUT retraining. P3.1 baseline: max |AUC-0.5| = 0.014.

| configuration | E_BUY max\|dev\| (n) | E_SELL max\|dev\| (n) |
|-------------|----------------------|------------------------|
| H8 TP1.0/SL0.5 | 0.0210 (25276) | 0.0108 (23645) |
| H8 TP2.0/SL1.0 | **0.0361** (19617) | 0.0225 (18599) |
| H16 TP2.0/SL1.0 | 0.0309 (23809) | 0.0163 (22368) |
| H24 TP2.0/SL1.0 | 0.0310 (25110) | 0.0169 (23559) |
| MI max | 0.0005 (E_BUY) / 0.0010 (E_SELL H24) | | |

```text
- Information content RISES ~2.5x from baseline (0.014 -> 0.036) when the label
  is aligned with the event (E_BUY, farther TP 2.0 ATR).
- E_BUY is consistently > E_SELL -> the long side is slightly more "predictable"
  by the frozen features against the event-aligned label.
- However, 0.036 is STILL below the practical threshold (~0.05) and not yet
  de-overlapped (part could be overlap artifact).
- Conclusion: the event-aligned label starts to "activate" feature information,
  but it is not yet material. This is partial evidence for "features informative
  against the right label" vs "features are truly uninformative" — not yet
  conclusive (needs de-overlap + final label).
```

---

## 12. LABEL QUALITY SCORECARD (P3.2 §14)

| Criterion | L1 directional | L2 MFE/MAE | L3 TP-before-SL | L4 time-to-outcome |
|-----------|:---:|:---:|:---:|:---:|
| Causal | PASS | PASS | PASS | PASS |
| SMC-aligned | FAIL | PASS | PASS | PASS |
| Execution-realistic | PASS | PASS | PASS | PASS |
| Event-specific | FAIL | PASS | PASS | PASS |
| Low overlap | FAIL (all bars) | PARTIAL (de-overlap) | PARTIAL (de-overlap) | PARTIAL (de-overlap) |
| Low ambiguity | PASS | PASS | PASS (1-3%) | PASS |
| Low censoring | PASS | PASS | PASS (H16+) | PARTIAL (survival) |
| Economically interpretable | FAIL (drift) | PASS (risk) | PASS (W/L) | PASS |
| Suitable for meta-labeling | FAIL | FAIL (not binary) | **PASS** | FAIL (survival) |

```text
L3 (TP-before-SL, event-based, de-overlapped, H16+) = the only family meeting the
meta-labeling requirements: causal, SMC-aligned, event-specific, low ambiguity,
low censoring, binary, economic.
L2/L4 are important for understanding setup characteristics, not as classification targets.
```

---

## 13. ROOT-CAUSE CLASSIFICATION (P3.2 §21/§22-M)

```text
B — LABEL IS MISALIGNED  : STRONGLY SUPPORTED
     - t_hit ~2 bars & typical MFE 3.8 ATR (P3.1): small threshold vs noise path
     - L1 (drift) vs L3 (TP-before-SL) give DIFFERENT profiles: L1 ~50/50,
       L3 LOSS>WIN for f9-confirm -> the 24-bar label measures the wrong thing
     - setups live 1-8 bars, not 24 bars (time-to-outcome)
C — LABEL FAMILY REQUIRED : PARTIALLY SUPPORTED
     - H4 censoring high -> survival/MFE-MAE needed for short horizons
     - for the 8-16 horizon, binary L3 is viable; L2/L4 as descriptive complements
A — FEATURE/SETUP FAILURE : REMAINS VALID (P3.1 finding, confirmed in P3.2)
     - after de-overlap NO event family has a payoff asymmetry
     - MFE ~= MAE; LOSS >= WIN on all existing setups
     - label redesign ALONE will not activate an edge: the existing setup
       semantics (f9-confirm, EQH/EQL as features) produce no asymmetry
```

**Official P3.2 classification: `B — LABEL IS MISALIGNED`** (for setup-validation
purposes), with the strong note that **label redesign is necessary BUT NOT
SUFFICIENT**: the existing setup/feature semantics also produce no payoff
asymmetry after de-overlap (confirmation of finding A from P3.1).

---

## 14. CANDIDATE LABEL CONTRACT (P3.2 §21 — PROPOSED, not deployed)

```text
NAME      : SGML_L3_EVENT (candidate, not yet frozen)
UNIT      : only SMC EVENT bars (D_EVENT), not all bars
ENTRY     : close of the decision bar (A); next-bar open (B) almost identical (0.008 ATR)
OUTCOME   : TP-before-SL
            TP = 1.5 x ATR(entry bar) ; SL = 0.75 x ATR(entry bar)  [semantic 2:1]
            (1.0/1.0 as robustness; 2.0/1.0 for far targets)
HORIZON   : 16 bars (4 hours) primary ; 8 bars (2 hours) short ; 24 bars (6 hours) long
CLASSES   : WIN / LOSS / UNRESOLVED / AMBIGUOUS
            (UNRESOLVED & AMBIGUOUS are NOT forced to LOSS)
DE-OVERLAP: lead event per cluster (gap > 16 bars) OR first event of a cluster;
            isolated/overlapping strata reported
INVALIDATION: opposite structure break (f8/f5 flip) recorded as a secondary
            metric, not a replacement for SL
PROVENANCE: dataset hash, contract hash, script hash, purged split, seed
```

IMPORTANT NOTE: this label is a CANDIDATE CONTRACT for correctly measuring setup
validation — NOT guaranteed to produce an edge. P3.2 evidence shows existing
setups most likely REMAIN edgeless under this label (MFE≈MAE, LOSS≥WIN). The
value of this label is a CORRECT measurement, not an edge creator.

---

## 15. RECOMMENDATION (P3.2 §22-O)

```text
1. The 24-bar/0.75 ATR label is NOT used as a setup-validation measure.
   (It remains valid as a directional-drift measure; not for SMC setups.)
2. Propose SGML_L3_EVENT (section 14) as a CANDIDATE LABEL CONTRACT.
   Review architecture/research first; DO NOT retrain before explicit approval.
3. Deeper findings for future research (not this session):
   - existing setup semantics (f9-confirm & EQH/EQL as features) have NO payoff
     asymmetry after de-overlap -> the problem is at the SETUP/FEATURE
     ARCHITECTURE level, not only the label (P3.1 A remains valid).
   - re-examine the true "candidate setup" definition for SMC (order block,
     breaker, FVG, displacement) as a feature-architecture revision — outside
     the P3.2 mandate (no hot-fix).
4. LSTM / regime / meta-label REMAIN UNTESTED until the label contract is
   approved and the feature re-audit against the new label is complete.
```

---

## 16. SUCCESS CRITERIA (P3.2 §20)

| # | Criterion | Answer |
|---|----------|---------|
| 1 | What does "setup succeeded" mean? | The expected outcome (TP) occurs BEFORE invalidation (SL / opposite break), within the setup-lifetime horizon |
| 2 | Does 24-bar/0.75 ATR measure that? | NO — it measures directional drift; threshold small vs noise (MFE 3.8); horizon >> setup lifetime (1-8 bars) |
| 3 | How long do setups live? | 1-8 bars (median tTP 2-8, tSL 1-5); low censoring at H16+ |
| 4 | Is TP-before-SL more representative? | YES (L3) — causal, SMC-aligned, event-specific; ambiguity 1-3%; low censoring at H16+ |
| 5 | How are MFE/MAE? | Symmetric for all families after de-overlap (MFE ≈ MAE) |
| 6 | How large is event overlap? | MASSIVE: median gap 1 bar; P(next within 16) 98%; cluster med 46 bars; isolated 0.05% |
| 7 | Censoring/ambiguity? | Ambiguity 1-3% (low); censoring H4 12-51%, H16 5-9%, H24 2-6% |
| 8 | Does feature information appear with an event-aligned label? | PARTIAL: max \|AUC-0.5\| 0.014 -> 0.036 (2.5x), E_BUY>E_SELL, still below the practical threshold & not de-overlapped |

---

## 17. EVIDENCE CLASSIFICATION

```text
TRUSTED      : all P3.2.1 tables (deterministic script, provenance recorded,
               dataset hash matches, de-overlap check recorded)
               massive-overlap findings (98%+), MFE≈MAE, LOSS≥WIN after de-overlap
INCONCLUSIVE : feature information under the new label (0.036, not yet de-overlapped) —
               whether features are truly informative against the final label is unresolved
INVALID      : the favorable E_EQH asymmetry in all-events (overlap artifact,
               gone after de-overlap)
SUPERSEDED   : none (P2.6/P3.1 baselines remain valid for their purpose)
```

---

## Appendix — P3.2 Artifacts & Provenance

```text
Script              : ml/p3/label_event_audit.py (SHA recorded in summary)
Output              : ml/p3/output/p3_2_label_events.json (per-event, H16, combo 1.5/0.75)
                      ml/p3/output/p3_2_label_summary.json (full summary)
                      ml/p3/output/p3_2_deoverlap_check.json (lead-event check)
Data                : Files\AlgoForge\Data\XAUUSD_*.npz (P3.1 hash)
P2.6 dataset        : e85a0861...
Frozen model        : 06df8452...
Entry semantics     : close of the decision bar (A); B checked (0.008 ATR med)
Horizons            : 4/8/16/24 M15 bars (1h/2h/4h/6h)
ATR                 : rolling 14-bar M15 (contract)
Random seed         : not used (deterministic)
```

**Closing note:** P3.2 completed without production changes. FEATURE_CONTRACT,
runtime semantics, and the baseline model were NOT touched. The candidate label
(SGML_L3_EVENT) is a research proposal, NOT a deployment. The finding that existing
setups have no payoff asymmetry after de-overlap is a forensic result that should
direct future research to a SETUP/FEATURE architecture review, not to more complex models.