SniperGold_ML/docs/SESSION_HANDOVER_2026-08-22_P3_SMC_SEMANTIC_VALIDATION.md

# SESSION HANDOVER — SNIPERGOLD_ML P3: SMC SEMANTIC VALIDATION

```text
Date       : 2026-08-22
Session    : P3.1 -> P3.2.1 -> P3.2.2 (feature/label forensic + de-overlap/survival) — CLOSED
Status     : RESEARCH / DIAGNOSTIC ONLY — NO production change
Forge      : https://forge.mql5.io/chiki2bum2/SniperGold_ML
```

---

## 1. SESSION SUMMARY

This session completed three consecutive forensic research phases:

| Phase | Output | Core result |
|------|--------|-----------|
| P3.1 | Feature + Label Forensic | 19 runtime-consistent features ≈ uninformative vs the 24-bar/0.75 ATR label (max \|AUC−0.5\|=0.014); label misaligned (t_hit ~2 bars, typical MFE 3.8 ATR); no temporal/regime evidence; root cause = MULTIPLE FACTORS (A dominant, B secondary) |
| P3.2.1 | Event-based label forensic | The 24-bar label measures directional drift, not setup validation; the existing event stream is heavily overlapped (98.5% within 16 bars); f9-confirm events LOSS > WIN; the apparent E_EQH asymmetry = overlap artifact |
| P3.2.2 | Formal de-overlap + survival | Retention after de-overlap ~1.5%; asymmetry gone after de-overlap; serial dependence gone (ac1 0.50 → 0); decision C — EVENT STREAM INVALID AS INDEPENDENT SETUP UNIT |

Commit & push complete:

```text
dc1faa9  research: checkpoint P3.1 feature-label audit and P3.2 event forensic (P3.2.1)
b41059d  research: P3.2.2 formal event de-overlap and survival diagnostic
CURRENT FORGE HEAD   = b41059d814c3dd812c8227dc0d34462f000af2b3
CURRENT LOCAL HEAD   = b41059d814c3dd812c8227dc0d34462f000af2b3
WORKING TREE         = CLEAN (no uncommitted/untracked in the publish mirror)
```

P3.2.2 remains the reference checkpoint. **No artifacts were deleted.**

---

## 2. P3.2.2 STATUS — COMPLETE

```text
P3.2.2 = COMPLETE

event retention after de-overlap  : ≈ 1.5%
97–99% of existing events         : FOLLOW-ON / clustered
median cluster                    : ≈ 46 bars (max 473)
lead-event outcomes               : LOSS >> WIN (SL ~2x to 4.7x TP)
median time-to-event              : ≈ 1–2 bars
~90% of outcomes within 8 bars ; >98% within 16 bars
H16: TP ≈ 0.31 | SL ≈ 0.67 | censoring ≈ 2% | ambiguity ≈ 0.5–1.6%
ALL outcome autocorrelation       : ≈ 0.50
LEAD outcome autocorrelation      : ≈ 0 (de-overlap successful)

DECISION: C — EVENT STREAM ITSELF INVALID (as an independent setup unit)
Clarification:
  SGML_L3_EVENT mechanism = technically viable as a labeling mechanism
  existing event stream    = NOT a valid independent setup unit
```

Evidence stored in `docs/P3_2_2_DEOVERLAP_SURVIVAL.md` + `ml/p3/output/p3_2_deoverlap_h{8,16,24}.json` + `p3_2_survival.json` (committed at `b41059d`).

---

## 3. IMPORTANT NEW RESEARCH CONCLUSION

Do not state anymore:

```text
"SMC has no edge"        — NOT YET PROVEN
"SMC setups are invalid" — NOT YET PROVEN
```

The correct conclusion:

> **The current SMC event implementation has not yet been proven to semantically represent one discrete trading setup.**

New hypotheses to carry into the next session:

```text
A = coding semantic error
B = SMC definition too simple
C = timeframe semantics incorrect
D = event-state treated as event onset
E = combination of A/B/C/D
```

Special note: the findings "median cluster 46 bars" and "events persist as long
state" are consistent with hypothesis D (event-state treated as event onset) —
events that should be discrete (onset) actually persist as long states, causing
repeated counting.

---

## 4. MQL LITE CAPABILITY CONSTRAINT

```text
MQL Lite / AI Assistant built into MT5
= DOES NOT have image analysis capability
```

Therefore the next session is FORBIDDEN from designing:

```text
MT5 AI -> visually reading screenshots/charts
```

as a validation method.

---

## 5. NEW VALIDATION ARCHITECTURE

```text
Historical MT5 data
        ↓
MQL5 SMC Engine
        ↓
Machine Annotation
        │
        ├───────────────┐
        │               │
        ↓               ↓
MT5 chart         OHLC/context
        │               │
        ↓               ↓
Human Annotation
        │
        └───────┬───────┘
                ↓
       Python Comparator
                ↓
 Semantic agreement/error analysis
                ↓
 Regression tests
```

Principle:

```text
MQL Lite is not a vision model.
Humans look at the chart.
The machine reads OHLC/SMC state.
Python compares the two.
```

---

## 6. GOLDEN DATASET PLAN

The next session starts:

```text
SMC SEMANTIC GOLDEN DATASET
```

Objective: test whether the algorithm truly implements the concepts:

```text
Liquidity Sweep | CHoCH / MSS | BOS | FVG | Order Block
Displacement | MTF Alignment | Candidate Setup
```

DO NOT audit all concepts at once.

**First phase: LIQUIDITY SWEEP.**

---

## 7. FIRST GOLDEN DATASET SCOPE

```text
Initial target : 50–100 cases
Must not contain only signal-positive examples.
The distribution must include:
  clear positive, clear negative, borderline,
  false-positive candidates, false-negative candidates,
  trend, range,
  high-volatility, low-volatility
Case selection NOT based on profit results.
```

---

## 8. CASE DEFINITION

Every case must have:

```text
case_id
symbol
decision_timestamp
decision_timeframe
available_data_end
H4 context
M30 context
M15 context
M3 context
```

Human data MUST stop at `decision_timestamp`. **No future candles allowed.**

---

## 9. MACHINE ANNOTATION

Produced from the existing code (no new Sweep definition; measure what the code currently does).

For Liquidity Sweep at minimum:

```text
liquidity_reference
reference_type
reference_price
sweep_high/low
excess_distance
close_after_sweep
rejection
timeframe
algorithm_decision
algorithm_reason
```

---

## 10. HUMAN ANNOTATION

Separate from the machine output. The reviewer answers:

```text
Liquidity Sweep: YES / NO / AMBIGUOUS
Reference: Equal High / Equal Low / Swing High / Swing Low / Other
Reason: free text
Confidence: HIGH / MEDIUM / LOW
```

Machine decisions are NOT shown before the human annotation is complete.

---

## 11. MULTI-ANNOTATOR RULE

```text
Minimum 2 independent human annotators (when possible).
Order: Human A vs Human B first.
Disagreement -> ADJUDICATION REQUIRED (do not treat either human as
                 absolute truth).
Record human agreement as the practical upper bound of agreement.
```

---

## 12. MACHINE VS HUMAN COMPARISON

```text
HUMAN vs MACHINE -> TP / FP / FN / TN / Precision / Recall / F1
Do not stop at numbers: also compare machine_reason vs human_reason.
```

---

## 13. FALSE AGREEMENT CHECK

```text
Human = YES & Machine = YES but different reasons
-> SEMANTIC FALSE AGREEMENT
Example: Human "equal-high swept & rejected" vs Machine "current high > previous swing".
Do not count YES==YES as full semantic success.
```

---

## 14. TIMEFRAME VALIDATION

```text
Compare where the human sees liquidity vs where the machine detects it:
H4 / M30 / M15 / M3.
Goal: find timeframe mismatches
(e.g. human: sweep = M30 event; machine: sweep = M15 event)
-> can indicate an architecture/implementation mismatch.
```

---

## 15. SEMANTIC AGREEMENT LEVELS

```text
Level 1 — Outcome agreement      : Human YES & Machine YES
Level 2 — Structural agreement   : reference level, timeframe, sweep direction,
                                   rejection, event context IDENTICAL
Level 3 — Reason agreement       : human reasoning & machine primitives
                                   substantively IDENTICAL
ONLY Level 3 = STRONG SEMANTIC AGREEMENT.
```

---

## 16. NO PRODUCTION CHANGE

The next session (initial phase) only: research / annotation / comparison / diagnostic.

Forbidden:

```text
modify production SMC logic
modify the feature contract
modify the MLP
modify runtime semantics
train LSTM
train a regime model
implement a meta-label gate
```

If a bug is found:

```text
DOCUMENT -> REPRODUCE -> CLASSIFY -> PROPOSE FIX
```

No hot-fixes during the annotation phase.

---

## 17. NEXT-SESSION ENTRY POINT

```text
First tasks of the next session:
1. Restore this state (Forge HEAD = b41059d8, working tree clean).
2. Read this handover + docs/P3_2_2_DEOVERLAP_SURVIVAL.md.
3. Start the SMC SEMANTIC GOLDEN DATASET — LIQUIDITY SWEEP:
   a. define case selection (50–100 cases, balanced distribution, no
      future candles, case_id + timestamps + 4-TF context);
   b. extract machine annotation from the existing code (no new definitions);
   c. separate human annotation (multi-annotator, adjudication if needed);
   d. Python comparator (TP/FP/FN/TN, Precision/Recall/F1 + reason comparison);
   e. false-agreement check + timeframe validation (Levels 1/2/3);
   f. classify hypotheses A–E (section 3).
4. Do not touch production; do not start P3.3 model work.
```

---

## 18. COMMIT & PUSH STATUS

```text
Commits made this session:
  dc1faa9  research: checkpoint P3.1 feature-label audit and P3.2 event forensic (P3.2.1)
  b41059d  research: P3.2.2 formal event de-overlap and survival diagnostic
Push: complete to https://forge.mql5.io/chiki2bum2/SniperGold_ML.git
FINAL FORGE HEAD  = b41059d814c3dd812c8227dc0d34462f000af2b3
HANDOVER COMMIT SHA = (filled after this handover is committed)
```

---

## 19. APPENDIX — P3.2.2 EVIDENCE LOCATION

```text
docs/P3_FEATURE_LABEL_FORENSIC.md           (P3.1)
docs/P3_2_LABEL_FORENSIC.md                 (P3.2.1)
docs/P3_2_2_DEOVERLAP_SURVIVAL.md           (P3.2.2)
ml/p3/*.py                                   (7+2 diagnostic scripts)
ml/p3/output/*.json                          (feature/label/candidate/temporal/regime/deoverlap/survival)
Large local artifacts (not committed, still available):
  ml/p3/output/p3_2_label_events.json        (17.8 MB, per-event H16 detail)
Key provenance: P2 commit 8d330343 | contract C44CC6F2 | model 06df8452
                | dataset e85a0861 | checkpoint dc1faa9 | P3.2.2 b41059d
```