SniperGold_ML/ml/p3/smc_semantic/README.md

# ml/p3/smc_semantic — SMC Semantic Golden Dataset (Phase 1: LIQUIDITY SWEEP)

Session: P3 SMC Semantic Validation (2026-08-22). **NO ML.**

## UPDATE P3-S.0 (2026-08-22) — f7 event lifecycle fix

- f7 (DetectLiquidityGrabs) corrected from persistent state -> event lifecycle:
  `f7_lifecycle()` with expiration `SEQ_WINDOW=40` (InpSeqWindow v4.4 — existing semantics).
- Test: `test_f7_lifecycle.py` (R1-R6 + historical) — **15/15 PASS**.
- Machine annotation rebuilt: `output/machine_annotations_f7_v2.csv`
  (22/60 cases changed, all persistent-state corrected; f10/f11 unchanged).
- Details: `docs/P3_S_F7_EVENT_LIFECYCLE_FORENSIC.md`.

## UPDATE P3-S.1 (2026-08-22) — Machine Freeze + Human Annotation prep

- `machine_annotations_f7_v2.csv` = FROZEN machine reference (source_commit b519a34).
- Integrity audit: `integrity_audit.py` -> `output/integrity_audit_f7_v2.json` — ALL PASS
  (case set 60/60 v1==v2; YES 45 / NO 15 all stale; diff 22/38/22/0 lifecycle-only;
  field contract complete).
- Blind package: `human_package/` (cases.csv, f7 template, protocol, context_m15/) +
  `machine_package/` (BLINDED — annotators must not open).
- `comparison.py` extended: inter-rater A/B + machine-vs-consensus + levels 1-3 +
  false agreement + timeframe/rejection agreement.
- Human annotation: **PENDING** (waiting for annotators A/B -> human_A_f7.csv / human_B_f7.csv).
- Details: `docs/P3_S1_HUMAN_MACHINE_F7_VALIDATION.md`.

## Objective

Prove whether the MQL5 implementation (FEATURE_CONTRACT v1.0) represents the SMC
**Liquidity Sweep** concept semantically the same way humans understand it.

## Architecture

```text
MQL5 (existing code)  -> machine_annotator.py  -> machine_annotations.csv
Trader (MT5 chart)    -> human_A.csv / human_B.csv   (BLINDED, up to decision_timestamp)
Python                -> comparison.py               -> comparison_report.json
```

## Files

| File | Role |
|------|------|
| `smc_semantic_common.py` | shared infrastructure (reuses p3_common + train_model; exact f7/f10/f11 semantics) |
| `sample_cases.py` | stratified sampling of 50-100 golden cases (state machine, vol, regime, year) |
| `machine_annotator.py` | machine annotation from existing code (v2: f7 lifecycle corrected) |
| `test_f7_lifecycle.py` | regression R1-R6 + historical before/after (P3-S.0) |
| `human_annotation_template.csv` | empty template for human annotators (A/B) |
| `comparison.py` | Human A vs B; Machine vs Human; false-agreement; timeframe audit |
| `audit_liquidity_sweep.py` | event-vs-state test + actual definition + spec regression test |
| `probe_state.py` | quick state-distribution probe (debug) |
| `review_output.py` | sanity check of annotation results |
| `output/` | cases.csv, cases_meta.json (BLINDED), machine_annotations.csv, audit JSON, comparison JSON |

## Usage

```text
1. python sample_cases.py 60 42        # create 60 golden cases -> output/cases.csv (+meta BLINDED)
2. python machine_annotator.py         # machine annotation -> output/machine_annotations.csv
3. python audit_liquidity_sweep.py     # event-vs-state audit -> output/audit_liquidity_sweep.json
4. [HUMAN] copy human_annotation_template.csv -> human_A.csv & human_B.csv;
   annotators view the MT5 chart ONLY up to decision_timestamp.
   MUST NOT view cases_meta.json / machine_annotations.csv before finishing.
5. python comparison.py human_A.csv human_B.csv   # -> output/comparison_report.json
```

## Key findings of this session (see docs/P3_SMC_SEMANTIC_GOLDEN_DATASET.md)

- f7 (DetectLiquidityGrabs) = **PERMANENT STATE**: active 99.95% of bars, NEVER
  resets to 0 after the first grab; a single grab dominates the state with median 94 bars (max 996).
- f10/f11 (DetectEQ) = monotonic state (35-43x repetition per onset).
- f10/f11 have **NO close-back/rejection** (definition chosen via AUC, DESAIN_MTF_v45.md).
- Machine "no sweep" exists only in 104 bars (0.05%) -> the golden set deliberately
  uses state age to expose the event-vs-state semantics.
- Windowed-vs-fullfeed f7 divergence: 0/1500 probes (empirically a non-issue).

## Discipline

Do not choose definitions based on backtests. STOP before production modification
if there is definition ambiguity / high human disagreement / event-state bug.