ml/p3/smc_semantic — SMC Semantic Golden Dataset (Phase 1: LIQUIDITY SWEEP)

Session: P3 SMC Semantic Validation (2026-08-22). NO ML.

UPDATE P3-S.0 (2026-08-22) — f7 event lifecycle fix

f7 (DetectLiquidityGrabs) corrected from persistent state -> event lifecycle: f7_lifecycle() with expiration SEQ_WINDOW=40 (InpSeqWindow v4.4 — existing semantics).
Test: test_f7_lifecycle.py (R1-R6 + historical) — 15/15 PASS.
Machine annotation rebuilt: output/machine_annotations_f7_v2.csv (22/60 cases changed, all persistent-state corrected; f10/f11 unchanged).
Details: docs/P3_S_F7_EVENT_LIFECYCLE_FORENSIC.md.

UPDATE P3-S.1 (2026-08-22) — Machine Freeze + Human Annotation prep

machine_annotations_f7_v2.csv = FROZEN machine reference (source_commit b519a34).
Integrity audit: integrity_audit.py -> output/integrity_audit_f7_v2.json — ALL PASS (case set 60/60 v1==v2; YES 45 / NO 15 all stale; diff 22/38/22/0 lifecycle-only; field contract complete).
Blind package: human_package/ (cases.csv, f7 template, protocol, context_m15/) + machine_package/ (BLINDED — annotators must not open).
comparison.py extended: inter-rater A/B + machine-vs-consensus + levels 1-3 + false agreement + timeframe/rejection agreement.
Human annotation: PENDING (waiting for annotators A/B -> human_A_f7.csv / human_B_f7.csv).
Details: docs/P3_S1_HUMAN_MACHINE_F7_VALIDATION.md.

Objective

Prove whether the MQL5 implementation (FEATURE_CONTRACT v1.0) represents the SMC Liquidity Sweep concept semantically the same way humans understand it.

Architecture

MQL5 (existing code)  -> machine_annotator.py  -> machine_annotations.csv
Trader (MT5 chart)    -> human_A.csv / human_B.csv   (BLINDED, up to decision_timestamp)
Python                -> comparison.py               -> comparison_report.json

Files

File	Role
`smc_semantic_common.py`	shared infrastructure (reuses p3_common + train_model; exact f7/f10/f11 semantics)
`sample_cases.py`	stratified sampling of 50-100 golden cases (state machine, vol, regime, year)
`machine_annotator.py`	machine annotation from existing code (v2: f7 lifecycle corrected)
`test_f7_lifecycle.py`	regression R1-R6 + historical before/after (P3-S.0)
`human_annotation_template.csv`	empty template for human annotators (A/B)
`comparison.py`	Human A vs B; Machine vs Human; false-agreement; timeframe audit
`audit_liquidity_sweep.py`	event-vs-state test + actual definition + spec regression test
`probe_state.py`	quick state-distribution probe (debug)
`review_output.py`	sanity check of annotation results
`output/`	cases.csv, cases_meta.json (BLINDED), machine_annotations.csv, audit JSON, comparison JSON

Usage

1. python sample_cases.py 60 42        # create 60 golden cases -> output/cases.csv (+meta BLINDED)
2. python machine_annotator.py         # machine annotation -> output/machine_annotations.csv
3. python audit_liquidity_sweep.py     # event-vs-state audit -> output/audit_liquidity_sweep.json
4. [HUMAN] copy human_annotation_template.csv -> human_A.csv & human_B.csv;
   annotators view the MT5 chart ONLY up to decision_timestamp.
   MUST NOT view cases_meta.json / machine_annotations.csv before finishing.
5. python comparison.py human_A.csv human_B.csv   # -> output/comparison_report.json

Key findings of this session (see docs/P3_SMC_SEMANTIC_GOLDEN_DATASET.md)

f7 (DetectLiquidityGrabs) = PERMANENT STATE: active 99.95% of bars, NEVER resets to 0 after the first grab; a single grab dominates the state with median 94 bars (max 996).
f10/f11 (DetectEQ) = monotonic state (35-43x repetition per onset).
f10/f11 have NO close-back/rejection (definition chosen via AUC, DESAIN_MTF_v45.md).
Machine "no sweep" exists only in 104 bars (0.05%) -> the golden set deliberately uses state age to expose the event-vs-state semantics.
Windowed-vs-fullfeed f7 divergence: 0/1500 probes (empirically a non-issue).

Discipline

Do not choose definitions based on backtests. STOP before production modification if there is definition ambiguity / high human disagreement / event-state bug.

4.2 KiB Raw Permalink Blame History