21 KiB
21 KiB
P3-S.2 LIQUIDITY SWEEP FORMAL SPECIFICATION & CODE-CONFORMANCE REPORT
Date : 2026-08-22
Session : P3-S.2 — Formal SMC Specification + Code-Conformance Audit
Scope : DetectLiquidityGrabs() (f7 sweep_dir) + semantic chain (f9/f18)
Principle : SPECIFICATION FIRST, CODE AUDIT SECOND
Status : NO CODE CHANGES (Phase D) — all findings DOCUMENTED
Artifacts : docs/SMC_LIQUIDITY_SWEEP_SPEC_v1.md (specification)
ml/p3/smc_semantic/spec_tests_liquidity_sweep.py (+ .json) (tests)
ml/p3/smc_semantic/output/spec_tests_liquidity_sweep_report.json
0. CHECKPOINT & PROVENANCE (session-start verification)
Forge HEAD : 0c8847b (handover commit P3-S.2; parent 8c1ef6c) VERIFIED
Working tree : CLEAN at start VERIFIED
P3-S.0 commit : b519a34 (f7 event lifecycle fix) VERIFIED
P3-S.1 commit : 8c1ef6c (machine freeze + human package) VERIFIED
P2 source SHA : 8d330343af688e2de2b2a1d12ce19a582714709a (commit) VERIFIED
P2.6 model SHA : 06df8452... (Include\SniperGold_ML_p26_corrected.mqh) VERIFIED
machine v2 hash : 6d7c0b41... (machine_annotations_f7_v2.csv) VERIFIED
case set hash : da3a0d01... (cases_meta.json) VERIFIED
FEATURE_CONTRACT : recorded C44CC6F2... (P2-era, "uncommitted" per P2 report);
current committed version = 7b908b12... — DIFFERENCE
DOCUMENTED (non-blocking; the file was committed at 8d33034
after the P2 hash was recorded)
DATASET_SHA : e85a0861... (P2.6-era) — current dataset file differs in
hash; DOCUMENTED (non-blocking)
HUMAN VERIFICATION : CANCELLED (human_A/B_f7.csv is NOT a validation gate)
A. SOURCE MATERIAL (Phase A — inventory & reconciliation)
| # | Source | Location | Role |
|---|---|---|---|
| 1 | FEATURE_CONTRACT v1.0 (f7) | docs/FEATURE_CONTRACT.md | Feature contract: "direction of the last liquidity grab (internal pivot)"; wick+close-back; window 8 |
| 2 | DESIGN.md (Algo Forge) | docs/DESIGN.md | Design intent: sweep as a signal element of Engine-2 agents N/E; attribution to SniperGold ProPlus 75466 |
| 3 | DESAIN_MTF_v45.md | Shared Projects\SniperGold_ML | Historical decision: f10/f11 close-back WITHDRAWN via AUC; v4.5 MTF structure; ML features stay M15 |
| 4 | v4.4/v4.5 DetectLiquidityGrabs | Indicators\Downloads\SniperGold_SMC_ProPlus_v4_4/4_5.mq5 | Legacy semantics (persistent state; InpSeqWindow=40 signal gate) |
| 5 | EA DetectLiquidityGrabs (current) | Experts\AlgoForge_Backtest_Baseline.mq5:365-394 | f7 runtime implementation (P3-S.0 fix at emission) |
| 6 | build_features_p2.py | ml/parity/build_features_p2.py | Training side (full-feed, parity-verified) |
| 7 | smc_semantic_common.py | ml/p3/smc_semantic/ | Code-semantics port (sweep_state + f7_lifecycle) |
| 8 | machine_annotator.py | ml/p3/smc_semantic/ | P3-S.1 machine annotation specification (operationalization) |
| 9 | AF_DetectSweep | Include\AlgoForge\AF_Engine2_Agents.mqh:261 | Second implementation (Engine 2, close-back REQUIRED, per-TF) |
| 10 | P3-S.0 forensic | docs/P3_S_F7_EVENT_LIFECYCLE_FORENSIC.md | State/event bug + lifecycle fix; trigger NOT changed |
| 11 | P3-S.1 doc | docs/P3_S1_HUMAN_MACHINE_F7_VALIDATION.md | Machine f7_v2 freeze |
| 12 | Golden dataset doc | docs/P3_SMC_SEMANTIC_GOLDEN_DATASET.md | Actual code definition; state/event findings; rejection ambiguity |
| 13 | audit_liquidity_sweep.json | ml/p3/smc_semantic/output/ | Measured behavior (permanent state 99.95%, etc.) |
| 14 | P2.2 EQH/EQL forensic | docs/P2_2_EQH_EQL_FORENSIC.md | EQ semantics (ATR@row, window, consecutive pairs) |
Three-layer semantic reconciliation:
Intended semantics : sweep = liquidity taken + price returns (intrinsic
rejection); direction opposite the taken side; event
-> CHoCH -> entry chain within InpSeqWindow (40).
Legacy semantics : identical wick+close-back trigger; PERSISTENT state (never
resets); 40 = signal-chain gate.
Current semantics : trigger identical; f7 emission uses the 40-bar lifecycle
(P3-S.0); internal g_swpDir remains persistent (used by
f9/f18 without expiry); f10/f11 = "break" definition without
rejection (AUC decision, DESAIN_MTF_v45).
B. PROJECT SEMANTIC SPECIFICATION (Phase B)
Full document: docs/SMC_LIQUIDITY_SWEEP_SPEC_v1.md (PROJECT SEMANTIC SPECIFICATION — not "universal SMC ground truth").
Spec decision summary:
B-1 Liquidity reference : f7 = internal swing pivot (fractal len=5) SINGLE
level; EQH/EQL & swing len=50 VALID for f10/f11;
HTF/external NOT VALID for f7 (bias only);
others out of scope.
B-2 Reference formation : fractal 5/5, equal allowed, confirmation <= t,
window p in [max(95, t-604), t-5]; MIN_BARS gate.
B-3 Sweep definition : wick break + close-back SAME BAR; b in (p,p+8];
excess >= 1 tick (strict >); no ATR/tick tolerance;
first-match = one onset per reference.
B-4 Direction : buy-side swept -> -1; sell-side swept -> +1.
B-5 Close-back : REQUIRED (intrinsic) for the f7 sweep event.
f10/f11 = NOT PART (state primitive, AUC decision).
B-6 Temporal : decision on the closed bar t; no future.
B-7 Event vs State : EVENT + VALIDITY WINDOW; SEQ_WINDOW=40 derived from
v4.4 InpSeqWindow (sweep->CHoCH->entry chain),
NOT a universal SMC constant.
B-8 Lifecycle : NO_SWEEP / SWEEP_ONSET / ACTIVE / EXPIRED /
SUPERSEDED; INVALIDATED & CONSUMED NOT modeled.
B-9 Timeframe : f7 = M15 ONLY; canonical project = SPECIFICATION
AMBIGUOUS (Engine 2/v4.5 per-TF, ML features M15).
Important spec notes (mathematically derived, data-verified):
- The nominal 8-bar window = effective 3 bars (b in [p+6, p+8]) because bars p+1..p+5 are the fractal confirmation bars — a break there cancels the pivot. Verified on real data: 2.837 onsets, 100% b-p in {6,7,8} (mean 7.02).
- The newest pivot that can be swept at decision bar t is p = t-6; the freshest possible onset = bar t (age 0).
C. FORMAL DEFINITIONS (Phase B — extract)
Sweep_event(ref=(p,L,isHigh), t) ->
exists b: p+6 <= b <= min(t, p+8) AND
( isHigh AND high[b] > L AND close[b] < L ) -- buy-side -> dir=-1
OR
( not isHigh AND low[b] < L AND close[b] > L ) -- sell-side -> dir=+1
f7[t] = dir(newest onset(b<=t)) if t - b <= 40, otherwise 0.
Temporal causality: pivot p valid only if p+5 <= t (confirmation done);
onset b > p; all bars <= t.
D. CURRENT CODE BEHAVIOR (Phase C — code audit)
DetectLiquidityGrabs (Experts\AlgoForge_Backtest_Baseline.mq5:365-394):
for s in g_ip: // internal pivots (ProcessStructure len=5)
p=g_ip[s].bar; lvl=g_ip[s].price;
last=min(total-1, p+GRAB_WIN); // GRAB_WIN=8
for b in (p+1 .. last]:
if isHigh && high[b]>lvl && close[b]<lvl:
if b>g_swpBar: g_swpBar=b; g_swpDir=-1; g_swpLevel=lvl; // buy-side -> -1
break;
if !isHigh && low[b]<lvl && close[b]>lvl:
if b>g_swpBar: g_swpBar=b; g_swpDir=1; g_swpLevel=lvl; // sell-side -> +1
break;
Emission (ComputeMLFeatures:487-489, P3-S.0 fix):
g_feat[7] = (g_swpBar>=0 && (tc-1-g_swpBar)<=AF_BT_SEQ_WINDOW) ? g_swpDir : 0;
Measured behavior (real XAUUSD M15 data, this session):
onset offset b-p : min 6, max 8, mean 7.02 — 100% within {6,7,8}
(effective 3-bar window; consistent with the spec)
state age (r-onset) : min 0, p50 46, p99 314, max 529 — the internal state
stays stale (expiry ONLY at the f7 emission)
Consumers:
f9 chochOK = (g_chochDir!=0 && g_chochBar>=g_swpBar && g_chochDir==g_swpDir); // NO expiry
f18 confluence : if(g_swpDir!=0) cb+=15; if(g_chochDir==g_swpDir) cb+=15; // NO expiry
E. CONFORMANCE MATRIX (Phase C)
Verdicts: CONFORMING / PARTIALLY CONFORMING / NON-CONFORMING / UNVERIFIABLE / SPECIFICATION AMBIGUOUS. (PASS/FAIL not used without semantics — see §J.)
| # | Requirement | Specification | Code (Function/Line) | Actual Behavior | Verdict |
|---|---|---|---|---|---|
| E-1 | Liquidity reference | Internal swing pivot len=5, single level (SPEC B-1) | DetectLiquidityGrabs EA:365-394; ProcessStructure(INTERNAL_LEN=5) EA:200-330; AddInternalPivot |
g_ip = fractal 5/5 (equal allowed), window [max(95,t-604), t-5] | CONFORMING |
| E-2 | Reference formation | Fractal 5/5, equal allowed, confirmation <= t (SPEC B-2) | IsPivotHigh/Low EA:157-169; begin=max(100,t-599) |
Pivot only if p+5 <= t; equal highs/lows do NOT cancel | CONFORMING |
| E-3 | Sweep trigger | wick break + close-back same bar; b in (p,p+8]; first-match (SPEC B-3) | EA:370-383 | high[b]>lvl && close[b]<lvl (buy) / low[b]<lvl && close[b]>lvl (sell); break on the first match |
CONFORMING |
| E-4 | Minimum excess | >= 1 tick, strict >, no ATR/tick tolerance (SPEC B-3) | EA:372,377 | strict inequality; no ATR threshold | CONFORMING |
| E-5 | Direction | buy-side -> -1; sell-side -> +1 (SPEC B-4) | EA:373,379 (g_swpDir=-1/1) |
matches comments: "buy-side swept (bearish intent)" / "sell-side swept (bullish intent)" | CONFORMING |
| E-6 | Rejection / close-back | REQUIRED (intrinsic) (SPEC B-5) | intrinsic to the EA trigger at 372,377 | same-bar close-back is part of the condition | CONFORMING |
| E-7 | Closed-bar / no future | all inputs <= t (SPEC B-6) | Engine 1 closed-bar lock; ComputeAll on closed bars |
LS-T13 PASS; scan bounded by min(total-1, p+8) |
CONFORMING |
| E-8 | Event lifecycle (output f7) | single onset/reference; valid b..b+40; expired -> 0; supersession (SPEC B-7/B-8) | emission EA:487-489 (P3-S.0) | (tc-1-g_swpBar)<=40 ? g_swpDir : 0; LS-T08/T09 PASS |
CONFORMING (only at the f7 emission) |
| E-9 | Event lifecycle (in-function) | the function must implement event semantics | DetectLiquidityGrabs itself |
the function is a STATE DETECTOR (g_swpDir/g_swpBar persist); lifecycle only at the caller | PARTIALLY CONFORMING |
| E-10 | Event lifecycle (consumers f9/f18) | event validity applies across the semantic chain (SPEC B-8 note) | f9 EA:470-471; f18 EA:510-516 | uses raw g_swpDir/g_swpBar WITHOUT an age bound (stale state still +15 / confirm) | NON-CONFORMING (BUG-P3S2-001) |
| E-11 | Per-bar precedence | explicit; same-bar = SPECIFICATION AMBIGUOUS (SPEC §12[3]) | guard b>g_swpBar EA:373,379 |
same-bar: the oldest pivot wins (strict >); unreachable with consistent data (proven) | PARTIALLY CONFORMING (undocumented tie; unreachable) |
| E-12 | Timeframe | M15 for f7 (SPEC B-9) | EA (M15 slot) | M15 only; HTF = bias (f0-f2) | CONFORMING |
| E-13 | Lookback window | 600 analyzed / 700 cache (SPEC B-2) | windowed EA vs full-feed training | THEORETICAL divergence if the last grab is from a pivot < r-699; empirical 0/1500 (P3-S.0 audit) | PARTIALLY CONFORMING (R4; documented risk) |
| E-14 | GRAB_WINDOW nominal vs effective | spec sets nominal 8, effective 3 (consequence of B-2) | window scan (p, p+8] | 8-bar scan but bars p+1..p+5 can never match (fractal confirmation) -> effective {6,7,8} | CONFORMING (code matches the contract; design note) |
F. CONFIRMED BUGS (Phase D — DOCUMENT ONLY, no code changes)
BUG-P3S2-001
SEVERITY : MEDIUM
LOCATION : ComputeMLFeatures f9 (chochOK) & f18 (confluence) — EA:470-471, 510-516
SPEC REQ : R-I / SPEC B-8 — event validity (SEQ_WINDOW) applies across all
semantic consumers
ACTUAL CODE : f9 & f18 use raw g_swpDir/g_swpBar, WITHOUT (t-g_swpBar)<=40
IMPACT : an event already EXPIRED for f7 stays "active" for f9 (choch
confirm) and f18 (+15 score) — stale state (measured age p50=46,
max=529) keeps contributing confluence weight; the semantic chain
is inconsistent (f7 event, f9/f18 state)
PROPOSED FIX : apply the same validity window at the f9/f18 consumption, OR
refactor the event into an onset object with a single validity
lifetime (one lifecycle source of truth)
BUG-P3S2-002
SEVERITY : LOW
LOCATION : DetectLiquidityGrabs guard `b>g_swpBar` (EA:373,379)
SPEC REQ : SPEC §12[3] — multi-reference precedence must be explicit
ACTUAL CODE : on the same bar, the FIRST (oldest) pivot wins (strict >)
IMPACT : only level attribution; direction is always identical for the same
bar; NOT REACHABLE with consistent data (fractal+confirmation
proves two valid pivots cannot both be swept on the same bar)
PROPOSED FIX : document it; or flip the precedence as a design note
BUG-P3S2-003
SEVERITY : LOW
LOCATION : training build_features_p2.py / smc_semantic_common.sweep_state
(full-feed) vs EA (700-bar windowed)
SPEC REQ : R-L / SPEC B-2 — runtime-training consistent lookback
ACTUAL CODE : training uses the full history (pivots >= bar 100); EA uses the
700-bar window (pivots [max(95,t-604), t-5])
IMPACT : THEORETICAL divergence if the last grab comes from a pivot
< r-699; empirical 0/1500 bars (P3-S.0 audit, XAUUSD M15)
PROPOSED FIX : align the window (or document the divergence tolerance)
BUG-P3S2-004
SEVERITY : LOW (documentation)
LOCATION : docs/FEATURE_CONTRACT.md §0 "absolute valid pivots [t-649, t-50]"
SPEC REQ : SPEC B-2 — the window text must be accurate per primitive
ACTUAL CODE : [t-649, t-50] = the SWING pivot window (len=50, f10/f11);
INTERNAL pivots (f7) = [max(95,t-604), t-5]
IMPACT : a contract reader can mistake the f7 window for [t-649,t-50]
PROPOSED FIX : revise the contract: separate the swing vs internal windows
BUG-P3S2-005
SEVERITY : INFO (cosmetic)
LOCATION : DetectLiquidityGrabs parameter `time[]` (EA:366) unused
PROPOSED FIX : remove the parameter or use it for context
Design note (not a bug): the nominal 8-bar grab window = effective 3 bars (b-p in {6,7,8}) — a mathematical consequence of the fractal confirmation rule; already documented in spec §4; whether InpGrabWindow=8 is really the design intent (two thirds of the window "wasted") is agenda K.
G. SEMANTIC AMBIGUITIES (Phase F — consistency)
G-1 CROSS-PRIMITIVE REJECTION (PROJECT)
f7: close-back REQUIRED; f10/f11: NONE (withdrawn via AUC,
DESAIN_MTF_v45.md). Two "sweep" definitions live in one model.
Not an internal f7 contradiction; an AMBIGUITY at the project level.
G-2 SAME-BAR PRECEDENCE
SPECIFICATION AMBIGUOUS (spec §12[3]); proven unreachable with consistent
data; no impact on the f7 direction.
G-3 CANONICAL SWEEP TIMEFRAME (PROJECT)
f7 = M15 (explicit). Engine 2 AF_DetectSweep = per-TF (H4/M30/M15/M3).
v4.5 computes MTF structure but the ML features stay M15.
Status: SPECIFICATION AMBIGUOUS for the "canonical project sweep".
G-4 "INTERNAL SWING" AS LIQUIDITY
f7 uses the internal level (len=5) — the weakest liquidity form per the
SMC concept (usually EQ / major swing). Valid per project; not
adjudicated as "SMC-correct" here.
G-5 EFFECTIVE 3-BAR WINDOW
Nominal 8 vs effective 3 (b-p in {6,7,8}) — the spec now establishes it;
the InpGrabWindow design intent not yet adjudicated.
G-6 "GRAB" vs "SWEEP" TERMINOLOGY
Used interchangeably in the project (DetectLiquidityGrabs = liquidity grab;
AF_DetectSweep = sweep). The spec establishes: both refer to the same concept.
H. SYNTHETIC TEST SPECIFICATION (Phase E)
Runner: ml/p3/smc_semantic/spec_tests_liquidity_sweep.py
Cases: ml/p3/smc_semantic/spec_test_cases_liquidity_sweep.json (17 cases)
Report: ml/p3/smc_semantic/output/spec_tests_liquidity_sweep_report.json
Principle: the expected result is derived from the SPEC (spec_oracle = truth); the code-port result (EA semantics) is reported as a conformance observation.
| ID | Case | Expected (SPEC) | Oracle | Code port | Note |
|---|---|---|---|---|---|
| LS-T01 | Valid buy-side sweep | onset @12 dir=-1 | PASS | match | trigger+direction |
| LS-T02 | No reference | NO_SWEEP | PASS | match | |
| LS-T03 | Touch without break (high==lvl) | NO_SWEEP | PASS | match | strict > |
| LS-T04 | Break without rejection | NO_SWEEP | PASS | match | close-back REQUIRED |
| LS-T05 | Valid sell-side sweep | onset @12 dir=+1 | PASS | match | direction + close-back |
| LS-T06 | Two opposite-direction references | last onset wins | PASS | match | supersession |
| LS-T07 | Two independent references | 2 events | PASS | match | |
| LS-T08 | Repeated condition after onset | only 1 onset | PASS | match | first-match |
| LS-T09 | Event after expiry | f7=0 @age41 | PASS | match | lifecycle |
| LS-T10 | Insufficient history | NO_SWEEP | PASS | match | |
| LS-T11 | Boundary exactly at the level | NO_SWEEP | PASS | match | |
| LS-T12 | Boundary 1 tick outside | SWEEP | PASS | match | min excess 1 tick |
| LS-T13 | Future-bar mutation | decision unchanged | PASS | match | closed-bar |
| LS-T14 | Price-scale invariance | same detection | PASS | match | |
| LS-T15 | Price-translation invariance | same detection | PASS | match | |
| LS-T16 | Two same-bar refs (tie) | same f7 (identical direction); tie only in level | PASS | match (f7) | unreachable with consistent data |
| LS-T17 | Reference window edge | p outside the window ignored | PASS | match | SPEC B-2 |
Result: 17/17 PASS — expected results consistent with SPEC v1; the code port matches the spec in ALL cases (including LS-T09/T13/T16/T17). This shows that for detection (trigger/direction/rejection/closed-bar/window), the EA port == spec.
I. REFERENCE IMPLEMENTATION (optional, secondary)
spec_oracle() in spec_tests_liquidity_sweep.py = the SPEC evaluator (truth).
code_port() = the faithful EA port (implementation under audit). Code B (spec
oracle) does NOT define the truth; the truth = FORMAL SPEC. When code A (EA) and
the spec oracle differ → investigate the SPEC (not assume B is right). Result: no
divergence on the tested cases.
J. FINAL VERDICT
PARTIALLY CONFORMING
Verdict semantics (not meaningless PASS/FAIL):
1. DETECTION CHAIN (E-1 to E-8, E-12, E-14): CONFORMING.
DetectLiquidityGrabs() really implements the formal Liquidity Sweep definition
for: internal swing len=5 reference, wick+same-bar close-back trigger, direction
(buy-side->-1 / sell-side->+1), excess >= 1 tick, closed-bar (no future), grab
window, M15 timeframe.
Proven: line-by-line code audit + 17/17 synthetic tests + real-data verification
(b-p in {6,7,8}, 100%).
2. EVENT SEMANTICS (E-9, E-10): PARTIALLY / NON-CONFORMING.
a) The function itself is a STATE DETECTOR (persistent), not an event emitter
— the lifecycle is applied only by the caller (f7 emission).
b) Consumers f9/f18 use raw state WITHOUT a validity window (BUG-P3S2-001) —
the semantic chain is NOT consistent with "EVENT + VALIDITY WINDOW".
Conclusion: the f7 implementation is CONFORMING on the DETECTION definition, but the
overall implementation (function + consumer chain) is only PARTIALLY CONFORMING
against the project's EVENT-lifecycle specification.
Mandatory interpretation: this verdict does NOT mean "the strategy has an edge" and
does NOT mean "the SMC concept is invalid". The verdict only answers: does the
implementation match the project specification? Answer: mostly YES (detection),
partly NO (lifecycle propagation to consumers).
K. REQUIRED FUTURE FIXES (documentation only — NO CODE CHANGE this session)
K-1 [MEDIUM] Apply the SEQ_WINDOW validity window to the f9/f18 consumption
(or refactor the event source) — closes BUG-P3S2-001.
K-2 [LOW] Document the same-bar precedence (BUG-P3S2-002); optionally flip it.
K-3 [LOW] Align the f7 runtime-training window (BUG-P3S2-003) OR document the
tolerance.
K-4 [LOW] Revise FEATURE_CONTRACT §0: separate the swing vs internal windows
(BUG-P3S2-004).
K-5 [INFO] Evaluate the InpGrabWindow=8 intent vs the effective 3-bar window
(G-5); if 3 bars is indeed the intent, consider documentation/adjustment.
K-6 [DESIGN] Adjudicate the f10/f11 rejection (AUC decision vs SMC semantics)
— G-1; and the canonical sweep timeframe (M15-only vs MTF liquidity) — G-3.
L. NEXT SEMANTIC AUDIT
After Liquidity Sweep receives its verdict (PARTIALLY CONFORMING — accepted as a
defensible verdict for this session), continue with the following concept order
(per brief §19):
1. CHoCH / MSS <- next (f8/f9, AF_DetectChoch)
2. FVG / OB (context features; AF_FindFVG / AF_FindOrderBlock)
3. Displacement (AF_DetectDisplacement)
4. MTF Alignment (roles of H4/M30/M3; G-3)
5. Candidate Setup (zone + confirmation = setup; DESIGN.md §9)
Each concept follows the same protocol: SPEC FIRST -> CODE AUDIT -> SYNTHETIC TESTS -> VERDICT. The Liquidity Sweep verdict must not be read as an "edge strategy"; the CHoCH verdict likewise.