20 KiB
20 KiB
P3-S.3 CHOCH / MSS FORMAL SPECIFICATION & CODE-CONFORMANCE REPORT
Date : 2026-08-22
Session : P3-S.3 — Formal SMC Specification + Code-Conformance Audit (CHoCH/MSS)
Scope : f8 (choch_dir) / f9 (choch_confirms) / f18 (CHoCH confluence leg)
+ legacy signal chain (InpRequireCHoCH / InpSeqWindow)
+ Engine 2 (AF_DetectChoch) + Display (AF_BuildStructLines)
Principle : SPECIFICATION FIRST, CODE AUDIT SECOND
Status : NO CODE CHANGES (Phase D) — all findings DOCUMENTED
Artifacts : docs/SMC_CHOCH_MSS_SPEC_v1.md (specification)
ml/p3/smc_semantic/spec_tests_choch_mss.py (+ .json) (tests)
ml/p3/smc_semantic/output/spec_tests_choch_mss_report.json
ml/p3/smc_semantic/output/audit_choch_state.json (forensic)
ml/p3/smc_semantic/output/parity_choch_pivot_strict_vs_equal.json
0. CHECKPOINT & PROVENANCE (session-start verification)
Forge HEAD : 595b4c9a622703b05228bd563b4979fef81a2ebf (P3-S.2) VERIFIED
Research checkpoint : c7a2b3f5e6acbf8deb748315dcbcc200046d0b84 VERIFIED
Branch : main VERIFIED
Remote : forge.mql5.io/chiki2bum2/SniperGold_ML.git VERIFIED
Working tree : CLEAN at start VERIFIED
P3-S.2 artifacts : spec/conformance/tests/report/handover INTACT VERIFIED
HUMAN VERIFICATION : CANCELLED (human annotation is NOT a validation gate) VERIFIED
A. SOURCE MATERIAL (Phase A — inventory & reconciliation)
| # | Source | Location | Role |
|---|---|---|---|
| 1 | FEATURE_CONTRACT v1.0 (f8/f9/f18) | docs/FEATURE_CONTRACT.md | Feature contract: f8 "last CHoCH direction (internal structure)"; f9 "CHoCH confirms sweep"; f18 +15 when chochDir==swpDir |
| 2 | DESIGN.md (Algo Forge) | docs/DESIGN.md | Design intent: "CHoCH/MSS" element of agent N (structure); E (confirmation); synonym labels |
| 3 | DESAIN_MTF_v45.md | Shared Projects\SniperGold_ML | v4.5 per-TF structure (BOS/CHoCH) + cache; ML features stay M15; AUC decisions only for f10/f11 |
| 4 | v4.3/v4.4/v4.5 ProcessStructure | Indicators\Downloads\SniperGold_SMC_ProPlus_v4_3/4_4/4_5.mq5 | Legacy CHoCH semantics: internal break + InpConfluenceFilter + InpSeqWindow signal gate |
| 5 | EA ProcessStructure (current) | Experts\AlgoForge_Backtest_Baseline.mq5:294-340 | f8/f9 runtime implementation (v4.4 copy) |
| 6 | build_features_p2.py | ml/parity/build_features_p2.py | Training side (build_structure_fast = equal-allowed pivots, parity-verified) |
| 7 | train_model.py build_structure | ml/train_model.py:161-229 | Streaming ProcessStructure replica (STRICT pivots: equal NOT allowed) |
| 8 | smc_semantic_common.py | ml/p3/smc_semantic/ | P3 forensic infra (uses TM.build_structure -> strict pivots) |
| 9 | AF_DetectChoch / AF_DetectBos | Include\AlgoForge\AF_Engine2_Agents.mqh:238-256 | Second CHoCH definition (Engine 2: fractal 2/2, no gate, per-TF) |
| 10 | AF_BuildStructLines | Include\AlgoForge\AF_Engine2_Display.mqh:137-192 | Third CHoCH definition (display: fractal 2/2, CHoCH/BOS labels) |
| 11 | P3-S.0 f7 forensic | docs/P3_S_F7_EVENT_LIFECYCLE_FORENSIC.md | State/event bug pattern + lifecycle fix (CHoCH comparison) |
| 12 | P3-S.2 spec/conformance | docs/SMC_LIQUIDITY_SWEEP_SPEC_v1.md, docs/P3_S2_... | Protocol, BUG-P3S2-001 (f9/f18 stale sweep), next gate CHoCH/MSS |
| 13 | Golden dataset doc | docs/P3_SMC_SEMANTIC_GOLDEN_DATASET.md | SMC concept list (Liquidity Sweep |
| 14 | audit_choch_state.json (NEW) | ml/p3/smc_semantic/output/ | Measured CHoCH state behavior (this session) |
| 15 | parity_choch_pivot_strict_vs_equal.json (NEW) | ml/p3/smc_semantic/output/ | Strict vs equal pivot divergence (this session) |
Three-layer semantic reconciliation:
Intended semantics : CHoCH = internal structure reversal (close break of the last
internal swing + opposite prior trend + non-opposing swing
gate); CHoCH/MSS = synonym labels (DESIGN.md); Sweep->CHoCH->
OB/FVG chain within InpSeqWindow (40); a separate MSS
(break+displacement) NOT defined.
Legacy semantics : trigger identical; the signal gate applies freshness
(curBar-chochBar)<=40; ML features f8/f9/f18 use a PERSISTENT
state without expiry (last CHoCH).
Current semantics : trigger identical; f8 = last CHoCH (persistent, per contract);
f9/f18 = raw-state consumption WITHOUT expiry (stale 34.7%/58%);
Engine 2 AF_DetectChoch = second definition (fractal 2/2,
no gate); display = third definition (fractal 2/2).
B. PROJECT SEMANTIC SPECIFICATION (Phase B)
Full document: docs/SMC_CHOCH_MSS_SPEC_v1.md (PROJECT SEMANTIC SPECIFICATION — not "universal SMC ground truth").
Spec decision summary:
S-1 Structure source : internal swing pivot fractal len=5 (equal allowed)
for f8/f9; swing len=50 only as the gate; Engine 2 fractal 2/2.
S-2 Swing definition : fractal 5/5, confirmation p+5, window [max(95,t-604),t-5],
equal allowed, no height threshold.
S-3 CHoCH : bullish: close[b] > last internal pivot high, prior
internal trend < 0, SwingTrendAt(b) >= 0.
bearish: close[b] < last internal pivot low, prior
internal trend > 0, SwingTrendAt(b) <= 0.
S-4 MSS : SAME AS CHOCH (project synonym label); no separate
primitive; the MSS+displacement definition NOT adopted.
S-5 Direction : +1 bullish (reversal from down), -1 bearish, 0 none.
S-6 Break semantics : CLOSE-CONFIRMED (wick-only insufficient; body not
required; displacement not required).
S-7 Precondition : prior internal trend != 0; non-opposing swing gate;
NO explicit HH/HL/LH/LL sequence.
S-8 Temporal : closed bar t; break b <= t; pivot p+5 <= b; no future.
S-9 Event vs State : EVENT (detection) + STATE (f8 last dir) + VALIDITY WINDOW
40 for chain consumers (f9/f18/signal gate).
S-10 Lifecycle : NO_CHOCH / ONSET / ACTIVE / EXPIRED / SUPERSEDED;
INVALIDATED & CONSUMED not modeled.
S-11 Timeframe : f8/f9 = M15 only; Engine 2 per-TF; canonical project =
SPECIFICATION AMBIGUOUS (MTF).
S-12 Sweep relation : detection independent; the consumption chain (f9/gate)
uses ordering + freshness.
C. FORMAL DEFINITIONS (Phase B — extract)
CHoCH_event(internal structure S, bar t) ->
exists internal pivot P=(p, L, isHigh) with p in [max(95,t-604), t-5],
p+5 <= b <= t, and P is the last pivot of that side before b, such that:
Bullish CHoCH (dir=+1):
isHigh(P) AND close[b] > L AND priorInternalTrend(b) < 0
AND SwingTrendAt(b) >= 0
Bearish CHoCH (dir=-1):
!isHigh(P) AND close[b] < L AND priorInternalTrend(b) > 0
AND SwingTrendAt(b) <= 0
priorInternalTrend(b) = direction of the last internal break < b (from the
structure engine; 0 if no break yet) — the first window break = BOS, not CHoCH.
One structure transition = one event (bar b). The f8 state = direction of the
LAST event; chain consumers (f9/f18) are valid only while (t - b) <= SEQ_WINDOW (40).
D. CURRENT CODE BEHAVIOR (Phase C — code audit)
D.1 Structure process (EA / v4.4 / v4.5 — identical)
// ProcessStructure(len=5, internal=true) — EA:294-340
for i in [begin, total): // begin = max(100, total-600)
p = i-len
if p>=len:
if IsPivotHigh(high,p,len): upTarget=high[p]; upBar=p; // fractal 5/5, equal allowed
if IsPivotLow (low, p,len): dnTarget=low[p]; dnBar=p;
// bullish break
if upBar>=0 && close[i]>upTarget:
choch = (trend<0)
allow = SwingTrendAt(i)>=0 // InpConfluenceFilter=true
trend = 1
if internal: g_inTrend=1; g_inLastBreak=i;
if allow && choch: g_chochDir=1; g_chochBar=i; // f8 event
// bearish break
if dnBar>=0 && close[i]<dnTarget:
choch = (trend>0)
allow = SwingTrendAt(i)<=0
trend = -1
if internal: g_inTrend=-1; g_inLastBreak=i;
if allow && choch: g_chochDir=-1; g_chochBar=i;
Emission (ComputeMLFeatures EA:487,500-501,522):
chochOK = (g_chochDir!=0 && g_chochBar>=g_swpBar && g_chochDir==g_swpDir); // f9, NO expiry
g_feat[8]= g_chochDir; // f8, last state
g_feat[9]= chochOK?1.0:0.0;
if(g_chochDir!=0 && g_chochDir==g_swpDir) cb+=15; // f18, NO expiry
D.2 Measured behavior (real XAUUSD M15 2017-2026, n=197.032)
CHoCH onsets (new events) : 2.197
active bars (f8 != 0) : 196.817 (99.89%)
repetition ratio : 89.6x (f7 = 69.5x)
state run median / max : 318 / 2.187 bars
state age (r - chochBar) : median 49, p99 242, max 418
state returns to 0 after event 1 : NEVER (same as f7)
f9 chochOK TRUE total : 51.105 bars
- fresh (CHoCH age <= 40) : 33.391
- STALE (age > 40) : 17.714 (34.7% of f9 TRUE)
f18 +15 leg (chochDir==swpDir) : 97.826 bars
- STALE (age > 40) : 56.761 (58.0% of the leg)
Runtime-training pivot divergence (parity diagnostic):
strict pivots (train_model.build_structure, equal NOT allowed) vs
equal-allowed (build_features_p2.build_structure_fast == EA):
choch_dir mismatch : 512 bars (0.26%)
choch_bar mismatch : 4.722 bars (2.40%)
onsets : 2.165 (strict) vs 2.197 (equal)
E. CONFORMANCE MATRIX (Phase C)
Verdicts: CONFORMING / PARTIALLY CONFORMING / NON-CONFORMING / UNVERIFIABLE / SPECIFICATION AMBIGUOUS.
| # | Requirement | Specification | Code (Function/Line) | Actual Behavior | Verdict |
|---|---|---|---|---|---|
| E-1 | Structure source | Internal pivot len=5, equal allowed (S-1) | ProcessStructure(INTERNAL_LEN=5,internal=true) EA:294-340; AddInternalPivot EA:181-184 |
g_ip = fractal 5/5; swing 50 only as gate (SwingTrendAt) | CONFORMING |
| E-2 | Swing definition | fractal 5/5, equal allowed, confirm p+5, window [max(95,t-604),t-5] (S-2) | IsPivotHigh/Low EA:157-169; begin=max(100,t-599) |
equal highs/lows do NOT cancel; confirm p+5; window matches | CONFORMING |
| E-3 | CHoCH trigger | close > last internal pivot high (bull) / < last pivot low (bear) (S-3) | EA:316-324 (bull), 326-335 (bear) | close[i]>upTarget / close[i]<dnTarget; target = last pivot |
CONFORMING |
| E-4 | Break semantics | CLOSE-CONFIRMED (S-6) | EA:316,326 | close (not wick) vs level; wick-only insufficient | CONFORMING |
| E-5 | Direction | +1 bullish / -1 bearish (S-5) | EA:322,332 (g_chochDir=1/-1) |
matches the reversal definition | CONFORMING |
| E-6 | Precondition | prior internal trend != 0 (S-7) | choch=(trend<0)/(trend>0) EA:318,328 |
first break (trend=0) = BOS, not CHoCH | CONFORMING |
| E-7 | Confluence gate | non-opposing SwingTrendAt (S-7/A-4) | allow = SwingTrendAt(i)>=0/<=0 EA:319,329 |
WEAK gate (>=0/<=0) vs the "align" comment (>=1/<=-1); weak/strong intent A-4 | PARTIALLY CONFORMING (weak-gate ambiguity) |
| E-8 | Closed-bar / no future | all inputs <= t (S-8) | Engine-1 closed-bar; ComputeAll; CH-T14 | no future visibility | CONFORMING |
| E-9 | Detector lifecycle | ONE event per transition (S-9/S-10) | ProcessStructure: g_chochDir set only at the break bar; full recompute per bar | detector = "last event" state; onsets 2.197 (no duplicate emission per se — state, not a stream) | CONFORMING (detector) |
| E-10 | f8 emission | last CHoCH direction, persistent per contract (S-9) | g_feat[8]=g_chochDir EA:500 |
last-event state; age p99 242 bars — MATCHES the contract | CONFORMING |
| E-11 | f9/f18 lifecycle | VALIDITY WINDOW 40 for chain consumers (S-9/S-10) | f9 EA:487; f18 EA:522 — NO expiry | stale 34.7% (f9) / 58.0% (f18 leg) — a stale event stays confirm/+15 | NON-CONFORMING (BUG-P3S3-001) |
| E-12 | Timeframe | M15 only for f8/f9 (S-11) | EA M15 slot; training M15 | M15 explicit; HTF only bias | CONFORMING |
| E-13 | Runtime-training parity | equal semantics runtime==training (S-1/S-2) | build_features_p2 _pivots_vec/build_structure_fast == EA (P2.5 parity 1e-9); train_model.build_structure & smc_semantic_common = STRICT |
parity feed matches; train_model/P3-infra diverge (0.26% choch_dir; 2.40% choch_bar) | PARTIALLY CONFORMING (BUG-P3S3-002) |
| E-14 | MSS distinct concept | NO MSS primitive (S-4) | no MSS code anywhere (grep 0 matches) | no code claims a separate MSS | CONFORMING (no claim) / label AMBIGUOUS (A-1) |
| E-15 | Engine 2 AF_DetectChoch | second definition: fractal 2/2, no gate, per-TF (S-1, spec §4.2) | AF_Engine2_Agents.mqh:238-247 (BuildSwing 196-230, TrendFromSwing 232-248); N agent 444-449; E agent 605-608 | close break vs highs[0]/lows[0] with the 4-swing trend; NO gate/displacement; per-slot TF | CONFORMING to the Engine-2 definition; AMBIGUOUS at the project level (two CHoCH definitions, A-2) |
| E-16 | Same-bar precedence (bull & bear break) | A-3: SPECIFICATION AMBIGUOUS | EA:316 & 326 both ifs (not else-if) | bullish executes first; unreachable with consistent data (analogous to P3-S.2 E-11) | SPECIFICATION AMBIGUOUS (unreachable) |
F. CONFIRMED BUGS (Phase D — DOCUMENT ONLY, no code changes)
BUG-P3S3-001
SEVERITY : MEDIUM
SOURCE : f9 (chochOK) & f18 (confluence leg) — EA:487, 522 (v4.4:951,986;
v4.5:920,938); training build_features_p2 choch_ok
SPEC REQ : S-9/S-10 — event validity (SEQ_WINDOW=40) applies to chain
consumers (f9/f18/signal gate); the detector is already correct
ACTUAL CODE : f9/f18 use raw g_chochDir/g_chochBar, WITHOUT
(t - g_chochBar) <= 40
IMPACT : an EXPIRED CHoCH event stays "confirmed" in f9 (34.7% of f9-TRUE
bars stale, measured 17.714 bars) and keeps +15 in f18 (58.0% of
the leg stale, 56.761 bars). The semantic chain is inconsistent:
the v4.4 signal gate applies freshness, the ML features do not.
Same bug class as BUG-P3S2-001 (sweep side) — P3-S.0 only fixed f7,
f9/f18 not yet.
PROPOSED FIX : apply (t - g_chochBar) <= SEQ_WINDOW at f9/f18 OR refactor the
CHoCH event into an onset object with a single validity lifetime
(one lifecycle source of truth for sweep & choch).
BUG-P3S3-002
SEVERITY : LOW
SOURCE : train_model.build_structure (STRICT pivots: equal NOT allowed)
vs EA/parity feed (equal allowed)
SPEC REQ : S-2 — equal-level handling consistent runtime-training
ACTUAL CODE : MQL5 IsPivotHigh: `if(h[idx-k]>v) return false` (equal OK);
Python _pivots_vec: `h >= np.roll(...)` (equal OK);
Python train_model.build_structure: `if h[p]<=h[p-k] ...` (equal FAIL)
-> build_structure_fast & EA match; train_model diverges.
IMPACT : choch_dir mismatch 512 bars (0.26%); choch_bar 4.722 bars (2.40%);
onsets 2.165 vs 2.197 (XAUUSD M15 2017-2026). smc_semantic_common
(P3 forensic infra) uses train_model -> forensic results differ
slightly from runtime on equal-level bars.
PROPOSED FIX : align the pivot predicate (use _pivots_vec/build_structure_fast on
all training & forensic paths), or document the tolerance.
BUG-P3S3-003
SEVERITY : INFO (documentation)
SOURCE : InpConfluenceFilter comment "Internal must align with swing trend"
vs the implementation SwingTrendAt(i)>=0/<=0
SPEC REQ : S-7/A-4 — the confluence gate must be explicit (weak vs strong)
ACTUAL CODE : gate = "swing NON-opposing" (>=0/<=0), NOT "aligned" (>=1/<=-1)
IMPACT : CHoCH is still recorded when the swing is flat (trend 0); the input
comment is misleading; spec A-4 flags AMBIGUOUS
PROPOSED FIX : revise the comment/documentation; adjudicate weak vs strong gate
G. SEMANTIC AMBIGUITIES (Phase F — consistency)
G-1 MSS vs CHOCH (TERMINOLOGY)
The project uses "CHoCH/MSS" as synonym labels (DESIGN.md, P3 concept list).
No separate MSS primitive (0 code matches). If "MSS" means break+displacement
(common SMC definition), the project does NOT define it.
Status: SPECIFICATION AMBIGUOUS (label) — but no code to audit.
G-2 TWO CHOCH DEFINITIONS IN ONE PROJECT
f8/f9 (fractal 5/5 + swing gate, M15) vs Engine-2 AF_DetectChoch (fractal 2/2,
no gate, per-TF H4/M30/M15/M3) vs Display (fractal 2/2, chart TF). All three
agree on direction (reversal) but differ in parameters & preconditions.
For ML features, the f8/f9 definition is authoritative.
G-3 WEAK CONFLUENCE GATE
SwingTrendAt(b)>=0/<=0 (non-opposing) vs "align" (aligned). A-4.
G-4 CANONICAL TIMEFRAME
f8/f9 M15 explicit; the HTF-structure role on LTF events not established.
Same as sweep G-3 (P3-S.2).
G-5 EVENT vs STATE IN CONSUMERS
f8 = state (per contract); f9/f18 should be event+window but use the state
(BUG-P3S3-001). The lifecycle consistency is not yet complete in the project.
H. SEMANTIC RELATIONSHIP WITH LIQUIDITY SWEEP (Phase C)
Detection : CHoCH f8 STANDS ALONE (no sweep required) — CONFORMING S-12.
Consumption: f9 = chain (chochBar>=swpBar & same direction) — CONFORMING for
ordering; freshness NOT applied (BUG-P3S3-001) — NON-CONFORMING.
Gate : v4.4/v4.5 signal gate uses InpRequireSweep+InpRequireCHoCH with the
40-bar window — CONFORMING to the legacy signal semantics.
Engine 2 : E agent ZONE + CONFIRMATION (sweep OR choch) — consistent with DESIGN.md.
NO detection dependency; f7 NOT changed to satisfy CHoCH (meets §17 instructions).
I. RUNTIME/TRAINING PARITY (Phase C — diagnostic, not a retrain)
f8/f9 (choch_dir/choch_confirms):
parity feed build_features_p2 (build_structure_fast, equal allowed) vs EA:
PARITY VERIFIED IN P2.5 (eps 1e-9, f0-f5/f7-f12/f18).
THE DEPLOYED-MODEL PATH (P2.6) = build_features_p2 -> runtime MATCHES.
train_model.build_structure (old path/P3 infra): STRICT pivots -> diverges
0.26% choch_dir / 2.40% choch_bar (BUG-P3S3-002) — documented.
NO retraining; the parity results are diagnostic only.
J. VERDICT
PARTIALLY CONFORMING
Detection semantics (E-1..E-10, E-12) : CONFORMING
Confluence gate (E-7) : PARTIALLY CONFORMING (A-4)
Event-consumer lifecycle (E-11) : NON-CONFORMING (BUG-P3S3-001)
Runtime-training pivots (E-13) : PARTIALLY CONFORMING (BUG-P3S3-002)
MSS distinct concept (E-14) : NO CLAIM (synonym label)
Engine 2 / Display (E-15) : CONFORMING to its definition; project AMBIGUOUS
Overall implementation : PARTIALLY CONFORMING
Verdict = conformance to the PROJECT SEMANTIC SPECIFICATION, NOT profitability/edge.
DO NOT replace with PASS just because the synthetic tests pass.
K. NEXT GATE
After the CHoCH/MSS verdict (per brief §22):
FVG -> OB -> Displacement -> MTF Alignment -> Candidate Setup.
DO NOT return to ML; DO NOT build a meta-label; DO NOT create new event labels.
L. CHECKPOINT & PROVENANCE (P3-S.3)
Forge HEAD (session start) : 595b4c9a (P3-S.2) — VERIFIED
P3-S.2 spec (immutable) : docs/SMC_LIQUIDITY_SWEEP_SPEC_v1.md
FEATURE_CONTRACT : C44CC6F2... (P2-era) / 7b908b12... (committed)
symbol/tf : XAUUSD / M15 (2017-01-01 .. 2026-08-20)
Human verification : CANCELLED
No production change : AF_Engine2_Agents.mqh / f8 / f9 / f18 / ProcessStructure
/ v4.4 / v4.5 / Engine 2 / Display / ML : NOT touched