16 KiB
16 KiB
P3-S.14 — F4 MTF / TRAINING ALIGNMENT (SNIPERGOLD_ML)
Date : 2026-08-23
Session : P3-S.14 — F4 MTF / Training Alignment
Baseline : P3_S13_SHA 1c0da62407fc905fb96732ec0f6ec5c43d136a8b (VERIFIED)
Contract : docs/SNIPERGOLD_CANONICAL_SETUP_CONTRACT_v1.md (frozen, P3-S.10)
Spec : docs/SMC_MTF_TRAINING_ALIGNMENT_SPEC_v1.md (this session, F4)
Prior MTF : docs/P3_S7_MTF_ALIGNMENT_CONFORMANCE.md, docs/SMC_MTF_ALIGNMENT_SPEC_v1.md
Prior P2 : docs/P2_SOURCE_OF_TRUTH_REPORT.md, docs/FEATURE_CONTRACT.md,
ml/parity/RUNTIME_TRAINING_PARITY_REPORT.md
Scope : SEMANTIC/ARCHITECTURAL alignment ONLY. No AUC/PF/backtest/ML retrain/
optimization. No production code change. No F1/F2/Candidate-Setup change.
Test report: ml/p3/smc_semantic/output/spec_tests_mtf_training_alignment_report.json
Human verification : CANCELLED (historical only).
F4 QUESTION (brief §31): Does the existing training/ML feature pipeline (D1/H4/H1 + M15, 19 features) represent the same MTF information and timing semantics as the frozen H4/M30/M15/M3 runtime and Candidate Setup architecture?
F4 ANSWER: NO — the two MTF models are semantically different. The ML feature pipeline is a legacy v4.4 feed (D1/H4/H1 bias + M15 structure) with no M30 and no M3 feature, while the canonical Candidate Setup relies on M30 as a HARD context gate and M3 as optional confirmation. The difference is architectural and documented (P3-S.7 D-5/S-PR), not a silent bug.
A. CANONICAL RUNTIME MTF (frozen)
H4 = Narrative / higher-TF CONTEXT STATE -> HARD direction-compatible GATE
M30 = Context STATE -> HARD direction-compatible GATE
M15 = Entry / CHAIN CARRIER -> setup layer (events/zones/condition)
M3 = Price Action / OPTIONAL MICRO CONFIRMATION -> no veto, no arithmetic role
Engine: AF_Engine2_Agents (AF_E2_TF_S1..S4 = H4/M30/M15/M3) + AFEngine1MTF cache
(closed-bar lock; bars[0] = newest closed).
Setup consumption (F3): H4 gate -> M30 gate -> fresh Liquidity EVENT (M15)
-> CHoCH after sweep (M15) -> unmitigated OB|FVG ZONE (M15)
-> M15 entry condition -> CANDIDATE_SETUP [optional M3 confirmation].
Source: AF_Defines.mqh:47-50, AF_Engine2_Setup.mqh (F3), SMC_MTF_TRAINING_ALIGNMENT_SPEC B.
B. CURRENT TRAINING MTF (legacy)
Base : M15 (every one of the 19 features is an M15-bar computation)
HTF bias : f0 = D1 bias, f1 = H4 bias, f2 = H1 bias
f3-f18 : M15 structure / zones / delta / distance / momentum / range / confluence
NO M30 feature, NO M3 feature.
Sources : FEATURE_CONTRACT f0-f18 (D1/H4/H1 + M15),
train_model.build_features (D1/H4/H1), build_features_p2 (P2.5 corrected,
NOT integrated into train_model.py), AlgoForge_Backtest_Baseline.mq5
(runtime ML path registers PERIOD_M15/H1/H4/D1).
C. FEATURE-LEVEL MTF MAP (f0-f18)
| Feature | Name | Runtime TF | Training TF | Runtime as-of | Training as-of | Match? | Notes |
|---|---|---|---|---|---|---|---|
| f0 | htf1_bias | D1 | D1 | 200 newest closed D1 @ tc | tf_bias_asof(E_ea) | ✅ (P2.1) | training only vs setup (no D1 role) |
| f1 | htf2_bias | H4 | H4 | 200 newest closed H4 @ tc | tf_bias_asof(E_ea) | ✅ (P2.1) | H4 is in BOTH, role differs (gate vs soft feature) |
| f2 | htf3_bias | H1 | H1 | 200 newest closed H1 @ tc | tf_bias_asof(E_ea) | ✅ (P2.1) | training only vs setup (no H1 role) |
| f3 | swing_trend | M15 | M15 | 700 cache/600 analyze @tc | 700-slice begin=100 | ✅ (P2.3) | shared base |
| f4 | internal_trend | M15 | M15 | same | same | ✅ | shared base |
| f5 | chart_bias | M15 | M15 | combined | combined | ✅ | shared base |
| f6 | eq_pos_norm | M15 | M15 | sw_high/sw_low [t-649,t-50] | same | ✅ (P2.3) | shared base |
| f7 | sweep_dir | M15 | M15 | DetectLiquidityGrabs + F1 SEQ window | inn pivots + window 8 | ✅ (P2.5) | M15-internal event |
| f8 | choch_dir | M15 | M15 | ProcessStructure internal | inn['choch_dir'] | ✅ | M15-internal event |
| f9 | choch_confirm | M15 | M15 | F1 chain (sweep+choch+dir) | same | ✅ | M15-internal event |
| f10 | eqh_swept | M15 | M15 | DetectEQ ATR@row | pairs + ATR@row | ✅ (P2.2) | shared base |
| f11 | eql_swept | M15 | M15 | same | same | ✅ (P2.2) | shared base |
| f12 | delta_sign | M15 | M15 | BTDelta | 10-bar loop | ✅ | shared base |
| f13 | delta_mag | M15 | M15 | same | same | ✅ | shared base |
| f14 | dist_high_atr | M15 | M15 | sw_high/ATR | same | ✅ (P2.3) | shared base |
| f15 | dist_low_atr | M15 | M15 | sw_low/ATR | same | ✅ (P2.3) | shared base |
| f16 | mom20_atr | M15 | M15 | c[·]-c[·-20]/ATR | c[i]-c[i-20]/ATR | ✅ (para) | shared base |
| f17 | range_atr | M15 | M15 | sw_high-sw_low/ATR | same | ✅ (P2.3) | shared base |
| f18 | confluence | M15 (derived) | M15 (derived) | from f0-f9 | same | ✅ | depends on f0-f2 (HTF) |
MTF-level observation: the ONLY HTF-bearing features in the ML set are f0/f1/f2 (D1/H4/H1). NONE of f0-f18 is computed on M30 or M3. The runtime H4 gate (frozen, hard, direction-compatible) is NOT the same information as f1 (a soft numeric H4 bias fed to an MLP). The ML set has no representation of M30 gate state, M3 confirmation, or H4/H30 gate direction-compatibility. Validated by F4-T08/T11/T12.
D. HTF BIAS MAPPING (runtime vs training)
| Aspect | Runtime (EA BTTFBias, post-P2.1) | Training (tf_bias_asof E_ea) | Match |
|---|---|---|---|
| Source TF | D1/H4/H1 (Engine-1 slots) | D1/H4/H1 (npz) | ✅ |
| Closed bar | closed-bar lock (forming dropped) | closed HTF bars only | ✅ |
| as-of timestamp | newest CLOSED bar at tc=t+900 | E_ea = searchsorted(ht, tc-per, 'right')-1 | ✅ |
| window | 200 NEWEST closed cache bars | 200 bars ending at E_ea | ✅ |
| pivot/break | fractal swing s=3 | same (verified 1e-9) | ✅ |
| warm-up | <120 -> "Neutral"(0) | m<s+2 -> 0 | ✅ (semantic) |
| lag bug (50-bar) | FIXED P2.1 (was GetBar(count-1-i)) | n/a (was corrected) | ✅ |
| gap at session-break | LEGACY DESIGN (Refresh-on-Bars-change) | handled by feed | ⚠ TOLERATED (P2.1) |
HTF bias: RUNTIME == TRAINING on equivalent semantics after the P2.1 fix (parity f0 1/14850, f1 1/14850, f2 0/14850; residual = session-break cache staleness, TOLERATED). The bias formula and as-of are aligned; the role in the overall system is not (gate vs feature, §C).
E. AS-OF SEMANTICS
For a decision at M15 bar time t (tc = t + 900):
Runtime H4_asof(t) = newest closed H4 bar close_time <= t
Runtime M30_asof(t) = newest closed M30 bar close_time <= t
Runtime M15_asof(t) = newest closed M15 bar (= decision bar)
Runtime M3_asof(t) = newest closed M3 bar close_time <= t
Training E_ea(t) = newest HTF bar close_time <= tc (D1/H4/H1)
Invariant : every consumed bar close_time <= t (runtime) / <= tc (training)
Both satisfy "no future information". Runtime enforced by the Engine-1
closed-bar lock; training by the E_ea searchsorted mapping. Validated
F4-T01..T07, T09, T13.
F. CLOSED-BAR SEMANTICS
- Runtime (Engine 1 Build): CopyRates(maxBars+1), drop the forming bar
(IsBarClosed: barTime + PeriodSeconds(tf) <= now). Cache holds ONLY closed bars.
- Training: builds features on closed M15 bars (tc = t+900) and closed HTF bars
(E_ea). No forming bar is used.
- Identical principle on both sides. No partial HTF candle leaks (F4-T09).
G. RUNTIME / TRAINING DIFFERENCES (semantic, classified)
| # | Difference | Classification |
|---|---|---|
| G-1 | ML set = D1/H4/H1+M15; setup = H4/M30/M15/M3 | LEGACY DESIGN (two MTF models, P3-S.7 D-5) |
| G-2 | ML has no M30 feature | LEGACY DESIGN (v4.4 feed predates M30 in setup design) |
| G-3 | ML has no M3 feature | LEGACY DESIGN |
| G-4 | ML has D1/H1 biases the setup layer does not use | LEGACY DESIGN (extra HTF) |
| G-5 | H4 role: setup = HARD gate; ML = soft numeric feature (f1) | INTENTIONAL DIFFERENCE (different layers) |
| G-6 | 50-bar HTF cache lag (old EA BTTFBias) | BUG -> FIXED P2.1 |
| G-7 | session-break cache staleness (Refresh-on-Bars-change) | LEGACY DESIGN, TOLERATED |
| G-8 | build_features (as-of open, lag-1) vs build_features_p2 (as-of close E_ea) | LEGACY DESIGN (p2 corrected feed NOT integrated into train_model.py; P2 §F) |
| G-9 | deployed SniperGold_ML.mqh trained on OLD v4.4 feed | LEGACY (frozen model, P2 §H risk #2) |
NOTE (G-8): train_model.build_features maps HTF bias via
idx = searchsorted(ht, t, 'right')-1 with htf_bias_series (lag-1),
while build_features_p2 (the P2.5 parity-corrected module) maps via
e = searchsorted(ht_, tc-per, 'right')-1 with tf_bias_asof(e).
These differ. The DEPLOYED MLP baseline was trained with
train_model.build_features (G-9). This is a DOCUMENTED legacy inconsistency
(P2 §F: "build_features_p2 not yet integrated into train_model.py"). It is
NOT silently fixed by F4.
H. ROOT CAUSES
R-1 ARCHITECTURAL BIFURCATION : the 19-feature MLP descends from legacy v4.4
(D1/H4/H1 TFBias + M15 structure). The canonical H4/M30/M15/M3 model is a
NEWER Algo Forge 3-engine design (P3-S.7+). The two were never reconciled
into one MTF model. [LEGACY DESIGN]
R-2 MASTER-DATA DRIFT : FEATURE_CONTRACT documents the ML path only.
P3-S.7 S-PR explicitly records the "FEATURE_CONTRACT MTF MISMATCH". The ML
contract and the setup contract each specify a different MTF set without a
common reconciliation. [DOCUMENTATION DRIFT]
R-3 MODEL FREEZE : the deployed MLP was trained on the v4.4 feed;
the corrected P2.5 feed was not integrated (P2 §F remaining risk #2/#4).
The model weights encode the legacy semantics. [HISTORICAL MODEL FREEZE]
R-4 NO M30/M3 SURFACE : the Feature Contract never defined M30/M3
features; the setup layer gained them in P3-S.7/S.10 design. [INTENTIONAL ML
ARCHITECTURE that has drifted from the setup design]
I. ALIGNMENT OPTIONS
OPTION A — Align ML directly to H4/M30/M15/M3 (drop D1/H1, add M30/M3).
Pro : one canonical MTF everywhere; runtime/training consistency; simpler long-term reasoning.
Con : existing 19-feature model becomes obsolete; new training dataset + new model;
old model non-comparable. Requires a FUTURE authorized retraining phase.
OPTION B — Keep ML feature architecture (f0-f18) but MAP it explicitly.
Pro : least disruptive; preserves the existing MLP and its historical evidence;
Document f1(H4 bias) -> runtime H4 gate mapping; document the M15 base
feature set == M15 setup carrier.
Con : two MTF abstractions remain; M30/M3 stay unrepresented in ML (documented gap).
OPTION C — Retire the current static ML architecture.
Consider only if evidence shows the frozen MLP cannot be meaningfully aligned.
Evidence exists (R-4: no M30/M3; G-9: trained on OLD feed), so Option C is
defensible on SEMANTIC grounds independent of AUC.
J. RECOMMENDED ARCHITECTURE (F4 decision)
PRIMARY : OPTION B now (map the existing ML feature architecture explicitly),
with an explicit DOCUMENTED GAP for M30/M3.
- Keep f0-f18 (D1/H4/H1 + M15) as the ML feed (do NOT force H4/M30/M15/M3
into the 19 features — §8 directive).
- Document: f1 (H4 bias) is the closest ML counterpart to the runtime H4
gate DIRECTION, but it is a SOFT numeric input, NOT a gate; the runtime
gate semantics (direction-compatible, non-zero, equal to chain direction)
are NOT represented in the ML feature vector.
- Document: M30 gate state and M3 confirmation have NO ML feature. The ML
cannot reconstruct Candidate-Setup gate content from f0-f18.
EXTENDED (future, authorized separately) : OPTION C (retire the static MLP)
OR OPTION A (align to H4/M30/M15/M3) — decision belongs to a FUTURE
implementation/training phase, never selected by AUC. F4 only records the
contract and the gap.
BRIEF DIRECTIVE (§8) : Do NOT force the four runtime TFs into ML. The ML
feature mathematically represents the information it was trained on (M15
structure + D1/H4/H1 bias). It does NOT represent the runtime H4/M30/M15/M3
gate semantics. The answer to "does the ML represent the same MTF information
as the setup?" is NO — recorded above. Adding raw M30/M3 columns would NOT
make it equivalent because the setup's gates are STATE (direction-compatible)
and the ML features are numeric inputs; equivalence requires semantic
definition, not column-count parity.
K. DEFERRED IMPLEMENTATION (NOT done in F4)
- NO ML retrain (MLP/LSTM/regime/meta-label) — documented only.
- NO feature-pipeline rewrite; NO new M30/M3 features built now.
- NO SniperGold_ML.mqh / model rebuild.
- NO change to FEATURE_CONTRACT.md (unchanged this session).
- NO change to train_model.py / build_features.py / build_features_p2.py.
- NO change to AF_Engine2_Setup.mqh (Candidate Setup frozen F3).
- NO change to F1 (event) / F2 (zone) semantics.
- NO runtime .mq5/.mqh change.
These become candidates for a SEPARATELY AUTHORIZED implementation/training
phase (P3-S.15), per the brief §17/§24/§31.
L. FINAL ALIGNMENT VERDICT
CLASSIFICATION : LEGACY / NOT COMPARABLE
(with PARTIAL ALIGNMENT of the M15 base layer and the H4 bias formula.)
- M15 base layer : ALIGNED (f3-f18 computed on M15 == setup M15 carrier)
- HTF bias formula+as-of (D1/H4/H1) : ALIGNED (P2.1-corrected, parity-verified)
- H4 role : NOT COMPARABLE (gate vs soft feature)
- M30, M3 : GAP — no ML feature (NOT comparable)
- D1, H1 : ML-only extra TFs (NOT comparable)
NOT "MODEL BAD": this is a SEMANTIC verdict about two different MTF
architectures, not a performance claim. Distinct from MISALIGNED: the two
models are different generations (legacy v4.4 ML feed vs newer candidate-setup
model) and are NOT reconciled — they are simply different objects.
M. IMPLEMENTATION DECISION (per brief §22)
DOCUMENTATION CHANGE REQUIRED
+ FUTURE FEATURE-PIPELINE / ML-PATH ALIGNMENT REQUIRED
(the ML path should eventually be aligned to the canonical setup MTF or be
retired — but that is a FUTURE authorized phase, NOT executed in F4.)
- NP CODE CHANGE REQUIRED in this session (F4 is audit/specification-only).
- The ML model is NOT retrained; the Candidate Setup layer is NOT changed.
N. REGRESSION
P3-S.2 Liquidity Sweep 17/17 PASS (untouched)
P3-S.3 CHoCH/MSS 20/20 PASS (untouched)
P3-S.4 FVG 20/20 PASS (untouched)
P3-S.5 Order Block 20/20 PASS (untouched)
P3-S.6 Displacement 20/20 PASS (untouched)
P3-S.7 MTF Alignment 22/22 PASS (re-run, PASS)
F1 Event Contract 8/8 PASS (untouched)
F2 Zone Contract 20/20 PASS (untouched)
F3 Candidate Setup 25/25 PASS (untouched)
F4 MTF/Training Align 16/16 PASS (NEW — this session)
No previous suite weakened; no oracle changed.
O. EXPLICITLY UNCHANGED AREAS
Candidate Setup : AF_Engine2_Setup.mqh, AFCandidateSetup, lifecycle, identity,
H4/M30 gates, M15 carrier, M3 confirmation — FROZEN (F3).
F1 (event semantics) : unchanged.
F2 (FVG/OB zone semantics): unchanged.
ML model (.mqh / weights / labels) : unchanged.
train_model.py / build_features.py / build_features_p2.py : unchanged.
FEATURE_CONTRACT.md : unchanged (F4 documents, does not modify).
Runtime .mq5/.mqh : unchanged.
P. PROVENANCE & CHECKPOINT
Forge HEAD (start) : 1c0da62407fc905fb96732ec0f6ec5c43d136a8b (P3-S.13)
Branch / remote : main == origin/main (VERIFIED clean @ start)
New artifacts : docs/SMC_MTF_TRAINING_ALIGNMENT_SPEC_v1.md
docs/P3_S14_MTF_TRAINING_ALIGNMENT.md (this report)
ml/p3/smc_semantic/spec_tests_mtf_training_alignment.py
ml/p3/smc_semantic/spec_test_cases_mtf_training_alignment.json
ml/p3/smc_semantic/output/spec_tests_mtf_training_alignment_report.json
docs/SESSION_HANDOVER_<DATE>_P3_S14_MTF_TRAINING_ALIGNMENT.md
Human verification : CANCELLED
End of P3-S.14 F4 MTF / Training Alignment record. No production/ML change.