346 lines
No EOL
16 KiB
Markdown
346 lines
No EOL
16 KiB
Markdown
# P3-S.14 — F4 MTF / TRAINING ALIGNMENT (SNIPERGOLD_ML)
|
|
|
|
```text
|
|
Date : 2026-08-23
|
|
Session : P3-S.14 — F4 MTF / Training Alignment
|
|
Baseline : P3_S13_SHA 1c0da62407fc905fb96732ec0f6ec5c43d136a8b (VERIFIED)
|
|
Contract : docs/SNIPERGOLD_CANONICAL_SETUP_CONTRACT_v1.md (frozen, P3-S.10)
|
|
Spec : docs/SMC_MTF_TRAINING_ALIGNMENT_SPEC_v1.md (this session, F4)
|
|
Prior MTF : docs/P3_S7_MTF_ALIGNMENT_CONFORMANCE.md, docs/SMC_MTF_ALIGNMENT_SPEC_v1.md
|
|
Prior P2 : docs/P2_SOURCE_OF_TRUTH_REPORT.md, docs/FEATURE_CONTRACT.md,
|
|
ml/parity/RUNTIME_TRAINING_PARITY_REPORT.md
|
|
Scope : SEMANTIC/ARCHITECTURAL alignment ONLY. No AUC/PF/backtest/ML retrain/
|
|
optimization. No production code change. No F1/F2/Candidate-Setup change.
|
|
Test report: ml/p3/smc_semantic/output/spec_tests_mtf_training_alignment_report.json
|
|
Human verification : CANCELLED (historical only).
|
|
```
|
|
|
|
**F4 QUESTION (brief §31):** Does the existing training/ML feature pipeline
|
|
(D1/H4/H1 + M15, 19 features) represent the same MTF information and timing
|
|
semantics as the frozen H4/M30/M15/M3 runtime and Candidate Setup architecture?
|
|
|
|
**F4 ANSWER:** **NO** — the two MTF models are semantically different. The ML
|
|
feature pipeline is a legacy v4.4 feed (D1/H4/H1 bias + M15 structure) with
|
|
**no M30 and no M3 feature**, while the canonical Candidate Setup relies on
|
|
M30 as a HARD context gate and M3 as optional confirmation. The difference is
|
|
**architectural and documented** (P3-S.7 D-5/S-PR), not a silent bug.
|
|
|
|
---
|
|
|
|
## A. CANONICAL RUNTIME MTF (frozen)
|
|
|
|
```text
|
|
H4 = Narrative / higher-TF CONTEXT STATE -> HARD direction-compatible GATE
|
|
M30 = Context STATE -> HARD direction-compatible GATE
|
|
M15 = Entry / CHAIN CARRIER -> setup layer (events/zones/condition)
|
|
M3 = Price Action / OPTIONAL MICRO CONFIRMATION -> no veto, no arithmetic role
|
|
Engine: AF_Engine2_Agents (AF_E2_TF_S1..S4 = H4/M30/M15/M3) + AFEngine1MTF cache
|
|
(closed-bar lock; bars[0] = newest closed).
|
|
Setup consumption (F3): H4 gate -> M30 gate -> fresh Liquidity EVENT (M15)
|
|
-> CHoCH after sweep (M15) -> unmitigated OB|FVG ZONE (M15)
|
|
-> M15 entry condition -> CANDIDATE_SETUP [optional M3 confirmation].
|
|
Source: AF_Defines.mqh:47-50, AF_Engine2_Setup.mqh (F3), SMC_MTF_TRAINING_ALIGNMENT_SPEC B.
|
|
```
|
|
|
|
---
|
|
|
|
## B. CURRENT TRAINING MTF (legacy)
|
|
|
|
```text
|
|
Base : M15 (every one of the 19 features is an M15-bar computation)
|
|
HTF bias : f0 = D1 bias, f1 = H4 bias, f2 = H1 bias
|
|
f3-f18 : M15 structure / zones / delta / distance / momentum / range / confluence
|
|
NO M30 feature, NO M3 feature.
|
|
Sources : FEATURE_CONTRACT f0-f18 (D1/H4/H1 + M15),
|
|
train_model.build_features (D1/H4/H1), build_features_p2 (P2.5 corrected,
|
|
NOT integrated into train_model.py), AlgoForge_Backtest_Baseline.mq5
|
|
(runtime ML path registers PERIOD_M15/H1/H4/D1).
|
|
```
|
|
|
|
---
|
|
|
|
## C. FEATURE-LEVEL MTF MAP (f0-f18)
|
|
|
|
| Feature | Name | Runtime TF | Training TF | Runtime as-of | Training as-of | Match? | Notes |
|
|
|---------|------|-----------|-------------|---------------|----------------|--------|-------|
|
|
| f0 | htf1_bias | D1 | D1 | 200 newest closed D1 @ tc | tf_bias_asof(E_ea) | ✅ (P2.1) | training only vs setup (no D1 role) |
|
|
| f1 | htf2_bias | H4 | H4 | 200 newest closed H4 @ tc | tf_bias_asof(E_ea) | ✅ (P2.1) | **H4 is in BOTH**, role differs (gate vs soft feature) |
|
|
| f2 | htf3_bias | H1 | H1 | 200 newest closed H1 @ tc | tf_bias_asof(E_ea) | ✅ (P2.1) | training only vs setup (no H1 role) |
|
|
| f3 | swing_trend | M15 | M15 | 700 cache/600 analyze @tc | 700-slice begin=100 | ✅ (P2.3) | shared base |
|
|
| f4 | internal_trend | M15 | M15 | same | same | ✅ | shared base |
|
|
| f5 | chart_bias | M15 | M15 | combined | combined | ✅ | shared base |
|
|
| f6 | eq_pos_norm | M15 | M15 | sw_high/sw_low [t-649,t-50] | same | ✅ (P2.3) | shared base |
|
|
| f7 | sweep_dir | M15 | M15 | DetectLiquidityGrabs + F1 SEQ window | inn pivots + window 8 | ✅ (P2.5) | M15-internal event |
|
|
| f8 | choch_dir | M15 | M15 | ProcessStructure internal | inn['choch_dir'] | ✅ | M15-internal event |
|
|
| f9 | choch_confirm | M15 | M15 | F1 chain (sweep+choch+dir) | same | ✅ | M15-internal event |
|
|
| f10 | eqh_swept | M15 | M15 | DetectEQ ATR@row | pairs + ATR@row | ✅ (P2.2) | shared base |
|
|
| f11 | eql_swept | M15 | M15 | same | same | ✅ (P2.2) | shared base |
|
|
| f12 | delta_sign | M15 | M15 | BTDelta | 10-bar loop | ✅ | shared base |
|
|
| f13 | delta_mag | M15 | M15 | same | same | ✅ | shared base |
|
|
| f14 | dist_high_atr | M15 | M15 | sw_high/ATR | same | ✅ (P2.3) | shared base |
|
|
| f15 | dist_low_atr | M15 | M15 | sw_low/ATR | same | ✅ (P2.3) | shared base |
|
|
| f16 | mom20_atr | M15 | M15 | c[·]-c[·-20]/ATR | c[i]-c[i-20]/ATR | ✅ (para) | shared base |
|
|
| f17 | range_atr | M15 | M15 | sw_high-sw_low/ATR | same | ✅ (P2.3) | shared base |
|
|
| f18 | confluence | M15 (derived) | M15 (derived) | from f0-f9 | same | ✅ | depends on f0-f2 (HTF) |
|
|
|
|
**MTF-level observation:** the ONLY HTF-bearing features in the ML set are
|
|
f0/f1/f2 (D1/H4/H1). NONE of f0-f18 is computed on M30 or M3. The runtime H4
|
|
**gate** (frozen, hard, direction-compatible) is NOT the same information as
|
|
f1 (a soft numeric H4 bias fed to an MLP). The ML set has **no representation
|
|
of M30 gate state, M3 confirmation, or H4/H30 gate direction-compatibility.**
|
|
Validated by F4-T08/T11/T12.
|
|
|
|
---
|
|
|
|
## D. HTF BIAS MAPPING (runtime vs training)
|
|
|
|
| Aspect | Runtime (EA BTTFBias, post-P2.1) | Training (tf_bias_asof E_ea) | Match |
|
|
|--------|-----------------------------------|-------------------------------|-------|
|
|
| Source TF | D1/H4/H1 (Engine-1 slots) | D1/H4/H1 (npz) | ✅ |
|
|
| Closed bar | closed-bar lock (forming dropped) | closed HTF bars only | ✅ |
|
|
| as-of timestamp | newest CLOSED bar at tc=t+900 | E_ea = searchsorted(ht, tc-per, 'right')-1 | ✅ |
|
|
| window | 200 NEWEST closed cache bars | 200 bars ending at E_ea | ✅ |
|
|
| pivot/break | fractal swing s=3 | same (verified 1e-9) | ✅ |
|
|
| warm-up | <120 -> "Neutral"(0) | m<s+2 -> 0 | ✅ (semantic) |
|
|
| lag bug (50-bar) | FIXED P2.1 (was GetBar(count-1-i)) | n/a (was corrected) | ✅ |
|
|
| gap at session-break | LEGACY DESIGN (Refresh-on-Bars-change) | handled by feed | ⚠ TOLERATED (P2.1) |
|
|
|
|
**HTF bias: RUNTIME == TRAINING on equivalent semantics** after the P2.1 fix
|
|
(parity f0 1/14850, f1 1/14850, f2 0/14850; residual = session-break cache
|
|
staleness, TOLERATED). The bias **formula and as-of are aligned**; the
|
|
**role** in the overall system is not (gate vs feature, §C).
|
|
|
|
---
|
|
|
|
## E. AS-OF SEMANTICS
|
|
|
|
```text
|
|
For a decision at M15 bar time t (tc = t + 900):
|
|
Runtime H4_asof(t) = newest closed H4 bar close_time <= t
|
|
Runtime M30_asof(t) = newest closed M30 bar close_time <= t
|
|
Runtime M15_asof(t) = newest closed M15 bar (= decision bar)
|
|
Runtime M3_asof(t) = newest closed M3 bar close_time <= t
|
|
Training E_ea(t) = newest HTF bar close_time <= tc (D1/H4/H1)
|
|
Invariant : every consumed bar close_time <= t (runtime) / <= tc (training)
|
|
|
|
Both satisfy "no future information". Runtime enforced by the Engine-1
|
|
closed-bar lock; training by the E_ea searchsorted mapping. Validated
|
|
F4-T01..T07, T09, T13.
|
|
```
|
|
|
|
---
|
|
|
|
## F. CLOSED-BAR SEMANTICS
|
|
|
|
```text
|
|
- Runtime (Engine 1 Build): CopyRates(maxBars+1), drop the forming bar
|
|
(IsBarClosed: barTime + PeriodSeconds(tf) <= now). Cache holds ONLY closed bars.
|
|
- Training: builds features on closed M15 bars (tc = t+900) and closed HTF bars
|
|
(E_ea). No forming bar is used.
|
|
- Identical principle on both sides. No partial HTF candle leaks (F4-T09).
|
|
```
|
|
|
|
---
|
|
|
|
## G. RUNTIME / TRAINING DIFFERENCES (semantic, classified)
|
|
|
|
| # | Difference | Classification |
|
|
|---|-----------|----------------|
|
|
| G-1 | ML set = D1/H4/H1+M15; setup = H4/M30/M15/M3 | LEGACY DESIGN (two MTF models, P3-S.7 D-5) |
|
|
| G-2 | ML has **no M30** feature | LEGACY DESIGN (v4.4 feed predates M30 in setup design) |
|
|
| G-3 | ML has **no M3** feature | LEGACY DESIGN |
|
|
| G-4 | ML has D1/H1 biases the setup layer does not use | LEGACY DESIGN (extra HTF) |
|
|
| G-5 | H4 role: setup = HARD gate; ML = soft numeric feature (f1) | INTENTIONAL DIFFERENCE (different layers) |
|
|
| G-6 | 50-bar HTF cache lag (old EA BTTFBias) | BUG -> **FIXED** P2.1 |
|
|
| G-7 | session-break cache staleness (Refresh-on-Bars-change) | LEGACY DESIGN, TOLERATED |
|
|
| G-8 | build_features (as-of open, lag-1) vs build_features_p2 (as-of close E_ea) | LEGACY DESIGN (p2 corrected feed NOT integrated into train_model.py; P2 §F) |
|
|
| G-9 | deployed SniperGold_ML.mqh trained on OLD v4.4 feed | LEGACY (frozen model, P2 §H risk #2) |
|
|
|
|
```text
|
|
NOTE (G-8): train_model.build_features maps HTF bias via
|
|
idx = searchsorted(ht, t, 'right')-1 with htf_bias_series (lag-1),
|
|
while build_features_p2 (the P2.5 parity-corrected module) maps via
|
|
e = searchsorted(ht_, tc-per, 'right')-1 with tf_bias_asof(e).
|
|
These differ. The DEPLOYED MLP baseline was trained with
|
|
train_model.build_features (G-9). This is a DOCUMENTED legacy inconsistency
|
|
(P2 §F: "build_features_p2 not yet integrated into train_model.py"). It is
|
|
NOT silently fixed by F4.
|
|
```
|
|
|
|
---
|
|
|
|
## H. ROOT CAUSES
|
|
|
|
```text
|
|
R-1 ARCHITECTURAL BIFURCATION : the 19-feature MLP descends from legacy v4.4
|
|
(D1/H4/H1 TFBias + M15 structure). The canonical H4/M30/M15/M3 model is a
|
|
NEWER Algo Forge 3-engine design (P3-S.7+). The two were never reconciled
|
|
into one MTF model. [LEGACY DESIGN]
|
|
R-2 MASTER-DATA DRIFT : FEATURE_CONTRACT documents the ML path only.
|
|
P3-S.7 S-PR explicitly records the "FEATURE_CONTRACT MTF MISMATCH". The ML
|
|
contract and the setup contract each specify a different MTF set without a
|
|
common reconciliation. [DOCUMENTATION DRIFT]
|
|
R-3 MODEL FREEZE : the deployed MLP was trained on the v4.4 feed;
|
|
the corrected P2.5 feed was not integrated (P2 §F remaining risk #2/#4).
|
|
The model weights encode the legacy semantics. [HISTORICAL MODEL FREEZE]
|
|
R-4 NO M30/M3 SURFACE : the Feature Contract never defined M30/M3
|
|
features; the setup layer gained them in P3-S.7/S.10 design. [INTENTIONAL ML
|
|
ARCHITECTURE that has drifted from the setup design]
|
|
```
|
|
|
|
---
|
|
|
|
## I. ALIGNMENT OPTIONS
|
|
|
|
```text
|
|
OPTION A — Align ML directly to H4/M30/M15/M3 (drop D1/H1, add M30/M3).
|
|
Pro : one canonical MTF everywhere; runtime/training consistency; simpler long-term reasoning.
|
|
Con : existing 19-feature model becomes obsolete; new training dataset + new model;
|
|
old model non-comparable. Requires a FUTURE authorized retraining phase.
|
|
|
|
OPTION B — Keep ML feature architecture (f0-f18) but MAP it explicitly.
|
|
Pro : least disruptive; preserves the existing MLP and its historical evidence;
|
|
Document f1(H4 bias) -> runtime H4 gate mapping; document the M15 base
|
|
feature set == M15 setup carrier.
|
|
Con : two MTF abstractions remain; M30/M3 stay unrepresented in ML (documented gap).
|
|
|
|
OPTION C — Retire the current static ML architecture.
|
|
Consider only if evidence shows the frozen MLP cannot be meaningfully aligned.
|
|
Evidence exists (R-4: no M30/M3; G-9: trained on OLD feed), so Option C is
|
|
defensible on SEMANTIC grounds independent of AUC.
|
|
```
|
|
|
|
---
|
|
|
|
## J. RECOMMENDED ARCHITECTURE (F4 decision)
|
|
|
|
```text
|
|
PRIMARY : OPTION B now (map the existing ML feature architecture explicitly),
|
|
with an explicit DOCUMENTED GAP for M30/M3.
|
|
- Keep f0-f18 (D1/H4/H1 + M15) as the ML feed (do NOT force H4/M30/M15/M3
|
|
into the 19 features — §8 directive).
|
|
- Document: f1 (H4 bias) is the closest ML counterpart to the runtime H4
|
|
gate DIRECTION, but it is a SOFT numeric input, NOT a gate; the runtime
|
|
gate semantics (direction-compatible, non-zero, equal to chain direction)
|
|
are NOT represented in the ML feature vector.
|
|
- Document: M30 gate state and M3 confirmation have NO ML feature. The ML
|
|
cannot reconstruct Candidate-Setup gate content from f0-f18.
|
|
|
|
EXTENDED (future, authorized separately) : OPTION C (retire the static MLP)
|
|
OR OPTION A (align to H4/M30/M15/M3) — decision belongs to a FUTURE
|
|
implementation/training phase, never selected by AUC. F4 only records the
|
|
contract and the gap.
|
|
```
|
|
|
|
```text
|
|
BRIEF DIRECTIVE (§8) : Do NOT force the four runtime TFs into ML. The ML
|
|
feature mathematically represents the information it was trained on (M15
|
|
structure + D1/H4/H1 bias). It does NOT represent the runtime H4/M30/M15/M3
|
|
gate semantics. The answer to "does the ML represent the same MTF information
|
|
as the setup?" is NO — recorded above. Adding raw M30/M3 columns would NOT
|
|
make it equivalent because the setup's gates are STATE (direction-compatible)
|
|
and the ML features are numeric inputs; equivalence requires semantic
|
|
definition, not column-count parity.
|
|
```
|
|
|
|
---
|
|
|
|
## K. DEFERRED IMPLEMENTATION (NOT done in F4)
|
|
|
|
```text
|
|
- NO ML retrain (MLP/LSTM/regime/meta-label) — documented only.
|
|
- NO feature-pipeline rewrite; NO new M30/M3 features built now.
|
|
- NO SniperGold_ML.mqh / model rebuild.
|
|
- NO change to FEATURE_CONTRACT.md (unchanged this session).
|
|
- NO change to train_model.py / build_features.py / build_features_p2.py.
|
|
- NO change to AF_Engine2_Setup.mqh (Candidate Setup frozen F3).
|
|
- NO change to F1 (event) / F2 (zone) semantics.
|
|
- NO runtime .mq5/.mqh change.
|
|
These become candidates for a SEPARATELY AUTHORIZED implementation/training
|
|
phase (P3-S.15), per the brief §17/§24/§31.
|
|
```
|
|
|
|
---
|
|
|
|
## L. FINAL ALIGNMENT VERDICT
|
|
|
|
```text
|
|
CLASSIFICATION : LEGACY / NOT COMPARABLE
|
|
(with PARTIAL ALIGNMENT of the M15 base layer and the H4 bias formula.)
|
|
|
|
- M15 base layer : ALIGNED (f3-f18 computed on M15 == setup M15 carrier)
|
|
- HTF bias formula+as-of (D1/H4/H1) : ALIGNED (P2.1-corrected, parity-verified)
|
|
- H4 role : NOT COMPARABLE (gate vs soft feature)
|
|
- M30, M3 : GAP — no ML feature (NOT comparable)
|
|
- D1, H1 : ML-only extra TFs (NOT comparable)
|
|
|
|
NOT "MODEL BAD": this is a SEMANTIC verdict about two different MTF
|
|
architectures, not a performance claim. Distinct from MISALIGNED: the two
|
|
models are different generations (legacy v4.4 ML feed vs newer candidate-setup
|
|
model) and are NOT reconciled — they are simply different objects.
|
|
```
|
|
|
|
---
|
|
|
|
## M. IMPLEMENTATION DECISION (per brief §22)
|
|
|
|
```text
|
|
DOCUMENTATION CHANGE REQUIRED
|
|
+ FUTURE FEATURE-PIPELINE / ML-PATH ALIGNMENT REQUIRED
|
|
(the ML path should eventually be aligned to the canonical setup MTF or be
|
|
retired — but that is a FUTURE authorized phase, NOT executed in F4.)
|
|
|
|
- NP CODE CHANGE REQUIRED in this session (F4 is audit/specification-only).
|
|
- The ML model is NOT retrained; the Candidate Setup layer is NOT changed.
|
|
```
|
|
|
|
---
|
|
|
|
## N. REGRESSION
|
|
|
|
```text
|
|
P3-S.2 Liquidity Sweep 17/17 PASS (untouched)
|
|
P3-S.3 CHoCH/MSS 20/20 PASS (untouched)
|
|
P3-S.4 FVG 20/20 PASS (untouched)
|
|
P3-S.5 Order Block 20/20 PASS (untouched)
|
|
P3-S.6 Displacement 20/20 PASS (untouched)
|
|
P3-S.7 MTF Alignment 22/22 PASS (re-run, PASS)
|
|
F1 Event Contract 8/8 PASS (untouched)
|
|
F2 Zone Contract 20/20 PASS (untouched)
|
|
F3 Candidate Setup 25/25 PASS (untouched)
|
|
F4 MTF/Training Align 16/16 PASS (NEW — this session)
|
|
No previous suite weakened; no oracle changed.
|
|
```
|
|
|
|
---
|
|
|
|
## O. EXPLICITLY UNCHANGED AREAS
|
|
|
|
```text
|
|
Candidate Setup : AF_Engine2_Setup.mqh, AFCandidateSetup, lifecycle, identity,
|
|
H4/M30 gates, M15 carrier, M3 confirmation — FROZEN (F3).
|
|
F1 (event semantics) : unchanged.
|
|
F2 (FVG/OB zone semantics): unchanged.
|
|
ML model (.mqh / weights / labels) : unchanged.
|
|
train_model.py / build_features.py / build_features_p2.py : unchanged.
|
|
FEATURE_CONTRACT.md : unchanged (F4 documents, does not modify).
|
|
Runtime .mq5/.mqh : unchanged.
|
|
```
|
|
|
|
---
|
|
|
|
## P. PROVENANCE & CHECKPOINT
|
|
|
|
```text
|
|
Forge HEAD (start) : 1c0da62407fc905fb96732ec0f6ec5c43d136a8b (P3-S.13)
|
|
Branch / remote : main == origin/main (VERIFIED clean @ start)
|
|
New artifacts : docs/SMC_MTF_TRAINING_ALIGNMENT_SPEC_v1.md
|
|
docs/P3_S14_MTF_TRAINING_ALIGNMENT.md (this report)
|
|
ml/p3/smc_semantic/spec_tests_mtf_training_alignment.py
|
|
ml/p3/smc_semantic/spec_test_cases_mtf_training_alignment.json
|
|
ml/p3/smc_semantic/output/spec_tests_mtf_training_alignment_report.json
|
|
docs/SESSION_HANDOVER_<DATE>_P3_S14_MTF_TRAINING_ALIGNMENT.md
|
|
Human verification : CANCELLED
|
|
```
|
|
|
|
*End of P3-S.14 F4 MTF / Training Alignment record. No production/ML change.* |