chiki2bum2/SniperGold_ML
Watch
1
0
Fork
You've already forked SniperGold_ML
 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions
SniperGold_ML/docs/P3_S15_ML_PATH_ARCHITECTURE_DECISION.md

482 lines
23 KiB
Markdown
Raw Permalink Normal View History

docs: define future ML path architecture from canonical Candidate Setup
2026-08-23 07:03:30 +07:00
# P3-S.15 ML PATH / TRAINING ARCHITECTURE DECISION — SNIPERGOLD_ML
```text
Date : 2026-08-23
Session : P3-S.15 — ML Path / Training Architecture Decision
Status : DESIGN / ARCHITECTURE DECISION ONLY. DOCUMENTATION ONLY.
NO code, NO model, NO training, NO Candidate Setup change,
NO FEATURE_CONTRACT change in this session.
Checkpoint : P3_S14_HEAD 8b29466e52ab810843768c083c1193e35e091e8e (VERIFIED
local == origin/main, branch main, working tree CLEAN)
Contract : docs/SNIPERGOLD_CANONICAL_SETUP_CONTRACT_v1.md (frozen, P3-S.10)
Handover : docs/SESSION_HANDOVER_2026-08-23_P3_S14_MTF_TRAINING_ALIGNMENT.md
Verification: P3-S.15 checkpoint VERIFIED — see §2.
Human verification : REMAINS CANCELLED (permanent research-path decision).
Decision : OPTION A final classification = "DESIGN DECISION REQUIRED -> A.
KEEP LEGACY MODEL AS REFERENCE + BUILD FUTURE SETUP-LEVEL ML"
(see §L). Resolved as an architecture decision, NOT by AUC/PF.
```
---
## 1. SESSION MANDATE (the single question)
> **Now that SniperGold has a repaired EVENT contract, repaired ZONE contract,
> a real Candidate Setup layer, and a canonical H4/M30/M15/M3 architecture,
> what should the future ML system learn from, and what should the current
> legacy MLP be considered?**
This session answers the architecture-level question above. It does NOT
compute AUC/PF, does NOT retrain, does NOT modify the Candidate Setup, and
does NOT touch feature contract frontiers. It decides the OBSERVATION UNIT,
LABEL DIRECTION, FEATURE ARCHITECTURE, TEMPORAL/REGIME model role, and the
FUTURE DATA PIPELINE for a separately-authorized future training phase.
---
## 2. CHECKPOINT VERIFICATION (Phase 0 — SESSION-HANDOVER-FIRST)
```text
A. Latest authoritrative handover found by repository search
(docs/SESSION_HANDOVER_*.md):
SESSION_HANDOVER_2026-08-23_P3_S14_MTF_TRAINING_ALIGNMENT.md
(newest date 2026-08-23, terminal phase P3-S.14; confirmed by git history
highest parent of current HEAD and by its own "Next gate: P3-S.15".)
B. Preceding handover for continuity:
SESSION_HANDOVER_2026-08-22_P3_S13_CANDIDATE_SETUP.md (F3, parent of S.14)
C. Checkpoint stated by the session brief (P3-S.14):
8b29466e52ab810843768c083c1193e35e091e8e
Checkpoint stated by the latest handover (P3-S.14 F4 test commit):
8b29466... (same object — the handover records it in the git log commit
"test: define F4 mtf-training-alignment session report generator").
D. Verified (against git):
local HEAD = 8b29466e52ab810843768c083c1193e35e091e8e OK
remote HEAD = 8b29466e52ab810843768c083c1193e35e091e8e OK (origin/main)
branch = main OK
working tree = CLEAN (git status empty) OK
local == remote == checkpoint OK
E. RESULT : repository state CONSISTENT with the latest handover.
PROCEED with the P3-S.15 architecture decision.
```
---
## 3. RECONSTRUCTED PROJECT STATE (from the latest handover — authoritative)
```text
Current phase : P3-S.15 — ML path / training architecture DECISION.
Completed phases : P3-S.2..S.8 semantic conformance; P3-S.9 architecture
review + root-cause matrix; P3-S.10 owner adjudication +
CANONICAL SETUP CONTRACT FREEZE; P3-S.11 F1 EVENT repair;
P3-S.12 F2 ZONE repair; P3-S.13 F3 CANDIDATE SETUP layer
(implemented, frozen, 25/25); P3-S.14 F4 MTF/training
alignment (verdict LEGACY / NOT COMPARABLE, 16/16).
Earlier P2.1-P2.6 (HTF parity, corrected baseline) and
P3.1/P3.2/P3.2.2 forensic evidence (feature audit,
label/event forensic, de-overlap/survival).
Frozen architecture : H4 (Narrative/Context gate) -> M30 (Context gate) ->
M15 (Entry/Setup carrier) -> M3 (optional confirmation).
Frozen Candidate Setup chain : H4 gate -> M30 gate -> fresh Liquidity EVENT ->
CHoCH/MSS EVENT after sweep -> active/unmitigated OB|FVG
ZONE -> M15 entry condition -> CANDIDATE_SETUP -> [M3].
Identity, lifecycle, one-setup-one-identity, one-entry,
score separation — all FROZEN (F3).
F1 / F2 : repaired and frozen (event contract; zone contract).
Current open decision:P3-S.15 (this session) — future ML/training architecture.
Production changes : NONE in P3-S.14/S.15 (docs only). Runtime unchanged.
Model status : SniperGold_ML.mqh v20260821_2head — FROZEN, UNCHANGED.
Legacy 24-bar directional MLP on D1/H4/H1+M15 (19 feats).
ML status : LEGACY / NOT COMPARABLE vs canonical setup (P3-S.14).
Known blockers : M30 and M3 have NO ML feature surface; virtual D1/H1 are
legacy-only; frozen model encodes the legacy v4.4 feed.
Next gate : P3-S.15 (this decision) -> per decision, P3-S.16 =
Setup-Level Dataset / Label Contract IMPLEMENTATION.
```
---
## 4. CURRENT ML MODEL STATUS (frozen, unchanged — §18)
```text
Object : SniperGold_ML.mqh (terminal + repo historical copies)
Version : v20260821_2head
Family : static MLP, 19 -> 12 -> 2 (long/short heads), 2-node aux-calibrated
Label : 24-bar forward direction drift | 0.75 x ATR threshold
(L1 DIRECTIONAL label, NOT setup outcome)
Feed : D1/H4/H1 HTF bias (f0-f2) + M15 structure/zone/delta/mom/confluence
(f3-f18) — the FEATURE_CONTRACT f0-f18 (P2 source of truth)
MTF : D1/H4/H1 + M15 (LEGACY generation)
AUC : validation 0.5005 | long 0.6270 | short 0.6207 (recorded header;
reported for status only — NOT used as a decision criterion)
Trained : 2026-08-21, XAUUSDc M15, on the OLD v4.4 feed semantics
(the P2.5-corrected build_features_p2 feed was NOT integrated
into train_model.py — a documented legacy inconsistency, P2 §F/G-8;
G-9: deployed model trained on the OLD feed).
Status classification (per §18, on established evidence only):
DEPLOYABLE : yes, it is wired and runs. [functional]
LEGACY : yes — pre-Candidate-Setup, bar-level, D1/H4/H1+M15 feed. [lineage]
REFERENCE : yes — usable as a historical baseline of the old generation.
SUPERSEDED : NO as "obsolete" for the CURRENT design intent; YES as
semantically non-equivalent to the canonical setup model.
FINAL : classify as LEGACY / REFERENCE benchmark. NOT the canonical
Candidate Setup model. It predicts directional drift (L1), which
is a DIFFERENT target from setup outcome. It stays intact.
```
The session's mandatory invariant (§18/§23): this model must remain the same
file/weights/architecture/calibration/header. It is preserved, not destroyed.
---
## 5. RUNTIME vs LEGACY ML ARCHITECTURE (§7, §17)
```text
RUNTIME (frozen, canonical) LEGACY ML / TRAINING (current)
----------------------------------- -----------------------------------------
MTF: H4/M30/M15/M3 MTF: D1/H4/H1 + M15
Setup unit: ONE Candidate Setup Unit: EVERY M15 bar
Chain: gate->gate->sweep->CHoCH->zone Unit class: per-bar feature vector
->M15 entry -> SETUP [->M3] Label: 24-bar forward DROFT (L1)
Identity: setup_id (causal) Label target: DIRECTION, not outcome
Lifecycle: full state machine No setup identity, no lifecycle
H4/M30 = HARD direction gates H4 = soft numeric bias (f1); D1/H1 = legacy
M30, M3 = REQUIRED gate / confirmation M30, M3 = ABSENT from feature set
Causality: as-of closed bars per stage Causality: as-of closed bars (aligned for M15/H4)
Overlap: M15 base layer + H4 bias semantics partially overlap (P3-S.14).
NOT COMPARABLE as the SAME predictor: different unit, different label target,
different MTF set, different consumer semantics.
The legacy model and the canonical setup are DIFFERENT OBJECTS, not the same
information written differently (P3-S.14 §L verdict).
```
---
## 6. ONE-OBSERVATION UNIT DECISION (§11, §12) — THE CORE ANSWER
### 6.1 Existing model unit
```text
one M15 bar -> future 24-bar DIRECTION drift
```
### 6.2 Future Candidate Setup unit
```text
one Candidate Setup -> setup OUTCOME (valid-before-invalidation)
```
### 6.3 Why the setup must be the observation unit
- **Semantic correspondence** (the decisive criterion): the runtime makes
decisions on WHOLE SETUPS (identity + lifecycle), not on arbitrary bars.
ML that learns per-bar cannot correspond to the runtime consumer.
- **Causality**: a setup is a causally-ordered chain with a defined creation
timestamp and lifecycle; an outcome can be measured causally from creation.
- **Training-unit correctness**: the runtime's decision granularity (one setup
per entry) forces the training unit to be the setup. Bar-level training
would optimize the wrong granularity.
- **Runtime/training consistency**: erased the P3-S.14 bifurcation (R-1).
- **Evidence (P3.2.2 §K)**: the L3 label MECHANISM is viable ONLY on a discrete
setup unit; the existing event/bar stream FAILED "unit validity" (97-99%
redundant, 1.5% retention). Setup-level units fix the unit-validity failure.
- **Setup identity is sufficient as an ML observation key** (§12): it has
setup_id, direction, sweep onset/dir, choch onset/dir, zone info,
creation timestamp, optional M3. This is a complete causal composite key.
### 6.4 Per-observation-consequence analysis (covers every required point)
| Dimension | Per-bar (legacy) | Per-setup (future) |
|---|---|---|
| Sample independence | FAIL — every bar overlaps its neighbours (outcome windows share) | IMPROVED — setups are discrete; dedup by identity removes per-bar redundancy (still need de-overlap between distinct setups, §6.5) |
| Label definition | direction drift (L1) — MISALIGNED for setups | setup-outcome (L3 TP-before-SL / risk-adjusted) — CAUSAL |
| Overlap | massive (98% +1-bar gap, P3.2.2 §J) | boundary: distinct setups may still overlap in TIME; need a min separation / lead-setup policy |
| Event clustering | bars cluster into long same-outcome runs | setups cluster into market episodes; de-overlap needed at the setup level |
| Setup deduplication | none (no identity) | BY CONSTRUCTION via setup_id (F3 one-setup-one-identity) |
| Time-to-outcome | horizon fixed 24 bars | setup lifetime (P3.2 §6/§8: 1-8 bars, ~90% <=8) — horizon must match lifetime |
| Censoring | — | must be handled (UNRESOLVED valid at H16+, low; survival for short H) |
| Class balance | near 50/50 direction | setup-outcome is LOSS-heavy (P3.2.2: LOSS ~2x WIN on existing event units); this is a REAL distribution fact, must be handled by the label design, not hidden |
| Feature leakage | fixed by closed-bar | must enforce closed-bar + no lookahead (setup created only when chain completes) |
### 6.5 De-overlap must be re-derived at the setup level
The P3.2.2 de-overlap was defined for EVENT stream. At the SETUP level the
dedup is by identity (§6.3), but two DISTINCT setups can still have
overlapping outcome windows (e.g. a second setup forms inside the first
setup's life). Future dataset must apply a setup-identity + temporal
separation policy (lead-setup-per-episode) analogous to P3.2.2 §B, defined in
the P3-S.16 label contract, NOT in this session.
**DECISION §6**: The future ML observation unit = **ONE Candidate Setup**.
This is the only unit that is semantically consistent, causally sound, and
matches the runtime consumer. (Section 6 answers §11, §12, and the STOP
condition "Candidate Setup cannot serve as a deterministic training unit" —
it CAN, because identity+lifecycle are deterministic and frozen.)
---
## 7. FUTURE LABEL CONTRACT — DIRECTION (§13, §14)
### 7.1 Explicit distinction (from P3.2 §0/§14 — the key forensic)
```text
BAR-DIRECTION label (L1) : one M15 bar -> future 24-bar drift.
Measures DIRECTIONAL PREDICTION, not setup validity.
This is what the CURRENT model encodes.
SETUP-OUTCOME label (L3+) : one Candidate Setup -> did the EXPECTED outcome
occur BEFORE invalidation?
Measures SETUP VALIDATION — what the runtime asks.
```
P3.2 (official classification B — LABEL IS MISALIGNED) established these are
DIFFERENT targets. P3.2.2 showed the L3 (TP-before-SL) mechanism is viable.
This session RECORDS the direction, does NOT implement labels.
### 7.2 Future label family (to be measured in P3-S.16; not implemented now)
```text
PRIMARY target : setup-outcome, TP-before-SL (L3 successor):
WIN = TP reached before SL within the setup-lifetime horizon.
LOSS = SL reached first (or invalidation/contrary structure first).
CENSORED/UNRESOLVED : neither within the lifetime; real censoring.
AMBIGUOUS : same-bar TP & SL (excluded/recorded, 1-3%).
Suggested semantic: TP = k*ATR, SL = m*ATR with a justified R:R (NOT
selected from performance; candidates in P3.2 §14 / P3.2.2 §0 can be
re-baselined against the FROZEN setup, not the event stream).
SECONDARY / descriptive (NOT classification targets):
- MFE / MAE excursion (L2) : setup risk/geometry profile.
- time-to-TP / time-to-SL (L4) : setup lifetime distribution.
- invalidation time : structure-break invalidation latency.
EXPLICITLY SEVERED from the legacy: the 24-bar DIRECTION drift label is NOT
reused as the setup-outcome target.
P3.2 evidence note: on the EXISTING event semantics the outcome is LOSS-heavy.
This is a measurement of the OLD setup units, not a judgment on the NEW
canonical Candidate Setup outcome distribution. P3-S.16 must re-measure on the
canonical unit. The label must not be engineered to fake balance; it should
measure truthfully.
```
---
## 8. FUTURE FEATURE ARCHITECTURE (§14) — PROPOSAL ONLY
The future setup-level model will require features aligned to the canonical
setup entity (NOT added to FEATURE_CONTRACT in this session):
```text
IDENTITY / TARGET block : setup_id, direction,
creation_bar / creation timestamp.
H4 context (gate) : H4 narrative STATE (as-of closed), direction.
M30 context (gate) : M30 context STATE (as-of closed), direction.
Liquidity EVENT : sweep onset, sweep direction, sweep freshness,
distance to sweep, sweep size relative to ATR.
CHoCH EVENT : choch onset, choch direction, choch-vs-sweep
order/latency.
ZONE (OB|FVG) state : zone type, zone age (vs zone formation),
zone width (ATR units), price-in-zone offset,
mitigation state at creation, zone-bound proximity.
Displacement attribute : value at chain completion (optional boost).
M15 entry state : M15 entry-condition boolean, premium/discount
filter state, distance-to-zone in ATR.
M3 confirmation : optional M3 confirmation presence + direction.
Geometry / time : zone geometry, time since event onset,
time since zone formation, setup lifecycle stage.
```
ALL features must be as-of the FROZEN MTF as-of contract (§L of the canonical
contract): every consumed bar closed before the setup creation decision bar.
NO partial candle, NO future information.
This is an ARCHITECTURAL PROPOSAL (Section 8 records the direction). The full
contract with numerics/lookbacks belongs to P3-S.16 (or a dedicated proposal),
where it will be created as a NEW proposal document, NOT by mutating
`docs/FEATURE_CONTRACT.md` (§19 policy respected).
---
## 9. TEMPORAL MODEL QUESTION (§15)
```text
Q: Would a future temporal model have a valid temporal sequence to learn once
Candidate Setup becomes the observation unit?
Bar-sequence view (old): M15 bars -> 24-bar drift. Sequences of bars,
but the "event" is drowned by bars (redundant, overlapping windows).
Setup-lifecycle-sequence view (future): the canonical lifecycle
NONE -> CONTEXT_VALID -> LIQUIDITY_TRIGGERED -> STRUCTURE_CONFIRMED ->
ZONE_READY -> ENTRY_ARMED -> CANDIDATE_SETUP -> [M3_CONFIRMED] ->
EXPIRED | INVALIDATED | CONSUMED.
Assessment:
- There IS a well-defined temporal object: the SETUP LIFECYCLE is a sequence
of dated states/events. A temporal model (e.g. sequence-of-stages) is
CONCEPTUALLY valid IF it models the lifecycle transition sequence per setup.
- However the per-setup lifecycle is SHORT (1-8 bars for confirm, then
terminal). Over a sequence of DISTINCT setups, the meaningful information is
the cross-setup context, not a long intra-setup series.
- REGIME/H4/M30 state provide the longer temporal context; M15 provides the
short lifecycle sequence.
- CONCLUSION: temporal modeling is JUSTIFIED ONLY as modeling the SETUP
LIFECYCLE SEQUENCE (stages as a sequence) and/or the cross-setup context
series. The old per-bar LSTM/Informer premise (learn on dense bars) is NOT a
valid temporal learning target after the unit becomes the setup, because it
re-introduces the overlap/redundancy failure (P3.2.2). A temporal model over
well-separated setup-lifecycle sequences would have a valid sequence to learn.
- NOT run here. This § answers the design question only.
```
---
## 10. REGIME MODEL QUESTION (§16)
```text
Q: What role should REGIME play in the future architecture?
Prior regime work (HMM/MSM/GARCH experiments) is NOT revived. The canonical
H4/M30 narrative+context gates already capture a coarse regime/state of the
market at the top of the MTF stack.
Preferred future role (documented, NOT implemented):
REGIME = LABEL-CONDITIONING VARIABLE / PRE-FILTER, layered on top of, and
defined consistently with, the H4/M30 narrative/context gates.
Rationale:
- H4 direction-compatible gate ALREADY conditions entry (a regime proxy).
- If used as a pre-filter, note the frozen OD-1/OD-4: regime must NOT
override the canonical gate semantics for the SETUP LAYER. Regime would
sit ABOVE the setup layer as an entry-quality conditioner, not as a new
setup gate.
- NOT a separate super-model; NOT a hard filter that contradicts H4/M30.
- A single regime SCORE could eventually be a SETUP FEATURE (context depth)
but should not be allowed to recreate the legacy "regime gate" that the
canonical contract removed (OD-4 gate-fail policy is authoritative for
the setup layer).
Preference ranking (as a documented default for future design):
1. Label-conditioning variable (most consistent with frozen gates)
2. Context candidate-setup feature (if it is meaningful)
3. Pre-filter (allowed only if it never contradicts H4/M30 gate semantics)
4. Separate model (NOT preferred — legacy direction, not revived)
A single preferred default will be fixed in the P3-S.16 contract.
```
---
## 11. FUTURE DATA PIPELINE (§17)
### 11.1 Future (recommended)
```text
Market Data (M15 base + H4/M30/M3 as-of)
-> SMC primitives (sweep/CHoCH/FVG/OB/displacement; F1/F2 repaired)
-> repaired Events & Zones (F1/F2 contracts; as-of lifecycle)
-> Candidate Setup (F3: deterministic chain, identity, lifecycle)
-> Setup SNAPSHOT (per-setup feature vector, §8)
-> Outcome Label (setup-outcome L3, §7, measured from creation bar forward)
-> ML Dataset (one row per Candidate Setup, de-overlapped by identity+time)
-> model (setup-level, see §12)
```
### 11.2 Old (legacy)
```text
Market Data (M15 bars)
-> features (D1/H4/H1+M15, f0-f18)
-> 24-bar label (directional drift)
-> MLP (19->12->2)
```
### 11.3 Comparability
```text
NOT comparable as the same predictor:
- observation unit : bar vs setup
- label target : 24-bar drift vs setup outcome
- MTF set : D1/H4/H1+M15 vs H4/M30/M15/M3
- consumer semantics : MLP numeric input vs runtime candidate-setup entity
They represent different generations. The legacy pipeline is preserved as
evidence; it is not the canonical training path.
```
---
## 12. ML ARCHITECTURE CLASSIFICATION (final — §21 with §26 guards)
```text
The three options (A / B / C) were evaluated on semantics only (§10):
A. KEEP LEGACY MODEL AS REFERENCE + BUILD FUTURE SETUP-LEVEL ML
B. ALIGN EXISTING ML PIPELINE TO CANONICAL RUNTIME
C. RETIRE CURRENT STATIC ML AND DESIGN A REPLACEMENT
P3-S.14 §I already recommended B-now + A/C-future for the FEATURE audience.
At the ARCHITECTURE level (this session) the deciding questions are:
(1) Can the legacy pipeline be "aligned" to the canonical runtime?
NO — one is bar-level (L1 drift), the other setup-level (L3 outcome).
Aligning the ONE-LINE pipeline (Option B) would change unit + label + MTF,
i.e. it would no longer be the legacy pipeline at all. So B is a label,
not a faithful "alignment" of the probe.
(2) Can the legacy model be retired NOW (Option C)?
It CAN be classified LEGACY/REFERENCE and preserved. "Retirement" as a
resolution is not required; preservation + reclassification is the honest
outcome. There is no architectural need to delete it.
(3) Is the current model a valid REFERENCE?
YES — as a historical baseline of the old generation (frozen, reproducible).
(4) What must the future build?
A NEW setup-level model (unit = Candidate Setup, target = setup outcome).
FINAL CLASSIFICATION : A.
KEEP the legacy static MLP as a frozen LEGACY / REFERENCE benchmark, AND
design/build the FUTURE SETUP-LEVEL ML (separately authorized, P3-S.16 and
beyond). The current model is NOT represented as the canonical Candidate
Setup model.
STOP-CONDITION CHECK (§26): None triggered.
- Candidate Setup CAN serve as a deterministic training unit (identity, F3).
- Future outcome CAN be defined causally (setup-outcome, §7).
- Legacy model CAN be preserved as a valid reference (§4).
- Future ML does NOT require changing frozen runtime semantics (it targets
the SAME canonical setup; it does not alter it).
- Selection is NOT based on AUC/PF (architecture/semantics only).
```
---
## 13. NET RECOMMENDATION
```text
CURRENT MODEL : SniperGold_ML.mqh v20260821_2head = LEGACY / REFERENCE
(bar-level directional MLP; frozen; NOT the canonical model).
FUTURE ML : Candidate-Setup-native. Observation unit = ONE Candidate
Setup; target = setup-outcome (TP-before-SL + censoring);
features aligned to H4/M30/M15/M3 canonical setup entity;
temporal model = setup-lifecycle sequence ONLY if warranted;
regime = label-conditioning variable consistent with gates.
NEXT GATE : P3-S.16 = Setup-Level Dataset / Label Contract
IMPLEMENTATION (docs+dataset contract proposal, per §7/§8,
preserving FEATURE_CONTRACT.md). No retrain in P3-S.15.
```
---
## 14. EXPLICITLY UNCHANGED AREAS (§18, §23)
```text
Candidate Setup : FROZEN (AF_Engine2_Setup.mqh, AFCandidateSetup) — NOT redesigned.
F1 (events) : unchanged.
F2 (zones) : unchanged.
ML model : SniperGold_ML.mqh — unchanged (weights/architecture/calib/header).
FEATURE_CONTRACT : docs/FEATURE_CONTRACT.md — NOT modified this session.
train_model.py / build_features.py / build_features_p2.py : NOT modified.
.model weights : NOT retrained / NOT re-exported.
production MQL5 : NOT changed. No .mq5/.mqh edit.
runtime semantics : NOT changed.
No AUC/PF/backtest/optimization used in this session.
```
*End of P3-S.15 ML path / training architecture decision. Documentation only.*
Reference in a new issue Copy permalink