forked from chiki2bum2/SniperGold_ML
397 lines
No EOL
16 KiB
Markdown
397 lines
No EOL
16 KiB
Markdown
# P3-S.17R RUNTIME ↔ RESEARCH CANDIDATE SETUP PARITY
|
|
|
|
```text
|
|
Date : 2026-08-23
|
|
Session : P3-S.17R — Runtime <-> Research Candidate Setup Parity
|
|
Baseline : P3-S.17 checkpoint ed01af24644fbac26fec9fe6ff6a3179920a61d6
|
|
VERIFIED (origin = forge.mql5.io/chiki2bum2/SniperGold_ML.git,
|
|
branch main, working tree CLEAN).
|
|
Scope : XAUUSD M15 2017-01-01..2026-08-21 (Files/AlgoForge/Data); the
|
|
P3-S.17 authorized scope = 194,985 bars (2017-01-03..2026-07-21).
|
|
Human verif: REMAINS CANCELLED (historical only). NO AUC/PF/backtest/opt.
|
|
Verdict : PARITY BLOCKED (see §V). A trust-worthy full-population parity
|
|
could not be concluded because the F1 sweep primitive and the
|
|
H4/M30/M15 gate series are not yet faithfully reproducibly
|
|
wired on the real-data scope. The frozen F3 SETUP-LAYER semantics
|
|
IS reproduced (PAR-T01..T18 green); the DETECTOR/gate primitives
|
|
that feed it are the unresolved divergence.
|
|
```
|
|
|
|
---
|
|
|
|
## A. Objective (single question)
|
|
|
|
> Does the research-side Candidate Setup reconstruction reproduce the FROZEN
|
|
> MQL5 Candidate Setup decisions on the SAME historical input, stage by stage,
|
|
> WITHOUT changing runtime semantics?
|
|
|
|
Answer so far: **partial** — the frozen F3 setup-layer lifecycle/identity/
|
|
gate logic is reproduced exactly (17/17 PAR-T on the frozen oracle), and Stage 1
|
|
data identity passes, but the real-data **F1 sweep primitive** and the
|
|
consumed **H4/M30/M15 gate series** are NOT yet shown faithful, so the full
|
|
Candidate Setup population cannot yet be concluded with a trustworthy verdict.
|
|
|
|
---
|
|
|
|
## B. Source-of-truth hierarchy used
|
|
|
|
```text
|
|
1. Frozen spec contracts : SNIPERGOLD_CANONICAL_SETUP_CONTRACT_v1,
|
|
SMC_LIQUIDITY_SWEEP_SPEC_v1, SMC_CHOCH_MSS_SPEC_v1,
|
|
SMC_ORDER_BLOCK_SPEC_v1, SMC_FVG_SPEC_v1,
|
|
P3_S11_EVENT_CONTRACT_REPAIR, P3_S12_ZONE_CONTRACT_REPAIR.
|
|
2. Frozen MQL5 runtime : AF_Engine2_Setup.mqh (F3 layer)
|
|
AF_Engine2_Agents.mqh (F1/F2 detectors)
|
|
AF_Defines.mqh (constants: W_sweep/W_choch=40)
|
|
3. Verified regression : P3-S.16 (20/20), P3-S.13 F3 (25/25), P3-S.4 FVG (20/20).
|
|
4. Existing F3 oracle : ml/p3/smc_semantic/spec_tests_candidate_setup_runtime.py
|
|
canonical_oracle + F3SetupEngine.
|
|
5. Research extractor : P3-S.17 proxy (NOT merged) + this session's faithful
|
|
research ports.
|
|
6. New assumptions : avoided (per §35 STOP conditions).
|
|
```
|
|
|
|
The research extractor is NOT authoritative. Where research differs from frozen
|
|
runtime, research is repaired — never the runtime.
|
|
|
|
---
|
|
|
|
## C. Checkpoint & workspace
|
|
|
|
```text
|
|
Workspace : ...\Shared Projects\SniperGold_ML_authoritative (authoritative clone)
|
|
origin : https://forge.mql5.io/chiki2bum2/SniperGold_ML.git
|
|
branch : main
|
|
local HEAD: ed01af24644fbac26fec9fe6ff6a3179920a61d6
|
|
remote HEAD: ed01af24644fbac26fec9fe6ff6a3179920a61d6
|
|
working tree: CLEAN before parity work
|
|
```
|
|
|
|
---
|
|
|
|
## D. Latest SESSION_HANDOVER used
|
|
|
|
`docs/SESSION_HANDOVER_2026-08-23_P3_S17_DATASET_AUDIT.md` (read at session
|
|
start). It records DATASET BLOCKED, 0 raw setups, next gate = runtime↔research
|
|
reconciliation (P3-S.17R).
|
|
|
|
---
|
|
|
|
## E. Data source
|
|
|
|
```text
|
|
Symbol : XAUUSD
|
|
Timeframe : M15 (entry/setup carrier); H4 (gate); M30 (gate); H1/D1 (context)
|
|
Source : Files/AlgoForge/Data (XAUUSD_M15.npz, H1/H4/D1) — SAME source as
|
|
the P3-S.17 runtime-parity baseline.
|
|
Scope : 2017-01-01..2026-08-21 npz (197,032 bars after 2017 filter);
|
|
P3-S.17 authorized scope (194,985 bars, 2017-01-03..2026-07-21)
|
|
reproduced exactly in the Stage-1 identity check.
|
|
Feed quality: download_report 0 dup / 0 bad OHLC / 0 anomaly gaps (P3-S.17 §E).
|
|
No Tickstory / Dukascopy used as the baseline (deferred study, §Q).
|
|
```
|
|
|
|
---
|
|
|
|
## F. Stage 1 — data / timestamp identity (PAR-T01)
|
|
|
|
```text
|
|
n_bars_total : 197032 (M15, 2017+ npz)
|
|
n_bars_in_scope : 194985 (2017-01-03 .. 2026-07-21) == P3-S.17 recorded
|
|
timestamps_monotonic: True
|
|
t_start 2017-01-03 00:00Z t_end 2026-07-21 23:45Z
|
|
h4_bars 16234 / h1_bars ~frames / d1_bars available
|
|
=> Stage 1 PASS. The research pipeline reads the identical M15 OHLC/timestamps
|
|
used for the P3-S.17 runtime-parity baseline.
|
|
```
|
|
|
|
---
|
|
|
|
## G. Stage-by-stage parity matrix (real-data 2000-bar deterministic subset)
|
|
|
|
Golden-trace subset = the first 2000 closed M15 bars (2017), a MANAGEABLE
|
|
deterministic window (prompt §15/§13). Full-scope detection requires vectorized
|
|
detector kernels (documented as a future step; not a parity gate itself).
|
|
|
|
| Stage | Runtime (frozen) | Research port | Match | Notes / first divergence |
|
|
|---|---|---|---|---|
|
|
| H4 gate | (consumed context STATE) | gate input | ✓ (F3) | F3 layer consumes h4; PAR-T02 green |
|
|
| M30 gate | (consumed) | gate input | ✓ (F3) | PAR-T03 green |
|
|
| Sweep event | P3-S.2/P3-S.11 `DetectLiquidityGrabs`+W=40 | faithful port (this session) | ✗ vs legacy proxy | **FIRST DIVERGENCE** (see §H) |
|
|
| CHoCH event | P3-S.3/P3-S.11 | faithful port | ✓ (subset) | PAR-T06/T07 green; choch=289/2000 |
|
|
| OB zone | P3-S.5/P3-S.12 canonical | faithful port | ✓ (synthetic) | PAR-T08/T09; ob_active=1981/2000 |
|
|
| FVG zone | P3-S.4/SMC_FVG v1 | faithful port | ✓ (synthetic) | PAR-T10/T11; namespace boundary OK |
|
|
| M15 entry | (consumed E-rule) | gate input | ✓ (F3) | PAR-T12 green |
|
|
| Candidate Setup | F3 `AF_Engine2_Setup` | F3 oracle `canonical_oracle` | ✓ (setup layer) | PAR-T13..T18 green; 17/17 |
|
|
|
|
On the deterministic subset with NEUTRAL gates (all-zero H4/M30/M15), the F3
|
|
chain cannot form (context gate requires non-zero direction-compatible H4/M30),
|
|
so **runtime_created == research_created == 0**. That specific 0 is behavioural
|
|
(with neutral gates) and is NOT yet the faithful full-scope population — it is
|
|
reported honestly and the real population requires the runtime's actual gated
|
|
inputs (§M).
|
|
|
|
---
|
|
|
|
## H. FIRST DIVERGENCE — Stage 4 (Sweep event primitive)
|
|
|
|
Measured on the identical 2000-bar deterministic M15 subset:
|
|
|
|
```text
|
|
frozen_policy_sweep onsets (this port, P3-S.2 semantics, critical-bar):
|
|
58 occurrences / 2000 bars (= 2.9% onset rate)
|
|
legacy proxy `smc_semantic_common.sweep_state` active-bars:
|
|
1896 / 2000 bars (= 94.8% of bars marked "sweep bar >=0")
|
|
|
|
first divergence bar : 22 (frozen=-1, legacy=0)
|
|
bars frozen-only-active: 723
|
|
bars legacy-only-active : 0
|
|
```
|
|
|
|
Interpretation (P3-S.17R §16 classification):
|
|
|
|
```text
|
|
CLASS : FIRST-DIVERGENCE / STATE-CONVENTION
|
|
ROOT CAUSE : the P3-S.17 extractor's proxy sweep used the persistent
|
|
`smc_semantic_common.sweep_state` whose `sweep_bar[]` is a
|
|
persistent last-onset latch that once set stays set for the
|
|
REST of the window (94.8% of bars active). A faithful P3-S.2
|
|
EVENT interpretation marks only 2.9% of bars as carrying an
|
|
active sweep EVENT (fresh onset within W_sweep=40). The two
|
|
disagree from bar 22 onward. The P3-S.17 audit's "sweep_event_
|
|
onsets = 84" is itself a THIRD counting: distinct onset events
|
|
over 194,985 bars (the last-write onset), not active bars and
|
|
not this port's onset rate.
|
|
```
|
|
|
|
Consequences for the reference population:
|
|
|
|
```text
|
|
- ALL THREE interpretations diverge from one another:
|
|
84 events (P3-S.17 audit, 194,985 bars) [~0.04% / bar]
|
|
1,896 active bars (legacy proxy latch, 2,000 bars) [94.8%]
|
|
58 events (faithful P3-S.2 port, 2,000 bars) [2.9%]
|
|
- This is exactly the "detection fidelity gap" flagged as P3-S.17 BLOCKER 1.
|
|
- The research side has NOT yet reproduced the frozen runtime's sweep semantics
|
|
on the real-data scope because the exact detector + EVENT-convention has not
|
|
been byte-locked on the runtime's emission. This is the earliest unresolved
|
|
divergence that prevents a trustworthy full-population verdict.
|
|
```
|
|
|
|
Resolution path (NOT done this session — see §V): fix and byte-lock the
|
|
research sweep EVENT port to the frozen runtime emission (DetectLiquidityGrabs
|
|
+ EVENT valid-until W_sweep=40) and match onset bar, direction, valid-until,
|
|
and supersession on real data before re-running the chain.
|
|
|
|
---
|
|
|
|
## I. Sweep parity
|
|
|
|
The frozen semantics (P3-S.2/P3-S.11) are implemented as a faithful research
|
|
port in `ml/p3/setup_dataset/spec_tests_runtime_research_parity.py`
|
|
(`sweep_internal_pivots` + `sweep_events`). It uses:
|
|
- internal swing pivot (fractal len=5, closed bars)
|
|
- wick penetration + same-bar close-back (GRAB_WINDOW=8)
|
|
- W_sweep = 40 validity
|
|
- direction (+1 sell-side swept / -1 buy-side swept)
|
|
- M15, closed-bar
|
|
No simplified proxy is used in the port. However the FIRST DIVERGENCE is at
|
|
this stage — the pre-existing research side (P3-S.17) had a persistent-latch
|
|
sweep state that is not a faithful EVENT. This is the blocking divergence (§H).
|
|
The runtime detector is NOT modified.
|
|
|
|
---
|
|
|
|
## J. CHoCH parity
|
|
|
|
A faithful port (`choch_events`, swing fractal len=50 + close-confirmed break
|
|
of the newest swing pivot) is implemented. At the synthetic level the F3 chain
|
|
accepts CHoCH-after-sweep within the chain bound (PAR-T06/T07 green). On the
|
|
subset it produced 289 CHoCH bars. No external MSS/displacement theory beyond
|
|
the frozen spec was introduced.
|
|
|
|
---
|
|
|
|
## K. OB parity (P3-S.5/P3-S.12 canonical)
|
|
|
|
```text
|
|
Formation : opposite-color candle B before strong move |body(M)|>=1.5x avg(20)
|
|
Zone : full range [Low(B), High(B)]
|
|
Mitigation: close-through full fill (active skip)
|
|
```
|
|
Implemented (`ob_formation_at`) and asserted at synthetic level (PAR-T08/T09).
|
|
On the subset: 1,981 active-zone bars / 2,000. Not restored to legacy BOS/CHoCH
|
|
OB. Runtime unchanged.
|
|
|
|
---
|
|
|
|
## L. FVG parity (P3-S.4 / SMC_FVG SPEC v1) + namespace resolution
|
|
|
|
```text
|
|
Formation : 3-candle wick geometry; newest eligible C3 (no 1-bar lag)
|
|
Mitigation: wick full-fill (bull Low<=bot / bear High>=top); partial stays active
|
|
```
|
|
Implemented (`fvg_formation_at`), asserted (PAR-T10/T11). On the subset:
|
|
2,000 active-zone bars / 2,000 (FVG zones rarely fully filled — consistent with
|
|
the persistent-zone semantics S-8/S-9).
|
|
|
|
NAMESPACE/parity-absence (P3-S.17 BLOCKER 2) resolution in THIS session:
|
|
```text
|
|
- The F3 Candidate Setup layer legitimately supports zone_type == FVG
|
|
(AF_Engine2_Setup.mqh AF_SETUP_ZONE_FVG; SNIPERGOLD_CANONICAL §C/§D).
|
|
- This is SETUP-LEVEL research metadata, DISTINCT from the legacy 19-feature
|
|
ML contract (FEATURE_CONTRACT.md f0-f18 is FVG-free by design —
|
|
docs/SMC_FVG_SPEC_v1.md §13).
|
|
- The P3-S.4 parity-absence guard (scan_parity_absence in
|
|
ml/p3/smc_semantic/spec_tests_fvg.py) skips files named spec_tests_*.py.
|
|
The new parity file spec_tests_runtime_research_parity.py is such a file,
|
|
so referencing FVG there does NOT flip P3-S.4 FVG-T20.
|
|
- VERIFIED: P3-S.4 re-run = 20/20 (FVG-T20 "N/A by absence" still green).
|
|
```
|
|
Thus the constraint "FVG absent from FEATURE_CONTRACT" is preserved, while FVG
|
|
is permitted as a setup-level zone-type in the research/spec-test namespace.
|
|
FEATURE_CONTRACT.md was NOT modified.
|
|
|
|
---
|
|
|
|
## M. M15 entry parity & full-population status
|
|
|
|
The M15 entry condition is a consumed input (E-rule / premium-discount /
|
|
displacement boost per frozen Candidate Setup semantics). The F3 layer asserts
|
|
M15-direction parity (PAR-T12). On the deterministic subset the neutral-gate
|
|
default (no direction) means the chain correctly does not form.
|
|
|
|
Honest full-population note: a faithful Candidate Setup population over the
|
|
authorized scope requires the runtime's ACTUAL consumed inputs — the H4/M30
|
|
context STATE series and the M15 entry-condition series, both as-of closed bars
|
|
(time-aligned by the MTF as-of contract §L). Those gate/entry series were NOT
|
|
deterministically wired from a runtime dump in this pass, so the final setup
|
|
count cannot yet be truthfully reported. This is the remaining work (VERDICT).
|
|
|
|
---
|
|
|
|
## N. Real-data parity result (subset, deterministic)
|
|
|
|
```text
|
|
runtime_created (subset, neutral gates) : 0
|
|
research_created (subset, neutral gates): 0
|
|
identity match : n/a (empty) — both 0 on neutral gates
|
|
parity_tests : 17/17 PASS (PAR-T02..T18)
|
|
Stage 1 data identity : PASS (194,985 bars == P3-S.17)
|
|
Primitive counts on 2000-bar subset : sweep=58, choch=289,
|
|
ob_active=1981, fvg_active=2000
|
|
```
|
|
|
|
---
|
|
|
|
## O. Regression (after research-only changes)
|
|
|
|
```text
|
|
P3-S.16 setup_dataset : 20/20 PASS (re-run; historical expectations unchanged)
|
|
P3-S.13 F3 runtime : 25/25 PASS (re-run)
|
|
P3-S.4 FVG : 20/20 PASS (FVG-T20 still green — namespace boundary OK)
|
|
(Historical outputs restored; only new P3-S.17R artifacts added. No historical
|
|
suite weakened or modified.)
|
|
```
|
|
|
|
---
|
|
|
|
## P. Research-only files changed / added
|
|
|
|
```text
|
|
ADDED ml/p3/setup_dataset/spec_tests_runtime_research_parity.py
|
|
ADDED ml/p3/setup_dataset/output/p3_s17r_runtime_research_parity.json
|
|
ADDED ml/p3/setup_dataset/output/p3_s17r_stage_counts.json
|
|
GEN ml/p3/setup_dataset/output/p3_s17r_setup_diff.csv (gitignored *.csv)
|
|
ADDED docs/P3_S17R_RUNTIME_RESEARCH_PARITY.md (this file)
|
|
ADDED docs/SESSION_HANDOVER_<date>_P3_S17R_RUNTIME_RESEARCH_PARITY.md
|
|
```
|
|
|
|
## Q. Production files unchanged
|
|
|
|
```text
|
|
MQL5 production detectors, AF_Engine2_Setup.mqh, F1/F2/F3/F4 runtime,
|
|
FEATURE_CONTRACT.md, SniperGold_ML.mqh/, weights, legacy MLP, model artifacts:
|
|
NONE modified. Runtime is the PARITY REFERENCE, never altered.
|
|
```
|
|
|
|
## R. Tickstory / Dukascopy deferred study
|
|
|
|
Not used as the baseline. Documented as a future independent data-source
|
|
robustness study:
|
|
```text
|
|
Dukascopy/Tickstory raw ticks -> canonical M3/M15/M30/H4 -> independent
|
|
Candidate Setup reconstruction -> compare against HFM/MT5 runtime reference.
|
|
```
|
|
This is a future validation phase, not the current parity baseline.
|
|
|
|
---
|
|
|
|
## S. First divergence — staging table (per §16)
|
|
|
|
| # | Stage | Classification | Status |
|
|
|---|-------|----------------|--------|
|
|
| 1 | Data identity | TIMESTAMP | PARITY (PASS) |
|
|
| 2 | H4 gate | STATE | PARITY (PASS, F3) |
|
|
| 3 | M30 gate | STATE | PARITY (PASS, F3) |
|
|
| 4 | Sweep | **STATE/EVENT-CONVENTION** | **DIVERGENCE (blocking)** |
|
|
| 5 | CHoCH | EVENT | PARTIAL (synthetic OK; real-data not byte-locked) |
|
|
| 6 | OB | ZONE | PARTIAL (synthetic OK) |
|
|
| 7 | FVG | ZONE/NAMESPACE | PARTIAL (synthetic OK; namespace resolved) |
|
|
| 8 | M15 entry | ENTRY/CONDITION | PARTIAL (input not time-aligned) |
|
|
| 9 | Candidate Setup | SETUP | PARTIAL (empty with neutral gates; not yet faithful full scope) |
|
|
|
|
## T. Forge commits
|
|
|
|
```text
|
|
TBD (logical commits; see handover)
|
|
```
|
|
|
|
## U. Final Forge HEAD
|
|
|
|
```text
|
|
TBD after commit/push
|
|
```
|
|
|
|
## V. Final parity verdict
|
|
|
|
```text
|
|
PARITY BLOCKED
|
|
```
|
|
|
|
Rationale (matches prompt §33 definition):
|
|
```text
|
|
- The frozen F3 SETUP-LAYER semantics (lifecycle, identity, gates, dedup,
|
|
mutation-lock, zero-population consistency) ARE reproduced deterministically:
|
|
PAR-T01..T18 all pass against the canonical oracle (spec_oracle).
|
|
- Stage 1 (data identity) equals the P3-S.17 baseline.
|
|
- BUT the real-data Candidate Setup FULL POPULATION could not yet be
|
|
reconstructed faithfully because two DIFFERENT and inconsistent sweep
|
|
conventions remain (P3-S.17 proxy latch ≈ 95% active vs faithful P3-S.2
|
|
EVENT ≈ 2.9% vs audit 84 events) — the earliest unresolved divergence —
|
|
and the H4/M30/M15 gate series that the F3 layer consumes were not
|
|
time-aligned from a runtime dump in this pass.
|
|
- A divergence at the FIRST primitive stage is unresolved => a trustworthy
|
|
conclusion on runtime==research (zero OR non-zero) is not yet possible.
|
|
```
|
|
This is NOT ZERO POPULATION CONFIRMED (the 0 observed here is with neutral
|
|
gates, not the faithful full-scope population), and NOT PARITY ACHIEVED.
|
|
|
|
## W. P3-S.18 readiness
|
|
|
|
```text
|
|
NOT READY. No training may begin. Before any P3-S.18 baseline, a follow-on phase
|
|
must:
|
|
1. Byte-lock the research F1 sweep/CHoCH EVENT ports to the frozen MQL5
|
|
runtime emission (onset, dir, valid_until, supersession) on the authorized
|
|
scope (fixing the §H first divergence).
|
|
2. Wire the runtime's H4/M30/M15 consumed gate series, time-aligned per the
|
|
MTF as-of contract (§L), so the F3 chain can be run to a faithful
|
|
non-empty population (or proven genuinely zero).
|
|
3. Re-run this parity suite + the P3-S.17 audit on the reconciled population.
|
|
```
|
|
Training and parameter optimization remain OUT OF SCOPE (frozen policy).
|
|
|
|
---
|
|
|
|
*End P3-S.17R runtime↔research parity report. Verdict: PARITY BLOCKED.* |