SniperGold_ML/docs/P3_S17R_RUNTIME_RESEARCH_PARITY.md

# P3-S.17R RUNTIME ↔ RESEARCH CANDIDATE SETUP PARITY

```text
Date       : 2026-08-23
Session    : P3-S.17R — Runtime <-> Research Candidate Setup Parity
Baseline   : P3-S.17 checkpoint ed01af24644fbac26fec9fe6ff6a3179920a61d6
             VERIFIED (origin = forge.mql5.io/chiki2bum2/SniperGold_ML.git,
             branch main, working tree CLEAN).
Scope      : XAUUSD M15 2017-01-01..2026-08-21 (Files/AlgoForge/Data); the
             P3-S.17 authorized scope = 194,985 bars (2017-01-03..2026-07-21).
Human verif: REMAINS CANCELLED (historical only). NO AUC/PF/backtest/opt.
Verdict    : PARITY BLOCKED (see §V). A trust-worthy full-population parity
             could not be concluded because the F1 sweep primitive and the
             H4/M30/M15 gate series are not yet faithfully reproducibly
             wired on the real-data scope. The frozen F3 SETUP-LAYER semantics
             IS reproduced (PAR-T01..T18 green); the DETECTOR/gate primitives
             that feed it are the unresolved divergence.
```

---

## A. Objective (single question)

> Does the research-side Candidate Setup reconstruction reproduce the FROZEN
> MQL5 Candidate Setup decisions on the SAME historical input, stage by stage,
> WITHOUT changing runtime semantics?

Answer so far: **partial** — the frozen F3 setup-layer lifecycle/identity/
gate logic is reproduced exactly (17/17 PAR-T on the frozen oracle), and Stage 1
data identity passes, but the real-data **F1 sweep primitive** and the
consumed **H4/M30/M15 gate series** are NOT yet shown faithful, so the full
Candidate Setup population cannot yet be concluded with a trustworthy verdict.

---

## B. Source-of-truth hierarchy used

```text
1. Frozen spec contracts : SNIPERGOLD_CANONICAL_SETUP_CONTRACT_v1,
                           SMC_LIQUIDITY_SWEEP_SPEC_v1, SMC_CHOCH_MSS_SPEC_v1,
                           SMC_ORDER_BLOCK_SPEC_v1, SMC_FVG_SPEC_v1,
                           P3_S11_EVENT_CONTRACT_REPAIR, P3_S12_ZONE_CONTRACT_REPAIR.
2. Frozen MQL5 runtime   : AF_Engine2_Setup.mqh  (F3 layer)
                           AF_Engine2_Agents.mqh (F1/F2 detectors)
                           AF_Defines.mqh        (constants: W_sweep/W_choch=40)
3. Verified regression   : P3-S.16 (20/20), P3-S.13 F3 (25/25), P3-S.4 FVG (20/20).
4. Existing F3 oracle    : ml/p3/smc_semantic/spec_tests_candidate_setup_runtime.py
                           canonical_oracle + F3SetupEngine.
5. Research extractor    : P3-S.17 proxy (NOT merged) + this session's faithful
                           research ports.
6. New assumptions       : avoided (per §35 STOP conditions).
```

The research extractor is NOT authoritative. Where research differs from frozen
runtime, research is repaired — never the runtime.

---

## C. Checkpoint & workspace

```text
Workspace : ...\Shared Projects\SniperGold_ML_authoritative  (authoritative clone)
origin    : https://forge.mql5.io/chiki2bum2/SniperGold_ML.git
branch    : main
local HEAD: ed01af24644fbac26fec9fe6ff6a3179920a61d6
remote HEAD: ed01af24644fbac26fec9fe6ff6a3179920a61d6
working tree: CLEAN before parity work
```

---

## D. Latest SESSION_HANDOVER used

`docs/SESSION_HANDOVER_2026-08-23_P3_S17_DATASET_AUDIT.md` (read at session
start). It records DATASET BLOCKED, 0 raw setups, next gate = runtime↔research
reconciliation (P3-S.17R).

---

## E. Data source

```text
Symbol      : XAUUSD
Timeframe   : M15 (entry/setup carrier); H4 (gate); M30 (gate); H1/D1 (context)
Source      : Files/AlgoForge/Data (XAUUSD_M15.npz, H1/H4/D1) — SAME source as
              the P3-S.17 runtime-parity baseline.
Scope       : 2017-01-01..2026-08-21 npz (197,032 bars after 2017 filter);
              P3-S.17 authorized scope (194,985 bars, 2017-01-03..2026-07-21)
              reproduced exactly in the Stage-1 identity check.
Feed quality: download_report 0 dup / 0 bad OHLC / 0 anomaly gaps (P3-S.17 §E).
No Tickstory / Dukascopy used as the baseline (deferred study, §Q).
```

---

## F. Stage 1 — data / timestamp identity (PAR-T01)

```text
n_bars_total        : 197032  (M15, 2017+ npz)
n_bars_in_scope     : 194985  (2017-01-03 .. 2026-07-21) == P3-S.17 recorded
timestamps_monotonic: True
t_start 2017-01-03 00:00Z   t_end 2026-07-21 23:45Z
h4_bars 16234 / h1_bars ~frames / d1_bars available
=> Stage 1 PASS. The research pipeline reads the identical M15 OHLC/timestamps
   used for the P3-S.17 runtime-parity baseline.
```

---

## G. Stage-by-stage parity matrix (real-data 2000-bar deterministic subset)

Golden-trace subset = the first 2000 closed M15 bars (2017), a MANAGEABLE
deterministic window (prompt §15/§13). Full-scope detection requires vectorized
detector kernels (documented as a future step; not a parity gate itself).

| Stage | Runtime (frozen) | Research port | Match | Notes / first divergence |
|---|---|---|---|---|
| H4 gate | (consumed context STATE) | gate input | ✓ (F3) | F3 layer consumes h4; PAR-T02 green |
| M30 gate | (consumed) | gate input | ✓ (F3) | PAR-T03 green |
| Sweep event | P3-S.2/P3-S.11 `DetectLiquidityGrabs`+W=40 | faithful port (this session) | ✗ vs legacy proxy | **FIRST DIVERGENCE** (see §H) |
| CHoCH event | P3-S.3/P3-S.11 | faithful port | ✓ (subset) | PAR-T06/T07 green; choch=289/2000 |
| OB zone | P3-S.5/P3-S.12 canonical | faithful port | ✓ (synthetic) | PAR-T08/T09; ob_active=1981/2000 |
| FVG zone | P3-S.4/SMC_FVG v1 | faithful port | ✓ (synthetic) | PAR-T10/T11; namespace boundary OK |
| M15 entry | (consumed E-rule) | gate input | ✓ (F3) | PAR-T12 green |
| Candidate Setup | F3 `AF_Engine2_Setup` | F3 oracle `canonical_oracle` | ✓ (setup layer) | PAR-T13..T18 green; 17/17 |

On the deterministic subset with NEUTRAL gates (all-zero H4/M30/M15), the F3
chain cannot form (context gate requires non-zero direction-compatible H4/M30),
so **runtime_created == research_created == 0**. That specific 0 is behavioural
(with neutral gates) and is NOT yet the faithful full-scope population — it is
reported honestly and the real population requires the runtime's actual gated
inputs (§M).

---

## H. FIRST DIVERGENCE — Stage 4 (Sweep event primitive)

Measured on the identical 2000-bar deterministic M15 subset:

```text
frozen_policy_sweep onsets (this port, P3-S.2 semantics, critical-bar):
       58 occurrences / 2000 bars  (= 2.9% onset rate)
legacy proxy `smc_semantic_common.sweep_state` active-bars:
       1896 / 2000 bars           (= 94.8% of bars marked "sweep bar >=0")

first divergence bar : 22   (frozen=-1, legacy=0)
bars frozen-only-active: 723
bars legacy-only-active : 0
```

Interpretation (P3-S.17R §16 classification):

```text
CLASS      : FIRST-DIVERGENCE / STATE-CONVENTION
ROOT CAUSE : the P3-S.17 extractor's proxy sweep used the persistent
             `smc_semantic_common.sweep_state` whose `sweep_bar[]` is a
             persistent last-onset latch that once set stays set for the
             REST of the window (94.8% of bars active). A faithful P3-S.2
             EVENT interpretation marks only 2.9% of bars as carrying an
             active sweep EVENT (fresh onset within W_sweep=40). The two
             disagree from bar 22 onward. The P3-S.17 audit's "sweep_event_
             onsets = 84" is itself a THIRD counting: distinct onset events
             over 194,985 bars (the last-write onset), not active bars and
             not this port's onset rate.
```

Consequences for the reference population:

```text
- ALL THREE interpretations diverge from one another:
      84 events  (P3-S.17 audit, 194,985 bars)  [~0.04% / bar]
      1,896 active bars (legacy proxy latch, 2,000 bars) [94.8%]
      58 events (faithful P3-S.2 port, 2,000 bars) [2.9%]
- This is exactly the "detection fidelity gap" flagged as P3-S.17 BLOCKER 1.
- The research side has NOT yet reproduced the frozen runtime's sweep semantics
  on the real-data scope because the exact detector + EVENT-convention has not
  been byte-locked on the runtime's emission. This is the earliest unresolved
  divergence that prevents a trustworthy full-population verdict.
```

Resolution path (NOT done this session — see §V): fix and byte-lock the
research sweep EVENT port to the frozen runtime emission (DetectLiquidityGrabs
+ EVENT valid-until W_sweep=40) and match onset bar, direction, valid-until,
and supersession on real data before re-running the chain.

---

## I. Sweep parity

The frozen semantics (P3-S.2/P3-S.11) are implemented as a faithful research
port in `ml/p3/setup_dataset/spec_tests_runtime_research_parity.py`
(`sweep_internal_pivots` + `sweep_events`). It uses:
- internal swing pivot (fractal len=5, closed bars)
- wick penetration + same-bar close-back (GRAB_WINDOW=8)
- W_sweep = 40 validity
- direction (+1 sell-side swept / -1 buy-side swept)
- M15, closed-bar
No simplified proxy is used in the port. However the FIRST DIVERGENCE is at
this stage — the pre-existing research side (P3-S.17) had a persistent-latch
sweep state that is not a faithful EVENT. This is the blocking divergence (§H).
The runtime detector is NOT modified.

---

## J. CHoCH parity

A faithful port (`choch_events`, swing fractal len=50 + close-confirmed break
of the newest swing pivot) is implemented. At the synthetic level the F3 chain
accepts CHoCH-after-sweep within the chain bound (PAR-T06/T07 green). On the
subset it produced 289 CHoCH bars. No external MSS/displacement theory beyond
the frozen spec was introduced.

---

## K. OB parity (P3-S.5/P3-S.12 canonical)

```text
Formation : opposite-color candle B before strong move |body(M)|>=1.5x avg(20)
Zone      : full range [Low(B), High(B)]
Mitigation: close-through full fill (active skip)
```
Implemented (`ob_formation_at`) and asserted at synthetic level (PAR-T08/T09).
On the subset: 1,981 active-zone bars / 2,000. Not restored to legacy BOS/CHoCH
OB. Runtime unchanged.

---

## L. FVG parity (P3-S.4 / SMC_FVG SPEC v1) + namespace resolution

```text
Formation : 3-candle wick geometry; newest eligible C3 (no 1-bar lag)
Mitigation: wick full-fill (bull Low<=bot / bear High>=top); partial stays active
```
Implemented (`fvg_formation_at`), asserted (PAR-T10/T11). On the subset:
2,000 active-zone bars / 2,000 (FVG zones rarely fully filled — consistent with
the persistent-zone semantics S-8/S-9).

NAMESPACE/parity-absence (P3-S.17 BLOCKER 2) resolution in THIS session:
```text
- The F3 Candidate Setup layer legitimately supports zone_type == FVG
  (AF_Engine2_Setup.mqh AF_SETUP_ZONE_FVG; SNIPERGOLD_CANONICAL §C/§D).
- This is SETUP-LEVEL research metadata, DISTINCT from the legacy 19-feature
  ML contract (FEATURE_CONTRACT.md f0-f18 is FVG-free by design —
  docs/SMC_FVG_SPEC_v1.md §13).
- The P3-S.4 parity-absence guard (scan_parity_absence in
  ml/p3/smc_semantic/spec_tests_fvg.py) skips files named spec_tests_*.py.
  The new parity file spec_tests_runtime_research_parity.py is such a file,
  so referencing FVG there does NOT flip P3-S.4 FVG-T20.
- VERIFIED: P3-S.4 re-run = 20/20 (FVG-T20 "N/A by absence" still green).
```
Thus the constraint "FVG absent from FEATURE_CONTRACT" is preserved, while FVG
is permitted as a setup-level zone-type in the research/spec-test namespace.
FEATURE_CONTRACT.md was NOT modified.

---

## M. M15 entry parity & full-population status

The M15 entry condition is a consumed input (E-rule / premium-discount /
displacement boost per frozen Candidate Setup semantics). The F3 layer asserts
M15-direction parity (PAR-T12). On the deterministic subset the neutral-gate
default (no direction) means the chain correctly does not form.

Honest full-population note: a faithful Candidate Setup population over the
authorized scope requires the runtime's ACTUAL consumed inputs — the H4/M30
context STATE series and the M15 entry-condition series, both as-of closed bars
(time-aligned by the MTF as-of contract §L). Those gate/entry series were NOT
deterministically wired from a runtime dump in this pass, so the final setup
count cannot yet be truthfully reported. This is the remaining work (VERDICT).

---

## N. Real-data parity result (subset, deterministic)

```text
runtime_created (subset, neutral gates) : 0
research_created (subset, neutral gates): 0
identity match                          : n/a (empty) — both 0 on neutral gates
parity_tests                            : 17/17 PASS (PAR-T02..T18)
Stage 1 data identity                   : PASS (194,985 bars == P3-S.17)
Primitive counts on 2000-bar subset     : sweep=58, choch=289,
                                          ob_active=1981, fvg_active=2000
```

---

## O. Regression (after research-only changes)

```text
P3-S.16 setup_dataset : 20/20 PASS  (re-run; historical expectations unchanged)
P3-S.13 F3 runtime    : 25/25 PASS  (re-run)
P3-S.4  FVG           : 20/20 PASS  (FVG-T20 still green — namespace boundary OK)
(Historical outputs restored; only new P3-S.17R artifacts added. No historical
 suite weakened or modified.)
```

---

## P. Research-only files changed / added

```text
ADDED ml/p3/setup_dataset/spec_tests_runtime_research_parity.py
ADDED ml/p3/setup_dataset/output/p3_s17r_runtime_research_parity.json
ADDED ml/p3/setup_dataset/output/p3_s17r_stage_counts.json
GEN   ml/p3/setup_dataset/output/p3_s17r_setup_diff.csv   (gitignored *.csv)
ADDED docs/P3_S17R_RUNTIME_RESEARCH_PARITY.md             (this file)
ADDED docs/SESSION_HANDOVER_<date>_P3_S17R_RUNTIME_RESEARCH_PARITY.md
```

## Q. Production files unchanged

```text
MQL5 production detectors, AF_Engine2_Setup.mqh, F1/F2/F3/F4 runtime,
FEATURE_CONTRACT.md, SniperGold_ML.mqh/, weights, legacy MLP, model artifacts:
NONE modified. Runtime is the PARITY REFERENCE, never altered.
```

## R. Tickstory / Dukascopy deferred study

Not used as the baseline. Documented as a future independent data-source
robustness study:
```text
Dukascopy/Tickstory raw ticks -> canonical M3/M15/M30/H4 -> independent
Candidate Setup reconstruction -> compare against HFM/MT5 runtime reference.
```
This is a future validation phase, not the current parity baseline.

---

## S. First divergence — staging table (per §16)

| # | Stage | Classification | Status |
|---|-------|----------------|--------|
| 1 | Data identity | TIMESTAMP | PARITY (PASS) |
| 2 | H4 gate | STATE | PARITY (PASS, F3) |
| 3 | M30 gate | STATE | PARITY (PASS, F3) |
| 4 | Sweep | **STATE/EVENT-CONVENTION** | **DIVERGENCE (blocking)** |
| 5 | CHoCH | EVENT | PARTIAL (synthetic OK; real-data not byte-locked) |
| 6 | OB | ZONE | PARTIAL (synthetic OK) |
| 7 | FVG | ZONE/NAMESPACE | PARTIAL (synthetic OK; namespace resolved) |
| 8 | M15 entry | ENTRY/CONDITION | PARTIAL (input not time-aligned) |
| 9 | Candidate Setup | SETUP | PARTIAL (empty with neutral gates; not yet faithful full scope) |

## T. Forge commits

```text
TBD (logical commits; see handover)
```

## U. Final Forge HEAD

```text
TBD after commit/push
```

## V. Final parity verdict

```text
PARITY BLOCKED
```

Rationale (matches prompt §33 definition):
```text
- The frozen F3 SETUP-LAYER semantics (lifecycle, identity, gates, dedup,
  mutation-lock, zero-population consistency) ARE reproduced deterministically:
  PAR-T01..T18 all pass against the canonical oracle (spec_oracle).
- Stage 1 (data identity) equals the P3-S.17 baseline.
- BUT the real-data Candidate Setup FULL POPULATION could not yet be
  reconstructed faithfully because two DIFFERENT and inconsistent sweep
  conventions remain (P3-S.17 proxy latch ≈ 95% active vs faithful P3-S.2
  EVENT ≈ 2.9% vs audit 84 events) — the earliest unresolved divergence —
  and the H4/M30/M15 gate series that the F3 layer consumes were not
  time-aligned from a runtime dump in this pass.
- A divergence at the FIRST primitive stage is unresolved => a trustworthy
  conclusion on runtime==research (zero OR non-zero) is not yet possible.
```
This is NOT ZERO POPULATION CONFIRMED (the 0 observed here is with neutral
gates, not the faithful full-scope population), and NOT PARITY ACHIEVED.

## W. P3-S.18 readiness

```text
NOT READY. No training may begin. Before any P3-S.18 baseline, a follow-on phase
must:
  1. Byte-lock the research F1 sweep/CHoCH EVENT ports to the frozen MQL5
     runtime emission (onset, dir, valid_until, supersession) on the authorized
     scope (fixing the §H first divergence).
  2. Wire the runtime's H4/M30/M15 consumed gate series, time-aligned per the
     MTF as-of contract (§L), so the F3 chain can be run to a faithful
     non-empty population (or proven genuinely zero).
  3. Re-run this parity suite + the P3-S.17 audit on the reconciled population.
```
Training and parameter optimization remain OUT OF SCOPE (frozen policy).

---

*End P3-S.17R runtime↔research parity report. Verdict: PARITY BLOCKED.*