ARIMA_SAX_Hybrid_Forecaster/results/R2-H/R2_H_PROTOCOL.md

417 lines
18 KiB
Markdown

# R2-H — HORIZON-BOUNDARY EXPERIMENT (FROZEN PROTOCOL)
**Experiment:** `R2-H`
**Protocol version:** `r2-h-v0.1.0-frozen`
**Status:** **DESIGN FROZEN — EXECUTION NOT AUTHORIZED.**
**Branch:** `main`
**Anchor commit (R1-R, immutable):** `67be79aa75ae98eace7c67b74c93797f4327d51a`
**Freeze session:** governance-controlled formal design-freeze (2026-08-26)
> This document is a formal, self-contained, frozen protocol. It defines the
> R2-H research family design. It freezes origin lists, model policy, cost
> model, metrics, inference, multiplicity control, no-lookahead gates, stop
> conditions, and the outcome matrix. **No forecasting model is executed, no
> performance is computed, and no prediction/evidence artifact is produced in
> this session.** Execution requires a separate, explicit EXECUTION
> AUTHORIZATION gate.
---
## A. Governance status
- Owner decision: **FORMAL R2-H FREEZE AUTHORIZED** (2026-08-26).
- Scope authorized: checkpoint identity resolution; protocol authoring;
protocol consistency checks; creation of R2-H design artifacts; committing
the frozen protocol and associated governance documentation.
- Explicitly NOT authorized: experiment execution; model fitting; forecasting;
backtesting; parameter optimization; statistical inference from newly
generated experimental results; economic evaluation; creation of
prediction/evidence artifacts.
- The repository checkpoint discrepancy is resolved: authoritative HEAD is
`67be79aa75ae98eace7c67b74c93797f4327d51a` (commit `research: finalize R1-R
robustness synthesis`). All other quoted hashes are transcription errors and
are invalid checkpoints.
- This session creates ONLY the formal R2-H design artifacts necessary for the
freeze (origin generator, frozen origin lists, protocol, manifest). No other
repository content is modified.
## B. R1-R anchor
R1-R is a **CLOSED and IMMUTABLE scientific anchor**. Within its defined scope
it established, on XAUUSDc M1/M5/M15:
- approximately 15-minute elapsed directional forecasting on XAUUSD;
- no useful gross directional signal;
- direction accuracy approximately 0.48–0.51;
- no non-naive model beat the naive baseline on the tested target;
- gross return approximately zero and mixed-sign;
- economics decisively negative (cost ≈ 0.8–3.4 ATR/trade vs gross ≈ 0–0.15
ATR/trade);
- no-lookahead, forensic, tick-validation, and reproducibility gates passed;
- deterministic origin selection repaired the historical R1-F.8 provenance
problem.
R2-H does **not** reopen, retune, reinterpret, or amend R1-R. R1-R evidence,
protocol, origin lists, predictions, and manifests remain untouched.
## C. Research question
> **Does the R1-R approximately 15-minute directional null persist, weaken, or
> invert at materially longer elapsed prediction horizons on XAUUSD when
> transaction-cost scaling is explicitly accounted for?**
The experiment is **not** framed as a search for a positive result. Valid
outcomes are: **NULL**, **BOUNDARY**, **PARTIAL / ECONOMICALLY NOT VIABLE**,
**INCONCLUSIVE**, **INVALID**.
## D. Hypotheses
- **Global null (FWER family {60, 240}):** At the 60-minute and 240-minute
elapsed horizons, the Drift model exhibits no gross directional signal
(direction accuracy ≤ 0.50), no positive gross return per trade, and no error
improvement over the naive baseline.
- **Global alternative:** At least one member of the FWER family exhibits a
gross directional signal that satisfies all three Q1 conditions (A, B, C).
- **Economic hypothesis:** Even if a gross signal exists, the net return under
the frozen 7.57 bps round-trip cost has a dependence-aware 95% CI whose lower
bound is ≤ 0 (Q2 fails → PARTIAL / ECONOMICALLY NOT VIABLE).
- The 15-minute H0 horizon is an **in-study replication/control** of R1-R and
provides **no independent evidence** for the new horizon-boundary claim.
## E. Horizon grid
| label | horizon (minutes) | base grid | role | FWER family |
|---|---|---|---|---|
| H0 | 15 | M1 | in-study R1-R control | **outside** (no independent claim) |
| H1 | 60 | M1 | primary | **in family** |
| H2 | 240 | M1 | primary | **in family** |
| H9 | 960 | M1 | **diagnostic-only** | **never** enters family |
Rules:
- H0/H1/H2 are the ONLY horizons in the primary multiplicity-controlled family.
- 960 minutes is DIAGNOSTIC-ONLY. It must NOT enter the FWER family, determine
success/failure, qualify as a boundary, justify reopening, or be adaptively
promoted after results are observed.
- No horizon may be added, removed, or substituted.
## F. Target definition
All horizons use the **M1 base grid**. The horizon is defined in **ELAPSED
TIME**, not bar count.
```
tau_H(o) = [ P(t_end) - P(o) ] / ATR20(o)
t_end = first valid M1 bar timestamp >= timestamp(o) + H minutes
P(x) = close price of M1 bar x
ATR20(o) = R1-compatible M1 close-only ATR(20) at origin o
```
- `ATR20` is the **deterministic close-change-based ATR proxy** retained for
cross-family comparability: simple rolling mean of |close[i] − close[i−1]|
over the trailing 20 M1 bars (see `src/forecasting/target.py`,
`rolling_atr` with high/low absent). It is **NOT** a Wilder ATR. This label
is frozen; no silent ATR methodology change is permitted.
## G. Data provenance
- Source: `results/R1_real_data/XAUUSDc_M1_raw.json` (MT5 terminal history,
broker symbol `XAUUSDc`, Gold Spot Cent; M1; 397,558 bars;
2025-06-02 01:00 → 2026-08-24 15:19 server trade time).
- **Fresh SHA-256 verification (R2-H provenance, computed in the freeze
session):**
```
file: results/R1_real_data/XAUUSDc_M1_raw.json
sha256: 5472357e67f70f7d1bfef8f9de67fcce91ee6f26208072d7a6766b06c158ee6e
```
- This hash matches the R1 manifest record (`R1_REAL_DATASET_MANIFEST.json`,
`dataset_hash_raw_json`) but is re-verified here as **new R2-H evidence**; it
is not silently reused. The R1-R close-only hash
`80e4b52b0df0c6e348ce2002348279eee9f6e8a61c3109efe82fe475ed3cbc19` remains
an R1-R property and is **not** re-used as R2-H evidence.
- The M1 file is used **directly**; no higher-timeframe aggregation is
performed in R2-H.
## H. Origin-selection rule (frozen, deterministic, model-independent)
Generator: `scripts/run_r2h_origins.py` (design artifact committed with this
protocol). Parameters (frozen):
| parameter | value |
|---|---|
| base grid | M1 |
| warmup | 2000 M1 bars (sufficient for ARIMA fit window 300, SAX window 24, drift lookback 40, ATR 20) |
| endpoint rule | `t_end` = first valid M1 bar timestamp ≥ timestamp(o) + H |
| gap tolerance | 5 minutes — any inter-bar delta > 5 min within (o, t_end] excludes the origin |
| origin spacing | 2 × H elapsed minutes between accepted origins |
| overlap | none (spacing 2×H with window H ⇒ non-overlapping target windows) |
| cap | none (no short-horizon cap applied) |
| selection | no human-selected origins; no model-dependent origins |
Gap/session policy (audited, frozen):
- An origin is usable **only** when the required elapsed-time target can be
resolved under the frozen gap rule.
- Any materially missing bar / gap that violates the frozen tolerance
(inter-bar delta > 5 minutes within (o, t_end]) **excludes** the origin.
- Weekend/session boundaries are **never** silently converted into ordinary
contiguous observations.
- All exclusions are deterministic and recorded (see manifest).
- No manual origin removal is permitted.
Generated origin lists (frozen, generated **twice**, byte-identical):
| file | horizon | origins | SHA-256 |
|---|---|---|---|
| `origins_H015.csv` | 15 min | 13,193 | `fb5a36e93c8f9c7b0fee638d35d551778f1a27cbc50e0727556cf6a1d8c5c7df` |
| `origins_H060.csv` | 60 min | 3,169 | `e7b5132c321f5350ec1e231d49d68dbc29468cdf9ee6a5014a2764aaf36eb9af` |
| `origins_H240.csv` | 240 min | 863 | `3ad65e94f46731e65b8d40cf2b7746a6c0bd2eafe43216178d9e6e6c6ebcb910` |
Two-run identity: the generator was run twice into separate directories; all
origin CSVs and the params manifest are **byte-identical** between the runs
(identical ordering, identical LF newline convention, identical contents,
identical SHA-256). Verified with `fc /b` and SHA-256. **Mismatch = HARD STOP**
— none occurred.
Recorded runtime parameters (per §9): warmup=2000; H ∈ {15, 60, 240}; step
(spacing)=2×H elapsed minutes; cap=none; gap rule tolerance=5 minutes; dataset
identity=`results/R1_real_data/XAUUSDc_M1_raw.json`
(sha256 `5472357e…`); software/config identity = generator file
`scripts/run_r2h_origins.py`, Python 3.14.6.
## I. Model policy
- **PRIMARY MODEL (frozen): Drift** — the sole model used for the primary
Q1/Q2 decision. No per-horizon "best model" selection exists.
- Secondary robustness/diagnostic models (never selected retrospectively as
"best"): ARIMA(1,0,0) (rolling fit window 300), SAX (word 8, alphabet 5,
window 24, top_k 10, min_analogs 5, neutral 0.25), Hybrid.
- **No model-shopping is permitted.** No model may be chosen based on observed
results. Secondary models cannot establish the primary boundary claim.
- Drift definition (existing, unchanged): linear slope over the last 40 closes
projected H bars ahead, normalized by ATR(20) (`src/baselines.py`,
`DriftBaseline`). Naive baseline = zero-return forecast
(`NaiveBaseline`), used as the error reference.
## J. Cost model (frozen, pre-registered)
- Headline round-trip cost: **7.57 bps**
- median tick spread ≈ **1.57 bps** (R1-R.4V tick validation, 103.3M ticks)
- commission assumption = **6.00 bps**
- total frozen cost = **7.57 bps**
- The 7.57 bps value is a **PRE-REGISTERED GLOBAL CONSTANT**. It is applied
identically to every origin and every primary horizon.
- Conversion to entry-ATR units:
```
cost_atr(o) = (7.57 / 10000) × close(o) / ATR20(o)
```
- Prohibitions: no future-looking or per-origin future-window spreads; no
spread derived from ticks after the origin; no cost tuning after results.
- Sensitivity: 0.5× and 2× cost multiples may be reported **descriptively
only**; they cannot alter the primary decision.
## K. Primary metrics
Primary directional metrics (computed on frozen origin lists, Drift model):
1. Direction accuracy (fraction of origins where the Drift direction sign
matches the realized sign of tau_H).
2. Mean gross return in ATR units (mean of sign-matched realized tau_H).
3. Mean net return in ATR units (mean gross − cost_atr).
4. Time-normalized / holding-period economic return (gross and net per unit of
elapsed holding time).
Secondary metrics (diagnostic only): MASE (vs naive); gross/cost ratio;
cost/range ratio; trade coverage; number of origins; effective sample size;
turnover / trade frequency; annualized or time-normalized economics;
chronological EARLY / MIDDLE / LATE descriptive results.
Reporting rules:
- Gross and net MUST always be reported separately.
- Longer horizons must NOT appear superior merely because they trade less
frequently — time-normalized economics are reported alongside per-trade
economics.
## L. Statistical inference (frozen)
- Method: **Moving Block Bootstrap (MBB)**.
- Block length: **B = 60** M1 observations.
- Bootstrap repetitions: **10,000**.
- Seed: **20260826**.
- Confidence level: **95%** (percentile CI).
- Effective sample size: **n_eff = floor(n_usable / B)**; the actual n and
n_eff are published for every horizon.
- Dependence-aware CIs are used for direction accuracy, gross return/trade,
net return/trade, and the error comparison vs naive.
- These parameters are fixed **before** execution. B, seed, and replication
count may not be modified after observing experimental results.
- A horizon that is statistically underpowered for the frozen decision rule is
classified **INCONCLUSIVE**. No significance is manufactured from a
low-n horizon.
Expected scale (audited, acknowledged, not a guarantee):
- H15: high usable-origin count (13,193 origins; n_eff ≈ 219).
- H60: moderate effective sample (3,169 origins; n_eff ≈ 52).
- H240: low effective sample (863 origins; n_eff ≈ 14).
H240 must not be claimed to have the same statistical power as H15; an
underpowered H240 result cannot establish a positive boundary. Power
limitations are reported as scientific results, not hidden caveats.
## M. FWER procedure (multiple-horizon inference)
- **Primary FWER family: {60, 240}.**
- H0 (15 min) is the in-study R1-R control and is **not** treated as
independent evidence for the new horizon-boundary FWER claim.
- Method: **joint cross-horizon block bootstrap** that preserves chronological
dependence **and** cross-horizon dependence (origins aligned on a common
M1/elapsed-time axis; blocks of 60 M1 observations resampled jointly across
horizons).
- Statistic: **max-statistic under a re-centered global null** (each horizon's
statistic re-centered to its null mean; the maximum over the family is the
test statistic). A simple independent per-horizon z-max approximation is
**not** used.
- Family-wise α = **0.05**.
- Per-horizon interpretation is permitted **only after** the global gate is
rejected.
- No horizon shopping; no post-hoc multiplicity changes.
## N. No-lookahead / forensic gates (frozen)
Gates F1–F9 (each must distinguish **OBSERVED / INFERRED / UNKNOWN /
DECISION**):
| gate | item |
|---|---|
| F1 | feature construction |
| F2 | target construction |
| F3 | ATR construction |
| F4 | SAX window construction |
| F5 | training data boundary |
| F6 | normalization |
| F7 | horizon endpoint determination |
| F8 | cost information |
| F9 | tick-derived information |
- Mutation testing is extended to **every primary horizon** (15, 60, 240).
Future-bar perturbation must not alter predictions at tested origins.
- Minimum spot checks: **≥ 3 origins per primary horizon**.
- Any no-lookahead failure = **INVALID / HARD STOP**.
- **DEFECT-001 SAX z-normalization invariant is preserved** (candidate windows
z-normalized before SAX encoding; regression test
`tests/test_sax_analog_zscore.py`). Any regression = **HARD STOP**.
## O. Economic analysis
- Q2 (economic survival) passes **only if** the dependence-aware 95% CI of net
return under the frozen 7.57 bps cost has lower bound **> 0**.
- A positive gross result that fails Q2 is **PARTIAL / ECONOMICALLY NOT
VIABLE**, not a successful boundary.
- Cost is applied at every origin identically; time-normalized economics are
reported to prevent horizon-frequency artifacts.
## P. Secondary diagnostics
- MASE (Drift vs naive) per horizon; gross/cost ratio; cost/range ratio; trade
coverage; n and n_eff per horizon; turnover/trade frequency; annualized or
time-normalized economics; EARLY / MIDDLE / LATE descriptive results.
- Secondary models (ARIMA, SAX, Hybrid) may be reported as robustness
diagnostics only; they cannot establish the primary boundary claim.
- 960-minute diagnostic results (if any) are reported **diagnostic-only** and
never enter the FWER family or the primary decision.
## Q. Stop conditions
Before committing the frozen protocol, STOP if any of the following occurs:
- authoritative HEAD mismatch;
- working tree not clean before freeze;
- origin/main mismatch;
- R2 namespace conflict;
- source dataset hash cannot be verified;
- protocol contradiction;
- unresolved statistical specification;
- unresolved cost specification;
- unresolved model specification;
- unresolved gap rule;
- unresolved multiplicity procedure;
- accidental R1-R modification;
- any attempt to execute the experiment;
- any attempt to generate forecasts/backtests;
- any attempt to tune parameters from results.
An unresolved scientific ambiguity is **not** "fixed" by improvisation — it is
returned to the owner.
## R. Outcome matrix
| outcome | definition |
|---|---|
| **NULL** | Global gate not rejected; no horizon in {60, 240} shows gross signal; consistent with R1-R null persisting. |
| **BOUNDARY** | Global gate rejected; ≥1 family horizon passes Q1 (A, B, C) AND Q2 (net CI lower bound > 0). |
| **PARTIAL / ECONOMICALLY NOT VIABLE** | Global gate rejected; ≥1 horizon passes Q1 but fails Q2. |
| **INCONCLUSIVE** | Underpowered horizon(s) prevent the frozen decision rule from resolving; or no-lookahead/specification ambiguity prevents classification. |
| **INVALID** | No-lookahead failure, DEFECT-001 regression, data provenance failure, or protocol violation → HARD STOP. |
Q1 conditions (Drift only, all required):
- **A.** Direction accuracy dependence-aware 95% CI lower bound > 0.50.
- **B.** Gross return/trade dependence-aware 95% CI lower bound > 0.
- **C.** Error vs naive satisfies the pre-registered error criterion: the
dependence-aware 95% CI of the paired error difference
(|e_naive| − |e_drift|) has lower bound > 0 (equivalently, Drift MASE < 1
with significance).
Q2 (economic survival): dependence-aware 95% CI of net return under frozen
7.57 bps has lower bound > 0.
## S. Reproducibility requirements
- A new R2-H manifest (`results/R2-H/R2_H_MANIFEST.json`) records: protocol;
configuration; source dataset; source hash; horizon definitions; origin
lists; origin hashes; code/config identity; statistical parameters; cost
parameters; seed; generated design artifacts.
- Origin lists are generated twice and verified byte-identical (see §H).
- All future execution artifacts (if later authorized) must be hash-linked to
this frozen protocol.
- R1-R evidence hashes are never reused as R2-H evidence.
## T. Execution authorization boundary
- This document freezes **design only**.
- **EXECUTION IS NOT AUTHORIZED** by this freeze. No model fitting,
forecasting, backtesting, parameter optimization, simulation, prediction,
metric computation from new experimental runs, or economic experiment may
begin until a separate **EXECUTION AUTHORIZATION** gate is granted by the
owner.
- The required next governance gate is **EXECUTION AUTHORIZATION**.
## U. Version / freeze metadata
| field | value |
|---|---|
| protocol id | `r2-h-v0.1.0-frozen` |
| status | DESIGN FROZEN — EXECUTION NOT AUTHORIZED |
| freeze date | 2026-08-26 |
| branch | main |
| R1-R anchor | `67be79aa75ae98eace7c67b74c93797f4327d51a` |
| dataset | `results/R1_real_data/XAUUSDc_M1_raw.json` |
| dataset sha256 | `5472357e67f70f7d1bfef8f9de67fcce91ee6f26208072d7a6766b06c158ee6e` |
| generator | `scripts/run_r2h_origins.py` |
| origin lists | `results/R2-H/origins_H015.csv`, `origins_H060.csv`, `origins_H240.csv` |
| cost | 7.57 bps global constant |
| bootstrap | MBB B=60, reps=10,000, seed=20260826, 95% CI |
| FWER family | {60, 240} minutes, α = 0.05, joint block bootstrap, max-statistic |
---
**DESIGN FROZEN — EXECUTION NOT AUTHORIZED.**
_End of R2-H frozen protocol._