ARIMA_SAX_Hybrid_Forecaster/RESEARCH_PROTOCOL.md

RESEARCH_PROTOCOL.md

1. Role

Authoritative governance for the ARIMA + SAX hybrid forecasting research project. Owned by chiki2bum2. This bootstraps a new, isolated repository and does not reference any existing project implementation state.

2. Central question

Does a hybrid ARIMA + SAX forecasting architecture provide statistically and economically meaningful incremental predictive information compared with appropriate baselines and with each component individually?

D > B, D > C, D > A, and — most importantly — whether any incremental information provided by D is stable out-of-sample rather than merely in-sample or restricted to selected historical periods.

3. Initial hypotheses (do not assume true)

• H0 — ARIMA + SAX does not provide materially better information than the strongest component or baseline after OOS evaluation and explicit costs.
• H1 — ARIMA + SAX provides statistically meaningful, stable incremental information.
• H1a — ARIMA captures structure beyond the naive baseline.
• H1b — SAX captures pattern structure beyond the baseline.
• H1c — ARIMA and SAX are partially complementary.
• H1d — Agreement has predictive value.
• H1e — Disagreement identifies unstable/uncertain regimes (rejection filter).

4. Design principle

Never average two raw price forecasts. Convert both components to a common target:

Y(t,H) = (Close[t+H] - Close[t]) / ATR[t]

Both models must express forecasts against the same: symbol, timeframe, forecast origin, forecast horizon H, target definition, normalization convention. The hybrid layer combines evidence, never incompatible raw prices.

5. Hard rules

1. Closed-bar data only for forecast evaluation.
2. All forecast inputs strictly before the forecast origin.
3. The evaluation outcome is revealed only after the prediction is frozen.
4. No future observations in model fitting.
5. No arbitrary parameter sweeps to make a backtest look good. Any search is pre-defined, bounded, documented, confined to train/valid, and frozen before final OOS evaluation.
6. Never report gross results as net results. Transaction costs are explicit.

6. Stop conditions (hard stop)

Immediately stop and report if any of the following is detected:

• lookahead bias
• future observations entering model fitting
• training/evaluation overlap
• data integrity failure
• non-reproducible result
• major implementation discrepancy
• metric definitions changed after observing results
• retrospective parameter changes to improve OOS

Do not silently repair a scientific-methodology violation.

7. Report separation

Every report separates:

• OBSERVED — what was actually measured
• INFERRED — what the measurements suggest
• UNKNOWN — what has not been established
• DECISION — the justified next research action

Never turn: high correlation / R² / low price error / high historical similarity
into a claim of trading edge without economic OOS evidence.

8. Scientific conclusion vocabulary

Only: SUPPORTED, INCONCLUSIVE, NOT TESTED, FAILED. Never: PROFITABLE, EDGE CONFIRMED (unless directly supported by an explicitly defined and reproducible experiment).

9. Correct outcomes are valid

ARIMA wins / SAX wins / hybrid wins / naive wins / none has useful information / hybrid works only in specific regimes / evidence is inconclusive — all are valid. Do not bias the implementation toward a preferred conclusion.