92 lines
No EOL
3.5 KiB
Markdown
92 lines
No EOL
3.5 KiB
Markdown
# RESEARCH_PROTOCOL.md
|
|
|
|
## 1. Role
|
|
|
|
Authoritative governance for the ARIMA + SAX hybrid forecasting research project.
|
|
Owned by `chiki2bum2`. This bootstraps a **new, isolated repository** and does not
|
|
reference any existing project implementation state.
|
|
|
|
## 2. Central question
|
|
|
|
> Does a hybrid ARIMA + SAX forecasting architecture provide **statistically and
|
|
> economically meaningful incremental predictive information** compared with
|
|
> appropriate baselines and with each component individually?
|
|
|
|
`D > B`, `D > C`, `D > A`, and — most importantly — whether any incremental
|
|
information provided by `D` is **stable out-of-sample** rather than merely
|
|
in-sample or restricted to selected historical periods.
|
|
|
|
## 3. Initial hypotheses (do not assume true)
|
|
|
|
* **H0** — ARIMA + SAX does not provide materially better information than the
|
|
strongest component or baseline after OOS evaluation and explicit costs.
|
|
* **H1** — ARIMA + SAX provides statistically meaningful, stable incremental information.
|
|
* H1a — ARIMA captures structure beyond the naive baseline.
|
|
* H1b — SAX captures pattern structure beyond the baseline.
|
|
* H1c — ARIMA and SAX are partially complementary.
|
|
* H1d — Agreement has predictive value.
|
|
* H1e — Disagreement identifies unstable/uncertain regimes (rejection filter).
|
|
|
|
## 4. Design principle
|
|
|
|
**Never** average two raw price forecasts. Convert both components to a **common target**:
|
|
|
|
```
|
|
Y(t,H) = (Close[t+H] - Close[t]) / ATR[t]
|
|
```
|
|
|
|
Both models must express forecasts against the **same**:
|
|
symbol, timeframe, forecast origin, forecast horizon `H`, target definition,
|
|
normalization convention. The hybrid layer combines **evidence**, never incompatible
|
|
raw prices.
|
|
|
|
## 5. Hard rules
|
|
|
|
1. Closed-bar data only for forecast evaluation.
|
|
2. All forecast inputs strictly before the forecast origin.
|
|
3. The evaluation outcome is revealed only **after** the prediction is frozen.
|
|
4. No future observations in model fitting.
|
|
5. No arbitrary parameter sweeps to make a backtest look good.
|
|
Any search is pre-defined, bounded, documented, confined to train/valid, and
|
|
frozen before final OOS evaluation.
|
|
6. Never report gross results as net results. Transaction costs are explicit.
|
|
|
|
## 6. Stop conditions (hard stop)
|
|
|
|
Immediately stop and report if any of the following is detected:
|
|
|
|
* lookahead bias
|
|
* future observations entering model fitting
|
|
* training/evaluation overlap
|
|
* data integrity failure
|
|
* non-reproducible result
|
|
* major implementation discrepancy
|
|
* metric definitions changed after observing results
|
|
* retrospective parameter changes to improve OOS
|
|
|
|
Do **not** silently repair a scientific-methodology violation.
|
|
|
|
## 7. Report separation
|
|
|
|
Every report separates:
|
|
- **OBSERVED** — what was actually measured
|
|
- **INFERRED** — what the measurements suggest
|
|
- **UNKNOWN** — what has not been established
|
|
- **DECISION** — the justified next research action
|
|
|
|
```
|
|
Never turn: high correlation / R² / low price error / high historical similarity
|
|
into a claim of trading edge without economic OOS evidence.
|
|
```
|
|
|
|
## 8. Scientific conclusion vocabulary
|
|
|
|
Only: `SUPPORTED`, `INCONCLUSIVE`, `NOT TESTED`, `FAILED`.
|
|
Never: `PROFITABLE`, `EDGE CONFIRMED` (unless directly supported by an explicitly
|
|
defined and reproducible experiment).
|
|
|
|
## 9. Correct outcomes are valid
|
|
|
|
ARIMA wins / SAX wins / hybrid wins / naive wins / none has useful information /
|
|
hybrid works only in specific regimes / evidence is inconclusive — **all are valid**.
|
|
Do not bias the implementation toward a preferred conclusion. |