65 lines
2.7 KiB
Markdown
65 lines
2.7 KiB
Markdown
# DEFECT_LOG.md
|
|
|
|
Every discovered implementation defect is recorded here with its impact,
|
|
fix commit, and re-run status. Original (invalid) evidence is preserved and
|
|
never overwritten; corrected runs are clearly re-labelled.
|
|
|
|
---
|
|
|
|
## DEFECT-001 — SAX candidate windows not z-normalized (frozen analog path)
|
|
|
|
**Status:** FIXED (commit `ff95e11`), re-run completed.
|
|
|
|
### Discovery
|
|
During R1 (real-data transfer), the fast SAX path (which z-normalizes candidate
|
|
windows, matching the SAX reference methodology) was validated against the
|
|
frozen `SaxAnalogForecaster`. The equivalence check FAILED: frozen output was
|
|
**constant across forecast origins** (e.g. identical median outcome and selected
|
|
analog set j = [23..32] at every origin), while the fast path produced
|
|
origin-dependent analog selection.
|
|
|
|
### Root cause
|
|
In `src/sax/analog.py` the candidate windows were encoded as:
|
|
|
|
```python
|
|
cand = close[j - W + 1:j + 1]
|
|
cand_word = sx.sax_encode(cand, ...) # NOT z-normalized
|
|
```
|
|
|
|
The query pattern was z-normalized, but the candidate windows were not.
|
|
SAX breakpoints are only valid for z-normalized series; encoding raw levels
|
|
collapses almost all candidate windows to the same symbol sequence, so
|
|
`mindist_midpoints` returns a near-constant distance for every candidate.
|
|
`argsort` then always selects the first `top_k` candidates (the earliest
|
|
bars), yielding identical, origin-independent outcomes.
|
|
|
|
### Impact
|
|
- **All SAX component outputs and all ARIMA+SAX Hybrid outputs produced before
|
|
this fix are INVALID.**
|
|
- Specifically affected evidence: E1–E8 synthetic benchmark (SAX and Hybrid
|
|
rows), any prior SAX unit/integration checks that relied on analog selection.
|
|
- NOT affected: Naive and Drift baselines, ARIMA component, hybrid state
|
|
classification logic itself, walk-forward machinery, no-lookahead guards.
|
|
|
|
### Fix
|
|
`src/sax/analog.py` now z-normalizes candidate windows before SAX encoding
|
|
(identical to the reference methodology and to the validated fast path):
|
|
|
|
```python
|
|
cand = sx.z_normalize(close[j - W + 1:j + 1])
|
|
cand_word = sx.sax_encode(cand, ...)
|
|
```
|
|
|
|
A regression test (`tests/test_sax_analog_zscore.py`) asserts that analog
|
|
selection is origin-dependent and that identical windows yield zero distance
|
|
while perturbed windows yield larger distances.
|
|
|
|
### Evidence re-run policy
|
|
Per the frozen governance: the affected experiments are re-run from the
|
|
beginning after the fix. Old result artifacts are preserved and marked
|
|
`INVALID (DEFECT-001)` in the relevant summary/manifest files.
|
|
|
|
### Re-run status
|
|
- [x] R1 (real XAUUSDc M1/M5/M15) — re-run after fix (this session)
|
|
- [x] E1–E8 synthetic benchmark — SAX/Hybrid rows regenerated after fix
|
|
(ARIMA/Naive rows unchanged and remain valid)
|