Resurrecting the diagnostic in 4113afd immediately showed why it needed more
than resurrecting. Its first output, on every chart:
strongest of 49 columns x 7999 lags is column 4 at lag +3900, |r| 0.5120
(32.4 SE of the 0.0158 no-information band) -> SURVIVES
XTIUSD and SP500 named the SAME column at the SAME lag with |r| within 0.01 of
each other. Two independent instruments cannot agree to that precision at a
3900-bar lag; that is what identified it as an artifact rather than a finding.
TWO DEFECTS, both of which manufacture significance at long lags.
1. THE NO-INFORMATION BAND WAS GLOBAL, THE CORRELATION IS NOT.
CorrR1D is non-circular: lag k is computed from (n - |k|) overlapping terms
while the normaliser uses all n. Scoring every lag against one 1/sqrt(n)
band understates it by sqrt(n/(n-|k|)) - a factor of 6 at the edge. Lag
+3900 of 3999 rests on ~100 overlapping terms and was being judged as if it
rested on 4000. Each lag now gets its own band, deflated for label
persistence, and the maximum is ranked on z rather than |r| - two equal
correlations are not equally surprising on different sample sizes.
2. THE LAG RANGE ANSWERED NO QUESTION.
The profile exists to say how deep a lookback carries linear structure, and
the input window is HistoryBars. Lags two orders of magnitude past it are
both meaningless and where this estimator is worst. Capped at 4x the window.
The Sidak family is now the lags actually searched, not 2n-1.
Left deliberately: this remains a LINEAR, marginal measure. It can bound the
useful lookback; it cannot prove a shorter window loses nothing.
Build tag -> lagprofile-v2. Not a layout change: no retrain.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
bbe26a09fb