Commit graph Warrior_EA/research/altdata/fred.py
Author SHA1 Message Date
AnimateDread
d83cecc011 feat(altdata): wire instrument-specific implied vol; fix FRED vintage path
Wires the screen_ivol survivors (41d726c). New per-symbol `ivolSeries` in the
catalog feeds a generic `ivol_chg5` feature from whichever CBOE vol index the
instrument owns, so one code path serves every symbol:

  XAUUSD  + ivol_chg5 (GVZ)  - MI|vol 0.01971 p=0.002, 3.6x the positive
                               control and 4.6x the vix_chg5 gold had alone.
                               vix_chg5 KEPT: this appends, it does not replace.
  EURUSD  + vix_chg5         - screened, incremental p<=0.006, and its first
                               real feature ever (it had only exploratory EIA).
  USDJPY  + vix_chg5         - screened, incremental.
  NAS100 / US30 / US2000 + ivol_chg5 (VXN / VXD / RVX) - exploratory by analogy.
  XTIUSD / XBRUSD + ivol_chg5 (OVX) - exploratory, no oil bars to screen yet.
  SP500 unchanged - its features already screened clean and VXN/VIX3M edging
  out VIX is a correlated within-family best-of-N, not a real ranking.

On EURUSD/USDJPY the screen put VXD marginally above VIX, but they are
near-duplicates and the gap sits inside the noise, so the tie is broken by a
rule rather than by the number: take the series already in the fetch path.

Also fixes a real collector bug: fetch_vintaged built ALFRED realtime windows
out to 2028, and FRED rejects realtime_end after today - so every REVISED
series (unemployment, CPI, GDP: exactly the ones needing the vintage path) was
unreachable, while unrevised series never noticed because they bail earlier.
UNRATE and CPIAUCSL now return first prints correctly.

Adds screen_macro.py (rates, curve, breakevens, Fed/ECB policy differential,
plus monthly country stats) with a `distinct` column that reports the honest
effective sample size - a monthly series pasted onto D1 bars is a step
function, and that column is what decides whether it can clear a gate at all.
Not yet run: the Market Data bars directory is being regenerated right now.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-16 17:14:03 -04:00
AnimateDread
4a56d8da48 feat(altdata): FRED/EIA collectors live + first MI screen - COT clears on USDJPY, VIX-change on SP500
- fred.py: ALFRED output_type=4 first prints, chunked realtime windows
  (2000-vintage cap), unrevised-series fallback (published=observed+1d);
  NFCI excluded from features (revised, no vintage archive)
- eia.py: 4 weekly petroleum series on disk (1982->now)
- screen.py: as-of joined alt features vs forward 5-bar range/ATR on D1,
  3x3 MI, circular-shift null, family-wise max bar, +/- controls

First readings (199 perms): SP500 vix_chg5 MI 0.047 (1.5x the positive
control) + usd_chg5 clear family bar; USDJPY 4 COT positioning features
clear family bar BEATING the positive control; XAUUSD vix_chg5 tops control
but sub-family-bar; EURUSD positive control FAILS -> table void per the
excursion-target rule, needs investigation.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-16 12:45:21 -04:00
AnimateDread
6d1ecb71ce feat(altdata): alternative-data collector package - COT shipped, FRED/EIA ready
Private-use pivot (marketplace dropped): DLL/Python/WebRequest now allowed.
- altdata/cot.py: CFTC COT, no key, 2010->now on disk for all 9 symbols
  (TFF: ES/VIX/BTC/EUR/JPY/CAD/GBP; Disagg: GC/CL); publication-lag stamping
  (Tuesday report -> Saturday 00:00 UTC availability)
- altdata/fred.py: ALFRED first-print vintages (needs free key)
- altdata/eia.py: weekly petroleum status (needs free key)
- DESIGN.md: source adjudication (corrections to the LLM source list),
  vintage + family-wise rules, EA file contract, staged EA-side plan

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-16 12:29:47 -04:00