Commit graph Warrior_EA/research/altdata/DESIGN.md
Author SHA1 Message Date
AnimateDread
40ddf06ce6 docs(altdata): adjudicate the NASA API trio - POWER queued behind NATGAS, GIBS unconsumable
POWER is the real find of the three: daily temperature -> degree days ->
natural-gas demand is the textbook gas fundamental, numeric and daily. But it
is point data needing construction into a national series (NOAA CPC ships that
ready-made), and its target symbol is not traded yet - fetch code written for
a chart nobody attaches first runs months later, unobserved, which is the
silent-FRED failure shape. Queued for the AvaTrade expansion, not refused.

FIRMS: re-raised, nothing changed since it was parked - point fire detections
behind the same unproven proxy chain. GIBS: imagery tiles, not numbers; our
CONV is 1D and NASA already sells the extracted products.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-16 17:35:58 -04:00
AnimateDread
1bee06c945 docs(altdata): FlashAlpha free tier probed - no validation possible, verdict hardens
Spent 3 of 5 daily requests. All three were informative:
  ETF data (SPY/QQQ/IWM) requires Basic - free tier is single stocks only.
  Full-chain GEX (all expirations) requires Growth - free and Basic must
    query one expiration per request, so even with history a full-chain
    backfill would be 24-54 requests per day of history.
  AAPL?expiration=2026-09-18 returned 200 with the right schema but a nearly
    empty payload: 13 of 93 strikes carried any open interest, total call OI
    4,296 against CBOE 373,253 for the same expiry, put OI zero, and every
    near-the-money strike blank.

So the construction could not be validated - not because the math disagreed
but because there was nothing to compare against. From outside it is not
possible to tell free-tier degradation from their flow-signed methodology,
and finding out costs $1,499/month.

Verdict hardens: the free CBOE CDN is strictly better than Basic for this
project - complete chains, every expiry and strike, gamma and open interest
populated, unlimited, $0. Our own AAPL figures were internally coherent
(+0.929 Bn/1% total, Sep-18 expiry +0.154 Bn, near-money gammas 0.013-0.019).
GEX stays externally unvalidated; if that ever matters, use a different vendor.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-16 16:43:23 -04:00
AnimateDread
9e49b2aeae docs(altdata): FlashAlpha backfill is dead - historical API is Alpha-tier only
Pricing checked: Free $0 (5/day), Basic $79 (250/day), Growth $299 (2,500/day),
Alpha $1,499 (unlimited) - and the Historical API is ALPHA-EXCLUSIVE. Basic and
Growth serve live data only.

The archive was the only thing worth buying from this vendor, so nothing in
budget helps: Basic would spend $79/month to make a once-a-day snapshot 15
seconds fresh instead of 15 minutes. Not subscribing.

The free key keeps one genuine use: a single live call to compare their GEX
against our CBOE-computed number, validating the recorder formula against a
commercial implementation (sign and magnitude only - they sign strikes from
classified tape, we use the standard open-interest assumption).

Recorded the EV argument for future sessions: the recorder banks this history
for free in ~12 months, and on this project base rate most alt-data families
die at the incremental gate. Paying four figures to test GEX a year early is a
poor trade. If revisited, price bulk ARCHIVE sellers, not analytics APIs.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-16 16:35:40 -04:00
AnimateDread
3abb2b1a7f feat(altdata): GEX forward recorder (CBOE delayed-quotes CDN, no key)
Option open interest is a snapshot source - no free history exists anywhere -
so the series only accrues from the day recording starts. That is why this
ships BEFORE the redeploy: every day the EA is not running is a day of history
that cannot be recovered later.

Records one row per weekday after 21:00 UTC to gex_{CANONICAL}.csv: net/call/put
dollar GEX per 1% move, call and put OI, the three nearest expiries and the
front expiry code. Feeds NOTHING - wiring a feature that is missing across ~100%
of the training sample would waste input width and hand batch-norm a constant.
It becomes a screening candidate at ~250 rows, gated like every other feature.

Thesis: dealer gamma is a RANGE mechanism (long gamma -> hedging sells rallies
and buys dips, range compresses; short gamma amplifies both ways), and range is
this project's one proven channel.

Verified in situ against the live SPX chain before writing any MQL5: 29,362
contracts, 20,993 with nonzero gamma, 54 expiries, total +90.7 Bn/1% (calls
+305.7, puts -215.0), and 100% of net GEX inside 5% of spot. The CDN publishes
per-contract gamma directly, so no pricing model - and no model risk - enters
the recorded data. Also verified the CDN does NOT gate on User-Agent (the old
"CBOE is UA-gated" note in DESIGN.md was a different CBOE path), so plain
WebRequest reaches it.

Dropped a zero-gamma "flip level" field: the probe returned a crossing above
spot while total GEX was strongly positive, which is incoherent - a static
gamma snapshot cannot give a flip level without repricing. Recording a
plausible-looking wrong number is worse than recording nothing.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-16 16:30:39 -04:00
AnimateDread
6d1ecb71ce feat(altdata): alternative-data collector package - COT shipped, FRED/EIA ready
Private-use pivot (marketplace dropped): DLL/Python/WebRequest now allowed.
- altdata/cot.py: CFTC COT, no key, 2010->now on disk for all 9 symbols
  (TFF: ES/VIX/BTC/EUR/JPY/CAD/GBP; Disagg: GC/CL); publication-lag stamping
  (Tuesday report -> Saturday 00:00 UTC availability)
- altdata/fred.py: ALFRED first-print vintages (needs free key)
- altdata/eia.py: weekly petroleum status (needs free key)
- DESIGN.md: source adjudication (corrections to the LLM source list),
  vintage + family-wise rules, EA file contract, staged EA-side plan

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-16 12:29:47 -04:00