feat(altdata): EIA wired, 24-instrument symbol catalog, mapping dialog for unknown symbols
EIA (user directive: "the NN might find patterns in it for both oil and regular
symbols"). Weekly Petroleum Status Report via the v2 API - crude stocks ex-SPR,
field production, refinery utilization - three features (1y percentile, 4w
change, utilization) on EVERY catalog symbol, not just oil. EIA screened NULL on
WTI's short 7y sample, so these ship as EXPLORATORY inputs: the deploy gate, not
the screen, decides whether a model trained on them trades. Publication stamp
observed+6d mirrors research/altdata/eia.py.
Symbol handling was hardcoded to three if-blocks; it is now a catalog of 24
instruments x alias lists covering The5ers/FTMO/AvaTrade/Dukascopy/OANDA/IC
Markets naming, with prefix matching for the broker suffix zoo (US500.cash,
XAUUSDm, EURUSD.r). Adding an instrument is one AddSpec row. COT caches are
named by CANONICAL so two brokers' names for one contract share a download.
Unrecognised symbol -> a chart dialog (Panel\AltDataMapDialog.mqh, CAppDialog +
dropdown) asks which instrument it is; the answer persists in symbol_map.cfg and
"No alternative data" is a recorded choice, not a nag. Non-blocking by design:
an unmapped symbol contributes 0 features and must never hold up a chart.
Also: UrlEncodePart now escapes '%' - SoQL like-predicates use it as the
wildcard and an unescaped one corrupts the query; docs/ gains the whitelist
URLs, an API-key backup, and the catalog reference.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-16 16:18:29 -04:00
|
|
|
WARRIOR EA - API KEYS (BACKUP COPY)
|
|
|
|
|
===================================
|
|
|
|
|
|
|
|
|
|
Kept here so the keys are never lost with a wiped data folder. Both are FREE,
|
|
|
|
|
personal, and rate-limited per key.
|
|
|
|
|
|
|
|
|
|
FRED (Federal Reserve Bank of St. Louis)
|
|
|
|
|
----------------------------------------
|
|
|
|
|
key: 9640c07ff6574c1c23a17393b735fd36
|
|
|
|
|
host: https://api.stlouisfed.org
|
|
|
|
|
register: https://fredaccount.stlouisfed.org/apikeys
|
|
|
|
|
feeds: VIXCLS (vix, vix_chg5), DTWEXBGS (usd_chg5)
|
|
|
|
|
limit: 120 requests/minute (the EA uses ~2 per hour)
|
|
|
|
|
|
|
|
|
|
EIA (U.S. Energy Information Administration)
|
|
|
|
|
--------------------------------------------
|
|
|
|
|
key: oeSZu7EaZxG5Icjm6q78yUIXaH2EKGhIwVsdTj76
|
|
|
|
|
host: https://api.eia.gov
|
|
|
|
|
register: https://www.eia.gov/opendata/register.php
|
|
|
|
|
feeds: Weekly Petroleum Status Report - crude stocks ex-SPR (WCESTUS1),
|
|
|
|
|
field production (WCRFPUS2), refinery utilization (WPULEUS3)
|
|
|
|
|
limit: 5,000 rows/request (the EA uses 1 request/week)
|
|
|
|
|
|
|
|
|
|
CFTC (Commitments of Traders)
|
|
|
|
|
-----------------------------
|
|
|
|
|
key: NONE REQUIRED - the Socrata endpoint is open
|
|
|
|
|
host: https://publicreporting.cftc.gov
|
|
|
|
|
dataset: gpe5-46if (Traders in Financial Futures, futures only)
|
|
|
|
|
|
2026-08-16 16:32:41 -04:00
|
|
|
CBOE (option chains for the GEX recorder)
|
|
|
|
|
------------------------------------------
|
|
|
|
|
key: NONE REQUIRED - open CDN, and it does not gate on User-Agent
|
|
|
|
|
host: https://cdn.cboe.com
|
|
|
|
|
path: /api/global/delayed_quotes/options/{TICKER}.json (_SPX, GLD, USO, ...)
|
|
|
|
|
feeds: the daily gex_{SYMBOL}.csv recorder (not a model feature yet)
|
|
|
|
|
|
|
|
|
|
FLASHALPHA (options analytics - RESEARCH ONLY, NOT USED BY THE EA)
|
|
|
|
|
-------------------------------------------------------------------
|
|
|
|
|
key: s0AmHthSZi3nenuwvARTz3aKd7DUAnMpOee617jh
|
|
|
|
|
auth: HTTP HEADER, not a query parameter: X-Api-Key: <key>
|
|
|
|
|
hosts: https://lab.flashalpha.com (live)
|
|
|
|
|
https://historical.flashalpha.com (point-in-time replay, 2018-04+)
|
|
|
|
|
register: https://flashalpha.com
|
|
|
|
|
plan: FREE - 5 requests per DAY, 15-minute delayed data
|
|
|
|
|
|
|
|
|
|
!! THE FREE QUOTA IS 5 REQUESTS A DAY. DO NOT SPEND THEM CASUALLY. !!
|
|
|
|
|
|
|
|
|
|
Registered 2026-08-16. NO CALLS HAVE BEEN MADE YET - the whole quota is
|
|
|
|
|
reserved for one specific question (below). This key is deliberately NOT
|
|
|
|
|
an EA input: the EA must never depend on a paid, rate-limited vendor in
|
|
|
|
|
its live path. It is for research-side backfill only.
|
|
|
|
|
|
|
|
|
|
Example call shape:
|
|
|
|
|
curl -H "X-Api-Key: <key>" \
|
|
|
|
|
"https://lab.flashalpha.com/v1/exposure/gex/AAPL?expiration=2026-06-19"
|
|
|
|
|
|
docs(altdata): FlashAlpha backfill is dead - historical API is Alpha-tier only
Pricing checked: Free $0 (5/day), Basic $79 (250/day), Growth $299 (2,500/day),
Alpha $1,499 (unlimited) - and the Historical API is ALPHA-EXCLUSIVE. Basic and
Growth serve live data only.
The archive was the only thing worth buying from this vendor, so nothing in
budget helps: Basic would spend $79/month to make a once-a-day snapshot 15
seconds fresh instead of 15 minutes. Not subscribing.
The free key keeps one genuine use: a single live call to compare their GEX
against our CBOE-computed number, validating the recorder formula against a
commercial implementation (sign and magnitude only - they sign strikes from
classified tape, we use the standard open-interest assumption).
Recorded the EV argument for future sessions: the recorder banks this history
for free in ~12 months, and on this project base rate most alt-data families
die at the incremental gate. Paying four figures to test GEX a year early is a
poor trade. If revisited, price bulk ARCHIVE sellers, not analytics APIs.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-16 16:35:40 -04:00
|
|
|
PRICING CHECKED 2026-08-16 - THE BACKFILL PLAN IS DEAD:
|
|
|
|
|
Free $0 5 req/day 15 min delay
|
|
|
|
|
Basic $79/mo 250 req/day 15 sec
|
|
|
|
|
Growth $299/mo 2,500 req/day 15 sec
|
|
|
|
|
Alpha $1,499/mo unlimited ~1 sec
|
|
|
|
|
HISTORICAL API ACCESS IS ALPHA-TIER ONLY ($1,499/month). Basic and Growth
|
|
|
|
|
are LIVE data only. Since the only thing we wanted from this vendor was
|
|
|
|
|
the ARCHIVE, no affordable tier helps: Basic would cost $79/month to make
|
|
|
|
|
a once-a-day snapshot 15 seconds fresh instead of 15 minutes, which is
|
|
|
|
|
worth exactly nothing for a daily EOD recording. DO NOT SUBSCRIBE.
|
|
|
|
|
Cancellation is anytime, if that ever changes.
|
2026-08-16 16:32:41 -04:00
|
|
|
|
docs(altdata): FlashAlpha free tier probed - no validation possible, verdict hardens
Spent 3 of 5 daily requests. All three were informative:
ETF data (SPY/QQQ/IWM) requires Basic - free tier is single stocks only.
Full-chain GEX (all expirations) requires Growth - free and Basic must
query one expiration per request, so even with history a full-chain
backfill would be 24-54 requests per day of history.
AAPL?expiration=2026-09-18 returned 200 with the right schema but a nearly
empty payload: 13 of 93 strikes carried any open interest, total call OI
4,296 against CBOE 373,253 for the same expiry, put OI zero, and every
near-the-money strike blank.
So the construction could not be validated - not because the math disagreed
but because there was nothing to compare against. From outside it is not
possible to tell free-tier degradation from their flow-signed methodology,
and finding out costs $1,499/month.
Verdict hardens: the free CBOE CDN is strictly better than Basic for this
project - complete chains, every expiry and strike, gamma and open interest
populated, unlimited, $0. Our own AAPL figures were internally coherent
(+0.929 Bn/1% total, Sep-18 expiry +0.154 Bn, near-money gammas 0.013-0.019).
GEX stays externally unvalidated; if that ever matters, use a different vendor.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-16 16:43:23 -04:00
|
|
|
TESTED 2026-08-16 - 3 of 5 daily requests spent. FINDINGS (all 403s and
|
|
|
|
|
the one 200 were informative, so nothing was wasted):
|
|
|
|
|
|
|
|
|
|
1. GET /v1/exposure/gex/SPY
|
|
|
|
|
-> 403 tier_restricted: "ETF data (e.g. SPY, QQQ, IWM) requires the
|
|
|
|
|
Basic plan or higher." Free tier is SINGLE STOCKS ONLY.
|
|
|
|
|
2. GET /v1/exposure/gex/AAPL (no expiration filter)
|
|
|
|
|
-> 403 tier_restricted: "Full-chain GEX (all expirations) requires
|
|
|
|
|
the Growth plan or higher." Free AND Basic must query ONE
|
|
|
|
|
EXPIRATION PER REQUEST.
|
|
|
|
|
3. GET /v1/exposure/gex/AAPL?expiration=2026-09-18
|
|
|
|
|
-> 200. Schema: symbol, underlying_price, as_of, gamma_flip, net_gex,
|
|
|
|
|
net_gex_label, strikes[] with per-strike call_gex/put_gex/net_gex/
|
|
|
|
|
call_oi/put_oi/volumes/oi_change.
|
|
|
|
|
BUT THE PAYLOAD WAS NEARLY EMPTY: only 13 of 93 strikes carried
|
|
|
|
|
any open interest, total call OI 4,296 against our 373,253 from
|
|
|
|
|
CBOE for the same expiry, and total put OI was ZERO. Every
|
|
|
|
|
near-the-money strike (290-320) showed call_oi 0 where CBOE shows
|
|
|
|
|
9,000-35,000 contracts.
|
|
|
|
|
So NO VALIDATION WAS POSSIBLE - not because our math disagreed,
|
|
|
|
|
but because there was nothing to compare against. Cannot tell from
|
|
|
|
|
outside whether that is free-tier degradation or a consequence of
|
|
|
|
|
their flow-signed methodology (they sign strikes from classified
|
|
|
|
|
tape, so strikes with no classified trades may simply be absent).
|
|
|
|
|
|
|
|
|
|
CONCLUSION: the free CBOE CDN is STRICTLY BETTER than FlashAlpha Basic
|
|
|
|
|
for this project - complete chains, every expiry, every strike, gamma and
|
|
|
|
|
open interest populated, unlimited requests, $0. Basic ($79/mo) would buy
|
|
|
|
|
ETF access that we already have for free, still one expiry per request,
|
|
|
|
|
and still no history. Nothing here is worth paying for at any tier below
|
|
|
|
|
Alpha ($1,499/mo), and Alpha is not worth it either (see DESIGN.md).
|
2026-08-16 16:32:41 -04:00
|
|
|
|
feat(altdata): EIA wired, 24-instrument symbol catalog, mapping dialog for unknown symbols
EIA (user directive: "the NN might find patterns in it for both oil and regular
symbols"). Weekly Petroleum Status Report via the v2 API - crude stocks ex-SPR,
field production, refinery utilization - three features (1y percentile, 4w
change, utilization) on EVERY catalog symbol, not just oil. EIA screened NULL on
WTI's short 7y sample, so these ship as EXPLORATORY inputs: the deploy gate, not
the screen, decides whether a model trained on them trades. Publication stamp
observed+6d mirrors research/altdata/eia.py.
Symbol handling was hardcoded to three if-blocks; it is now a catalog of 24
instruments x alias lists covering The5ers/FTMO/AvaTrade/Dukascopy/OANDA/IC
Markets naming, with prefix matching for the broker suffix zoo (US500.cash,
XAUUSDm, EURUSD.r). Adding an instrument is one AddSpec row. COT caches are
named by CANONICAL so two brokers' names for one contract share a download.
Unrecognised symbol -> a chart dialog (Panel\AltDataMapDialog.mqh, CAppDialog +
dropdown) asks which instrument it is; the answer persists in symbol_map.cfg and
"No alternative data" is a recorded choice, not a nag. Non-blocking by design:
an unmapped symbol contributes 0 features and must never hold up a chart.
Also: UrlEncodePart now escapes '%' - SoQL like-predicates use it as the
wildcard and an unescaped one corrupts the query; docs/ gains the whitelist
URLs, an API-key backup, and the catalog reference.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-16 16:18:29 -04:00
|
|
|
WHERE THESE KEYS ARE USED
|
|
|
|
|
-------------------------
|
|
|
|
|
1. EA inputs (the live source of truth):
|
|
|
|
|
FredApiKey, EiaApiKey - Variables\Inputs.mqh, defaults = the keys above
|
|
|
|
|
They travel WITH the EA, so wiping Common\Files\Warrior_EA cannot break the
|
|
|
|
|
downloads. This is deliberate: on 2026-08-16 a folder wipe took the old
|
|
|
|
|
keys.txt with it and the FRED leg silently stopped, leaving models training
|
|
|
|
|
with zero alt-data features.
|
|
|
|
|
|
|
|
|
|
2. Fallback file (only read when an input is left BLANK):
|
|
|
|
|
%APPDATA%\MetaQuotes\Terminal\Common\Files\Warrior_EA\AltData\keys.txt
|
|
|
|
|
Format, one per line:
|
|
|
|
|
fred=9640c07ff6574c1c23a17393b735fd36
|
|
|
|
|
eia=oeSZu7EaZxG5Icjm6q78yUIXaH2EKGhIwVsdTj76
|
|
|
|
|
|
|
|
|
|
3. Research side (Python, never reads the EA inputs):
|
2026-08-16 16:32:41 -04:00
|
|
|
..\Market Data\altdata\keys.json (outside the repo, gitignored)
|
|
|
|
|
{"fred": "...", "eia": "...", "flashalpha": "..."}
|
|
|
|
|
Read via research/altdata/common.py api_key(name), which checks the
|
|
|
|
|
environment variable ALTDATA_<NAME>_KEY first.
|
feat(altdata): EIA wired, 24-instrument symbol catalog, mapping dialog for unknown symbols
EIA (user directive: "the NN might find patterns in it for both oil and regular
symbols"). Weekly Petroleum Status Report via the v2 API - crude stocks ex-SPR,
field production, refinery utilization - three features (1y percentile, 4w
change, utilization) on EVERY catalog symbol, not just oil. EIA screened NULL on
WTI's short 7y sample, so these ship as EXPLORATORY inputs: the deploy gate, not
the screen, decides whether a model trained on them trades. Publication stamp
observed+6d mirrors research/altdata/eia.py.
Symbol handling was hardcoded to three if-blocks; it is now a catalog of 24
instruments x alias lists covering The5ers/FTMO/AvaTrade/Dukascopy/OANDA/IC
Markets naming, with prefix matching for the broker suffix zoo (US500.cash,
XAUUSDm, EURUSD.r). Adding an instrument is one AddSpec row. COT caches are
named by CANONICAL so two brokers' names for one contract share a download.
Unrecognised symbol -> a chart dialog (Panel\AltDataMapDialog.mqh, CAppDialog +
dropdown) asks which instrument it is; the answer persists in symbol_map.cfg and
"No alternative data" is a recorded choice, not a nag. Non-blocking by design:
an unmapped symbol contributes 0 features and must never hold up a chart.
Also: UrlEncodePart now escapes '%' - SoQL like-predicates use it as the
wildcard and an unescaped one corrupts the query; docs/ gains the whitelist
URLs, an API-key backup, and the catalog reference.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-16 16:18:29 -04:00
|
|
|
|
|
|
|
|
IF YOU EVER SELL OR SHARE THIS EA
|
|
|
|
|
---------------------------------
|
|
|
|
|
Blank the FredApiKey / EiaApiKey input defaults before building the
|
|
|
|
|
distributable, and delete this file from the shipped copy. A shared key is
|
|
|
|
|
rate-limited against YOUR account and can be revoked by the provider.
|
|
|
|
|
|
|
|
|
|
To rotate a key: register a new one at the URL above, update the input default
|
|
|
|
|
in Variables\Inputs.mqh, this file, and Market Data\altdata\keys.json.
|