Two literal duplications the scan found, both of the kind where a divergence is
silent:
- Training.mqh stashed the era-loop resume context at FOUR yield points, seven
identical assignments each (pass 1 differing only in i-1). A field missed at
one of them resumes the next chunk against a different era than the one that
yielded, and nothing reports it until the numbers drift. Now StashEraResume().
- AltDataFetch grew its five parallel arrays inline in three places. They are
one record split across five buffers, so a resize missed on any one reads out
of range on the NEXT append, not at the site of the mistake. Now
AltSeriesAppend(), which returns the new index and zero-fills; callers set
only the columns their source has.
Braces balance across every in-scope file.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The //| box blocks were excluded from 0b06f8e and 5efdb48 and were what
remained: 160 of them ran to 10+ lines, the longest to 88. Compressed to their
leading topic sentences - 5 lines for a function header, 8 for a file header -
keeping the box format and the standard MQL5 name/author lines verbatim.
Verified at the BYTE level this time, across every in-scope file: the list of
non-comment lines is byte-identical to HEAD and braces balance. The first check
compared a locale-decoded 'git show' against a UTF-8 read and flagged 25 files
that had not changed at all - every BOM and every non-ASCII line mismatched.
47,696 -> 40,665 lines in scope; comment share 38% -> 26%.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Same pass as 0b06f8e, applied file by file: comment runs of 4+ lines compressed
to their leading topic sentences, capped at 4 lines, whole sentences only.
Warning sentences (NEVER / MUST / trap / would-have) survive the budget.
Every file was checked the same way before committing: the list of non-comment
lines is byte-identical to HEAD, and braces balance. No code was touched.
Panel/, Enumerations/ and the already-terse System headers needed little or
nothing - PooledGate, TradeChecks, BinomialStats and Random came through with
no blocks over the threshold at all.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Verified dead by grep across all first-party sources (references/, Scripts/,
research/ excluded): EraCount, HiddenLayersCount, LstmHiddenSize, ConvFilterCount,
HistoryBars and MinTrainYear setters, PendingBatchSamples, getPrevOutIndex,
BaseCurrency, QuoteCurrency, CurrencyCount, IsLoaded, LastFiredDirection,
DBConfidence, SpecIndex, and the conv Step/WindowOut shape accessors. Every
backing member stays - each is still read internally and several are pinned by
the positional .cfg layout - so this removes surface, not behaviour.
Two comments were asserting the opposite of the code and are now true: the
"No setter: the taper's endpoints are derived" note was directly above three
setters, and the conv shape block claimed EnforceTopologyContract reads all
three accessors when CNet::FirstConvWindow only ever calls Window().
CTrailingATR::CheckTrailingStopLong/Short were byte-identical but for Bid vs Ask
and the isLong flag; both now delegate to one CheckTrailingStop body.
Deliberately NOT removed: the fractal-target branch (TrainTargetFractal,
IsFractalTarget and their label machinery). It reads as dead because the
TrainingTarget input was withdrawn, but Warrior_EA.mq5:836 documents it as a
parked option with a three-line restore path - that is a product call, not a
refactor.
Not compiled - MetaEditor compile pending.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Two things, one of which was a compile error.
1. ExitPolicy() was declared in the protected block but is pushed in from
Warrior_EA.mq5:770. Moved to public beside the other EA-facing setters.
2. Chart objects surviving OnDeinit. The 4,500 ms teardown budget is measured
from the STOP REQUEST, not from OnDeinit's first line, and OnDeinit cannot
begin until whatever is in flight returns - so a scan still running after
_StopFlag is raised does not delay the cleanup, it SPENDS it, and the purge
never gets its turn.
New CExpertSignalAIBase::ShutdownRequested() = IsStopped() || m_shutdownInProgress.
Deliberately NOT m_trainingStopRequested: that latches, and a latched flag
would permanently disable scans that must run again on the next Start.
Guarded, longest first:
- TuneIndicatorsByFilter - per candidate, restoring the OPERATOR's settings
on the way out (best[] is mutated in place; the tuner otherwise keeps the
last trial's parameters, which nothing chose).
- ReportBarrierGeometryScan - per pairing, breaking to ONE restore point so
m_barrierScanLiveLabels can never be left true (that makes ComputeLabelForBar
read the last candidate's multiples as the configured geometry).
- ReportFeatureLabelInformation / ReportExcursionInformation / lag profile -
nulls ABANDON rather than truncate: fewer draws is not a smaller null, it
is a wrong one, and p shifts toward significance. m_dirEvidence staying
false is the safe direction.
- SimulateExitPolicyOutcomes - zeroes its accumulators so the divergence line
is dropped instead of latching a partial expectancy as the run's only report.
- ReportGeometryExpectancyScan - per ladder rung.
- HttpGet - one choke point for up to a dozen blocking WebRequests per
first-pass Update(). An in-flight request cannot be cancelled; refusing to
start another is the whole remedy.
- PollTraining, OnChartEventHandler's study event, TuneIndicatorsAndTrain -
entry points, so a queued event cannot open an era during teardown.
TuneIndicatorsAndTrain's guard is the first statement, ahead of the
m_tuneFilterDone / g_ensembleChartTuneDone latches.
- OnTick / OnTimer / OnChartEvent.
Training's own bar loops already honoured this (pass 1 per bar, passes 2/2.5/3
yield on a 120 ms budget); the warm-up scans did not, and they are the longest
uninterruptible stretches the EA has.
StopTraining() is unchanged: the operator's Stop still finalises synchronously.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Log review of the 18:12 attach. The wiring works: VIX, dollar index, COT, all
seven macro series fetched and SP500_D1.csv rebuilt with its full 13 features
on the first pass. Three findings from the same log, fixed:
KEY LEAK: the EIA failure line echoed the first 80 chars of the URL, which
included most of the api_key. Every URL-echoing error path now goes through
MaskUrl(). The key itself is unchanged - it was printed to a local journal,
not transmitted - but rotate it if that log ever leaves the machine.
EIA HTTP 1003: an MT5 transport-layer code, not a server response. Requests
now carry a User-Agent (gateways reject empty-UA at the edge; the CBOE probe
showed no-UA is fine THERE, but EIA fronts differ) and 1xxx codes are
explained in the log line. Retries were already hourly.
UNBOUNDED BACKFILL: an empty cache fetched full series history - CPIAUCNS
goes back to 1913, whose pre-1970 dates are outside MQL5 datetime range and
whose 1913-era levels sat below the plausibility band, producing 157
scary-but-meaningless REJECTED lines. All FRED fetches now start at 2005
(5y of lookback margin ahead of the 2010 grid). DTWEXBGS staleness horizon
raised to 10 days to match its weekly H.10 publication lag.
Also confirmed from the log: the running build predates the H4 fallback, so
the H4 panels still show 0 features - resolved by the recompile this commit
requires anyway.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Systematic audit of the alt-data stack against "any symbol, any timeframe",
prompted by the H4 surprise. Findings, each fixed:
CAPACITY: ALTDATA_MAX_FEATURES was 16 with FX symbols already at 15 - the next
added column would have been silently truncated by a MathMin. Raised to 32,
pin-chars 512 -> 1024.
SYMBOL NAMES: the panel reload path re-derived symbol/timeframe by splitting
the file path on its FIRST underscore - mis-parsing every symbol containing
one (OANDA-style EUR_USD and US_500 are in our own alias lists) and knowing
only three timeframes. It now stores the (symbol, period) Load() was called
with and reuses them verbatim. Path-hostile characters in broker symbols
("EUR/USD") are sanitized by a shared AltDataFileSymbol() used by the panel,
the fetcher and TunedPeriods, so a slash cannot route a write into an
unintended subfolder.
DOWNLOAD VALIDATION: every FRED-family fetch now enforces a per-series
plausibility band (VIX 1-200, yields -5..30, CPI index 20-1000, ...) -
StringToDouble on transport garbage returns 0.0, and one absurd value poisons
every change/percentile feature computed across it (the BatchNorm NaN-latch
incident came from exactly one huge-but-finite input). Rejected rows are
counted and reported, never dropped silently.
LOUD EMPTINESS: a successful response with zero observations on an empty
cache now says so - naming the series (wrong id / format drift) or the COT
predicate (the unverified like-clauses) instead of leaving 0-filled features
unexplained. GEX gains a truncation guard: a day-over-day contract-count
collapse >50% is the fingerprint of a partial 13 MB download, not of markets,
and is skipped rather than recorded as a plausible-but-wrong number.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The user attached H4 charts and the panel found no {SYM}_H4.csv - the fetcher
only writes _D1 files - so the run trained with ZERO alt features, silently.
The daily file is timeframe-agnostic by construction (rows are as-of daily
values, and Features() joins published <= bar open per bar), so the panel now
falls back to {SYM}_D1.csv on any timeframe, logging the substitution. A
per-TF file still takes precedence if one ever exists.
Per-symbol subfolders (the user suggestion) are NOT the fix for multi-chart
concerns: filenames are already symbol-keyed, and the shared caches
(raw_VIXCLS etc.) are shared deliberately - one download serves every chart.
The REAL races were: (1) two charts of one symbol (D1+H4) each caching their
own last-GEX date and double-appending the same day - UpdateGex now re-reads
the file date before spending the download; (2) whole-file rewrites were
truncate-then-write, so a concurrent reader could parse a torn file - SaveRaw
and RebuildFeatures now write a temp and FileMove-swap it into place.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The user whitelisted the hosts and still got the alert - because the alert
never said WHICH request failed. Two hosts (api.eia.gov, cdn.cboe.com) were
added to the EA after the original whitelist instruction, so any build newer
than the whitelist raises 4014 on the new hosts while the message implied the
old ones were the problem.
Three defects fixed: the popup and journal now print the exact blocked host as
a copy-paste whitelist line; the stale "three URLs" text is gone (the full
four-line reference prints once per session); and the backoff is per-host
instead of global - one missing entry no longer silences the whitelisted
sources for an hour per miss. Popup fires once per host per session; hourly
retries log one quiet line naming the host.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Owner decision (stated twice): available data gets wired; the networks judge
usefulness; the deploy gate remains the arbiter of what trades. Implemented:
MACRO block (6) on every symbol: 10y yield 20d change, curve slope, 5y
breakeven 20d change, Fed-ECB policy gap, CPI yoy, unemployment 12m change.
Screened null vs forward range on all four research symbols - recorded as
the honest prior in the catalog comment, wired regardless.
RISK block (3) extended to every symbol (FX majors, metals, energy, BTC all
now carry vix/vix_chg5/usd_chg5).
IVOL pair extended with the level alongside the change.
Vintage integrity kept where it is free: CPI is fetched as CPIAUCNS (NSA,
essentially never revised) so the plain-FRED backfill stays first-print-clean;
yields/curve/breakevens/policy rates are unrevised by nature. UNRATE is the
one exception (seasonal refits, ~0.1-0.2pp) - the EA cannot run the ALFRED
protocol, accepted and documented at the declaration site.
UpdateFred gains a staleDays parameter so the monthly series do not fire a
pointless fetch attempt every hour for three weeks after each print.
FeatureValue now takes the day and does its own as-of lookups - adding a
source no longer widens a parameter list. Feature counts: 12-15 per symbol;
symbol feature-order changed, safe only because no models exist yet.
export.py mirrors the new catalog for the five research symbols (13-15
features), smoke-tested: all five CSVs written, 6,072 daily rows each.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Wires the screen_ivol survivors (41d726c). New per-symbol `ivolSeries` in the
catalog feeds a generic `ivol_chg5` feature from whichever CBOE vol index the
instrument owns, so one code path serves every symbol:
XAUUSD + ivol_chg5 (GVZ) - MI|vol 0.01971 p=0.002, 3.6x the positive
control and 4.6x the vix_chg5 gold had alone.
vix_chg5 KEPT: this appends, it does not replace.
EURUSD + vix_chg5 - screened, incremental p<=0.006, and its first
real feature ever (it had only exploratory EIA).
USDJPY + vix_chg5 - screened, incremental.
NAS100 / US30 / US2000 + ivol_chg5 (VXN / VXD / RVX) - exploratory by analogy.
XTIUSD / XBRUSD + ivol_chg5 (OVX) - exploratory, no oil bars to screen yet.
SP500 unchanged - its features already screened clean and VXN/VIX3M edging
out VIX is a correlated within-family best-of-N, not a real ranking.
On EURUSD/USDJPY the screen put VXD marginally above VIX, but they are
near-duplicates and the gap sits inside the noise, so the tie is broken by a
rule rather than by the number: take the series already in the fetch path.
Also fixes a real collector bug: fetch_vintaged built ALFRED realtime windows
out to 2028, and FRED rejects realtime_end after today - so every REVISED
series (unemployment, CPI, GDP: exactly the ones needing the vintage path) was
unreachable, while unrevised series never noticed because they bail earlier.
UNRATE and CPIAUCSL now return first prints correctly.
Adds screen_macro.py (rates, curve, breakevens, Fed/ECB policy differential,
plus monthly country stats) with a `distinct` column that reports the honest
effective sample size - a monthly series pasted onto D1 bars is a step
function, and that column is what decides whether it can clear a gate at all.
Not yet run: the Market Data bars directory is being regenerated right now.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Option open interest is a snapshot source - no free history exists anywhere -
so the series only accrues from the day recording starts. That is why this
ships BEFORE the redeploy: every day the EA is not running is a day of history
that cannot be recovered later.
Records one row per weekday after 21:00 UTC to gex_{CANONICAL}.csv: net/call/put
dollar GEX per 1% move, call and put OI, the three nearest expiries and the
front expiry code. Feeds NOTHING - wiring a feature that is missing across ~100%
of the training sample would waste input width and hand batch-norm a constant.
It becomes a screening candidate at ~250 rows, gated like every other feature.
Thesis: dealer gamma is a RANGE mechanism (long gamma -> hedging sells rallies
and buys dips, range compresses; short gamma amplifies both ways), and range is
this project's one proven channel.
Verified in situ against the live SPX chain before writing any MQL5: 29,362
contracts, 20,993 with nonzero gamma, 54 expiries, total +90.7 Bn/1% (calls
+305.7, puts -215.0), and 100% of net GEX inside 5% of spot. The CDN publishes
per-contract gamma directly, so no pricing model - and no model risk - enters
the recorded data. Also verified the CDN does NOT gate on User-Agent (the old
"CBOE is UA-gated" note in DESIGN.md was a different CBOE path), so plain
WebRequest reaches it.
Dropped a zero-gamma "flip level" field: the probe returned a crossing above
spot while total GEX was strongly positive, which is incoherent - a static
gamma snapshot cannot give a flip level without repricing. Recording a
plausible-looking wrong number is worse than recording nothing.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
EIA (user directive: "the NN might find patterns in it for both oil and regular
symbols"). Weekly Petroleum Status Report via the v2 API - crude stocks ex-SPR,
field production, refinery utilization - three features (1y percentile, 4w
change, utilization) on EVERY catalog symbol, not just oil. EIA screened NULL on
WTI's short 7y sample, so these ship as EXPLORATORY inputs: the deploy gate, not
the screen, decides whether a model trained on them trades. Publication stamp
observed+6d mirrors research/altdata/eia.py.
Symbol handling was hardcoded to three if-blocks; it is now a catalog of 24
instruments x alias lists covering The5ers/FTMO/AvaTrade/Dukascopy/OANDA/IC
Markets naming, with prefix matching for the broker suffix zoo (US500.cash,
XAUUSDm, EURUSD.r). Adding an instrument is one AddSpec row. COT caches are
named by CANONICAL so two brokers' names for one contract share a download.
Unrecognised symbol -> a chart dialog (Panel\AltDataMapDialog.mqh, CAppDialog +
dropdown) asks which instrument it is; the answer persists in symbol_map.cfg and
"No alternative data" is a recorded choice, not a nag. Non-blocking by design:
an unmapped symbol contributes 0 features and must never hold up a chart.
Also: UrlEncodePart now escapes '%' - SoQL like-predicates use it as the
wildcard and an unescaped one corrupts the query; docs/ gains the whitelist
URLs, an API-key backup, and the catalog reference.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The AltData folder in Common\Files gets wiped before every fresh test, and
keys.txt died with it (2026-08-16 silent-FRED incident). The credential now
travels with the EA: FredApiKey input, owner key as default; keys.txt demoted
to a fallback consulted only when the input is blanked. EiaApiKey stored the
same way - reserved, nothing consumes it since the WTI screen came back null.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The 2026-08-16 first live fetch looked complete but was not: COT (keyless)
downloaded 1053 reports, then FredKey() hit the absent keys.txt, returned ""
with no log line, UpdateFred bailed, and the SP500_D1.csv rebuild - gated on
all three raw series - never happened. The panel stayed at 0 features with
nothing in the journal explaining why.
FredKey() now logs loudly when keys.txt is missing, and only latches once a
key is actually FOUND: the file is re-read on each hourly-throttled attempt,
so dropping keys.txt in after attach recovers without a restart.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
- MI_TUNE_TARGET = MI_TARGET_EXC_RANGE: the coordinate sweep scored
candidates against the barrier label - measured noise - so it climbed a
flat landscape and the gate rightly rejected every winner. It now
selects indicator settings for MI vs realised RANGE (4x null, positive
control), the channel the excursion head consumes these features for.
Winner gate re-tests on the same target. Barrier-label report unchanged.
- AltDataFetch 4014 handling: Alert popup + once-per-session walkthrough
with the two whitelist URLs on their own journal lines (copy-paste
ready); hourly-backoff retry instead of a permanent latch, so the
whitelist fix takes effect without re-attaching.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
System\AltDataFetch.mqh: the EA backfills missing alt-data history at
attach and keeps appending forward while deployed - online learning never
depends on an external process. CFTC Socrata API (no key, 2006->now, one
GET per symbol; ES name variants verified, max-OI dedupe) + FRED (VIXCLS/
DTWEXBGS, key from AltData\keys.txt). Identical publication stamps and
fixed a-priori transforms as research/altdata/export.py; rebuilds the
same {SYM}_D1.csv files, so Python and EA interoperate on one format.
OnTimer hook (30-min staleness check, in-memory compares when current;
never in tester - cache files serve there) + AltDataReload() on signals.
Classic-signal removal CANCELLED per user (vote experiment later).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
