NEWS_REFRESH_SEC documents an hourly refresh, but g_newsExport.Update()
was called only from WarmExternalData() in OnInit. The exporter would
therefore have run once per attach and never again, and the calendar is
exactly the file that keeps changing after startup - a release lands and
the terminal's base gets its actual. OnTimer's alt-data upkeep block was
already calling the alt-data fetcher under the same tester guards, so the
exporter joins it there. Its own hourly throttle makes the call cheap and
means the two cadences do not have to agree.
Also lands the per-tick order expiry clock (CExpertCustom::ExpirePendingOrders)
that was still sitting uncommitted, with its reasoning corrected: it argued
from a "sixty-second order", and the entry window became a count of BARS in
eb2a443. The sixty seconds came from a fast-fill result that turned out to be
87% session-gap fills booked at a price never offered, so the deadline's
length carries no edge - only the honouring of it does, which is what this
function exists to guarantee. The defect it fixes is unchanged and is about
the CLOCK, not the length: with Expert_EveryTick off, Refresh() declines every
intrabar tick, so the earliest any deadline could be enforced was the next bar
and a one-bar window silently became a two-bar one.
Build tag three-1 -> books-1 so the OnInit line identifies the binary.
Compile-verified in _claude_stage: 0 errors, 0 warnings.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
No book counts an entry window in seconds. The Forex book offers next-bar-only, next-2, next-3 or
unlimited and deliberately refuses to fix one (p199); the Bitcoin book fixes the immediately
following bar and says the setup is dead if it does not trigger there (p117).
The sixty seconds in this class was fitted to a measurement artefact, not read from a book: the
fast-fill population it selected for was 87% session-gap fills booked at the mother-bar high while
the market had opened far above it, which is where the whole apparent +0.398 R came from. Removing
the artefact removes the parameter.
m_entryWindowBars defaults to 1 - the tightest reading either book offers - and the expiry is
computed from the START of the bar the order is placed on. The per-tick expiry clock and the GTC
fallback in CExpertCustom stay: those were correct and are general to any setup with a window.
The class header still advertised the sixty seconds as the validated rule; corrected.
Compile-verified in _claude_stage: 0 errors, 0 warnings.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The operator found a paid MQL5 product that exports historical news, and drew the right
conclusion: MT5 carries the calendar itself and exposes it to MQL5, so the EA can export it the
same way it already maintains alt data. Verified before writing anything - a probe script
compiled clean against CalendarValueHistory / CalendarEventById / CalendarCountryById and the
full MqlCalendarValue field set on this terminal.
THE CONSTRAINT THAT SHAPES IT: the calendar API returns nothing inside the Strategy Tester. So
this follows the alt-data contract exactly - a LIVE chart writes
Common\Files\Warrior_EA\News\calendar.csv, and backtests, research scripts and models all read
the FILE. In the tester it says so once and leaves the existing file alone, because a backtest
must never truncate what a live session wrote.
WHAT IT UNBLOCKS, and it is three things rather than one:
1. The two NEWS SETUPS - two of the Forex book's six, unbuildable until now. The News Straddle
carries the only explicit R-multiple in the whole book (~8:1).
2. The news AVOIDANCE rule both Walsh books state and we have never honoured: flat five minutes
before a release, nothing new until fifteen after (forex s13). Every backtest so far has
been holding through releases it should have been flat for.
3. News as FEATURES for the networks - time to the next release, its importance, and the
surprise itself (actual minus forecast), which is an axis price data does not carry.
SCALING, STATED HONESTLY. MqlCalendarValue carries actual/forecast/previous as longs and
MetaQuotes documents them as the real value times a million, with LONG_MIN meaning "no value".
That is documentation, not something this code has observed, so every row carries BOTH the
scaled reading and the raw long, plus digits/unit/multiplier - the first live export settles the
convention and nothing downstream has to trust it in the meantime.
Written atomically through AtomicFile, refreshed hourly beside the alt-data fetch.
Compile-verified in _claude_stage: 0 errors, 0 warnings.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
There is one day-trading book PER ASSET CLASS: the Stocks book (applied to indices), the Bitcoin
book for crypto, and the Forex book. A pattern is only accountable on the class its book was
written for. If it also works on another class, keep it there; if it does not, that is not a
failure of the pattern and must not count against it - it simply is not traded there.
DECLARED, NOT INFERRED. WarriorAssetClass() reads SYMBOL_PATH and falls back to SYMBOL_CALC_MODE,
which is a good guess and still a guess: a broker filing US500 under "CFD" would hand the chart
the wrong book silently. ASSET_AUTO keeps the detection and any other value overrides it, set
when the EA is attached. WarriorAssetClassResolved() is the single place that resolution happens,
so the cost model, the logs and any future book gate cannot disagree about what this symbol is.
OnInit now states it either way - auto-detected (with a note to set the input if the broker files
the symbol oddly) or declared, printing what the detector would have said.
Compile-verified in _claude_stage: 0 errors, 0 warnings.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Operator: "it's okay to miss certain entries as well, if we do not get filled, we log and wait
for the next setup. that's discipline."
This entry requires the market to trade UP THROUGH the mother bar's high. When the session has
already opened beyond it - on SP500 that is the 01:00 open after the daily break, and it is
most of them - a buy stop is impossible and the trade layer routed a buy LIMIT instead: a
pullback entry the books never described and the research never measured. LongCondition now
declares the setup VOID in that case, logs it with its full 48-feature context, and waits.
That is not a safety measure, it is the better book. Measured on SP500 M60 2019-2026:
all signals chased +0.049 (n=3,244) -> disciplined +0.060 (n=2,518, missed 22%)
the rule chased +0.063 (n= 185) -> disciplined +0.066 (n= 102, missed 45%)
Every occurrence of the pattern is now written to Adapt\features_{SYM}_{PERIOD}.csv with a
status - taken, void_gapped, void_network - so what we passed on is exactly as analysable as
what we took. The passes are data; that is the point of the journal.
The class header carried the old headline (+0.116 / +0.285 out of sample, "the order expiry is
the load-bearing part"). It has been rewritten to what is actually true: 87% of the rule's
minute-0 fills were gap-throughs booked at a price never offered, the SP500 92.6% win rate was
the tell, and honestly priced the book is +0.044 R at 52%. EnableInsideBarGap stays FALSE.
Compile-verified in _claude_stage: 0 errors, 0 warnings.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
TESTER-FOUND, NOT YET TESTER-CLEAN. Compile-verified (0 errors) in _claude_stage; run 7
(SP500 H1, 2022-2026) shows buy stops shaped 1.25R/8R/60s, the 15-minute time stop
closing them, and the network gate active. Two defects remain open - see the handoff
memory project_handoff_20260906_opus.
WHAT WAS WRONG. The parent CExpertSignalCustom never called a child's OpenLongParams
or CheckCloseLong: once the inside-bar setup voted, the parent shaped a MARKET order
from its own ATR rule (2 ATR stop, no target) and closed it on the vote 9-35 hours
later, or on the very next tick through the protective path. And with the fifteen
classic voters on, the setup's 100 netted to ~11 against a threshold of 15.
Expert/ExpertSignalCustom.mqh
OwnsTrade() / SetupArmed(isLong) virtuals; ArmedSetupOwner() scans m_filters.
OpenParams() asks the armed owner for price/sl/tp/expiration first and records
m_tradeOwner + m_shapedExpiration; the ATR path clears them.
CheckClosePosition() with an owner returns the owner's CheckCloseLong/Short only.
Direction(): an armed setup is decisive (+/-100), never diluted by the classics.
Expert/ExpertCustom.mqh
OpenPosition(): on TRADE_RETCODE_INVALID_EXPIRATION resend GTC and set
CExpert::m_expiration to the requested expiry. KNOWN DEFECT: that clock is only
read on the new-bar path (Expert_EveryTick=false) so orders lived 30 min, not 60 s.
Signals/SignalInsideBarGap.mqh
OwnsTrade/SetupArmed overrides, m_armedBar; CheckCloseLong no longer chains to the
vote exit; CNNFilter + CInsideBarFeatures wired as the confluence gate (file
Common\Files\Warrior_EA\Adapt\_ALL__inside_bar_u.nn, feature names checked one by
one); nn_gate/nn_a/nn_b/nn_agree in the journal context; verbose-mode feature dump
to Adapt\features_{SYMBOL}_{PERIOD}.csv for research/check_feature_parity.py.
Features/InsideBarFeatures.mqh (new)
The 48 network inputs computed on a chart from closed bars, in the file's order.
First parity pass on 13 bars: 30/48 agree, 18 drift - NOT trusted yet.
System/NNFilter.mqh (new) two groups of dense nets, agreement = both >= threshold.
System/AltData*.mqh
cot_am_idx3y / cot_lm_idx3y / vix_rank500 appended to the catalog (export v3),
raw COT cache widened to asset-manager long/short (old caches refetch), HasColumn().
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
THE VOTER FIX, found in the strategy tester and worth stating plainly because it
will bite the next standalone setup too. CExpertSignalCustom weights each child
by VoteCapableWeight(), which is m_weight only when GetPatternCount() > 0 and
ZERO otherwise: a child with no patterns is a news/session-style VETO with no say
in the consensus. CSignalInsideBarGap declared none, so its first tester run
fired 192 times over 2019-2022 and placed nothing - it voted 100 with a capable
weight of 0 and the accumulator divided by zero into 0.00 on every bar. It now
carries one pattern at weight 100, an ID, and its used-series mask, exactly as
every classic module's constructor does. Tester validation of the fixed build is
in progress; this commit is compile-verified only (0 errors, 0 warnings, in a
scratch copy inside the MQL5 tree - the repo path cannot resolve Network.cl).
THE JOURNAL. Two TEXT columns, added IN PLACE with ALTER TABLE (new
EnsureColumn on the database managers) rather than a dbVersion bump, which wipes
the directory:
context key=value;... the setup's view at signal time, the same terms the
research conditioned on, named identically (PublishContext). Accepted
by the journal only when published within 180 s of the position
opening, so an expired order's line cannot be inherited by a later
position from another signal.
passages first-touch MINUTE of every stop and target level on the research
grid plus signed R at each time-stop horizon - the exact fields
research/pricing.py prices an exit from. A synthetic round-trip
proved the live format re-prices the traded exit identically to the
research (max |diff| 4e-7 R) once the horizons carry six decimals.
One honest limit is written into the file header: a live trade is closed by its
own barriers, so from live rows an exit can only be re-selected TIGHTER.
THE ADAPTIVE EXIT. CSignalInsideBarGap reads
Common\Files\Warrior_EA\Adapt\{SYMBOL}_{PERIOD}_inside_bar_u.cfg once per server
day, written by research/journal_adapt.py with the research's discipline (choose
on the older trades, confirm on the newer, placeable stop, stable after dropping
the best 5%). No file means the validated defaults; a file outside the research
grid is refused and logged.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
M60 inside bar, buy stop at the MOTHER bar's high, risk down to the inside bar's
low, stop 1.25R, target 8.0R, 15-minute time stop, and a SIXTY-SECOND ORDER
EXPIRY that carries the edge.
Validated in Warrior_Research (RESULTS.md, tag promising-results-20260906) on
2003-2026 across eight instruments, selected pre-2019 and measured on 2019+:
unfiltered +0.116 R/trade (t 4.57, n 1,203)
with the context rule +0.285 R (t 6.45, n 412), and 2022+ +0.309 - stronger
recently than in sample
The expiry is the finding, not the pattern. By time-to-fill, orders filling in
the first minute paid +0.398 R in sample and +0.116 out; every later fill paid
-0.004 and -0.015. Only ~4% fill that fast. Being an ORDER parameter rather than
a model feature, it is causal by construction and needs no run-time inference.
Maps onto the existing architecture without changing it: LongCondition() fires on
the closed bar, and OpenLongParams() returns a price ABOVE the market, which is
what makes CExpert place a pending buy stop - the order type the research
measured, and one that fills at its own level instead of crossing the spread.
The expiration is set absolutely to 60 s rather than the base class's whole
chart period, because that is the whole point.
Definitions match the research exactly and are pinned in the class comment, since
they are easy to "tidy up" into something that no longer matches what was
measured: entry is the MOTHER bar's high, and valid_test is Wyckoff p118 - volume
below EACH of the two preceding bars, a two-bar lookback and not a moving average.
EnableInsideBarGap DEFAULTS TO FALSE. Unlike the vote modules it sits beside, this
fires at weight 100 and names its own entry, stop and target, so enabling it
changes what the expert trades rather than how it weighs an opinion. Shorts return
0 deliberately: the mirror setup measured +0.054 R against the long side's +0.285.
Compiles clean - the expert built with exactly the same single pre-existing error
(Network.cl resource path, unrelated) and zero warnings, before and after.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
DumpSymbolSpecs exports symbol specifications and deal history so the research
cost model uses the account's real commission and swap rather than assumptions.
ExportIndicatorBuffers dumps every AD/Wyckoff indicator buffer over full history,
so the research conditions on the PRODUCTION detectors rather than a Python
re-implementation of them - which is what made the Wyckoff verdict a verdict on
the indicators rather than on my approximation of them.
Both are read-only: handles, CopyBuffer, and writes under Common\Files. No orders,
no chart changes, no writes to any model or AltData file.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
I introduced two duplications in 3a925fa and this removes them before they set.
THE SAVE/EVAL/RESTORE BRACKET WAS WRITTEN TWICE. HistoricalNetVote() had it inline and
ClassicVoteAt() got a second copy. It is six statements that must stay together - SaveVoteState,
EvalShift(idx), Direction(), EvalShift(0), read the ladder weights, RestoreVoteState - and the
consequence of getting one wrong is not a compile error: Direction() writes the state the signal
database journals for a real fire, so an unbracketed replay of a three-week-old bar overwrites what
the live tick just decided and records history as though it happened now. Two copies is two chances
to drop a line. Now ReplayClassicWeightAt(), used by both.
It returns the RAW ladder difference and leaves ModuleWeight() to the caller, which is the honest
seam rather than a convenience: HistoricalNetVote applies it because it is reconstructing a VOTE
where a module opinion is weighted by trust, and the feature path must not because a model input
cannot move for reasons unrelated to the bar it describes. Encoding that choice inside the helper
would have forced one caller to undo it.
THE CLASSIC-VOTER PREDICATE WAS WRITTEN THREE TIMES. "not an AI filter and has patterns" appeared in
ClassicVoteWidth, ClassicVoteAt and again in HistoricalNetVote. Width and values MUST agree on which
children count or the feature block silently misaligns against its own column names - a fault that
produces plausible numbers rather than an error. Now IsClassicVoter(), four callers.
A MISTAKE WORTH RECORDING: my first attempt used line-mode `perl -pi` with embedded newlines against
a 3,000-line file and expanded it to 268,141 lines. Caught by the match count reading 262,029
instead of 2, restored with git checkout, redone with precise edits. The deployed build was the good
commit throughout, so nothing reached the running fleet. Structured multi-line edits do not belong
in line-oriented regex.
No behaviour change: same brackets, same predicate, same arithmetic, one copy each.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The fifteen classic modules now feed the neural nets as features instead of only voting beside
them. One signed column per pattern-bearing module, replayed AT EACH HISTORICAL BAR so the network
sees what the rule-based primaries said there, not what they say now.
WHY FIFTEEN COLUMNS AND NOT SIXTY. Every module OVERWRITES rather than sums (verified across all
fifteen), so exactly one pattern weight is returned per call - and the weights are DISTINCT within
every module (verified: no collisions anywhere). The signed net vote is therefore a lossless code
for which pattern fired and in which direction: RSI +10 rising, -10 falling, +20 oversold reversal,
-20 overbought, +-60 divergence, +-90 double divergence, 0 nothing.
And it works as a SCALAR, which a categorical code normally does not. Feeding codes to a network as
a number line is usually a modelling error - it interpolates between values that have no order - and
the fix is one-hot at ~60 columns of width. This escapes that because the ladder was built ordered
BY CONVICTION: 10 is a weak confirming state, 95 is rare and strong. Magnitude carries meaning
independently of identity, so the net can read signed strength before it ever learns the codebook.
Both properties exist ONLY while the weights are frozen priors. If the database is ever allowed to
re-score them this column becomes an unreadable moving quantity.
SCALED /100 on the way in. The raw ladder runs to +-95 while the rest of the vector sits within a
few units, and one column a hundred times wider than its neighbours owns the first layers gradients
and the covariance matrix with it - the exact failure that cost a session when a Wyckoff sentinel
read -580.
THE REPLAY IS THE EXISTING ONE. HistoricalNetVote() already brackets a filter evaluation in a
six-field SaveVoteState/EvalShift/Direction/RestoreVoteState sandwich, because Direction() writes
the live journaling state and replaying a three-week-old bar without it would overwrite what the
live tick just decided and journal history as if it had happened now. Same bracket, new consumer.
CACHED ON THE PARENT, not per member. The four AI models on a chart share the same filter objects,
so a naive implementation replays fifteen ladders over 179k bars four times - and worse, could let
members see different vectors. One computation per bar, served to all four: an ensemble voting on
different evidence is not an ensemble.
RSI, MACD AND TIME FEATURES ARE BACK ON. They measured 0/1, 0/3 and 0/6 unanimous under the
leg-ride label and I cut them last night on that evidence. Under this architecture they are CONTEXT
rather than predictors - an RSI-oversold MACD cross is a different trade from a bare one, and
day-of-week conditioning is exactly the conjunction a meta-labeler is meant to find. Standalone
predictiveness and conditioning value are different questions and the width cut answered only the
first.
Width is now retrain-forcing through m_neuronsCount as always, so the models re-key themselves.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Five more modules, 16 patterns, recovered from 1073262 where they had already been ported to
CExpertSignalCustom - so this is restoration rather than a fresh port. Same dead
SweepPrepareIndicator override stripped as before.
WEIGHTED ON THE SAME LADDER, and deliberately IDENTICALLY where the construct is identical: CCI,
Stochastic and WPR repeat the RSI shape - direction 10, overbought/oversold reversal 20-25,
divergence 60, double divergence 90 - because they measure similar things by similar means. Giving
one a higher number than another would invent a distinction the literature does not make, and would
be the first step back toward fitting the weights.
SAR is the exception worth noting: its pattern 0 is a persistent STATE (the dots are on one side and
stay there for many bars, 10) while pattern 1 is the SWITCH - a discrete, dated event and the only
thing SAR offers as an entry, so 50.
Running total: fifteen voting modules, 60 patterns, all on fixed priors. Ten are classic/stdlib
constructs and five are the Bill Williams suite.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Four new modules, 11 patterns, built on the stdlib CiAlligator/CiFractals/CiGator/CiBWMFI
indicator classes and ported to CExpertSignalCustom like the rest.
ALLIGATOR (4). Three smoothed averages of median price, displaced forward - 13/8, 8/5, 5/3, which
is Williams specification rather than a choice. Patterns encode his own reading: ordered lines are
context (10), a lips/teeth cross is the trigger (40), price clear of the lips with the mouth still
opening is the trend running (45), and AWAKENING - previous bar intertwined, this one ordered -
carries the most weight (55) because it times the transition instead of reporting a state that may
have been true for twenty bars. A SLEEPING alligator produces no vote at all rather than a weak one.
FRACTALS (3). A five-bar pattern, and the two-bar confirmation delay is the whole difficulty: a
fractal at bar i is only knowable at i+2, so reading it at i is exactly the lookahead that faked a
+4 sigma reading in bd076dd and had to be found twice. FRACTAL_CONFIRM_BARS is named rather than
inlined and every scan starts at from+2.
GATOR (2). The Alligator restated as two magnitude histograms, so it carries NO direction - it says
whether the market is trending or resting, never which way. Direction is borrowed crudely from the
bar close and the weights are low to match. Deliberately a regime filter, not an entry.
BWMFI (2). Williams four bar states from the movement/volume pair - green, fade, fake, squat. Only
green (20) and squat (50) vote; fade and fake are explicitly not trades. HONEST LIMIT: on FX and
CFDs MT5 reports TICK volume, not traded size, so green means many ticks rather than much money.
That weakens the construct and the weights are set below what an exchange-traded instrument would
justify. VOLUME_TICK is not a preference either - VOLUME_REAL is zero on these feeds and the index
would divide by it.
With the six restored modules this is 44 patterns across ten voting modules, all on fixed priors,
with the database recording outcomes and no longer writing weights back.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
WHAT AND WHY. ed91919 removed the four classic votes because all 26 patterns measured AT CHANCE as
standalone entries - pre-registered, nothing fitted, and the +4 sigma that had once appeared was two
bars of lookahead. That result stands. It is also not the claim being made here.
That test measured UNCONDITIONAL edge: fired blind, does this pattern beat a coin on average. No -
after the lookahead fix MACD_p4 read -0.02pp at -0.02 sigma, flat rather than weak. A signal that is
zero on average can still be strongly positive on a SUBSET, and finding that subset is exactly what
meta-labelling is for. The primary supplies direction and an entry BAR and is judged on RECALL; the
net decides which firings to take.
THE MEASURED PROBLEM THIS ATTACKS. Under the leg-ride label the primary was "the ZigZag leg in
progress", so entry landed on an ARBITRARY bar inside a move already underway. Measured 2026-09-05
on the converged fleet: the ORACLE ride is +4.2 to +4.6 ATR per leg on every chart, and the models
captured +0.08 to +0.38 over a 14-17 bar hold - under 10%, because most of the leg was gone before
entry. Always-ride scores -0.14 to +0.10, i.e. an arbitrary entry inside a leg is worth nothing. A
pattern fires at a CHOSEN bar with the move ahead of it.
RESTORED: SignalMA (4 patterns), SignalRSI (4), SignalMACD (6), SignalIchimoku (12),
OscillatorDivergence, plus Bill Williams SignalAO (4) and SignalAC (3) recovered from 1073262.
33 patterns across six voting modules. The only edit needed was dropping SweepPrepareIndicator(),
whose base-class method no longer exists.
THE WEIGHTS ARE FIXED PRIORS (Variables\ClassicSignals.mqh), set from structural strength and rarity
in the spirit of the standard library's own ladder, NOT fitted to returns:
10 confirming state ("price is on the right side") - every module's pattern 0
15-30 simple state or weak-grade event (overbought reversal, cross in a poor location)
30-50 a crossing or completed pull-back
50-70 structural: divergence, cloud breakout, strong-grade cross
80-100 rare confluence: double divergence (90), Sanyaku Kouten (95)
33 numbers tuned against measured returns would be 33 free parameters and would hand back the
family-wise problem this project keeps rediscovering. They are priors and must stay priors.
AND THE FEEDBACK LOOP IS CUT. DB_RankingFeedsWeights=false splits the signal database in two:
ProcessBufferedSignals() still RECORDS every firing, direction, entry, exit and outcome - that
corpus is the input to the meta-labelling work - while UpdateSignalsWeights() no longer writes win
rates back into the pattern weights. That loop is why the classic votes were never evaluable: the
same setup contributed a different amount at different times, so the vote drifted era to era under
the model and nothing could be measured against it. UseDatabaseRanking gates BOTH halves, so turning
the master switch off would have stopped the collection too.
RARITY CUTS BOTH WAYS, recorded rather than glossed: double divergence earns 90 BECAUSE it is rare,
which also means the net will see very few examples and can say little about it. The prior carries
those patterns; the model will learn mostly about the common, low-weight ones.
Compiles clean. Not yet deployed - the Bill Williams suite (Alligator, Fractals, Gator, BWMFI) and
the stdlib ports land next, and one restart should carry all of it.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
THE PREMISE NEEDED CORRECTING FIRST, and the correction is the useful part of this commit.
The intent was to stop "the database system that changes patterns and signals weights". The signal
database does not change the NN's weights. `ApplyPatternWeight()` opens with
if(m_tiersSelfRanked) return;
and every AI model sets that flag the first time it ranks itself, so the hourly DB win-rate pass
(UpdateSignalsWeights -> ApplyPatternWeight) weights CLASSIC FILTERS and nothing else. Its own
comment says so: "THE SIGNAL DB DOES NOT OUTRANK THE HOLDOUT... without this the ranking pass would
silently undo every era's self-ranking within the hour."
WHAT ACTUALLY MOVES AN AI SIGNAL'S WEIGHTS is RankTiersFromOos() (Lifecycle.mqh), at EVERY pass-3
completion:
w = ShrunkRatePct(effHits, effN, trustPct, TIER_PRIOR_EFF_N); // per tier
ApplyTierWeight(t, wi); // four tier weights
Weight(trustPct / 100.0); // ensemble trust weight
Both are recomputed from the model's own purged held-out fires, era after era. So the same feature
vector maps to a different signal strength depending on WHEN it was evaluated, and a dataset
collected across eras mixes several mappings - any pattern statistics taken from it measure the
ladder as much as the market. That is the thing to freeze for stable collection, and turning off
UseDatabaseRanking would not have touched it.
IT RANKS ONCE, THEN HOLDS - not "never ranks". A member that has never self-ranked is not admitted
to the vote at all (the m_tiersSelfRanked guard at Lifecycle.mqh:566, and the "deployed but WITHOUT A
TIER LADDER (so it cannot vote)" path), so a freeze applied before the first ranking would silence
the ensemble rather than stabilise it. The overlay snapshot still runs unconditionally, so the chart
view keeps refreshing; only the WEIGHTS are held, and one throttled line says so per model.
ALSO WORTH RECORDING: two comments in the tree claim UseDatabaseRanking "ships FALSE"
(TradeJournalManager.mqh:333, Warrior_EA.mq5:1210). It ships `input bool UseDatabaseRanking = true`.
Not changed here - it is a separate question from this freeze, and it also gates the RECORDING half
(table creation and Direction()'s per-tick buffering, Warrior_EA.mq5:2532), so flipping it would
stop collecting the data rather than stop weighting it.
Not in any fingerprint: models resume from disk, nothing retrains.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
THE PROBLEM THIS FIXES IS IN MY OWN EVIDENCE. The width cuts (95adc79, 4092a94) chose which feature
blocks to keep from a keep-screen measured on EURUSD, USDJPY, NAS100, XAUUSD and XTIUSD - and were
then judged on the books of those same five charts. The screen selects on MI between column and
label while the book is mean ATR per call, so it is not circular; but it is not INDEPENDENT either.
A column set chosen to be informative on five instruments should be expected to look informative on
those five. That is precisely the shape of result this project keeps promoting and failing to
replicate, and it does not stop being that shape because the numbers moved the right way:
chart 552 inputs -> 372 -> 294 net of ~0.035 ATR cost at 294
XAUUSD +0.538 +0.343 +0.364 +0.329 (29/29 eras positive)
EURUSD -0.009 +0.141 +0.133 +0.098
USDJPY +0.064 +0.023 +0.073 +0.038
NAS100 -0.208 -0.086 +0.036 +0.001 (breakeven, from -0.25)
XTIUSD -0.152 -0.047 -0.161 -0.196 (the one that regressed)
Three of five clear cost and a fourth is at breakeven, up from two at the start of the night. But
every one of those five helped choose the features being tested.
GBPUSD, USDCAD and DAX40 took no part in that selection. They contribute nothing to the screen that
produced the current 49 columns, so their books answer the question the other five cannot: does a
feature set chosen on one panel of instruments pay on instruments it never saw? If it does not, the
cut fitted the panel rather than the market.
GBPUSD was originally dropped from the fleet for being the most correlated with EURUSD - which made
it a poor POOLING candidate, since it adds few independent observations. That is a different role
from the one it has now: as a HELD-OUT test its correlation with the selection panel is a weakness
only if it makes the test too easy, and USDCAD and DAX40 are there to cover that.
NO WIPE AND NO RE-KEY: FLEET_EXPANSION_SYMBOLS is not part of BuildModelFingerprint, so the five
existing charts resume from disk (USDJPY came back at era 25) and only the three new ones start at
era 0. Verified in situ: eight charts, 294 inputs (6 bars x 49 features), zero casting errors, zero
out-of-range, 27 GB headroom.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
SAME EVIDENCE BASE AS 95adc79, which is why this did not need re-measuring: the keep-screen run on
five charts across two independent cold starts. Both blocks have ZERO columns clearing on all five
charts in EITHER run.
ichimoku 0/8 unanimous, 5/8 dead on every chart in both runs
wyckoffFail 0/5 unanimous, 2/5 dead
ICHIMOKU'S REMOVAL IS THE CONCLUSION OF ITS OWN EXPERIMENT, not a reversal of one. It was re-enabled
on 2026-09-04 (a225dca) specifically to be measured under the leg-ride label, after a year in which
the only thing wrong with it turned out to be our double-shift, not the indicator. It has now been
measured, twice, on five instruments: it carries nothing that survives on all of them. Keeping it
would have meant re-enabling a block to measure it and then ignoring the measurement.
SPREAD IS DELIBERATELY KEPT despite also being 0/2 unanimous: neither of its two columns is dead on
any chart, and two columns is not worth a retrain to remove. "No column clears EVERYWHERE" and "dead
everywhere" are different verdicts and only the second is decisive on its own.
RESULT: 294 inputs (6 bars x 49 features), from 552 (6 x 92) this morning - a 47% cut. Verified in
situ: five charts training, zero casting errors, zero out-of-range, zero stalls, and no Ichimoku
fallout (WarriorPrimeIchimokuBuffers is load-bearing only when the feature is ON).
WHAT THE FIRST CUT DID, recorded before this one buries it - measured over 9-42 vote eras each, so
provisional:
chart mean book before -> after net of ~0.035 ATR cost
XAUUSD +0.538 -> +0.343 +0.308
EURUSD -0.009 -> +0.141 +0.106
USDJPY +0.064 -> +0.023 -0.012
XTIUSD -0.152 -> -0.047 -0.082
NAS100 -0.208 -> -0.086 -0.121
All three failing charts improved and the strongest fell: the books COMPRESSED toward each other,
which is what removing the capacity to fit chart-specific noise looks like. Two of five clear cost.
A LEVER THAT LOOKED PROMISING AND IS NOT: certifying a higher vote rung. The per-rung ride book is
not monotone and not stable - XTIUSD read 1%->+0.101 one era and 1%->-1.446 the next, XAUUSD
2%->+0.565 then 2%->-0.138 - and the higher rungs run on n=28-140. Trading coverage for expectancy
is the right IDEA under a net-expectancy bar, but this curve cannot support the choice.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
THE BINDING CONSTRAINT ON THIS PROJECT IS INPUT WIDTH, and this is the first change that actually
moves it. The first-layer budget effN/(width+1) went from 1.6-6.6 to 7.0-9.7 across the fleet,
against a rule-of-thumb 10-30 it had never been close to.
WHAT WAS MEASURED. The per-column keep-screen (MI against the leg-ride label, Benjamini-Hochberg at
q=0.10) was run on all five charts TWICE, on two independent cold starts with every model wiped in
between - so the second reading shares no weights, no pool and no cache with the first.
* The screen is sound: 0.0147-0.0173 nats/feature against a shuffled-label null of
0.0021 +/- 0.0002, p=0.005 over 200 permutations; strongest single column 0.14 vs a null max of
0.0068. That is 7-8x the null.
* The screen REPLICATES: per-chart hex masks differ between the two runs by one or two nibbles,
29 columns clear on all five charts in BOTH runs, 28 clear on none in either, and only 5 of 92
columns disagree at all.
SWITCHED OFF (each has ZERO columns clearing on all five charts, in both runs):
ma 0/5, rsi 0/1, macd 0/3 - dead on every chart in both runs
swing 0/7, fracdiff 0/3, volume 0/4, time 0/6, atr 0/1
30 columns, 180 inputs. As `const`, not a mask - the indicators are never created and never
computed, because "a mask that hides output is not a switch that stops work".
WHAT CARRIES THE SIGNAL, all 29 unanimous columns: cumdelta 6/6, wyckoffEvent 12/16, alt 7/12,
sot 2/4, wyckoffBarInv 2/5. Every CLASSIC technical block contributes nothing that survives on all
five instruments; the Wyckoff suite, cumulative delta and the alt block carry it.
THAT REVERSES THE 2026-09-03 READING, where `ma` was the strongest block (5/5 on five charts) and
wyckoffEvent was dead on 5/5 - and the reversal has a cause rather than being drift: ADWyckoffEvent-
Stream had a PROVABLY UNREACHABLE continuation branch until it was fixed hours ago, and the chart
set changed.
EnableMAFeature and EnableSwingContext were `input` and are now `const` with the other six. They
move m_neuronsCount, field 4 of the model fingerprint, so they are retrain-forcing - and MT5 stores
inputs PER CHART, so changing an input default reaches no already-attached chart.
A MAPPING ERROR I MADE AND CORRECTED, in the file rather than quietly: my first pass assumed
fracdiff was 8 columns and alt was 7. FRACDIFF_COLUMNS is 3 and alt emits 12, so every block after
fracdiff was attributed columns shifted by +5. The eight blocks below are unaffected - each is 0/N
unanimous under both mappings, and 92-30=62 emitted columns confirms the corrected layout - but the
verdicts I first wrote for ichimoku (claimed 3/8, actually 0/8), spread (claimed 2/2, actually 0/2),
cumdelta (claimed 4/6, actually 6/6) and wyckoffEvent (claimed 8/16, actually 12/16) were wrong.
Verify a block layout against an emitted width before trusting a column index.
VERIFIED IN SITU: 372 inputs (6 bars x 62 features), five charts training, zero casting errors, zero
out-of-range, zero stalls, models re-keyed (PAI-19ba / CONV-f3c2 / LSTM-aa25 / HYB-d8d7).
STILL OPEN: ichimoku (0/8, five columns dead), spread (0/2) and wyckoffFail (0/5) also carry no
unanimous column and are the next candidates - left ON for now so this cut can be judged alone.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
IT HAS BEEN INERT SINCE BEFORE THIS SESSION. `EnableCrossAsset` is `const bool ... = false`, the only
thing that ever calls `UseCrossAsset(...)` is Warrior_EA.mq5:819 passing exactly that constant, and
`BuildCrossAssetPanel()` returned at its first line. Its 6 features were never added to the 92, and
its fingerprint tag `|XA:` was never appended - so REMOVING IT CHANGES NO FINGERPRINT AND NO FEATURE
VECTOR. Nothing retrains because of this commit.
WHAT WENT:
System\CrossAsset.mqh 564 lines, deleted outright
the panel build, the per-bar emit block, the feature-table entry (FeatureBuilder)
the member, the accessors, the persistence fields (ExpertSignalAIBase)
three view interfaces + their two implementations
the topology feature-count contribution and the fingerprint tag
the blocking warm-up in OnInit, the sweep-guard rule, the input constant
THE PERSISTENCE FORMAT LOSES A FIELD, and that is safe ONLY because every model was wiped an hour
ago. The .cfg carried a length-prefixed cross-asset pin between the direction-confidence threshold
and the alt-data name list. Both the writer and the two readers (the loader and the cfg walker) drop
it together, so the remaining fields stay aligned - but a .cfg written by the PREVIOUS build would
now have its alt-data pin read from the cross-asset slot. There are no such files.
WHY DELETE RATHER THAN LEAVE IT SWITCHED OFF: it was 117 references across 22 files that every future
reader had to understand before concluding it did nothing - I spent part of tonight doing exactly
that, twice, and got the reasoning wrong the first time (I claimed UseCrossAsset(true) was never
called anywhere; it is called, with a const false). Dead code that takes two passes to prove dead is
not free.
NOT A MEMORY FIX. It was proposed as one and it is not: the panel allocated nothing at runtime. The
27 GB was the indicator tuner (b31f462).
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
THREE NEW OPTIMIZERS, APPENDED TO ENUM_OPTIMIZATION (never inserted - the ordinal is written into
every .nnw and hashed into the weights filename, so renumbering would silently reinterpret every
saved model). SGD=0 and ADAM=1 keep their values.
WHAT EACH ACTUALLY COSTS, because two of the three are nearly free and it is worth saying why:
RMSPROP = the EXISTING Adam kernel with beta1 = 0. With no first moment mt collapses to the raw
gradient and the step becomes lr*g/sqrt(v), which is RMSprop exactly. Not one kernel
was added in any backend - it is a scalar, not an algorithm. The bias correction had
to move into WarriorOptLearnRate() because the literal (1 - pow(beta1,t)) denominator
is (1 - 0^t) = 1 for t>=1 but ZERO at t==0, a division by zero on an unadvanced
counter.
NESTEROV = the momentum kernels' `optimizer` argument, which had been retained-but-unused since
the DFA entry was deleted, finally selects something. Heavy-ball applies the velocity;
Nesterov applies lr*g + mu*v_new (the Bengio/PyTorch reformulation, which expresses NAG
without evaluating the gradient at a shifted point). No new state, no signature change
on three of the four momentum entry points.
AMSGRAD = the only one needing extra state: a non-decreasing denominator needs the running max of
the second moment (Reddi et al. 2018 - Adam can fail to converge because a rare large
gradient inflates v and then DECAYS away, so the effective step GROWS again just after
the event that should have shrunk it).
IT REUSES DeltaWeights AS THE v-MAX. The Adam family already frees that buffer, it is
the same element count, already zeroed by ZeroOptimizerBuffer, already saved and
loaded. Net memory versus SGD: zero. That mattered: an extra per-tensor allocation is
exactly what took this machine down twelve hours ago.
FOUR TIERS, IN LOCKSTEP, WHICH IS THIS FILE'S OWN STANDING RULE:
* MQL5 host (NeuronCPU.mqh): all three, with CConnection gaining a vMax field (persisted
unconditionally - CConnection cannot know which optimizer owns it, and a conditional would make
the record length depend on state the reader does not have yet).
* CPU DLL (WarriorCPU.cpp + .h): Nesterov in all four momentum kernels; a nullable vm pointer in
all four Adam kernels (vmHandle < 0 leaves it null, so every non-amsgrad caller is
byte-identical to before). Rebuilt and deployed.
* MQL5 host FALLBACK inside ApplyAccumToBlock: the same amsgrad branch. This is the path the DLL
drops to when the element-wise apply fails, and a model that trained differently depending on
whether the DLL was healthy would be unreproducible.
* OpenCL (Network.cl): Nesterov via the same `optimizer` argument; matrix_vm + an amsgrad flag on
all four Adam kernels, bound to a harmless dummy buffer when off so there is still ONE kernel
per operation, matching the DLL's shape rather than forking a second set.
THE OPENCL TIER IS UNVERIFIED AND SAYS SO AT RUNTIME. This machine has no OpenCL device ("cannot get
OpenCL platforms / opencl.dll not found"), so the CPU-DLL tier is what actually executed these. The
kernels are written and wired; they have never run. A throttled line names that on first use rather
than implying a parity that was never tested.
THE BINDINGS THAT WOULD HAVE BROKEN SILENTLY, and why the predicates exist:
`optimization == SGD` and `== ADAM` appeared at 34 sites and meant three different questions - which
buffers to allocate, which to persist, and which step to take. An unlisted optimizer would have
taken the ADAM branch for its buffers and the SGD branch for its persistence. They are now
WarriorOptUsesMoments / WarriorOptUsesDelta / WarriorOptIsMomentumStep, and the allocation and
persistence sites ask TWO INDEPENDENT questions instead of one if/else, because AMSGRAD is the first
optimizer that needs both families.
ASSIGNED PER ARCHITECTURE, so the ensemble's members now differ in their optimizer as well as their
shape - four members that fail in correlated ways average to nothing, and member correlation
(printed every era, r 0.19-0.36) is the measurement that says whether this bought diversity:
Perceptron/MLP -> NESTEROV (no structural prior, so a non-adaptive step is the implicit
regulariser that makes it the conservative ANCHOR vote; the lookahead
damps heavy-ball's overshoot at zero extra state)
Conv -> ADAM (unchanged - AdamW already)
LSTM -> AMSGRAD (the most heavy-tailed gradients here: shared weights across timesteps
mean one bad window contributes a burst of correlated updates, which
is the regime AMSGrad was derived for)
ConvLSTM -> RMSPROP (two very differently-scaled gradient sources; per-weight
normalisation without a first moment stops the conv stage's momentum
dragging the recurrent one)
VERIFIED IN SITU, not by compiling: 5 charts, 18 eras, zero UpdateWeights failures, zero fallback
warnings, one fingerprint per architecture (PAI-0bd8 / CONV-5327 / LSTM-44d1 / HYB-452a). Commit
3,975 MB against 29,982 MB headroom.
COST: every model re-keys and retrains, the optimizer being a field of the weights filename.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
DiagnoseADBuffers.mq5 reports, per indicator per buffer per chart, the share of
EMPTY_VALUE, the share of exact zeros, the single most common value share, and the
standard deviation - flagging NEVER-FILLED / ALL-ZERO / CONSTANT / NEAR-CONSTANT.
It answers the question that has to be settled BEFORE any "this feature carries no
signal" verdict: is the buffer computing anything at all. A constant column is a code
fault, not a weak feature - a real but weak signal still varies.
FIRST RUN, 40,000 H1 bars x 5 charts: ADWyckoffEventStream buffers 12 and 13
(Reaccumulation, Redistribution) are identically zero with sd exactly 0 on ALL FIVE
charts. Tracing it gives a proof rather than a suspicion:
line 465 accumulation ranges open only under tr<=0, calling OpenRange(+1, bi, tr)
line 392 continuation needs (dir>0 && trendAtOpen>0), and trendAtOpen IS that tr
so the reaccumulation branch is unreachable by construction, and redistribution is
unreachable symmetrically. gRIsContinuation can never be true.
The conceptual error is one Trend(bi) reading serving two different questions - the
climax gate asks about the trend the climax TERMINATES, reaccumulation asks about a
larger trend the range sits INSIDE. Different horizons; collapsing them makes the
conditions mutually exclusive.
This corrects my own earlier reading. I reported the block as "dead at every parameter
set" from a 41-candidate sweep on two bands; no parameter can make an unreachable
branch fire, so that search was never a test of the concept. The feature set is
unchanged - the operator is having the suite reviewed for correctness.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
REMOVED, NOT DISABLED. TuneIndicatorsByFilter() and ScoreCurrentParamsByMI() are gone, along with
the block in TuneIndicatorsAndTrain() that invoked them. AutoTune.mqh 701 -> 473 lines.
WHY REMOVAL AND NOT A FIX. The leak was the cheap half of the problem. The expensive half is that
the search CANNOT WIN:
* It scored candidates by per-column MUTUAL INFORMATION against an overlapping label. That
statistic sits at the noise floor on this data, so 324 candidates per model moved 0.00236 ->
0.00236 on all three AD configs and 0.00370 -> 0.00370 on PAI, and the family-wise gate
correctly refused every winner it ever produced.
* It searched PER CHART. A per-chart winner is a per-chart overfit, and this project ships one
parameter set for every instrument.
* It leaked one terminal indicator instance per candidate (CIndicator::Create overwrites the
handle, it does not release). Measured 2026-09-04: 20,076 live full-history buffers, 27.4 GB
committed and never touched, 202,125 "VirtualAlloc failed in large allocator" lines in a day.
Fixing only the leak would have bought a cheaper way to lose.
KEPT DELIBERATELY, and this is the part a wholesale delete would have broken:
* FeatureColumnMI / BuildMiSample / ScoreMiSample stay. They are the sampling layer under the MI
DIAGNOSTIC, and FeatureScreen.mqh calls all three - that screen produced the per-column table
showing cumdelta clears 4/6 columns on all five charts while wyckoffEvent clears 0/16. The
sampling was never the problem; using it to SELECT was.
* CADIndicatorTuner stays. It still holds the ACTIVE indicator settings ReInitADIndicators builds
handles from. It is now a settings holder, not a search.
* The .nnw indicator-parameter field stays. Dropping it would change the persistence format and
force a second retrain on top of the one the const AutoTuneIndicators already triggered.
WHAT REPLACES IT: Scripts\ResearchADParams.mq5, run once, offline, output edited into the _DEF
constants by hand. It changes the three things that were actually wrong:
1. BLOCK-LEVEL, NOT PER-COLUMN. Fits a block's k buffers jointly against the label, which is what
the network does with them and has far more power than k univariate tests.
2. ATR PER CALL, NOT NATS. The metric is the mean forward RIDE RETURN of the bars the fit ranks
highest - the same currency as the book the deploy gate judges. MI was never convertible to it.
3. SELECT IN-SAMPLE, REPORT OUT-OF-SAMPLE, on a PURGED split. Max-of-N is biased upward by
construction, so the IS number is printed but explicitly labelled as not evidence.
Ranking is MAXIMIN ACROSS CHARTS - a candidate is only as good as its WORST instrument - because a
mean lets one strong chart carry a set that is useless on the other four, which is precisely how
this project has previously promoted results that then failed to replicate.
Every iCustom handle is released in the iteration that created it.
A FLAW IN ITS OWN VERDICT LINE, recorded rather than quietly patched: the script prints "clears
every chart OOS" when maximin OOS > 0. That test is nearly free, because every chart's no-model
baseline is already POSITIVE (+0.09 to +0.74 ATR - the forward ride pays a drift). The meaningful
quantity is the EDGE over that baseline, which the per-chart lines do print. Read the edges, not
the verdict.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
MEASURED, NOT INFERRED. The commit charge was 43,221 MB against a 44,498 MB system limit - 97% of
everything this 32 GB box can commit - while the WORKING SET was 1,625 MB. A 26x gap means memory
that is committed and never touched, which is a reservation, not a leak in the usual sense.
HOW IT WAS FOUND, after four wrong guesses (MaxBars, OpenCL, our own arrays, the history cache -
each refuted by measurement, none of them it):
1. Region enumeration of the live process (VirtualQueryEx) instead of more theorising:
20,076 private allocations of EXACTLY 1,433,600 bytes = 179,200 doubles = 27,448 MB
10,038 private allocations of EXACTLY 319,488 bytes = 39,936 doubles = 3,058 MB
179,200 doubles is one full-history buffer at this bar count, rounded to MQL5's 1024-element
allocation granularity. So: twenty thousand live full-history buffers.
2. Same terminal, same 5 charts, same history, EA renamed so it could not load: 154 MB.
That is the whole terminal-side cost. Every one of the missing 33 GB is ours.
3. Commit did NOT scale with chart count (3 charts 33,349 MB, 5 charts 32,189 MB), which killed
the standing "super-linear in charts" theory. It is a fixed cost paid per tuner run.
4. The TERMINAL JOURNAL - logs\, not MQL5\Logs\, which is where I had been looking all day and
the reason this took as long as it did:
202,125 x "VirtualAlloc failed in large allocator" in one day
~155,000 AD custom indicator instances torn down: 33,309 ADCumulativeDelta, 30,424
ADWyckoffEventStream, 30,424 ADShorteningOfThrust, 30,406 ADWyckoffFailedStructure,
30,289 ADWyckoffSignificantBarInversion
Removals arrive in bursts of 6,000-12,600 inside a single minute. A burst that size means that
many were LIVE AT ONCE - churn would be spread out. That is the 20,076 buffers.
[[project_indicator_handle_leak]] diagnosed this exact mechanism on 2026-08-08, wrote down the
diagnostic recipe that found it again, and concluded "AutoTuneIndicators=false is both the unblock
and the right default". It was set to true. This makes it stick.
WHAT THE TUNER BOUGHT FOR 27 GB: nothing, ever, on this data. It has never returned an accepted
winner - 0.00236 -> 0.00236 on all three AD configs, 0.00370 -> 0.00370 on PAI. Its objective sits
at the noise floor ([[project_mi_noise_floor_verdict]]), so the family-wise gate correctly refuses
every candidate. It is a search that cannot win by construction.
const, NOT input bool ... = false. MT5 stores inputs PER CHART, so an attached EA keeps its saved
value and ignores a changed default - the trap that cost a deploy cycle in 2679b02. As a const it
also drops out of the .chr and .tpl input lists, so the fleet template cannot carry the old true.
MEASURED AFTER: 2,850 MB commit / 1,875 MB working set, flat over 5 minutes, 33.8 GB headroom.
Commit now TRACKS the working set instead of standing 20x above it, which is the real tell that the
untouched reservation is gone.
TWO FATAL CHART-KILLERS FIXED ALONGSIDE, both the same shape: an unchecked allocation under the
memory pressure this was causing, then an out-of-range read, which in MQL5 is fatal and REMOVES THE
EXPERT while the terminal keeps running and logging - the silent-dead-chart mode of
[[project_fleet_selfheal_dead_charts]]. Both were observed killing charts today.
* Labels.mqh (666,32), killed XTIUSD at 21:31:51. The prebuild's two tail loops bounded their
reads by m_labelPrebuildBars - the bar count the pass STARTED with - rather than by the array
they were about to index. The identical guard already existed on the loops at 574 and 605; these
two were missed. Also: m_labelCacheBars is the length every reader trusts, and it was set to the
REQUESTED bars even when the resize came back short. It is now set to what was actually
allocated, so a short allocation shortens the window instead of corrupting memory.
* BaselineComparator.mqh, killed XAUUSD at 21:08:45. ALGLIB allocates without checking and then
indexes what it did not get: CDForest asserted "rows(xy)<npoints", the forest then reported a
PERFECT 0.000 OOB error (a failed build, not a good one), and MLPCreateC1 died on "index out of
range in dataanalysis.mqh (8779,41)". These baselines are a diagnostic that nothing downstream
depends on, so they now refuse to start unless TERMINAL_MEMORY_AVAILABLE covers 4x the matrix,
and verify the matrices' actual dimensions before writing a single row.
NOT THE CAUSE, checked because it was proposed: CrossAsset. UseCrossAsset(true) is never called
anywhere - the only writer is the setter itself - so m_useCrossAsset is permanently false,
BuildCrossAssetPanel returns at its first line, Warm() is flag-guarded, and its 6 features are never
added to the 92. It costs nothing at runtime. It is dead code and worth removing for that reason,
but removing it would have recovered zero bytes.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
TWO OPERATOR REQUESTS.
1) THE OPTIMIZER IS NO LONGER AN INPUT. One input applied one optimizer to all four ensemble
members, which is the opposite of what a voting ensemble wants - members that fail in correlated
ways average to nothing. Each model class now names its own via the new virtual
PreferredOptimizer(), so the choice lives with the architecture instead of in a global switch.
It also had to stop being an input on this project's own rule: it is retrain-forcing (a field of
the model filename), and retrain-forcing values are not inputs, because MT5 stores inputs PER
CHART and an already-attached EA ignores a changed default - the exact trap that cost a deploy
cycle in 2679b02.
ASSIGNED WITHIN WHAT IS ACTUALLY IMPLEMENTED, and no further: only SGD (heavy-ball momentum) and
ADAM exist here, and ADAM already carries DECOUPLED weight decay - it IS AdamW. RMSprop, Nesterov
and amsgrad are NOT implemented and are not pretended to be.
Perceptron / MLP -> SGD, the deliberate odd one out. No structural prior, so it overfits
fastest and adaptive steps make that worse; SGD is an implicit
regulariser and makes it the conservative ANCHOR vote.
Conv / LSTM / ConvLSTM -> ADAM (AdamW).
The learning rate follows the optimizer: InitialEtaForOptimizer() now TAKES it instead of reading
the global, which would have handed every member the same LR and re-created exactly the
uniformity this change removes.
THIS IS ALSO THE CHEAPEST TEST OF THE DIVERSITY HYPOTHESIS. Member correlation is already printed
every era - currently r 0.19-0.36 across the fleet, i.e. 4 models worth ~2.2 independent ones. If
varying the optimizer moves that number, implementing RMSprop/Nesterov/amsgrad is justified by
measurement; if it does not, the hypothesis is refuted for the price of one config line. I expect
a small effect: the members already differ by ARCHITECTURE and still correlate at 0.29 on
average, so the shared input vector and shared label look like the real cause.
2) ERA CAP 100 -> 10000. 100 was arbitrary and far too low - an era here is a full pass over ~179k
bars, and networks of this shape are normally given thousands of them. It no longer terminates
anything either (that was removed the same day), so all a low cap did was fire a redundant deploy
and a log line every 100 eras while charts sat past era 260.
DB key note: the chart-level config fingerprint now hashes TrainingOptimizerDefault rather than any
per-member value, which is correct - per-member differences live in each model FILENAME - and leaves
every existing database key byte-identical.
COST: PAI re-keys to era 0 on every chart, and the ensemble lead cap will hold its siblings until it
catches up. Done now deliberately: the terminal is already restarting for an unrelated MT5 live
update, so nothing converged is being discarded that was not restarting anyway.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The previous commit split m_trainingComplete into "may trade" and "has stopped learning", but the
.nnw carries a SINGLE boolean written before that split. PersistLoadNetOnce read it straight into
m_trainingComplete, so any model that deployed under the old build was resurrected frozen.
Observed directly after the restart: the three charts that had deployed earlier (NAS100, XAUUSD,
XTIUSD) completed ZERO eras while the three that never deployed (EURUSD 29, GBPUSD 28, USDJPY 4)
trained normally. Same build, same terminal, same minute - the only difference was what their file
said.
Load now translates a persisted "complete" into m_deployedLive with m_trainingComplete FALSE: the
model comes back TRADING and CARRIES ON LEARNING. Save writes (m_deployedLive || m_trainingComplete)
into the same slot, because writing only m_trainingComplete would silently lose deployed status
across a restart now that deployment no longer sets it - the model would return unable to trade.
Old files migrate on first load. Nothing needs deleting and no era count is lost.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
StartLabelCachePrebuild refused to arm on `m_zigZag.GetData(0, 0) == EMPTY_VALUE`, reported as "the
handle is COLD or DEAD". But a HEALTHY ZigZag is empty at almost every bar - it carries a value only
AT a pivot - so the test fired whenever the newest bar simply was not one. While ticks arrive, bar 0
keeps moving and the condition clears by luck. The moment the market CLOSES, bar 0 freezes on a
non-pivot and the prebuild can NEVER arm again.
MEASURED, same evening, same build otherwise:
* 16:47 restart (broker ~20:47, market open): 0 such messages, first era completed in 50 seconds.
* 19:26 restart (broker ~23:26, week closing): 150 such messages and NOT ONE chart completed a
single era in 15 minutes.
The handle was never cold. The terminal journal reported "custom indicator ZigZag loaded succesfully"
on all six charts, and the LEG STATE REPLICA verified against that same buffer across 1299 pivots
while the prebuild was calling it dead.
BarsCalculated() <= 0 is the actual readiness signal - it means the indicator has produced nothing,
which is what "cold or dead" was meant to catch, and it does not depend on whether any particular bar
happens to be a pivot.
WHY IT MATTERS BEYOND TONIGHT: it is Friday. Without this the fleet sits idle for the entire weekend
while appearing healthy - the terminal alive, all six charts logging, no errors, no rejections. The
only symptom is the absence of eras, which is exactly the class of silent failure this project keeps
getting caught by.
I first suspected my own lifecycle change from the preceding commit. The diff was clean and the
market-open/market-closed comparison above is what separated the two.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Operator instruction, given more than once: "let the nn learn, there should not be a gate. we only
filter inputs by correlation."
m_trainingComplete meant TWO things at once - "this model may trade" (LongCondition/ShortCondition
return 0 without it) and "this model has stopped learning" (trainingPending gates on it). So
deploying froze the chart. Measured today: THREE of six charts had stopped training, and two of them
froze on books the gate itself had refused - XTIUSD deployed after -0.198/-0.013/+0.026/-0.121, all
four NOT DEPLOYABLE. A frozen chart also stops publishing rows to the shared training pool, which is
the one lever that ADDS independent observations, so freezing quietly undid the reason for running
six instruments.
Split into m_deployedLive (may trade) and m_trainingComplete (has stopped learning). Deployment now
sets the first and leaves the second alone, so a later era can still promote a better checkpoint.
Four sites changed:
* ENSEMBLE DEPLOY: goes live, no FinalizeTrainRun, and the once-only guard is now keyed to
m_deployedLive - keying it to m_trainingComplete would have re-fired the branch every call and
returned before training ran, a HARDER freeze than the one being removed.
* era-end deploy: no era.stop, no completion; announces once.
* LongCondition/ShortCondition: m_deployedLive also permits trading.
* the lifecycle if/else became TWO independent tests, so a deployed model refreshes its live signal
AND keeps training. A merely-complete model still takes only the first branch, as before.
THE ERA CAP IS GONE AS A STOP, and it also stopped asking. PromptContinuePastEraCap opens a
MessageBox in a LIVE terminal - on an unattended chart that blocks the EA thread waiting for a click -
and on "no" it set m_trainingComplete and ended the run for good. Both halves are gates on learning.
The cap now does the only useful half of its old job: it takes the best checkpoint live and resets
the cap window so training continues.
No m_trainingComplete = true remains anywhere in Training.mqh.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
MEASURED, and the estimate that justified the expansion was wrong. I sized the step 3 -> 6 at ~1 GB
per chart from the 3-chart steady state. The real figure is ~5.8 GB per chart COMMITTED: six charts
reached 34.8 GB and left 4.2 GB of system headroom, against the 2.6 GB at which the operator VS Code
was dying this morning.
THE COST IS ADDRESS SPACE, NOT RAM. Working set was 7.4 GB against 34.8 GB committed - about 1.2 GB
touched per chart against 5.8 GB reserved. Windows charges the reservation against the commit limit
(RAM + pagefile), so the unlock is a LARGER PAGEFILE, not fewer charts. That needs admin this session
does not have.
AND THE LIMIT ITSELF MOVES: 49,650 -> 44,498 -> 48,113 MB across today, because the pagefile is
auto-managed. Headroom can therefore fall by several GB without anything being allocated, which is
why 4.2 GB is not a safe margin to leave unattended even though it was stable while watched.
GBPUSD is the one dropped: most correlated with EURUSD in the set, so it adds the fewest INDEPENDENT
observations, which is the entire point of pooling. Remaining set spans an FX major, JPY, an index,
a metal and an energy.
NOTE: this only stops GBPUSD being RE-ADDED. FleetExpansion has no close path - it opens and revives
only - so the already-open chart stays until closed by hand or by a deliberate contraction feature.
I did not add self-closing charts unattended; a bug in that path closes the fleet.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Everything that removes PARAMETERS was tried and measured this session and none of it reaches the
budget:
* correlation pruning: 92 -> 54 columns even at |r| >= 0.60, which already deletes genuinely
different features
* STANDARDISED PCA: 45-46 of 92 components carry 95% of the variance. The "8 components" figure the
PCA plan was built on came from RAW-variance PCA and was almost entirely a scale artefact - on
the correlation matrix the data genuinely spans ~45 dimensions. k=8 would discard real
information and k=45 does not cut enough.
So there is no redundancy left to reduce away, and the constraint is the SAMPLE, not the encoding.
Adding an instrument is the only lever that ADDS independent observations instead of removing
parameters - the capacity line has said so all along.
Pooling is already working (each chart adopts 8-16k rows from its two peers), so this multiplies
something that is already plumbed rather than starting it.
WHY 6 AND NOT BACK TO 10. The fleet was cut to 3 this morning for an OOM whose root cause is now
fixed and measured - eleven masked-out feature blocks were still loading and computing, commit
27.8 GB -> 3.3 GB, headroom 2.6 GB -> 32.7 GB. The operator instruction was "max 3 charts IF it is an
OOM crash"; the premise has changed but the caution has not, so this steps 3 -> 6 (~1 GB per chart,
leaving ~27 GB headroom) and measures before going further.
CHOSEN FOR INDEPENDENCE. Pooling correlated instruments adds rows without adding much information,
so the set spans FX majors, JPY, an index, a metal and an energy. XAUUSD and XTIUSD are also the two
the leg-ride label actually paid on in its first-run verdict (+0.467 on gold, positive long book on
oil) while FX was ~zero and the index negative.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The "8 of 92 columns carry 95% of the variance" figure that the PCA plan is about to be built on
came from PCABuildBasis run on RAW columns, so it ranks directions by RAW variance - and these
columns are not on a common scale. FEATURE HEALTH measures widest/median at ~21x today, and before
the Wyckoff zone sentinel was fixed it was 1137x, at which point a SINGLE COLUMN was the entire
first component and the report said "1 of 92 carry 95%, top component 100%". A raw-variance
component count therefore partly measures our unit choices rather than the information.
Standardising each column to unit variance first makes PCA operate on the CORRELATION matrix, which
is the scale-free version and the only defensible basis for choosing k. It matters more now than it
would have a week ago: the supervised keep-screen is gone, so PCA runs on unscreened columns and
nothing else is left to stop scale deciding the answer.
Both counts are printed, because the GAP is the diagnostic - if standardising moves the number a
lot, the raw figure was measuring units. Constant columns are left at 0 rather than divided by a
~0 sd, which keeps them out of the basis instead of inventing a direction for them.
The report also states the rule the transform will have to follow: k comes from the STANDARDISED
number, and the transform must standardise with the SAME per-column mean and sd it was fitted with -
fitting on one scaling and applying another is the silent version of this same bug.
Measurement only. No feature vector changes, no fingerprint change.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Shipped an hour ago asserting "the encoder is at its minimum hidden size, so the only term left to
cut is FEATURES PER BAR". The live readings are H=16 and H=32 against a floor of 8, so that is
simply false, and it made the line understate the options: at H=32 it reported F<=0 for alpha=2,
which reads as impossible, when dropping H to the floor affords F<=48 on the same budget.
An LSTM costs 4*H*(H+F+1). H and F trade against each other and BOTH are levers. The line now prints
the affordable F at the CURRENT H and at the floor, and says plainly that cutting H costs
representational depth while cutting F costs information - so which gives way is a judgement, not an
arithmetic result. What the arithmetic does say is that at alpha=0.40 both have to move, and no
choice of H alone reaches the band while F stays at 92.
I wrote the false claim from the earlier chart where H was 8 and generalised it without checking the
others.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The capacity line now reports a real whole-network weight count, and the first honest readings are
11018 weights = 3.03 per independent observation on the widest model and 1738 = 0.48 on the
narrowest. The old line said 0.2, because it counted the first dense layer only.
"3.03 weights per observation" only means something against a target, so the standard bound is
stated with it: total parameters <= N_eff / alpha, alpha in [2,10]. At N_eff ~3763 that is 1882
weights at alpha=2, 753 at alpha=5, 376 at alpha=10 - and the widest model implies alpha=0.33, i.e.
six to thirty times outside the band.
THE LINE ALSO NAMES THE ONLY REMAINING LEVER, because the ratio alone does not. Both stages are
ALREADY budgeted against effN individually - ComputeFirstLayerWidth solves H1 = N_eff/(N_in+1) and
ComputeLstmHiddenSize solves 4*H*(H+inputs+1) <= N_eff - so this is not a missing constraint, it is
two constraints applied SEPARATELY whose sum is 2-3x either one. And the encoder is already at
LSTM_HIDDEN_MIN on these charts, so hidden units cannot come down further.
That leaves FEATURES PER BAR. An LSTM costs 4*H*(H + F + 1), so at the minimum H the affordable F
follows directly: F<=50 at alpha=2, F<=14.5 at alpha=5, against the 92 emitted now. The line prints
those numbers per chart so the feature set can be designed against a budget instead of an opinion.
That also gives PCA a target rather than a guess: 8 components carry 95% of the variance (measured
after the Wyckoff sentinel fix), and 8 features per bar sits inside the alpha=5 budget of 14.5.
Required IFeaturesView::LstmHiddenSize, since the budget arithmetic needs the encoder width.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
TWO FIXES, THE FIRST OF WHICH IS AN OUTAGE I CAUSED.
1. FATAL CAST. The weight walk did `CNeuronBase *n = layer.At(i)` - an implicit downcast off
CLayer::At(), which returns CObject*. A layer can hold CNeuronBaseOCL, a SEPARATE hierarchy that
does not derive from CNeuronBase, so the cast raised "incorrect casting of pointers in
Network.mqh (683,22)". That is FATAL in MQL5 and REMOVES THE EXPERT FROM THE CHART: all three
charts stopped training at 14:56 and stayed down for 30 minutes, with the terminal alive and the
log still writing, which is exactly the signature project_fleet_selfheal_dead_charts describes.
Every cast in the walk is now a dynamic_cast, which returns NULL on a mismatch.
2. THE COUNT THEN RETURNED 0, because these models run on the DLL/OpenCL stack and every
dynamic_cast to the CPU hierarchy legitimately missed. The report printed "0 trainable weights =
0.00 per independent observation" - the most reassuring lie this line could tell, on the exact
question it exists to answer. WarriorNetTotalWeights now handles BOTH hierarchies (CNeuronBase
via WeightCount, CNeuronBaseOCL via getWeights) and lives at the END of Network.mqh because it
must see a class declared after CNet. When it still cannot count, the line says capacity is
UNKNOWN rather than zero.
The lesson is the one this file keeps teaching: a diagnostic that fails silently to a flattering
value is worse than no diagnostic. A zero here is a failure to count, not a measurement, and it must
read as one.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
It reported "first dense layer is 9 x 16 = 144 weights against ~868 independent observations = 0.2
weights per observation" - a comfortable number - while IS/OOS said the same models were overfitting
13-90x. The two disagreed because the diagnostic was measuring a SLICE: on a sequence model the
first DENSE layer sits AFTER the encoder, so it sees ~9 inputs, not the 552 that enter the network.
A capacity figure that ignores the encoder is not a capacity figure, and this one was quietly
reassuring on the exact question it existed to answer.
Added a virtual CNeuronBase::WeightCount(), overridden where the weights do not live in Connections:
CNeuronPool counts its OutputLayer, CNeuronLSTM counts its FOUR GATE LAYERS plus the output layer.
CNet::TotalTrainableWeights() walks the stack. The CAPACITY line now prints both - the first-layer
figure it always printed, and the whole-network total with its own weights-per-observation - and
says plainly which one to judge capacity on.
I also have to correct myself: earlier today I asserted the first dense layer was "~17,600 weights
against 3,763 observations". That was wrong for the sequence models - it assumed the raw 552 inputs
fed a dense layer directly. The direction of the concern survives (the encoder holds thousands of
weights nobody was counting) but the number I gave did not, and the fix is to measure it rather than
to argue about it.
Context for why this matters now: correlation pruning cannot rescue the width. Sweeping the prune
threshold gives 92 -> 82/78/69/59/54 columns at |r| >= 0.95/0.90/0.80/0.70/0.60, i.e. 552 -> 324
inputs even at an aggressive 0.60 that starts deleting genuinely different features. Redundancy is
not the whole story, so the real parameter count is the thing that has to be known.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
A prune at one threshold answers "how much is duplicated at 0.90". It does not answer the question
that matters: whether correlation pruning ALONE can cut the input width far enough to matter against
~3,763 independent legs. At 0.90 it drops 15 of 92 columns - 552 -> 462 inputs, a 16% cut - which is
nowhere near enough for an IS/OOS gap of 13-90x.
So the report now sweeps |r| in {0.95, 0.90, 0.80, 0.70, 0.60} and prints columns and resulting input
width at each. If even the aggressive end leaves the width large, redundancy is NOT the whole story
and the rest has to come off by ROTATION (PCA) or by shortening the lag window - pruning can only
remove information that is DUPLICATED, never information that is genuinely there.
What the 0.90 pass already found is worth recording: two pairs at |r| = 1.000.
* cumdelta[1]~cumdelta[0] - not our bug. ADCumulativeDelta defines
pressure = normalizedCumulativeDelta + 0.25*initiative - 0.15*absorption, so buffer 0 is a
near-copy of buffer 1 and the adjustments are negligible against the variance.
* wyckoffEvent[6]~wyckoffEvent[0] - event direction and struct direction share a sign exactly.
Feeding both of each pair is pure waste, and the prune found it without being told what to look for,
which is the point of filtering on the INPUTS rather than on the label.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The redundancy report counted redundant PAIRS and named the worst by INDEX ("worst is columns
54/60"), which nobody can act on. It now names them and runs the prune the operator asked for:
filter on CORRELATION - a question about the INPUTS, needing no label - rather than on marginal
significance, which is the network job and which structurally cannot see a feature that is useless
alone and useful in combination.
GREEDY, LOWEST-INDEX-WINS: walk columns in emitted order, drop any that correlates above the
threshold with one already kept. Deterministic, no fitted basis to persist, no IS/OOS split, no
fingerprint of its own - unlike PCA, which rotates the space and must carry a basis with the model.
Start here; reach for PCA only if this does not cut enough. Reports what a prune WOULD leave
(columns, lags, resulting input width) and the dropped columns with the twin each duplicates.
MEASURED, NOT APPLIED.
WHY NOW, and it is the most urgent number on the page: IS/OOS has blown out to 13-90x. EURUSD books
+2.1..+3.4 ATR per call in-sample against +0.06..+0.09 out; USDJPY +1.5..+2.2 against +0.03..+0.16;
NAS100 is NEGATIVE out-of-sample on 4 of 5 recent eras. Memory records this gap as 3-5x, and the
input width tripled (180 -> 552) in between. That is textbook over-parameterisation against ~3,763
independent legs, and redundancy reduction is the only lever that cuts width WITHOUT discarding
information.
Required a small interface addition: ITrainingData::FeatureSlotName, so a report living in the
comparator can name a column owned by the feature builder.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The stop frontier on all three charts says a TIGHT stop improves the ATR book: EURUSD -0.125 ->
-0.075, USDJPY -0.055 -> -0.014, NAS100 +0.040 -> +0.121, all at a 0.50 ATR stop. That is the
opposite of the mechanism it was built to test - it is cutting LOSERS early, not letting winners run
- and it rests entirely on the one thing the report does not model.
At an 80-85% hit rate, slippage lands on five of every six trades. A caveat is not good enough for a
number that would otherwise look like the best result of the day, so it is now falsifiable:
BEslip = advantage over no-stop / hit rate
the slippage PER HIT that erases a rung entire advantage. A rung whose whole advantage is 0.05 ATR
at an 84% hit rate dies at 0.06 ATR of slippage, which on an H1 stop-out is an ordinary number
rather than a pessimistic one. If BEslip comes back smaller than a realistic slip, the rung is an
artefact of the frictionless assumption and must be read as one.
This is the same discipline the cost-split and horizon reports were held to: state the assumption
that is doing the work, then print the number at which it breaks.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Operator asked whether the NN magnitude signal can drive dynamic SL/TP to stop cutting winners
early. Two prior findings shape the answer, and both are respected here rather than re-litigated:
* DIRECTION IS CLOSED, but excursion SIZE clears at 3.9-5.4x its null on three instruments. So
magnitude is the right thing to lean on - for RISK CONTROL, which is what the excursion verdict
already says it is good for.
* THE RATIO IS EV-NEUTRAL for a driftless walk (reaching +m before -k has probability k/(k+m),
which IS break-even). Geometry cannot manufacture alpha, so nothing here claims it does.
What CAN be true is the operator premise: with Exit_On_Leg_Flip and NO take-profit, the stop is the
ONLY thing that can end a trade early, so a stop inside the ride typical adverse excursion silently
converts winners into losses. Nothing in the era report could see that - g_ensVoteUpR/DnR are
excursions over a FIXED horizon, not over the ride.
So the missing measurement is what each ride had to SURVIVE. LegRideLabel already walks every bar
between entry and the flip to find it; it now also tracks the worst adverse excursion on that walk
(lowest low for a long, highest high for a short, from the entry close, in ATR at entry) at one
comparison per bar and no extra pass. Cached per bar, carried onto each scored vote row.
The STOP FRONTIER then reports, at the certified rung: the book with NO stop at all, and the book
with a stop at each of 9 distances, with hit rate. THE COUNTERFACTUAL IS EXACT, not modelled - a
stop at S fired iff the measured ride MAE >= S.
REPORTED IN R, NOT ATR, and that is the point. A wider stop means a proportionally SMALLER position
at the same risk percentage, so comparing stop distances in ATR compares trades of different size
and always flatters the widest. Dividing by the stop distance is what the account actually
experiences.
Slippage is not modelled - a hit is priced at exactly -S - so the TIGHT rungs are flattered, and the
report says so rather than hiding it.
ALSO CONFIRMED, NOT BUILT: the operator second request (reject a trade when even the smallest lot
breaches the risk threshold) ALREADY EXISTS and is correct - MoneyRiskBase.mqh:147, "BELOW-MINIMUM
MEANS NO TRADE, NOT A BIGGER TRADE", refusing rather than letting TCNormalizeVolume round a
risk-derived 0.05 up to a 0.10 minimum and double the intended risk.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
FEATURE HEALTH now reports SCALE, and the first thing it found was severe:
median column range 2, widest/median 1137x
wyckoffEvent[4]=2274 (x1137) wyckoffEvent[5]=2265 (x1133)
ZoneTop and ZoneBottom ARE ATR-normalised - the defect is that the indicator reports "no active
zone" as 0, and `(0 - close) / atr` then evaluates to -close/atr. That ratio is scale-free, so it is
about -580 on EURUSD and -580 on NAS100 alike, injected into a vector where every other column lives
in +/-2.
WHY IT MATTERS MORE THAN A COSMETIC OUTLIER:
* It owns the COVARIANCE MATRIX. This is the entire explanation for the redundancy report claiming
"1 of 92 columns carry 95% of the variance (top component alone 100%)". The data is not
one-dimensional; one column is three orders of magnitude wider than the rest. Every PCA or
correlation-prune decision taken on that report would have been taken on an artefact.
* It owns the FIRST LAYER GRADIENTS. A column that much larger dominates every weight update, so
the network was learning mostly from a sentinel and drowning the other 90 columns.
Same defect class as CArrayDouble::At returning DBL_MAX for a negative index, found earlier today: an
out-of-band marker that arithmetic consumes without complaint. Guarded via WarriorZoneDistAtr, which
returns a NEUTRAL 0 for an absent level - the honest encoding of "there is no zone here" - and also
rejects EMPTY_VALUE and non-finite input, because an indicator with no data must not be read as a
price of zero. Width contract unchanged.
The SCALE half of FEATURE HEALTH is the durable part. The report has always named CONSTANT and
mostly-zero columns and never said how WIDE they are, so a scale outlier was invisible to it while
being the most damaging thing a column can be. It now prints the median range, the widest/median
ratio and the five widest columns by name, and says plainly that columns must be comparable in
magnitude before PCA or a correlation prune means anything.
Still outstanding from the same report: ichimoku[3,4,6] at 17-21x the median. Plausible for real
(close-cloud)/atr on small-ATR bars rather than a sentinel, so it is left measured and unpatched.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The label horizon has been called "the only lever that raises evidence" for weeks and was never
measured. This measures it, from PRICE and the leg replica alone - no network, no training, no era.
What it computes per ZigZag depth: legs (which IS the effective sample size, because every bar
inside a leg shares that leg outcome), mean leg life (= the label overlap), the ORACLE ride a
perfect caller takes, the round-turn cost over the same window, the break-even capture, and the
total at 2/5/10% capture.
EURUSD, 60k bars:
depth 4: legs 8994 life 6.7 oracle 2.055 BE-capture 1.62% @2%=+71 @5%=+625 @10%=+1549
depth 6: legs 6780 life 8.8 oracle 2.556 BE-capture 1.33% @2%=+116 @5%=+635 @10%=+1502
depth 8: legs 5330 life 11.3 oracle 3.046 BE-capture 1.13% @2%=+141 @5%=+628 @10%=+1440
depth 12: legs 3763 life 15.9 oracle 3.865 BE-capture 0.90% @2%=+160 @5%=+597 @10%=+1324
depth 16: legs 2887 life 20.8 oracle 4.581 BE-capture 0.75% @2%=+165 @5%=+562 @10%=+1223
depth 24: legs 1662 life 36.1 oracle 6.347 BE-capture 0.52% @2%=+156 @5%=+473 @10%=+1000
THREE RESULTS, TWO OF WHICH KILL MY OWN FRAMING:
1. Cost never binds. Break-even capture is 0.39-1.62% at every horizon, against rides of 2-8 ATR.
The "shorter horizon sells payoff to buy evidence" trade-off this report was designed around
barely exists.
2. At the ~2% capture this fleet has demonstrated, the horizon is nearly FLAT: +141 to +165 across
depths 8-36, with the SHIPPED depth 12 already within 3% of the peak. Shortening to depth 6 makes
it WORSE (+116). The horizon is not the lever.
3. Capture rate is first-order - 2% -> 5% roughly quadruples the total at any depth - and shorter
horizons only win once capture is high (at 10%, depth 4 is worth 2x depth 16).
The first cut of this report ranked by the ORACLE and therefore picked depth 4 on every chart, which
is simply wrong: it credits a horizon with money no model here has ever taken. Ranking now uses the
demonstrated capture, and the capture columns are the ones to read.
Also fixed while here: LEG_STATE_DEPTH was NOT in the model fingerprint, though it sets where every
pivot falls and therefore which leg each bar belongs to, its direction, its ride, and whether it is
Buy/Sell/Neutral at all. Two models at depth 12 and depth 6 train on completely different targets and
were sharing a key. Now TGT:LEG1:<minride>:D<depth>. The ZigZag replica takes depth/deviation/backstep
as arguments defaulting to the shipped #defines, so every existing caller - including the in-situ
verification against the stock indicator - is byte-identical.
WHERE THIS POINTS: capacity is dominated by WIDTH, not horizon. Depth 12 gives 3763 independent legs;
the mask-off change took inputs 180 -> 552, so the first dense layer is ~17,600 weights against 3,763
observations. Halving the horizon buys 1.8x observations; tripling the width already spent 3x. The fix
is the redundancy filter - correlation between INPUTS, no label involved - which is designed in
project_pca_reduction_plan and never built: measured worst pair |r|=0.999, PCA takes obs/param 0.49 ->
11.9.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
CiIchimoku serves Tenkan and Kijun (buffers 0/1, no offset) perfectly and always has. The two SPAN
buffers carry Offset(kijun), which makes CIndicatorBuffer::Refresh issue
CopyBuffer(handle, num, -m_offset, m_size, ...) - a NEGATIVE start_pos - and THAT is the only path
that has ever failed here. Six hypotheses died to measurement before this was accepted as the fix:
stdlib cannot serve the buffers - false, a probe read them fine
full-history count at negative start - false, returns 179048 err=0
a second Create corrupts the buffers - false, after 2 Creates spanA still ok
primer/tuner parameter mismatch - false, tunedKijun=26 matched the primer
the primer was missing - false, it ran and the spans were still EMPTY
buffer starvation from re-init - live state was Total=10 spanA(0)=EMPTY
The live object fails in a way a constructed one will not reproduce, and I could not pin it. So the
dependency is removed instead of diagnosed further: the spans are copied ONCE PER ERA from the raw
handle at start_pos 0 - measured reliable on all three charts, 179022 values, err=0 - into our own
series-indexed arrays, and the kijun shift is applied by the READER. Same semantics the wrapper
Offset(kijun) provided, now stated where it can be read instead of inside a base class.
The raw handle is created with the SAME TUNED parameters as the wrapper, because a different triple
is a different indicator instance - reading 9/26/52 does nothing for a chart whose tuner moved to
9/30/52 - and the old handle is released first so ReInitADIndicators cannot leak one per call.
VERIFIED LIVE at the full 552-input width: 0 rejections, 0 stalls, all three charts training
(EURUSD era 14 in 18s, USDJPY era 3 in 11s, NAS100 era 6 in 10s), combined vote scoring at era 13.
Also: compile staging now excludes .venv/.git/docs/pdb/obj, which cuts the per-compile copy from
505 MB to 7.4 MB. It does NOT speed the compile - MetaEditor genuinely takes ~83s on this codebase -
it just stops copying a 386 MB Python virtualenv into the terminal folder every time.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The keep-screen scored each column MARGINAL mutual information against the label and dropped what
did not clear Benjamini-Hochberg at q=0.10. That is univariate feature selection, and it
structurally cannot see a column that is worthless alone and valuable in combination - which is
most of the interesting ones. sin(hour) is independent of "will this leg continue"; a breakout AT
THE LONDON OPEN is not the same event as a breakout at 03:00. Volume alone is noise; volume crossed
with range is not.
The header of FeatureMask.mqh already stated this blind spot - "the screen tests each column
MARGINAL information, so it cannot see a column that is useless alone and useful in combination" -
and the mask was built on it anyway, so every pass deleted precisely the features a network exists
to combine. We were using an algorithm to do the model job, in a single pass, and then recording
the result as "this data is worthless".
FEATURE_MASK_VERSION 12 -> 0, which restores the full column set of every enabled block exactly, as
that knob was designed to do. Width 6x36 -> 6x92 = 552 inputs. Re-enabled: time (hour/day/month,
cyclic), volume, ATR, fracdiff, alt data, spread, and the four AD/Wyckoff blocks. Block selection is
now a toggle decision made on data value and engineering cost, never on a per-column significance
test.
THE LEGITIMATE FILTER IS REDUNDANCY, NOT SIGNIFICANCE. Two columns carrying the same information is
a correlation question between INPUTS and needs no label at all - worst measured pair is |r|=0.999.
That screen stays and is the next thing to build; the supervised one is gone.
ONE BLOCK REMAINS OFF AND NOT ON A VERDICT: cross-asset loads FOUR EXTRA SYMBOLS of full history per
chart, which was the largest contributor to the 27.8 GB of committed address space that was crashing
the machine. It returns when there is headroom to measure it in.
MEASURED COST OF THIS CHANGE, stated plainly: commit 3.7 GB -> 33.3 GB, headroom 26 GB -> 5.1 GB.
The AD/Wyckoff iCustom instances are that cost. Porting them to direct source inclusion is the fix
that buys back both the memory and the throughput.
Baseline to judge the new features against (pre-time, mask v11): USDJPY book +0.042/+0.067/+0.062
with alpha -0.077/-0.052/-0.058; EURUSD -0.070/-0.155.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The operator was right that Ichimoku is a stdlib indicator like every other, and it is. Three
hypotheses of mine died to measurement today:
1. "MT5 will not serve the Senkou buffers" - false; the wrapper reads them fine
2. "the full-history count at a negative start_pos fails" - false; returns 179048, err=0
3. "a second Create corrupts the buffers" - false; after 2 Creates, Total=10, spanA still ok
What actually correlates with the fix is the one thing the probe started DOING rather than
reporting. While it copied 64 values the feature path still failed every bar with spanA/spanB
EMPTY and no chart completed an era. The moment it began issuing FULL-HISTORY CopyBuffer calls on
buffers 2 and 3 at init, the feature path started working on all three charts - 0 rejections, 0
stalls, eras in 21-34s, combined vote scoring - with no other functional change between those two
builds.
The reading: the Senkou plots are shifted kijun bars FORWARD, and that shifted region needs one
full-range materialisation before CIndicatorBuffer::Refresh - which asks at start_pos = -offset for
m_size values - will serve it. Priming costs two CopyBuffer calls per init. Not priming cost the
fleet hours across two days.
CAUSALITY IS INFERRED FROM SEQUENCE, NOT PROVEN BY ISOLATION, and the header says so. It is renamed
from WarriorProbeIchimokuBuffers to WarriorPrimeIchimokuBuffers and marked DO NOT DELETE AS
SCAFFOLDING, because I was one turn away from removing it as spent diagnostics - which would have
re-broken the fleet and left no trace of why. Buffer 3 is primed alongside buffer 2 even though
nothing reports on it: the feature block reads SenkouSpanB every bar exactly as it reads SenkouSpanA.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
I put a raw CopyBuffer plus a ten-argument StringFormat on the Ichimoku EMPTY-read branch to
capture the one number that would have prevented an earlier misdiagnosis. That branch runs once per
FAILING BAR, and transient rejections are entirely normal during a cold pass-1 scan, so it became a
terminal API call per bar.
Measured cost, on a fleet that had been healthy minutes before:
pass 1 throughput ~15,000 bars/s -> ~1 bar/s
EURUSD LSTM bar 1536 of 179042 after 1180s
USDJPY CONV bar 1024 of 178952 after 1277s
combined-vote eras scored in 20 minutes: ZERO
Every chart stopped completing eras and the ensemble never scored, so nothing could be certified.
The branch was invisible before this only because the feature had been disabled - re-enabling
Ichimoku is what exposed a cost that had been sitting on a path nobody was walking.
The raw-handle comparison it was meant to capture already lives in the one-shot init probe, which
answers the same question once per start instead of once per rejected bar.
THE RULE: a diagnostic belongs where the diagnosis is READ, not where the failure is DETECTED. A
rejection path in a per-bar loop is a hot path, and it does not stop being one because the feature
that walks it happens to be off today.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
I switched Ichimoku off earlier today on the reasoning that "MT5 will not reliably serve the Senkou
buffers", from the symptom that spanA/spanB read EMPTY at every index while tenkan/kijun read fine
on the same bar. That conclusion was wrong. The operator pushed back - it is a stdlib indicator like
every other - and an init-time probe settled it by measuring the thing I had only inferred:
wrapper: BufferResize(39502)=ok BarsCalculated=39502
at idx 5 -> tenkan=ok kijun=ok spanA=ok spanB=ok
raw: spanA CopyBuffer(start=0)=64 err=0
spanA CopyBuffer(start=-26)=64 err=0 spanA[0]=29432.895
On all three charts. CIndicatorBuffer::Refresh reads shifted buffers with
`CopyBuffer(handle, num, -m_offset, m_size, m_data)`, and that NEGATIVE start_pos is deliberate and
works - it is how the forward-plotted cloud region is addressed. The wrapper is coherent and the
indicator serves data.
What was actually broken is the double-shift fixed in 5cd93bf: CiIchimoku::Initialize already applies
Offset(kijun_sen) to buffers 2 and 3, and the feature block added kijunShift a SECOND time, reading
raw index idx + 2*kijun. Re-enabled to test whether that alone accounted for it, and it does -
since the re-enable: ZERO Ichimoku rejections, zero feature rejections of any kind, zero stalls, and
both EURUSD and USDJPY completed era 0 with the block live. Width back to 180 (6 bars x 30),
FEATURE_MASK_VERSION 10 -> 11.
THE DIAGNOSTIC GAP THAT CAUSED THE WRONG CALL, now closed. CDoubleBuffer::At returns EMPTY_VALUE for
any index >= m_data_total, so "EMPTY at every index" is equally consistent with a bad handle and with
a short buffer - and I never measured which. The failure path now prints a raw CopyBuffer result and
the TUNED kijun beside the EMPTY flags, so the number that separates those two cases is in the log
rather than inferred a second time. A temporary one-shot init probe (WarriorProbeIchimokuBuffers,
marked for deletion) prints the same comparison at startup.
The lesson is not subtle: when a symptom is consistent with two causes and one of them is "the
standard library is broken", measure before choosing that one.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Indices carry no commission, so costPx is 0, every row costR is 0, the median is 0, and
`costR <= median` is true for EVERY row - the whole population lands in the cheap half and the
other reports n=0. Measured on NAS100: "CHEAP n=595, DEAR n=0", with the cost multiple printing
0.00x because a book divided by a zero cost is undefined, not infinitely profitable.
Refusing to form the split is what makes the silence honest. A degenerate split that still prints
two halves invites the reading that one half "wins", when one half is the entire sample. The test
is the TOP of the distribution: if the dearest row still costs nothing, there is nothing to
compare.
Found because the fleet result came in and the NAS100 lines were unreadable next to EURUSD and
USDJPY. The measurement itself has now closed its own question - the two charts that CAN be split
disagree in sign - but the guard matters for every future cost-denominated statistic.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The keep-mask decides which EMITTED columns survive. It does NOT stop a block from RUNNING. Eleven
blocks that FEATURE_MASK_VERSION 10 discards were still loading history, creating indicator handles
and computing values on every bar so the result could be thrown away.
MEASURED, on a restart with nothing else changed:
terminal64 committed 27,817 MB -> 3,309 MB (-88%)
system commit headroom ~2,600 MB -> 26,491 MB (10x)
That 23.6 GB was being committed in the FIRST SIXTY SECONDS after launch, before a single era
completed - which is what finally localised it, after the training loop had been the obvious suspect
all morning. It is also the direct cause of the operator's VS Code dying overnight: Windows charges
COMMITTED pages against the commit limit whether or not they are ever touched, and at 94.6% of a
49.7 GB limit any process asking for a few hundred MB fails. The working set was never more than
2.1 GB, which is exactly why Task Manager looked healthy the whole time.
Switched off, each one named by the mask decision that already discarded it: volume (v6), time, atr,
fracdiff (v5), sot, wyckoffEvent, wyckoffFail, wyckoffBarInv, crossasset (v6), spread, alt (v5).
crossasset was the expensive one - it loaded FOUR EXTRA SYMBOLS of history per chart - and the four
Wyckoff blocks are iCustom, which is the per-bar cost.
THE EMITTED VECTOR DOES NOT CHANGE. FeatureBlockTable already sizes a disabled block to width 0 and
the mask already dropped these columns, so what reaches the network after masking is byte-identical.
Only the work disappears. Verified live: input 132 (6 bars x 22), zero feature rejections, zero
stalls, all three charts training.
const, NOT input, for the seven that were inputs. They move m_neuronsCount and are therefore
retrain-forcing, and this project's rule is that retrain-forcing values are not inputs - MT5 stores
inputs PER CHART, so an already-attached EA ignores a changed default and would have kept every one
of these pinned true. That is the trap that cost a deploy cycle in 2679b02.
THE COST, stated honestly: the keep-screen can only report on columns that are EMITTED, so these
blocks can no longer be re-measured without re-enabling them. Each was measured and dropped on
evidence across the v5-v8 screens. Re-enable deliberately for a measurement pass; do not leave them
on as a standing subscription, which is precisely what they had become.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The two Senkou plots are shifted kijun bars into the FUTURE, and CopyBuffer from position 0 across
a forward-shifted plot does not dependably fill. Measured on NAS100: `spanA=EMPTY spanB=EMPTY` at
series index 5 (raw index 31) while `tenkan=ok kijun=ok` at the SAME bar, same handle, same
refresh, BarsCalculated=39500. Buffers 0 and 1 healthy; buffers 2 and 3 essentially unpopulated.
WHAT IT COST. With every cloud read failing, not one of 39494 scanned bars produced a usable
feature window, so every era was discarded and restarted from scratch, forever. It idled charts
for 66 minutes on 09-03 and for 18 more on a FRESHLY STARTED terminal on 09-04 - so it is not the
revive path and a restart does not cure it. The tell is the member split: PAI reached era 4 while
CONV/LSTM/HYB sat at era 0, because only the sequence models carry a 6-bar lookback window and one
dead column kills the whole window.
THE TOGGLE IS WHAT HAD TO MOVE, NOT THE MASK, and getting this wrong would have shipped a no-op.
The keep-mask decides which EMITTED columns are kept; CFeatureBuilder's Ichimoku block is gated by
UseIchimoku() (Warrior_EA.mq5:806) and would still have RUN and still returned false on an EMPTY
span, failing the whole window before the mask was ever consulted. Both are moved here: the flag
stops the block executing, and FEATURE_MASK_VERSION 9 -> 10 drops the columns and forces the
retrain.
AND THE EVIDENCE FOR KEEPING IT WAS NEVER VALID. The 7/8 and 3/8 screen votes v8 cites were
measured while the block DOUBLE-APPLIED the cloud shift (fixed hours earlier, 5cd93bf), so the
screen voted on columns lagged a full kijun from what they claimed to be. There is no trustworthy
measurement that Ichimoku earns its eight columns. Dropping them takes the vector from 30 to 22
per bar, 180 to 132 inputs, which also buys back capacity on the one constraint this project has
identified as binding.
REVERSIBLE, AND THE WAY BACK IS NOT THIS FLAG. Senkou A is (Tenkan + Kijun) / 2 by definition and
buffers 0/1 read perfectly, so the cloud can be reconstructed from those plus a highest/lowest over
the Senkou-B period. Do that if a screen ever asks for it back; do not re-enable the buffer reads.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
151f2bc fixed CTrainPool::Add() - it capped a reserve that was `16384 * m_width` and it checked the
ArrayResize return. CTrainPoolReader::Take(), forty lines away, still had both, and it is the side
that runs the moment a SECOND chart publishes a matching pool.
THE RESERVE. `16384 * m_width` is a reserve of 16384 ROWS, so a wider feature vector bought a
proportionally larger allocation for no benefit: 23.6 MB at the current width of 180, per reader,
per member, per chart. The reserve is an allocation hint and has no business being a function of the
row width. Capped in ELEMENTS at 65536, the same constant and the same reasoning as the writer.
THE UNCHECKED RESIZE, which is the serious half. ArrayResize returns -1 on failure and the loop then
wrote at m_rowCount * m_width regardless. An array-out-of-range is FATAL in MQL5 and REMOVES THE
EXPERT FROM THE CHART - which is exactly the failure that left five charts idle for 77 minutes
through two terminal restarts, invisible because fleet expansion only ever asked whether a chart
existed. The writer was hardened against it and the reader was left behind.
ON FAILURE THE ROW IS STILL CONSUMED, and this is the part worth reading twice. Adopt() advances the
file by exactly one record per iteration and relies on Take() having read m_width doubles; the
else-branch beside the call already reads them explicitly "to stay on the record boundary". Returning
early without reading would desynchronise every subsequent row in the file - turning an allocation
failure into silently corrupt training data adopted from a peer. Skipping a sample is affordable;
losing the record boundary is not.
Found while tracing why the terminal commits 32.6 GB of address space against a 2.0 GB working set.
This is NOT that 30 GB - the pool is capped at TRAINPOOL_MAX_ROWS 60000, which is 86 MB at width
180 - and the over-commit source is still open. It is a real crash path on its own.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
AnimateDread