NOT COMPILED - user compiles.
Root cause of the Neutral collapse. Logit adjustment (Menon et al. 2020) makes a
classifier Bayes-optimal for BALANCED error by subsidising rare classes. It was
wired here when Neutral was the DOMINANT class - the "big move up / big move down
/ nothing much" era, where the correction pulled the model off the majority.
The triple-barrier relabel (b4a704d) inverted the distribution. The barriers are
now the EA's own SL/TP, so ~89% of bars RESOLVE and only timeouts are Neutral.
Measured on SP500 H4, from the EA's own log:
measured priors Buy 48.26% Sell 41.13% Neutral 10.61%
log-prior spread 1.52 | tau 1.00 CAPPED to 0.79
Neutral became the RAREST class, so the correction started subsidising it - by
tau*(log pB - log pN) = 1.20 logits. With no directional edge to overcome that
(direction is closed at best-of-999, p=1.0000), the model took the free lunch:
OOS recall Buy:1% Sell:0% Neutral:100%
OOS raw out spread avg 0.9993 (softmax saturated, near one-hot)
dW/W bn1 0.000% bn3 2.0% bn5 6.5% (input weight block frozen; head twitching)
The anti-collapse mechanism was the collapse. The recall gate needs >=40% on all
three classes, so nothing could ever deploy and the plateau ladder burned eras.
Present in both runs today (b6b5 froze bn1 by era ~719, 17ae by ~169), so it
predates this week's work.
FIX: the correction now spans the DECIDABLE classes only, Buy against Sell,
centred on their midpoint, with Neutral pinned at offset 0. Neutral is the
ABSTAIN outcome and abstention already has a better owner - m_dirConfThreshold,
refitted every era on the held-out calibration band against a coverage floor and
the measured break-even. Subsidising the abstain class does that job twice and
spends the whole correction suppressing the only decisions that can pay.
What still gets corrected is real: a trending symbol resolves more long barriers
than short, and uncorrected the model inherits that as a standing directional
bias. Here it is log(0.4826)-log(0.4113) = 0.16, so the offsets are tiny - the
correct answer, not a broken one. The two traded classes were already balanced;
the old spread of 1.52 only ever described how rare a timeout is.
Everything is derived from the measured distribution, as requested - offsets from
the priors, cap from the resulting spread. tau itself is deliberately NOT fitted:
tuning it against the same data that selects the checkpoint would add another
search dimension to a project that has been burned by exactly that. tau=1 is the
theory value and the cap (now ~9.5x looser at spread 0.16) will rarely bind.
Log line now reports both spreads and, when the abstain class is the rarest, says
how much the old form would have boosted it. Fingerprint |LA:<tau> -> |LA:<tau>:BS
so models trained under the all-three form re-key instead of resuming.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
b197999 gated the repaint on "this era beat the best checkpoint". That was not
what was asked for and it changes policy rather than fixing a bug, so it is
reverted - behaviour is now byte-identical to before b197999. Only a comment
recording what was verified remains.
What the check found: there is NO repaint or erase at the start of an era. With
NMS on, passes 1, 2.5 and 3 only RECORD predictions into m_arrowSignalCache -
pass 1's own comment says "NMS on: record only - the era-end sweep is the SOLE
renderer, so no raw (un-declustered) arrow is ever drawn mid-era" - and
PruneDirectionalClusters runs once, at the end of pass 3, which is the end of the
era. The requested behaviour was already the implemented behaviour.
So the arrows vanishing at the era boundary is not a timing fault. That sweep
DELETEs the arrow on any bar the era scored Neutral, and the model is currently
scoring Neutral on 98-100% of bars (see the logit-adjustment finding: Neutral is
the RAREST class at 10.6% and the imbalance correction is subsidising it by ~1.2
logits). The chart is reporting the model accurately; the model is the problem.
One thing that CAN clear arrows at era 0, and did on the first attach after the
|ALTW re-key: ClearPersistedChartSignals("fresh topology at era 0 - arrows belong
to a previous model"). That fires once per fresh model, not per era.
Compile-verified: 0 errors, 0 warnings.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
User report: "as soon as the next era training begins the chart signals are
erased, they should persist for as long as they are accurate."
Cause: PruneDirectionalClusters ran unconditionally at the end of pass 3, and it
DELETEs the arrow on any bar the CURRENT era scored Neutral. A model exploring
away from its best therefore wipes the chart every era even though the best
checkpoint still calls those turns. On the run that prompted this the model sat
at OOS recall Neutral:100% for 40+ consecutive eras, so essentially every arrow
was deleted at every era boundary.
The render is now deferred to the era-end block - the first point that knows
whether the era beat the best checkpoint - and only a new best repaints. On eras
that did not improve, the previous best's arrows stay untouched. Two exceptions
keep the chart from ever showing nothing: before the first checkpoint exists
there is no best to preserve, so early eras still paint; and a finishing run
repaints unconditionally, because FinalizeTrainRun is about to restore the
deployed weights and the chart must describe THOSE.
Recorded for ensemble members too. A member's own best era is not the deployable
one (the joint checkpoint decides that), but it is still the most accurate thing
that member has drawn, and the alternative is a chart that empties itself.
Also verified against the log, since two other symptoms were reported alongside:
era cadence PAI-b6b5 (before these changes) 3.76 s/era
PAI-17ae (after) 3.53 s/era
topology both "2 dense from 16 units | input 800 (16 bars x 50)"
calibration both fitted on 1699 held-out bars
So training speed is unchanged - the ~6% is the ranking slice removing 20% of
pass 3's bars. It only FEELS fast because this is a single PAI chart taking the
whole 120 ms budget, not four ensemble members sharing it behind an era barrier.
The Neutral collapse is also pre-existing, not new: b6b5 ran at Neutral 94-100%
with 1-5% directional calls for all 723 of its eras, before any of this work.
That is the known neutral-collapse/recall-gate failure mode, and it is what
"barely drawing signals" actually is. Worth watching, separately: b6b5 reached
best-bal 34.2% by era 723 while 17ae is at 13.6% after 77 - too early to read,
but it is the number to check once 17ae has run comparable eras.
Compile-verified: 0 errors, 0 warnings.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
User's call before deploy: "I would rather avoid lagging so the NN finds
accurate patterns." Correct instinct, and it picks the conservative variant.
110b384 deduplicated the WHOLE window, so every distinct reading survived at one
slot. The flaw is which slot: it depends on where the calendar-day boundary falls
inside that particular window, and on H4 that boundary cycles through ~6 phases.
A dense layer holds a separate weight per (slot, feature), so a given lag would
have landed on a different coordinate from one window to the next - turning a
stable lagged input into a moving one.
Now it blanks only bars carrying a BYTE-IDENTICAL copy of the anchor's reading
and stops at the first bar that differs. An as-of lookup into a daily file is a
step function in time, so those copies are exactly the contiguous run of bars
sharing the anchor's calendar day. Everything older keeps its natural replicated
run, in the same slots it always occupied - whatever the net learned to read
there, it still reads there.
Why the anchor's reading is the right one to isolate: the window's newest slot IS
the bar being predicted (BuildFeatureWindow's final iteration lands on r, and
pass 3 grades that same index), so it is the reading contemporaneous with the
decision - and the only one the alt screens ever validated. They measured the
CURRENT reading's MI against forward range and never tested lags, so the lagged
content is unproven, which is a reason to leave it undisturbed rather than a
licence to rearrange it.
What is still fixed: the anchor's reading reaches the first layer on one
coordinate instead of once per bar of its day, removing the ~16x gradient
upweight for the validated signal. And this is IDENTICAL to full dedup exactly
where replication was worst - on M15/H1 the whole window sits inside one calendar
day, so the anchor's run is the whole window - and a no-op on D1, where the bar
before the anchor is already a different day and the loop breaks immediately.
The two differ only on middle timeframes, and there this is the safe side.
Fingerprint |ALTW:1 -> |ALTW:2 so nothing trained under the hour-old full-dedup
semantics can silently resume under these.
Compile-verified: 0 errors, 0 warnings.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
aba9bd2 kept only the newest slot's copy of the external block. That is right on
every intraday timeframe and WRONG on D1: there the 16 window bars are 16
distinct calendar days, the daily alt file returns a different row for each, and
blanking 15 of them destroyed real information instead of a copy of it. Caught
while extending the measurement to the other instruments.
Now compares values instead of slot positions: walk newest -> oldest, keep the
last DISTINCT reading, blank a slot only when it repeats one a newer slot
already carries. Exact on every timeframe with no timeframe test, and it also
handles weekends, holidays and publication gaps, where a window spans fewer
distinct rows than calendar days. How much collapses falls out of the data:
M15 x 16 bars = 0.17 calendar days -> 1 distinct row -> 15 of 16 blanked
H1 x 16 bars = 0.67 calendar days -> 1 distinct row -> 15 of 16 blanked
H4 x 16 bars = 2.67 calendar days -> ~3 rows -> ~13 of 16 blanked
D1 x 16 bars = 16 calendar days -> 16 rows -> NOTHING blanked
OTHER INSTRUMENTS - the question that prompted this. Per-symbol exports for
USDJPY/XAUUSD/EURUSD are not on disk (written only when that chart is attached),
but they are not needed: CAltData reads a DAILY file for every symbol, so bars
sharing a calendar day are byte-identical by construction everywhere. What
varies per symbol is only WHICH sources, and the catalog (AltDataFetch.mqh
AddSpec rows) gives:
SP500 13 = risk(3) + cot_spec_net(1) + eia(3) + mac(6)
EURUSD 15 = cot(3) + risk(3) + eia(3) + mac(6)
USDJPY 15 = cot(3) + risk(3) + eia(3) + mac(6)
XAUUSD 14 = risk(3) + ivol(2, GVZCLS) + eia(3) + mac(6)
Measured observation-date gaps in the raw sources on disk: VIX, USD index,
DGS10, T10Y2Y, T5YIE, DFF, ECBDFR all 1 day; COT and EIA 7 days; CPI and UNRATE
31 days. NO per-bar source exists anywhere in the catalog - the tick-activity
survivors from the flow screen are an in-terminal feature block, not alt data,
and are untouched by any of this. So the redundancy is universal across
instruments; only its magnitude varies, and by timeframe rather than by symbol.
Compile-verified: 0 errors, 0 warnings.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Measured on the live SP500 D1 export (6073 rows, 13 features, 5888 simulated
16-bar windows):
distinct values per feature per window : 1.7 - 2.7 of 16 slots
variance in the first 13 PCs : 96.5 - 97.0%
components for 95% / 99% : 12 / 17-19
effective rank (entropy) : ~11.5
208 inputs carrying about 12 dimensions. Only 6 of the 13 features move daily
(VIX complex, USD, the rates trio); 5 are weekly (COT, EIA, output gap) and 2
monthly (CPI, unemployment). The lookup is as-of by bar open time into a DAILY
file, so bars sharing a calendar day are byte-identical by construction.
The cost is NOT overfitting capacity - collinear copies span ~12 directions,
not 208, so an earlier claim that this wasted 26% of the model overstated it.
It is GRADIENT WEIGHTING. Batch norm standardizes each of the 208 coordinates
independently; that rescales the copies without decorrelating them, so one
factor arrives on 16 unit-variance coordinates, each weight takes a full-size
step, and the factor's aggregate coefficient moves ~16x faster than a per-bar
price feature's. The network was biased toward the external block by a factor
of the window length - and pointing the wrong way, since these features cleared
only a marginal incremental screen while price is the base signal.
Zeroed at WINDOW ASSEMBLY, not in BufferTempData: that output is cached PER BAR
and a bar sits at slot 15 of one window and slot 0 of the next, so a
slot-dependent value there would poison the cache or force a recompute per slot.
The cache keeps true values; only this window's copies are cleared. Width
contract untouched - same count, same positions - so conv/LSTM/HYBRID keep their
bar-major rectangle unchanged and the block arrives at the newest bar, which for
the LSTM is the final timestep. Zero-variance coordinates are safe through batch
norm (divisor is MathMax(MathSqrt(var + BN_EPSILON), BN_MIN_STD)).
Fingerprint gains |ALTW:1 when alt data is on. Same width and same .cfg, so
nothing else would have caught a model trained under the replicated layout
resuming under this one. Conditional append per the existing rule: configs
without alt data keep their fingerprints and their trained models.
NOT the concat branch. CNet is a strictly linear stack (CLayerDescription has no
input-source field; NetBuild wires i to i+1 and stores layer L's weights on
L-1), so a real two-tower model needs a new multi-input layer type across
WarriorCPU, WarriorDML and the OpenCL kernels plus an .nnw format change - the
highest-risk change in this repo, in the code that produced the transposed dense
gradient, the Adam second-moment bug and the reversed LSTM window. This captures
the part of that idea the measurement actually supports, at no engine risk.
Compile-verified: 0 errors, 0 warnings.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
TOPOLOGY - reverts the two constants and drops CausalHiddenLayerFloor.
The MQL5 article's 30%-per-layer cut and floor of 20 are load-bearing on ITS
first-layer width of 1000 (1000->300->90->27 needs a floor to stop). This
codebase MEASURES that width, and on the live SP500 H4 config it is 16 units -
already floored, with the budget printing "11360 estimated in-sample bars
cannot support a 800-wide input ... roughly 1.1 weights per training bar -
expect overfitting". At 16 units a floor of 20 makes lastHidden >=
m_initialNeuronsCount, so ComputeHiddenLayerCount returns on its first branch
and the width taper - the only part derived from this symbol's data - became
dead code on all four ensemble members, with depth (2 -> 4) set entirely by
counting feature domains. ComputeLayerWidths had already rejected this exact
pair of constants in its own comment.
The causal floor's premise does not hold either: layers are not inference
steps. The "1 layer linear / 2 nonlinear / 3 multi-connected" result is
Lippmann 1987 and is about hard-threshold units; with sigmoid/ReLU, Cybenko
1989 and Hornik 1991 give universal approximation from a single hidden layer.
Depth buys parameter efficiency for compositional functions, not reasoning
hops. ForceHiddenLayers remains for measuring depth directly.
RANKING SLICE - the backfill no longer reads the window it is judged on.
The deployed checkpoint is CHOSEN as the best-scoring era on the OOS window,
so win rates measured back over it are selection-inflated, and the backfill
was writing exactly those into the table filter weights rank on: the
selection set consumed twice, beside a deploy gate that applies a Sidak
correction for that effect. The newest RANK_SLICE_PCT_OF_OOS (20%) of the OOS
window, plus a label-horizon purge, is now reserved and graded by nothing -
not pass 3, not checkpoint selection, not the gate. The backfill reads only
that. The gate keeps ~80% of its measurement (power goes as the square root,
so ~10% of a sigma), and the slice is the newest data, which is the regime
about to be traded. RankSliceBars returns 0 when no honest slice fits and the
backfill then REFUSES and says so, rather than falling back to the scoring
window and looking like a success.
SHRINKAGE - per-tier win rates are shrunk toward the filter's own pooled rate
by MIN_TRADES_FOR_WIN_RATE pseudo-trades before becoming weights. The raw
ratio at the minimum sample count carries a ~15pp standard error, so a tier
that went 8-2 was handed weight 80 and outranked a tier measured over
hundreds of calls at 55 - the ranking was being driven by which small tier got
lucky. Opt-in per call site (priorWeight 0 keeps the raw behaviour).
Compile-verified: 0 errors, 0 warnings.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Four defects in 64c5dd5/1a05e63, found by review + a baseline compile.
Goals 1-8 of that session are unchanged; this makes 6 and 8 actually reachable.
1. HEAD DID NOT COMPILE - 6 errors. CControlPanel::Minimize/Maximize were
declared `virtual bool ... override`, but CAppDialog declares both as
`virtual void` (Controls\Dialog.mqh). errors 265 + 404 on each, plus 151
on `bool ok = CAppDialog::Minimize()`. Return type is void now; there was
never a success flag to forward. Verified: 0 errors, 0 warnings.
2. THE BACKFILL COULD NEVER ADVANCE, and neither could the OOS continual
simulation (that one has been dead since it was written). Both are armed
at the instant convergence is declared, and both advance only from inside
Train(), one chunk per call. But ScheduleTrainingIfNeeded's only per-tick
ArmStudyEvent site sits in the `else` of a branch taken whenever
m_trainingComplete is set and m_trainRunActive is clear - which is exactly
the state FinalizeTrainRun() leaves behind one line before they are armed.
Train() was never called again, so the walks sat at their start index
forever: no "simulation complete" line, and not one row written to the DB
this feature exists to fill. Only a manual Resume/Retrain unstuck them.
Both flags now keep the model schedulable.
3. IN AI_HYBRID - the mode this ships in - the backfill was never even armed.
Ensemble members deploy at Train() ENTRY and return immediately (so no era
is wasted), which skips the era-end block the backfill was started from.
All four members were a no-op for a second, independent reason. Armed on
the ensemble deploy path too, from m_resumeBars/m_resumeOosCutoff.
4. RE-RUNS DUPLICATED ROWS. RegisterSignal inserts unconditionally - no key,
no duplicate check - and m_dbBackfillDone is in-memory, so every later
attach that retrained to convergence wrote a second full set of rows for
the same bars. The ranking would count one bar once per model that ever
deployed, weighting superseded opinions as heavily as the live one. A
.dbfill marker stamps the deployed era; written only on completion (an
interrupted walk redoes itself rather than ranking a partial window) and
deleted with the other sidecars on reset-weights.
Also: WarmBlocking's timeout was silent, which restored the exact silent
pin failure it was added to prevent - it now says so in the journal, and
returns true for "no reference pairs to wait for" so the warning stays rare
enough to be read.
Not addressed, needs a decision: the backfill scores the OOS window with the
checkpoint that was SELECTED as best on that same window, then writes those
win rates into the table filter weights rank on - the selection set consumed
twice, undiscounted, while the deploy gate right next to it applies a
family-wise correction for exactly that effect. The rows are also simulated
triple-barrier outcomes at today's spread sharing a table with realised
fills. The completion log line now states both plainly.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The unit of evaluation in ensemble mode becomes the vote, because the
vote is what trades (user: "at the end of the day they will vote
together during live trading so that would make sense").
Four decisions move from the member to the ensemble:
* which era is "best" -> the era whose COMBINED VOTE scored best
* what is checkpointed -> a JOINT snapshot: every member's weights
at that one era
* when the run gives up -> one shared plateau ladder
* whether it may deploy -> family-wise gate on the vote
WHY THE JOINT CHECKPOINT IS THE POINT: per-member selection picks each
net's own best era, and those eras differ. The resulting quartet was
never measured together at any instant, so the vote it casts live is a
configuration no OOS number ever described. Capturing all four at the
era whose vote won makes the deployed ensemble exactly the measured one.
Correct because of the era barrier (b77e7b4): Train() runs at most one
era per call and a member that finished era N is held until every member
reaches N, so when the last member scores the vote no member's weights
have advanced past end-of-era-N. That makes the deferred simultaneous
capture a guarantee rather than a race. Each snapshot is era-STAMPED and
deploy requires every stamp to equal the winning era - otherwise a member
whose capture failed would still hold an older snapshot and the deployed
quartet would again be one nothing measured. Partial capture rolls the
era back out of "best" so the search continues instead of freezing
behind a checkpoint that does not exist.
Statistics mirror the per-member gate one for one - same coverage floor
(MIN_COVERAGE_FRACTION_OF_BASE_RATE), same always-call-one-direction
chance reference, same EDGE_MIN_SIGMAS margin, same Sidak correction over
the eras ranked (DEPLOY_FAMILY_WISE_ALPHA). Only the population differs:
the bars the VOTE fired on, at Min_Vote_Open, rather than the bars one
member called. Two-sidedness is required of the vote itself - a vote that
never goes short IS the always-long model the chance reference prices in.
Members keep their own per-era statistics and their own learning-rate
dynamics (regression restore, eta decay); those are per-net training
mechanics, not deployment decisions. The shared ladder is mirrored onto
each member so per-era log lines report the state that actually governs
them. Solo charts are untouched on every path.
Verified: full MetaEditor compile, 0 errors 0 warnings.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Four user-reported/requested items, one root cause chain:
1) DEAD CONTROL PANEL in AI_HYBRID mode. All members posted custom event
id 1 and handled id 1001, and CExpertCustom broadcasts every chart
event to every filter - so each posted event ran a train chunk in ALL
N members (N*N chunks per round) and the chart thread never idled
long enough to deliver clicks/drags. profiling.csv: 99.45% of time in
OnChartEventHandler. Fix: per-instance study-event ids
(STUDY_EVENT_ID_BASE + construction order, offset above the Controls
library's ON_* codes - id 1 was also ON_DBL_CLICK, so panel
double-clicks fired training chunks). ArmStudyEvent() is the single
post site; lost-event watchdog replaces the accidental
sibling-clears-my-flag rescue.
2) WARM-UP DUPLICATION. The auto-tune sweep is deterministic over
identical features/labels, and it ends in the full MI diagnostic
suite, which the MI-share gate never intercepted on the sweep path -
four members ran four identical ~36s sweep+report blocks. First
member publishes outcome (g_ensembleChartTuneDone/Installed/Settings);
the rest apply it and skip both.
3) DEINIT STRANDED PANEL+ARROWS (user repro 18:52). Root cause from the
log: the 4,500ms budget runs from MetaTrader's stop REQUEST - a heavy
autosave in flight ate it, OnDeinit got ~430ms and died in the first
member's arrow persist ("Abnormal termination" 432ms in). Fix: early
visible-UI sweep (native prefix deletes for status/panel/dialog)
right after ClearStatusLabel, and a fast path for still-training
models - their arrows are re-rendered every era, so they get one bulk
purge instead of scan+atomic-write in the death window.
4) ENSEMBLE FEATURES (user requests): era BARRIER - members advance era
by era together; a member ahead of the slowest still-training member
declines Train() calls and its chunk budget is donated
(TRAIN_TIME_BUDGET_MS = 120/activeTrainers, UI headroom constant).
COMBINED-VOTE OOS SCORE - each member's pass-3 scan contributes its
adjusted per-bar decision (0.0 on abstain) to a shared row buffer;
the last member to finish the era scores the averaged vote vs the
mirrored Min_Vote_Open against the same target-before-stop outcomes
members grade themselves on, publishing an "Ensemble vote" line on
the aggregated panel. Member headlines now carry their lifetime win
rate with break-even.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The ensemble chart UI had a shared-namespace defect that answered the user
question "what do the arrows represent?" with "a bug": all four members drew
arrows under the same WarSig_<bartime> object names, so the chart showed
whichever member rendered LAST, one member Neutral deleted another member Buy
at the same bar, each member init sweep wiped the arrows the previous member
had just restored, and SaveChartSignals - which rebuilds the sidecar by
SCANNING the chart - persisted every other member arrows into its own history
(the exact cross-model laundering its own header warns about, now happening
BETWEEN ensemble members).
Arrows are now namespaced per member (WarSig_PAI_, WarSig_CONV_, WarSig_LSTM_,
WarSig_HYB_): draw, delete, restore, prune, member init sweep, destructor
purge and the sidecar scan are all member-scoped, and the tooltip names the
model. Global purges keep matching the bare WarSig_ prefix, which covers all
member namespaces plus old-format leftovers from earlier builds.
Labels: the ensemble panel header no longer says "HYBRID ensemble" (HYBRID is
one member; the header is the ensemble) and the CONVLSTM member displays as
ConvLSTM instead of Hybrid. Its SHORT id stays HYB deliberately - it names the
model folder and changing it would orphan every model trained under that path.
Deinit: the alt-data mapping dialog namespace (WarriorAltMap_) joins
WarriorChartPrefixes, so both the OnInit purge and the deinit final sweep now
cover it - it was in neither list, so a dialog starved of its own Destroy()
left its controls on the chart permanently.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
- PeriodMA/MA_Type/PeriodRSI: input -> const seeds (closing the set: every
indicator parameter is now tuner-owned)
- Variables\TunedPeriods.mqh: chart-level tuned-period state. A gated
install writes TunedPeriods_{SYM}_{TF}.cfg; next attach reads it BEFORE
the DB fingerprint and classic-signal config, so classic votes, DB key,
and tuner seeds always describe the same indicators regardless of
classic/AI/hybrid use. Restart-grained adoption by design (no mid-run
handle churn); new periods re-key the signal DB (semantics rule).
- EnableAltData input in AI Input Features (consumption gate only;
collection keeps running); |ALT DB-fingerprint token; opt-out on an
alt-trained model correctly starts fresh via the width compare.
- Defaults: all four classic votes OFF (AI-first; WARRIOR_MARKET_BUILD
branches collapsed with the marketplace pivot), order-flow/Wyckoff NN
features OFF (alt data is the default information diet; toggles stay).
Compiles 0 errors / 0 warnings.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The 18 inputs added 2026-08-08 (when the tuner defaulted off and the
values needed an operator path) become compile-time aliases of their own
defaults - same names, zero consumer churn, byte-identical values. The
tuner is now the only path by which these values move: it defaults ON
(the 08-08 off-flip was measured against the direction target's flat
landscape; the objective is now RANGE, which has signal), searches from
the seeds under the Sidak family-wise gate, and persists winners in the
.nnw beside the weights. ADP fingerprint token retired (deviation now
impossible by construction; tuned values were never its job).
Menu shrinks 102 -> 84 inputs. Compiles 0 errors / 0 warnings.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
- MI_TUNE_TARGET = MI_TARGET_EXC_RANGE: the coordinate sweep scored
candidates against the barrier label - measured noise - so it climbed a
flat landscape and the gate rightly rejected every winner. It now
selects indicator settings for MI vs realised RANGE (4x null, positive
control), the channel the excursion head consumes these features for.
Winner gate re-tests on the same target. Barrier-label report unchanged.
- AltDataFetch 4014 handling: Alert popup + once-per-session walkthrough
with the two whitelist URLs on their own journal lines (copy-paste
ready); hourly-backoff retry instead of a permanent latch, so the
whitelist fix takes effect without re-attaching.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The derived stop/target were quantiles of EVERY bar''s excursions over a
fixed horizon - q75 adverse gave a 2.6-3.5*ATR stop against a ~1.7*ATR
target (user: "looks limiting"). That pooled measurement was correct
when direction was dead (any subset of bars had the same distribution)
and is provably mis-sized now that the gate certifies the label carries
information: the bars the model trades are the labeled bars, and their
excursions differ from the pool.
FractalDirectionLabel now records, for every Buy/Sell-labeled IS bar
during the prebuild, the favourable and adverse travel in ATR units
over exactly the LEG the label points at - entry close through the next
fractal extreme (user request: "from a fractal to the next for maximum
accuracy"). DeriveBarrierGeometry reads the same q75-adverse/q50-
favourable quantiles off that conditional sample instead of the pool,
with a logged fallback to pooled when fewer than the minimum legs
exist. Quantiles kept over averages deliberately: a mean MFE is
dominated by runaway legs and would set an unreachable target.
No circularity: the fractal label does not depend on SL/TP (the barrier
label does - this path must never feed it). Recording stops the moment
geometry is derived and pinned, so pass 2 relabels and later bars
cannot silently re-shape a certified pair.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Two user-reported ensemble regressions, one cause each:
- "getting ready is very long": every member ran the full MI diagnostic
suite (headline MI, positive control, alignment, lag profile,
geometry scan + winner test - ~200 permuted draws per line) on
IDENTICAL features and labels, reporting the same numbers four times.
First member runs it, the rest adopt with one log line. Documented
caveat: if the geometry scan ever ADOPTS a winner under its gate
(it never has), the adoption becomes donor-only and the gate must be
revisited.
- "panel not responsive": four members chunks queue back-to-back on the
one chart thread - 4 x 120ms = 480ms worst-case click latency, the
exact regime the 200ms note in Training.mqh already documents as
broken. Ensemble members now use a 30ms chunk budget, restoring solo
UI latency at slightly higher dispatch overhead.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
All four ensemble members previously wrote their full multi-line panels
to the SAME global label objects - an ensemble chart would flicker
between four stacked panels covering the chart side (user request:
aggregate). Every AI-side SetStatusLabel call site now routes through
CExpertSignalAIBase::PublishStatus - solo charts draw the full panel
exactly as before; an ensemble member claims a slot and contributes
only its HEADLINE to one combined block ("HYBRID ensemble - N models",
then one line per model; the live line leads with the model current
signal). The combined render skips unchanged text and enforces its own
minimum redraw interval so four publishers cannot multiply
ChartRedraw() cost.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The user is right that no special combination logic is needed: the AI
signals are ordinary voting filters, and the aggregate already has
union semantics - abstaining filters do not dilute the average, so an
ensemble chart trades whenever ANY deployed member clears the vote
threshold and disagreeing members net out. What the ensemble preset
actually adds:
- AI_CHOICE value 4 renamed AI_CONVLSTM (the name says the front-end);
enum VALUES stable, CSignalHYBRID class and State\HYBRID\ folder kept,
so saved configs and trained models keep their identity.
- New AI_HYBRID = 6: enables PAI+CONV+LSTM+CONVLSTM together on one
chart - replaces four separate charts of the same symbol. Each member
trains and self-gates independently; only certified members ever vote.
- |ENS1 fingerprint token on every member, so an ensemble member's
weight files can never collide with a solo model of identical
settings on another chart of the same symbol (the duplicate-chart
guard would otherwise correctly fight over one .nnw).
- Private default AIType = AI_HYBRID: one D1 drop now yields every
topology's gate verdict for that symbol.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
User direction (2026-08-15): back to predicting swing turns, D1 charts,
fractals over ZigZag pivots (their call - balances classes, matches the
reference library target, and a 5-bar fractal confirms 2 bars after its
extreme so labels resolve nearly to the present with no repaint embargo).
- TRAINING_TARGET enum + TrainingTarget input: TARGET_BARRIER (Market
default - existing models keep their meaning and fingerprints) or
TARGET_FRACTAL (private default).
- FractalDirectionLabel (Labels.mqh): per-bar 3-class label = direction
from the bar close to the next confirmed strict 5-bar fractal extreme,
costs charged in the same bid-series convention as the barrier label,
Neutral when the move cannot clear max(2 spreads, 0.10 ATR) or on an
outside bar (both-extreme bars are unorderable within OHLC).
- The barrier walk still runs in full: measured SL/TP geometry, the
expectancy scan, excursion caches and the era gate all keep scoring
what a trade at the EA's own stop/target actually collected - only the
TRAINING label changes. NOT the pre-b4a704d "is this bar the pivot"
form; that target's 31:1 imbalance stays retired.
- Fingerprint token |TGT:FRA1 so switching targets trains a separate
model; AI_META unaffected (guarded setter).
- Private defaults: AIType back to AI_HYBRID (direction topology needed)
+ TrainingTarget=TARGET_FRACTAL = drop-on-D1-chart workflow.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
A resumed META model hot-looped pass 1 (0->100% scan oscillation, silent for
3 minutes until the stall reporter fired) because EVERY window failed at the
first AD/Wyckoff feature: the init-time param adoption called
ReInitADIndicators unconditionally, destroying five freshly-calculating
indicator instances to recreate them with BYTE-IDENTICAL params (verified by
parsing the .nnw header - the MI tuner had kept the configured settings), at
process start, on a box with 1 GB free of 31. The replacements sat cold for
6+ minutes while full-history resweeps starved the indicator threads harder.
- AdoptIndicatorParams: installs a loaded param set into the tuner and
rebuilds handles ONLY when the set actually differs from what the live
indicators run. Both call sites (resume init + panel reload) use it.
- Resumed models get the same 3 warm-up passes as fresh ones. The skip was
the shared root cause of the cold-ATR (ba13eef), cold-AD (2026-08-11) and
this incident - custom indicators recompute from scratch every process
start regardless of what the .nnw proves.
- Cold-sweep backoff: a pass-1 sweep in which every window failed on a
TRANSIENT cause arms a 5s era-start pause instead of an immediate
full-history resweep, so the retry loop stops consuming the CPU/memory the
warming indicators need. The stall reporter names the backoff branch.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
350-era S2 verdict on SP500 H1: the meta head carries REAL ranking skill
(+1.0-1.3pp mean over base, 101/350 eras clear their own 2-sigma bar, traded
subset wins 66.1% at <30% coverage vs 64.5% base) but 0/350 eras produced a
positive cov x (p - BE): the candidate stream sits 3pp under the derived
geometry's 67.5% break-even and ~2.6pp of recovered skill cannot bridge it.
Skill plateaued by mid-run (1.28pp -> 1.05pp), so more eras only buy
multiplicity, and the deploy gate correctly shipped nothing.
The aggregate can hide a deployable subset (one family/side clearing BE
blended with junk), so the META era line now decomposes the SAME traded
population into MA/RSI/MACD/Ichimoku x LONG/SHORT cells, each as
traded/candidates base->traded win rate. 32 cells is a best-of-N search by
construction - any candidate cell faces the family-wise rule before belief.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The NN now has a target that is not per-bar direction (closed, best-of-999
p=1.0000): P(win | this journaled candidate, at the EA's own SL/TP, net of
cost). One net for all 52 pattern-sides, AIType=AI_META.
- NetForward.mqh: the host-side softmax+CE gradient generalized total==3 ->
2||3 on both backprop paths; a 2-class softmax IS a logistic head, and no
compute backend changes.
- SignalMETA.mqh (new): corpus loaded read-only from the LARGEST signal DB on
disk (decoupled from the config fingerprint that burned four S1 runs); the
GMT->server offset is measured PER ROW against entryPrice vs bar open
(DST-immune, histogram logged); a window-span regime filter drops the
pre-2017 daily-backfill rows; 31-feature setup descriptor appended at the
input (26 one-hot + side + tanh netVote + SL/TP ATR + spread/ATR).
- Training.mqh: candidate-queued pass 1, binary-target pass 2, per-candidate
calibration (2.5) and OOS (3) walks. Counter mapping win->Buy / loss->Sell
lets checkpoint selection, the edge floor, the plateau ladder and the
family-wise deploy gate run UNCHANGED: precision reads as win rate among
traded candidates, chance as the base win rate, recalls as sensitivity/
specificity. Era-end META line: coverage x (p - break-even) vs the null.
- Labels are the side-conditional triple-barrier win caches - never the DB's
stop-and-reverse outcome. Logit adjustment deliberately skipped (~40% base
rate). Live inference + online learning guarded off until S3.
- Fingerprint: conditional |TGT:META1; State\META\ folder + 2-output filename
slot keep meta models fully separate from direction models.
Compiles clean (0 errors, 0 warnings). S2 run = attach a chart with
AIType=AI_META; S3 wires the votes via the per-side hooks.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The warning lived inside the VerboseMode-gated corpus report, so a
forgotten wipe silently voided an entire 18-year corpus run - the
outdated-row guard rejected the whole replay against leftover rows
and the run appended 35 rows instead of building a corpus. The check
now runs unconditionally at tester OnInit (MetaCorpusStaleCheck): 52
quiet one-row newest-key probes vs the test start, with a loud stop-
wipe-rerun instruction when the DB is newer than the test. Absent
tables probe quietly via FetchNewestTimeKey''s new quiet flag.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The first corpus build produced 3,681 rows, all 2026, from an 18-year
backtest: ProcessSignal''s outdated-row guard rejects any registration
older than a row its table already holds (correct for a live stream),
so a tester run starting before the leftover rows'' dates silently
registers nothing for the overlap. The corpus report now prints a
loud WARNING when running in the tester with DB rows newer than the
test''s start: corpus builds start from an empty DB.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Implements stage S1 of Meta_Labeling_Design.md, superseding the
original "training-time ladder sweep": the per-side journaling from
652bf81/195be20 already produces the exact candidate stream a sweep
would compute - every pattern instance the live ladders fire, both
sides, uncensored, with netVote and touchable entry price - so the
corpus is READ from the DB instead of re-implementing 26 ladder
conditions in training code. That eliminates the silent-divergence
trap outright: the corpus is by construction identical to live
behaviour. Accepted costs are documented in the module and the doc:
coverage equals the populating backtest, and sampling is one
candidate per fire-stretch (the right dedup for training anyway).
- Expert\AIBase\MetaCorpus.mqh: CMetaCorpus reader (52 tables ->
SMetaCandidate rows) + VerboseMode OnInit report: volume/closed/
S&R-win-rate per family, span, and the GMT->server bar-offset
match table (offsets +0..+3h) that S2''s label plumbing pins to -
measured, not assumed.
- DB_MaxRowsPerTable input (default 1000 = old MAX_TABLE_ROWS): a
corpus build raises it (e.g. 20000) so a 15-20 year backtest
isn''t pruned; wired through CExpertSignalCustom::MaxTableRows().
- Report-only stage: nothing downstream consumes the corpus yet.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The reference-pair set was re-discovered from Market Watch on every
build, so adding or removing a terminal symbol silently changed what a
trained model's six cross-asset features meant - the last open
train/serve parity gap from the 2026-08-11 audit. The set a model's
FIRST successful build actually used is now stamped into its .cfg
(append-and-length-guard, adopt-don't-compare - the derived-barrier
pattern) and every later build constructs the panel from exactly that
list; a pinned pair that is temporarily unavailable is skipped, never
substituted.
Also warms SymbolSelect/SeriesInfo for every reference symbol at
InitNeuralNetwork, so the terminal's ~minute of async cross-symbol
download starts at init instead of when the first Build() trips over
an unselected symbol - the source of the startup 'only 0 usable
reference pairs' console failures.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
On a CFD whose base and quote currency match (SP500 -> USD/USD) the FX
encoding degenerated: base and quote strength were the SAME series twice
and the divergence feature collapsed to the symbol's own 20-bar return.
Index mode re-encodes the six slots: denomination-currency strength
(fast/slow), a risk-proxy currency's strength (JPY by fixed preference
order - deterministic across rebuilds), and divergence as own move minus
what the denomination alone implies. FX-pair symbols are untouched.
Fingerprint gains :IDX2 for base==quote symbols only, so index models
trained under the degenerate encoding re-key while FX models keep their
filenames. FORCES RETRAIN on index/CFD charts.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
passTrail demanded m_excTrailScored >= EXCURSION_MIN_SCORED (500), but since
e2c9593 the trail race only scores DISJOINT bars: m_excTrailScored is bounded
by m_excScoredD (~OOS/horizon ~= 256 on SP500 H1) minus the post-ring-clear
warm-up (~8), so every chart failed "[trailing incumbent not warm enough to
race]" at 247-248 of a possible ~256 forever - observed live 2026-08-11 on
all four charts. The counter's statistical population is the same disjoint
sample passDj gates on, so it now takes the same minimum
(EXCURSION_MIN_DISJOINT, 200), reachable with margin after warm-up.
Compile: 0 errors, 0 warnings.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Three findings from the 2026-08-11 audit:
1. The excursion head's trailing-quantile ring was deliberately never cleared
between eras ("a rolling estimate of the market, not of the era") - but
pass 3 re-walks the SAME OOS window every era, so at each walk's restart
the ring still held the outcome masks of the newest OOS bars from the
previous walk: the chronological FUTURE of the bars about to be scored.
For the first ~window+horizon pushes of every era the "trailing" incumbent
was partly a leading one - conservative for the gate (an informed incumbent
is a harder hurdle) but exactly the self-made-artifact class 06d4785 hunts.
The ring now clears at era-score reset; the warm-up bars simply don't score
the trail race, which the m_excTrailN gating already accounts for.
2. skillTrail compared the head's FULL-block Brier (pro-rated by coverage)
against the incumbent's subset sum - valid only if head skill is uniform
across the OOS walk, while the trail-scored subset systematically excludes
each era's warm-up bars. The audit also found m_excBrierHeadD/BaseD/
m_excOosHitsD declared, zeroed and never accumulated (dead since e2c9593
made every scored bar disjoint). The dead trio is replaced by
m_excBrierHeadT: the head's Brier accumulated only on the bars the warm
incumbent also scored, so the race now compares both predictors on an
identical bar set.
3. The AD/Wyckoff feature blocks read GetData with no EMPTY_VALUE guard; a
cold (still-calculating) indicator returns EMPTY_VALUE everywhere, the
sanitize loop rewrote that to 0.0, and the bar SUCCEEDED - so
BufferTempData cached an all-zero Wyckoff block as a success for the whole
bar frame: the one path the f6150ee only-cache-successes rule cannot see,
because it never fails (the ba13eef class, arriving through values that
never fail; a resumed model's era-0 prebuild starts milliseconds after
OnInit). ADIndicatorCold() probes the NEWEST bar - EMPTY_VALUE there means
async warm-up (transient reject, retried), while deep bars beyond the
buffered depth keep the sanitize loop's neutral-fill so degraded history
still trains. Also fixed m_featureCacheValid's declaration comment, which
still described the pre-f6150ee cached-miss semantics.
Compile: 0 errors, 0 warnings.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
RefreshLatestSignal ran at the first tick after a bar opens and built its
window at r=0: series index 0 at that instant is a candle with one tick of
data - (close-open)/atr ~ 0, high ~ low, degenerate volume, indicators on a
1-tick bar. Training never produces such a window (every labeled bar is fully
closed, entry at that bar's CLOSE), so the deployed model's final timestep -
the one the LSTM/HYBRID output is keyed to - was out-of-distribution on every
live decision, and pass 3's deploy-gate OOS scores measured a different query
than live executed. The parity index is r=1: the newest CLOSED bar, whose
close IS the current price - the exact instant the label's hypothetical entry
happens. Single backtests shared the old skew (same r=0), which is why the
tester agreed with live while both disagreed with training.
Bookkeeping split that the index change forces: m_lastBarTime/dtStudied stay
anchored to the FORMING bar's open (they gate against SERIES_LASTBAR_DATE;
anchoring at bar 1 would re-fire the refresh every tick), while bt - the
arrow, its High/Low placement, and NMS declustering - anchors to the decision
bar, now matching the rescan path's convention.
Also: a failed refresh no longer trades the previous bar's signal for the
whole bar. RefreshLatestSignal returns success, zeroes dPrevSignal on failure
(no opinion beats a stale one), and RefreshConvergedSignal advances dtStudied
only on success so the next tick retries - the tester path (m_lastBarTime)
already worked this way; this is the live path catching up.
FORCES RE-VALIDATION of deployed models: the effective live query distribution
changes. Bundled with the backprop transpose fix's retrain.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Pass 1 already skipped its feedForward on QUEUED bars, because pass 2 redoes
them. The same argument covers two more bands it was still forwarding:
OOS window (30% of bars) - pass 3 re-forwards every one of them
calibration band (~10% of bars) - pass 2.5 re-forwards every one of them
All three passes derive their bounds from the same helpers and apply the
identical eligibility test, so the bar sets are equal by construction, not by
coincidence. Only the two purge bands and the ineligible edge bars are visited
in pass 1 and nowhere else - those keep their forward pass.
The scan's copy was never the one that survived. Its arrow-cache write was
overwritten by pass 3's (with the thresholded, post-training decision), its
status-label paint was transient, and its predicted-class tally measured
last era's weights. Those tallies move to pass 2.5 and pass 3, on the raw
argmax exactly as pass 1 and pass 2 count it, so the population behind the
panel's "Predicted -> Buy/Sell/Neutral" line is unchanged and stays comparable
with the "Actual" line beside it, which pass 1 still accumulates over every
labelled bar.
Verified unaffected by the cut: dPrevSignal and m_lastBarTime are both written
last by bars 0/1, which are label-ineligible and therefore still forwarded, so
FinalizeTrainRun's `dtStudied = m_lastBarTime` and Lifecycle's newBarPending
sentinel read the same values as before.
Correctness, not just speed: batch norm is UNFROZEN during pass 1 (passes 2.5
and 3 freeze it deliberately), so every scan-time forward on a held-out bar was
advancing the BN running mean/variance from data the model is graded on. Those
running statistics are inference-time model state. It is the mild,
unsupervised kind of leakage - feature statistics, not labels - but it fed the
weights pass 3 then scored, and it is now gone.
Cost: ~40% of all bars lose one forward pass per era, ~16% of net time once
pass 2's backward pass is weighted in. Per-dispatch, so it lands on every
backend.
Both variants compile 0 errors / 0 warnings. Build tag scan-nofwd-v5.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Measured on exc-race-v3: LSTM era 300s -> 1087s (net 272->748s, "other"
30->337s). My estimate had been "single-digit percent". The cost is
per-DISPATCH, not per-FLOP, and therefore hits EVERY backend: the head is
19k weights and ~2.4 GFLOP an era - seconds of arithmetic - but ~48k
forward/backward calls x several layer submits each, and its 760-wide
layer exceeds the CPU DLL's inline threshold so each one pays a real
handoff. The classifier's own net time tripled too, from contention with
a second pool on an already-full box.
Three changes, all backend-neutral because they remove submits rather
than tune threads:
SCORE ONLY DISJOINT WINDOWS (~64x). Adjacent bars share all but one bar
of their horizon, so 16k consecutive bars were always ~250 independent
observations - the full-sample tally was never worth more than the
disjoint one, it just quoted an n that was ~64x too large. Dropping it
costs nothing statistically and removes 63 of every 64 forward passes.
The two parallel tallies collapse into one, which is also less code.
The trailing ring still advances on every bar: it needs the outcome
SEQUENCE, and that is array lookups, not a forward pass.
TRAIN ON EVERY 4th PRIMARY BAR (4x). The target is low-dimensional and
strongly autocorrelated - neighbouring bars carry near-identical
excursion information - so per-bar training buys resolution the target
does not have. Strided on ATTEMPTS, not acceptances, so a stretch of
unlabelled bars cannot silently change the spacing.
OWN TIMING COLUMN. The head's passes were landing in the era line's
"other" bucket, which is how a 3.6x regression read as an unexplained
jump in the one column nobody attributes. A cost that cannot be seen in
the timing line cannot be traded off against anything.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Leftover objects survived deinit because the cleanup list had drifted.
PurgeChart()'s own comment said it removed "our namespaced signal arrows
plus the status-label objects" while the code removed arrows ONLY, and
the panel prefix was swept at OnInit and nowhere else - so an ordinary
deinit left the status line, and any panel straggler, on the chart.
Three scattered call sites and a comment cannot be kept in step. There is
now ONE list - WarriorChartPrefixes() - covering arrows, status label and
panel, and one sweep, WarriorPurgeChartObjects(), used by every path.
Add a prefix there when a new object family appears and every cleanup
picks it up.
Two call sites added:
OnInit, before ANYTHING is drawn (including the status label it would
otherwise delete). Chart objects live in the chart PROFILE, not in the
EA, so they outlive the process: a deinit force-terminated at
MetaTrader's ~4,500 ms budget, a crash, a terminal kill, or an .ex5
replaced while attached all strand objects no later deinit will ever
own - and deleting the EA's files does not remove them, which is why
they read as corruption. Arrows are included: LoadChartSignals restores
them from their sidecar moments later and already opens with its own
arrow sweep, so this only removes orphans the sidecar does not account
for - the ones SaveChartSignals would otherwise ADOPT, since it rebuilds
that sidecar by scanning the chart.
OnDeinit, after ExtPanel.Destroy. Destroy walks an unbounded control
tree and ClearStatusLabel clears text rather than guaranteeing object
removal; either can leave a straggler and nothing looked afterwards.
Bounded work - three prefix deletes and one object-list scan - so it
respects the ordering rule that keeps the cheap visible cleanup ahead
of the heavy save. Arrows excluded: ShutdownChartCleanup already
persisted and removed them and re-deleting would race that write.
The two are complementary: the deinit sweep closes the ordinary case, the
OnInit purge closes the case where MetaTrader never let us finish. Only
the second can help after a starved shutdown.
Both sweeps rescan by name across EVERY object type and delete what the
bulk call missed. ObjectsDeleteAll's return has already been observed
disagreeing with a by-name scan of the same chart microseconds apart, and
object commands are queued on the chart rather than applied inline, so a
returned count is not evidence the objects are gone.
Panel create site now uses WARRIOR_PANEL_PREFIX instead of a literal, so
the name cannot drift away from the list that cleans it up.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Beating a frozen global constant is the weakest admissible bar for
replacing a global constant. The honest incumbent is a rolling rung
frequency: it adapts to the volatility regime - exactly what the head
claims to predict - and needs no model, no 760 inputs and no training.
Implemented as a ring of per-bar outcome bitmasks (32 rungs fit one
ulong), sized horizon + EXCURSION_TRAIL_WINDOW. The newest `horizon`
entries are held back UNRESOLVED: a bar's rung outcomes are only known
one horizon later, so using them would be lookahead and would flatter the
incumbent into an opponent the head could never fairly beat. Pass 3 walks
oldest-to-newest, so "pushed more than horizon bars ago" is exactly
"resolved by now". Each push is O(rungs), not O(window).
The head's decision-rung Brier is pro-rated to the trailing estimate's
coverage before the ratio, since the incumbent only scores bars where its
window is warm.
This line is worth reading on its own, independently of the head: if the
trailing quantile beats the global constant, that is a cheap risk-control
win available with no machine learning at all - and it is the same number
either way, so the run answers both questions in one pass.
The ring is deliberately NOT reset per era - it estimates the market, not
the era, and re-warming 500 bars every era would leave the incumbent
unusable over the first chunk of every scoring pass, handing the head a
free win on exactly those bars.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Second-opinion review killed the +4.2% far-rung result, correctly, and
the mechanism is my own bug. A head trained toward {0.05,0.9} converges
to 0.05+0.85p, so its bias is 0.05-0.15p: negative where p is near 1,
POSITIVE where p < 1/3, growing monotonically as the rung gets farther.
Against a baseline frozen at the IS rate, an upward-biased head scores
positive Brier skill whenever the OOS rate merely sits above the IS rate.
Predicted signature: huge negatives near, ~zero at p=1/3, growing
positives far. Observed: -82% ... -0.6% ... +1.2/+2.7/+4.2. The far rungs
were not the clean end of a distorted measurement, they were the other
face of the same artifact. Everything before 25aca83 is void.
The gate was a bare `skill >= 2%` point estimate over 8 rungs x 4
topologies x N eras, reported per era - a best-of-~300 with no interval
and no multiplicity control, which is the shape of the four traps already
documented here. It now needs FOUR things at once:
DECISION RUNGS only the rungs ExcursionQuantile actually reads at the
live geometry (target 1.62, stop 3.31 ATR), fixed
before looking. Skill at 5 ATR is skill about a
distance no order is placed at - and the TARGET side
currently interpolates 1.5/2.0, which measured -2.2%
and -1.3%.
DISJOINT SAMPLE one bar per horizon. Adjacent bars share 63 of 64
horizon bars, so ~16k scored bars is ~250 independent
ones and every SE over the full set is ~8x understated.
VS ORACLE the best constant achievable ON THE SCORED BLOCK,
closed form from H and n (Brier = H*(1-H/n)). A head
that learned only a LEVEL nearer the OOS rate than the
frozen IS constant scores positive against the old
baseline and <= 0 here. This is the control that
separates per-bar skill from base-rate drift.
MONOTONE CURVE P(reach k) must be non-increasing in k. Nothing
constrained 8 independent sigmoids to obey that, and
ExcursionQuantile returns the FIRST crossing - so a
tangled curve is misread exactly where the head is
least sure. Counted and reported, not silently used.
The pass message now also states what a pass would and would not buy:
expectancy is -costs at zero directional edge whatever the stop distance,
and under prop DD limits LOWER variance also lowers P(reach target before
limit), so "better drawdown" is a choice of failure mode, not a win.
Still owed before any Stage 2: a race against a trailing-quantile
incumbent and a vol-feature logistic. Beating a frozen global constant is
the weakest admissible bar for replacing a global constant.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
ExcursionTargets built its 32 binary targets from the classifier's
LABEL_SMOOTH_HIGH/LOW (0.9/0.05). That caps what the head can ever output
at 0.9, and the near ladder rungs have base rates close to 1.0 - almost
every bar travels 0.5 ATR inside a 64-bar horizon. The Brier comparison
is then decided before the net learns anything:
constant at 0.99 -> 0.99*(0.01)^2 + 0.01*(0.99)^2 = 0.0099
head at 0.90 -> 0.99*(0.10)^2 + 0.01*(0.90)^2 = 0.0180 skill -82%
Which is what the first run reported at rung 0.50: PAI -61.8%,
CONV -146%. A property of the target encoding, not of predictability.
Smoothing earns its place on the 3-class head, where it stops one logit
running away inside a softmax competition. There is no competition here
and this head is scored on calibration, so it has to be free to say 0.99
when the answer is 0.99. Hard 1/0 is safe against the runaway smoothing
guards: this is an MSE-on-sigmoid gradient (calcOutputGradients) whose
(target - output) term vanishes as the output approaches the target, not
the unbounded-logit cross-entropy the classifier uses.
The far rungs, where the artifact is smallest, already showed positive
skill on the two topologies with a sequence stage (LSTM 3.00:+1.2%
4.00:+2.7% 5.00:+4.2%, HYBRID similar), so the verdict was being decided
by the most distorted end of the ladder.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
ba13eef cached a miss unless it was flagged transient, and flagged
exactly two guards: the EMPTY_VALUE open and the cold ATR. Every other
rejection in BufferTempDataCompute - an indicator buffer not yet
calculated, a panel not yet built, a series not yet loaded, a failed Add -
still cached as PERMANENT.
Observed 2026-08-11: the MI pre-scan runs ~3 s after OnInit and touches
all 54k bars while the indicators are still warming. The log announced it
immediately and unmistakably:
feature/label information - ... (0 samples 19 bars apart
= 0 independent blocks over a 64-bar horizon, 0.0s)
Zero usable rows, four seconds in. Training then stalled at era 0 for an
hour with "NOT ONE of 54681 scanned bars produced a usable feature
window" on all four charts. Both charts reporting cross-asset PRESENT and
both reporting ABSENT got 0 samples, so the optional block was not the
discriminator - the cache was.
Enumerating which rejections are "really" permanent is the wrong shape of
fix: it is a list that must be re-audited every time a feature block is
added, and being wrong once costs the whole run silently - which is
exactly how the two-guard version failed. Caching only successes needs no
list and cannot be wrong.
Cost is bounded and small: in steady state the only bars that still fail
are the handful at the deep end of history inside the indicators' own
warm-up, so an era recomputes ~ind_Periods bars rather than 54k.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Era 0 stalls with "NOT ONE of 54681 scanned bars produced a usable
feature window, windows ok=0 failed=54681" and nothing else. That line
reads identically for a cold ATR, a conditionally-missing optional
feature block and an out-of-range index, so it cannot be diagnosed
without one restart per hypothesis.
Two changes:
1. WIDTH CONTRACT in BufferTempData. Every enabled block must emit
exactly m_neuronsCount values on EVERY bar. A block that emits its
values on some bars and skips them on others (indicator, panel or
series unavailable for that bar) does not merely shorten the window -
it SHIFTS every feature after it into the wrong slot, and the net
then trains on silently misaligned inputs that still look like a
valid window to everything downstream. Now rejected, rolled back and
reported once, naming the optional blocks (XA / SPR / swing context)
as the ones carrying an availability test. Worth having independently
of the current stall.
2. BuildFeatureWindow records WHICH lookback slot rejected and how much
of the window was assembled, and the pass-1 stall report renders it:
"slot 0 of 20 REJECTED (window had 0 of 760)" is an indicator warm-up
or history-edge read; "every lookback bar ACCEPTED and the window was
still short: 640 of 760" is a missing 6-value block.
No behaviour change on a healthy run: the width check is an equality
that already holds, and the diagnostics render only inside the
total-failure branch.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Direction is closed - normalised asymmetry fails on three instruments
with a working positive control, and the classifier's own best-of-999
era-cap test agrees (+0.9pp = 1.48 sigma, family-wise p=1.0000). SIZE is
a different question and RANGE clears at ~4x its null.
Checked the denomination before building on that, since the source memo
warns to: m_excUpCache holds (maxHigh - fill)/ATR, so "RANGE is
predictable" is a claim about travel RELATIVE to current ATR, not a
restatement of "ATR is autocorrelated". It is exactly the part a fixed
multiple (stop 3.31*ATR, target 1.64*ATR) discards.
A second small CNet, 760 -> 24 -> 32 sigmoid outputs = P(price reaches
ladder rung k) upward and downward. Survival parameterisation rather than
regressing the multiple, because it needs nothing new from CNet: sigmoid
outputs and the per-neuron delta the `total != 3` branch already applies
(a quantile head would need a linear activation and a pinball gradient in
Network.mqh, Network.cl and the DirectML path, on a class four topologies
share). Targets are free - m_ladderUpAt already records first-touch age
per rung with 0 meaning never reached.
Separate net, not extra outputs on the classifier: more outputs would
change m_outputNeuronsCount, the .nnw shape and the fingerprint, and push
the count off 3 - the exact condition backProp uses to select the joint
softmax gradient the 3-class head depends on. The classifier is
bit-for-bit unaffected and this is removable without trace.
STAGE 1 PLACES NO ORDERS. It reports a Brier skill score against the
constant per-rung base rate - the baseline a fixed ATR multiple already
assumes - with both predictors fitted IS and evaluated OOS, so neither
gets a look at the test set. Positive skill justifies Stage 2 (drive
SL/TP and sizing off ExcursionQuantile, which is defined and deliberately
uncalled). Zero or negative means ATR already carries everything and
Stage 2 must not be built.
Trains only on primary occurrences: the replay queue oversamples for
CLASS balance, and a direction-balanced sample is a biased SIZE sample.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
coveragePct, dirPrecPct and the declustered TRADED tally were all computed
from oPrevSignal - the RAW argmax - while the live order, the arrow and the
panel all run on oDeploySignal, which is argmax AFTER the confidence
threshold. The gate was certifying a strategy the EA does not trade.
Invisible until now: the threshold sat at ~0.02, so the two populations
were the same set. The held-out calibration slice (2189316) moved it to
0.14-0.40 and the gap opened immediately - PAI era 256 graded 100% coverage
while its traded population was 21% (3,399 of ~16,200 OOS bars).
Consequences that were being hidden:
- coveragePct >= minCoveragePct was tested against the wrong population,
so a model whose TRADED coverage falls under the 24.8% floor still read
as clearing it
- precSE = sqrt(p(1-p)/n) used n ~16,000 instead of n ~3,400, so the
EDGE_MIN_SIGMAS bar was ~2.2x too lenient on the real evidence
- the NMS replay declustered a different, larger stream than live, so
threshold-rejected bars consumed cluster slots and set alternation state
Gate quantities now read m_oosBuyFired/m_oosSellFired (the thresholded
population, already tracked for the live-precision line) and the NMS replay
runs on oDeploySignal. The threshold can only turn a direction into Neutral,
never flip a side, so the fired set is a strict subset and every per-bar
outcome is the one already computed.
Recall and logBuyPrecPct deliberately stay on the raw argmax: they measure
intrinsic class separation, and thresholding them would conflate "cannot
separate the classes" with "declines to act on the separation it found".
This is the 9a7c37f defect class, and the NMS block carried a comment
warning about it while committing it three lines above.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
FitDirConfThreshold harvested its margin histogram from pass 2's own
backprop samples. Pairing every fit against the same era's OOS result
shows what that measured:
PAI era 1 IS 25% cov @ 66.1% (-0.8pp) -> OOS 64% (-3pp) gap +2.1pp
PAI era 76 IS 90% cov @ 79.6% (+12.7pp) -> OOS 65% (-2pp) gap +14.6pp
LSTM era 9 IS 77% cov @ 81.6% (+14.6pp) -> OOS 63% (-4pp) gap +18.6pp
The gap grows monotonically while OOS stays flat, so within a handful of
eras the curve stops describing behaviour on unseen bars. That is fatal
here specifically, because the objective branches on the SIGN of
(p - break-even): the memorized curve reads +12pp at 95% coverage, so
coverage x (p - p0) correctly maximises coverage and returns ~0.02 - fire
on every bar. The "p < p0 -> get more selective" branch, which is the
actual regime and the entire point of 983a6a3, could never fire because IS
never showed p < p0.
Carve a calibration slice out of the IS span - DIR_CONF_CALIB_PCT_OF_IS,
purged from backprop by one label horizon on BOTH sides (the far-side
purge is not optional: without it the newest training bars carry labels
partly decided by price action inside the slice, putting the memorization
straight back into the curve). Score it in a new chunked pass 2.5, after
pass 2 has trained and before pass 3 grades - the only position where the
histogram is simultaneously not-trained-on, not-graded, and current with
the weights it will be applied to.
Costs 15% of the training data. Worth it beyond honesty: the deploy gate
needs dirPrecPct > chance + EDGE_MIN_SIGMAS*SE, and a threshold pinned
near zero dilutes any edge concentrated in the confident bars across every
bar the model calls, driving dirPrecPct toward chance by construction. A
threshold that can be selective is the only mechanism by which a small,
concentrated edge could ever clear that gate.
Also: a sparse histogram now KEEPS the previous threshold instead of
resetting to 0.0. A failed measurement must not decay to the most exposed
setting in the range.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
SignalClusterWindow 3 -> 10 for all topologies. On H1 a 3-bar window
collapsed only the tightest runs and left visible clusters at every
turn; 10 bars is closer to the spacing of genuinely distinct setups.
ALTERNATION. Rule 1 only collapses a same-direction run INSIDE the
window; past it a second Buy is emitted with no Sell between, giving
Buy/Buy/Buy/Sell. With both directions tradeable that sequence is the
model re-entering a move it is already in rather than finding a new
one. The kept sequence must now alternate: the first signal passes,
and after that a direction passes only if the last KEPT signal was the
opposite one.
Added to ALL THREE consumers, with identical logic, because they must
agree:
- NmsLiveAccept -> the live trade
- pass 3's OOS replay -> the tally the deploy gate grades
- PruneDirectionalClusters -> the drawn history
A rule applied to only some of these certifies one strategy and trades
another - the same defect class as the geometry the gate certified
while OpenParams placed something else (9a7c37f) - and would draw the
user arrows the EA would never have taken.
Deliberately NOT applied to the LABEL. The barrier target has no "must
flip" invariant: consecutive Buy labels are routinely correct, and an
earlier alternation gate was removed with the triple-barrier relabel
for exactly that reason. This filters what is ACTED ON, which is what
"applies to training" can honestly mean here - pass 3's declustered
tally is the training-side number that decides deployment.
BothDirectionsTradeable() is the stated precondition (with one side
disabled there is no opposite to wait for, so alternation would
suppress everything after the first call). This build has no
long-only/short-only input, so it is constant true - kept as a named
predicate so a future direction restriction has one place to change
rather than three call sites silently assuming both sides.
Build tag -> nms-alternate-v4.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
FitDirConfThreshold walked from the most selective bin down to bin 0
keeping `precPct >= bestPrec`, with the stated intent that a plateau
should walk toward more coverage. The failure mode is the models that
need a threshold most: a net with no edge scores its base rate at
EVERY threshold - a perfect plateau - so the walk ran all the way to
bin 0 and returned 0.0, i.e. fire on every bar.
Reported as PAI "overshooting signals" while the other three stayed
selective. PAI has the flattest plateau because its margin
distribution is the most degenerate: its OOS outputs span the full
0.000..1.000 where CONV sits at 0.214..0.814, so nearly every call
lands in the top bins and precision barely moves as the walk descends.
The deeper problem is that precision is not the money quantity. For a
k:m barrier with p0 = m/(m+k),
EV = (p - p0) * (k + m) => EV per bar = coverage * (p - p0) * (k+m)
and (k+m) is constant across thresholds, leaving coverage * (p - p0).
That objective needs no tie-break and behaves correctly everywhere:
p > p0 everywhere -> takes the coverage (the old outcome, now for a
reason rather than as a plateau artifact)
p flat at p0 -> every point scores 0, the coverage floor decides
p < p0 everywhere -> the LEAST coverage loses the least, so it gets
MORE selective instead of trading everything
The last case is the current reality for all four models (-1 to -4pp
against break-even) and is the exact opposite of what the old rule
did. The comparison is sound: the histogram is already fitted on wins
(qTradeWon), not label agreement, so precision and break-even measure
the same quantity.
Ties now keep the more selective point - the loop reaches it first and
the test is strict >.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The risk/reward explanation was a second, wrapped line and cost more
vertical space than it earned - that detail belongs in the journal,
where the geometry is already logged in full.
The comparison itself stays, in two words: "64% (unseen data, need
67%)". Without it the win rate reads as skill when it is the barrier
geometry's own base rate, which is exactly how four models sitting at
chance came to look like four models at 65%.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Both made the run structurally unable to succeed, independently of any
signal in the data. Found by reading the 13:01 log.
RECALL GATE. m_objectiveMet required Buy, Sell AND Neutral OOS recall
each >= 40%. First-touch resolution (ce52654) collapsed Neutral from
the ~94% majority it was under exact-pivot labels to a same-bar-tie
residue - 250 of 38,261 bars, 0.65% - so the floor was asking the model
to identify 40% of coin-flip ties before it could converge. Measured:
CONV, LSTM and HYBRID all logged "Neutral:0% (need >=40% each)" on
every era. No model could ever satisfy it; every run was destined for
the plateau ladder or the era cap.
Only the DIRECTIONAL floors are load-bearing for the anti-collapse job
the gate exists to do: an all-Neutral model shows Buy and Sell recall
at 0% and is blocked by them. Neutral's own floor guarded the mirror
bias (over-calling Buy/Sell at Neutral's expense), which was real at
94% prevalence and is not at 0.65% - there, almost never calling
Neutral is correct rather than biased.
Prevalence-guarded rather than hardcoded off, so it returns by itself
if a future label rule makes Neutral substantial again. Deliberately
NOT extended to Buy/Sell: exempting a thin directional class reopens
the era-44-46 hole, which directionalRecallMeasured only half-covers -
it checks those classes were MEASURED, not that they passed.
ETA DECAY. A regressing era restored the checkpoint, reset the
optimizer and cut eta - all on the FIRST regression. The next era then
started from an identical state with a smaller step, regressed again,
and got the same treatment. The loop is self-sustaining and cannot
discover anything, because rolling the weights back is exactly what
removes the exploration that would end it.
Measured on PAI: eras 2-11 every one a regression against era 1, eta
0.000594 -> 0.000024, dW/W 0.000%/0.000% from era 2 onward. Ten eras,
~45s each, reproducing era 1 exactly and unable to do anything else.
Now requires ETA_DECAY_PATIENCE_ERAS consecutive regressions - the
standard ReduceLROnPlateau formulation. A single bad era is noise, and
an improving era clears the counter so alternating runs never
accumulate into a decay.
Build tag -> gate-patience-v3. It had not moved in six commits, which
is why the running binary could not be identified from its own log.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Chart objects live in the MT5 chart PROFILE, not in this EA's files.
They survive a terminal restart, a recompile, and deleting every
.nnw/.cfg/.stats/.arrows on disk. Only a deinit that RUNS TO COMPLETION
removes them - and MetaTrader force-terminates OnDeinit at roughly
4,500 ms, so a run killed mid-cleanup orphans them permanently with no
owner left to clean up after. That is the "deleted every file,
recompiled, restarted, old arrows and a stale panel still there"
report: nothing was wrong with the files and deleting them could not
have helped.
Both halves are fixed.
STOP OVERRUNNING THE BUDGET. OnDeinit used to finalise the in-flight
run (StopTraining -> FinalizeTrainRun: checkpoint restore, live-state
re-seed) and then write two full nets per chart. On four charts that is
the bulk of the budget, spent to preserve a PARTIAL era that was never
scored, never checkpointed and never deployable. FlushTrainRun()
discards it instead - drop the resumable bookkeeping, leave the net
neutral (unfreeze BN, flush the batch, batch size 1), skip the save -
and training resumes from the last completed era, which the era-end
save and the periodic autosave have already put on disk. What is
discarded is bounded by one era.
A CONVERGED model keeps the old finalise-and-save path: its weights can
carry online-learning updates made since the last era boundary, and for
a deployed model no further era boundary is coming to persist them.
MAKE CLEANUP SELF-HEALING. Every purge sat behind a branch - no model
loaded, sidecar missing - so the common paths returned leaving whatever
the previous instance stranded. LoadChartSignals now sweeps the arrow
namespace unconditionally before restoring, so the post-init chart
holds exactly what the sidecar holds whichever branch runs, and the
panel gets the same treatment before Create() (CAppDialog namespaces
its controls, so a killed Destroy strands the lot and the next attach
draws a second panel on the corpse).
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
