forked from MrBaro75/Warrior_EA
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
AnimateDread
|
2b5d0f8355 |
feat(measurement): fix zero-skill denominator, publish the deploy bar, measure lifespan per rung, add a MEASURE scale objective
The last run could not have demonstrated an edge either way, and nothing in the
log said so. Four changes so it does.
1. THE ZERO-SKILL LINE DIVIDED BY THE WRONG DENOMINATOR. m_oosWinLongTotal resets
every era; m_oosSamples only resets on a full model reset. So 'always-long %'
decayed as ~1/era: a run whose true rate is 37% printed 1.2% at era 33 and
0.0% at era 2219. This is the SAME bug already found and fixed for
logBuyPredPct thirty lines above ('era-15 Buy:2% that was really ~30%'), left
in the one line whose whole job is to be the reference every other number is
read against. Correct at era 1, wrong everywhere after - including the '62%
zero-skill' figure in the 2026-08-16 notes. Now per-era, and always-short is
finally readable.
2. THE DEPLOY GATE STATES ITS OWN BAR. 'edge -1pp' era after era cannot separate
'short by a hair' from 'short by an amount no strategy could cover'. The era
line now prints the required win rate, the SE, the effective n and the
lifespan it was deflated by; above 100% it says UNREACHABLE. At 4,738 OOS bars
and L=75.6 there are ~63 independent observations, putting the bar near 66% at
typical coverage.
3. LIFESPAN MEASURED PER RUNG. The first-passage cache already stores touch ages
at every ladder level, so each candidate geometry's resolution time is
readable without training on it - L-vs-width becomes a measurement across the
whole ladder in ONE run rather than a second chart. Each rung reports L,
n_eff, min provable edge and min provable EV.
4. SCALE OBJECTIVE IS PHASE-AWARE, defaulting to MEASURE. Width and detectability
are opposed: labels overlap by L, L grows like m*k = width^2 at fixed ratio,
so min provable EV ~ width^2 while the cost saving from width is only linear.
Doubling width quadruples the smallest EV you can prove. DEPLOY (widest that
clears reachability) is right once an edge is known; MEASURE (narrowest that
keeps round-trip spread under BARRIER_MAX_COST_FRACTION_PCT) is right while it
still has to be shown. The direction does not depend on the exponent, and
item 3 makes the exponent checkable.
Fixed in review: m_lastRungLifespan is cleared on every LadderWinShare entry or a
rejected rung reports the previous rung's lifespan as its own; per-rung
detectability is labelled IS-sample based (the deriver may not see the holdout),
so absolute figures are optimistic while the ranking is unaffected.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
||
|
AnimateDread
|
1540ba8e64 |
fix(labels): overlapping-label sample correction + horizon cap on the scale ladder
Three defects, all surfaced by the 2026-08-17 SP500 H4 run that shipped
stop 4.86 / target 9.71 (width 14.57*ATR, horizon 384).
1. EVERY STANDARD ERROR ASSUMED INDEPENDENT SAMPLES. Triple-barrier labels
started one per bar overlap by the label's lifespan, so n calls are worth
~n/L independent observations (Lopez de Prado, AFML ch. 4 - sample
uniqueness). All three sqrt(p(1-p)/n) sites divided by the RAW count.
The tell: the operating point's null-of-the-maximum gate is family-wise and
should fire on ~5% of eras under the null. Measured fire rates - PAI 47/73
(64%), ConvLSTM 9/24, LSTM 8/21 (38%), CONV 4/62 (6%). CONV, the only model
whose margin distribution admits few bins, sat on the null; the rest cleared
a bar that was too low by ~sqrt(L). PAI's deployed threshold consequently
alternated between the ENDS of its own range era to era (0.10 -> 0.88 ->
0.86 -> 0.66; coverage 16% <-> 73%).
TripleBarrierLabel now records when each label became KNOWABLE - the first
winning touch, or both stops, or the timeout - and the prebuild accumulates
the mean. EffectiveSampleSize() feeds the operating point, the member deploy
gate and the ensemble vote gate. Conservative by construction (n/L is an
upper bound on the damage); gates get harder, never easier.
2. THE SCALE LADDER RAN AWAY, again. Horizon scales as swingMedian*sl*tp, and
since
|
||
|
AnimateDread
|
bc57aca15d |
fix(geometry): the target was small BY CONSTRUCTION - ratio is now policy, scale is measured, ladder ceiling removed
The derivation read the stop from q75 of ADVERSE travel and the target from q50
of FAVOURABLE travel. Over one horizon those distributions are broadly the same
shape, so q75 > q50 MECHANICALLY - the target came out smaller than the stop no
matter what the market did. SP500 H4 shipped stop 3.07 / target 1.70: a 0.55:1
payoff needing 64.3%. That was never a measurement, it was two mismatched
constants.
The reachability line printed beside it - "target on 50.0% of bars, stop on
25.0%" - is exactly 1-q50 and 1-q75. Tautological. It cannot disconfirm
anything, and it read as validation.
WIDTH AND RATIO ARE INDEPENDENT AND ONLY ONE PAYS. EV = edge x width;
ratio is EV-neutral (a driftless walk reaches +m before -k with probability
k/(k+m), which IS break-even). Width is what buys cost efficiency: the spread
is a fixed 0.047*ATR here, so the shipped 4.77*ATR width paid it 21 times per
unit of travel. So:
RATIO = policy. BARRIER_TARGET_RR = 2.0 (user's 1:2). Break-even 33.3%.
SCALE = measured. The stop quantile is chosen from a ladder, WIDEST FIRST,
taking the first rung whose implied 2x target is still reached often
enough to be a trainable class.
That last clause is the difference from the min-reward:risk raise removed in
2026-08-09, which forced target = 2 x stop with NO reachability test, landed on
6.66*ATR reachable on 3.3% of bars, and trained the model to predict something
that essentially never happened. Same ratio; the scale now retreats until the
data says the target is attainable. Every rung is logged.
LADDER CEILING REMOVED. BARRIER_LADDER stopped at 5.00 and the expectancy scan's
"best resolvable pair on width alone" came back as stop 5.05 / target 4.95 - it
pinned to the top rung. A recommendation landing exactly on the edge of its own
search space is a boundary, not a finding: it cannot tell "5 ATR is optimal"
from "5 ATR is all we allowed". Extended to 20*ATR (8 -> 14 rungs). Nothing else
needs editing - every consumer is parameterised by BARRIER_LADDER_COUNT - and
the horizon constraints (decided >= 60%, reachability floor) now bind instead of
a constant.
THE SCAN COULD NOT SEE THE SHIPPED GEOMETRY. ReportBarrierGeometryScan looked
the configured pair up in its integer grid, and DeriveBarrierGeometry produces
CONTINUOUS multiples (3.07/1.70) that can never equal a grid point - so
cfgExcess stayed at its -1.0 sentinel and the report printed "configured 3:2
scores -1.00000", which reads as a catastrophic score and actually means "never
evaluated". Worse, the grid skipped target<stop entirely because it "inverts the
trade's whole premise" - while the derivation was shipping exactly that. The
incumbent is now always scored as a peer (never crowned; it is already in force
and is not an enum pairing the scan could adopt).
BREAK-EVEN NOW INCLUDES THE SPREAD. Every report quoted the frictionless
SL/(SL+TP). On SP500 H4 that read 64.3% while the MEASURED zero-skill rate was
62.1% - a 2.2pp gap that IS the cost, and that made every model look 2.2pp
better than it was. CostAdjustedBreakEvenPct() prices a win at (TP - spread) and
a loss at (SL + spread), matching the expectancy scan's convention exactly so
the two reports cannot disagree.
It also feeds FitDirConfThreshold, which is the correctness half: the operating
point subtracts break-even from precision, so the frictionless figure made every
candidate threshold look better by the width of the spread - 2.2pp against a
measured edge of 2.3pp, i.e. very nearly all of it.
Era line now carries both: "break-even 64.3% frictionless, 66.6% AFTER SPREAD".
Forces a full relabel and retrain. Requested.
NOT COMPILED - user compiles.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
||
|
AnimateDread
|
d30420e3f2 |
fix(batchnorm): bound the normalized value - a constant input feature was amplified 1e4x and pinned PAI's head to its rails
BN_MIN_STD = 1e-4 caps the per-unit gain at 1/1e-4 = 1e4, and the comment above
it states that as though it were a safety property. It is not. A unit whose
running variance is ~0 is a CONSTANT feature carrying no information, and
dividing its rounding noise by 1e-4 hands the next layer an activation of
several hundred. BN's contract is "output has ~unit variance"; a unit that
cannot supply that must contribute nothing, not the largest signal in the layer.
MEASURED, 2026-08-17 SP500 H4, four topologies on identical separate charts:
model spread Neutral CHOSE Neutral TIED rail
CONV 0.386 0.68% 0.10% 0.48%
LSTM 0.392 0.63% 0.00% 0.00%
HYB 0.376 1.79% 0.00% 0.01%
PAI 0.192 0.09% 80.63% 99.99%
bn1's cached nx normed 1.38e4 over 800 units. PAI's SIGMOID head was on its
rails on 99.99% of bars, with Buy and Sell landing on the SAME rail so they
compared exactly equal, and ApplyClassificationSoftmax()'s strict-majority rule
reported that tie as Neutral on ~80% of bars.
So the long-running "PAI is heavily biased toward Neutral" was never a
class-prior problem: the net CHOSE Neutral on 0.09% of bars. It was float
equality on a saturated head. The
|
||
|
AnimateDread
|
331ab29c56 |
feat(diagnostics): split a reported "Neutral" into CHOSE vs TIED - they need opposite fixes
ApplyClassificationSoftmax() requires a STRICT majority over both rivals and
sends every tie, 2-way or 3-way, to Neutral. So "OOS recall Neutral:100%" is
two completely different events sharing one label:
CHOSE - the net genuinely ranks Neutral highest. A class-prior/label problem.
TIED - the top two are EXACTLY equal, so the net expressed no preference and
the tie-break reported Neutral. A SATURATION problem: the head is
SIGMOID, and a saturated sigmoid returns exactly 0.0f or 1.0f in the
DLL's float32, so two classes pinned to the same rail compare equal
and the bar is silently discarded.
Nothing in the logs could tell them apart, and the fixes point opposite ways.
Eras 1-25 of the 2026-08-17 solo PAI run read "Neutral 100%" at spread avg 0.99
- fully saturated - and broke out at era 27 as the spread fell to 0.75. That is
consistent with EITHER story. The user reports the Neutral phase on most runs,
so it is worth four longs to stop guessing.
Four per-era counters on the pass 3 OOS walk, reported as:
| Neutral CHOSE 12.4% / TIED 38.1% (of which B=S 1204) | rail 61.2%
m_oosNeutralStrict - Neutral strictly highest
m_oosNeutralTie - no strict winner; the tie-break produced Neutral
m_oosTieBuySell - the costly subset: Buy and Sell tied AT the top, i.e. a
DIRECTIONAL reading thrown away by float equality
m_oosRailBars - any raw output sitting on a sigmoid asymptote, the
saturation that makes exact ties possible at all
Read on the RAW logits, before ApplyClassificationSoftmax() overwrites TempData
in place. Legitimate because softmax is strictly monotone: it cannot change the
ordering and cannot break a tie either, so the raw reading and the decision
always agree. Placed alongside the existing min/max/spread capture so all the
output diagnostics describe the same values.
Measurement only - no decision path reads these.
NOT COMPILED - user compiles.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
||
|
AnimateDread
|
208da4cbaa |
fix: drop the ranking slice for the calibration band; un-collapse the tiers
NOT COMPILED - user compiles. (1) THE RANKING SLICE IS GONE. It reserved 20% of the OOS window so the pattern-DB backfill would read bars the deployed checkpoint was not SELECTED on. That objection stands; carving a new region to answer it did not. The calibration band already has every property the slice was buying: never trained on | never graded by pass 3 (which walks [0, oosCutoff) and so never reaches it) | never seen by the deploy gate | purged by a full label horizon on BOTH sides | and larger besides - 1,684 bars vs the ~970 carved So the backfill now walks [calibLo, calibHi) and pass 3 goes back to grading the entire OOS window, exactly as before any of this. The gate gets its full sample back (~10% of a sigma), the split loses a region, and the failure mode found an hour ago - a reserved region silently blanking ~10 months of chart arrows, because arrows are only drawn on bars pass 3 grades - becomes impossible. One impurity, stated in the completion log rather than hidden: m_dirConfThreshold is FITTED on that band and the walk applies it to decide which bars fired, so coverage there is mildly optimistic. One scalar under a coverage floor, against checkpoint selection over hundreds of eras. This backfill IS the deploy-time warm-up: it runs right after FinalizeTrainRun() restores the deployed weights, so it scores with exactly what is about to trade. (2) EVERY CALL WAS TIER 0, AND IT WAS ARITHMETIC. ConfidenceTier() quartiles [floorConf, 1] where floorConf = 1/3 - the lowest magnitude a 3-way softmax winner can hold. But it was fed CalibratedConfidenceMagnitude(), which multiplies by m_confidenceCalScale, clamped to [0.3, 1.5]. That lower clamp is BELOW 1/3. Whenever calibration bottoms out, t goes negative and MathMax(0, ...) pins every call to tier 0. Which is what the live run does. m_confidenceCalScale is EMA'd toward empiricalAccuracy / avgClaimedConfidence; with the model over-calling Neutral, 3-class agreement sits near 10% against a claimed confidence near 0.9, so the ratio is ~0.11 and clamps to 0.3 every era. Logged: tier prec T0:72%(828) T1:n/a(0) T2:n/a(0) T3:n/a(0) 828 calls, one bucket - the four tier weights and the entire per-tier pattern-DB ranking reduced to a single number. The backfill was feeding a mechanism that structurally could not rank. Tiering now reads the RAW head magnitude, which genuinely lives on the [1/3, 1] range these bounds were written for. Calibration keeps its real jobs - AIConfidence() for MM sizing and SignedAIConfidence() for the vote are unchanged. STILL OPEN, deliberately not touched here: the calibration TARGET itself. empiricalAccuracy is 3-class agreement, which is the wrong quantity to scale a DIRECTIONAL confidence against - it counts a Neutral class that is 0.19% of labels. The honest target is the win rate on the calls the confidence describes (directional precision), with the claimed-confidence average taken over those same called bars. That needs a new accumulator and it interacts with the Neutral over-calling being fixed elsewhere, so it wants one clean run first. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
AnimateDread
|
75d23e9b82 |
fix(gate): move the ranking slice to the OLD end - it walled off the recent chart
NOT COMPILED - user compiles. User: "there is quite some trading going on, but absolutely nothing on the recent area of the chart, like there is a hard wall starting around november 2025." That wall is 7caf2f6's ranking slice, and it was placed at the wrong end. Chart arrows are only ever drawn on bars pass 3 GRADES, and the slice reserved the NEWEST 20% of the OOS window plus a label-horizon purge. At the live sizing - ~4,860 OOS bars, 128-bar horizon - that is ~1,100 H4 bars withheld from grading, about ten months back from today, exactly where the wall appears. The invisible cost was worse than the visible one: it handed the deploy gate the OLDEST 80% of the OOS window and withheld the most recent regime from the single decision that has to generalise forward. Both fixed by putting the reserve at the oldest end instead: [0, oosScoreHi) OOS - graded by pass 3 (NEWEST, arrows restored) [oosScoreHi, rankLo) purge - one label horizon [rankLo, oosCutoff) RANKING - backfill only, graded by nobody [oosCutoff, calibLo) purge [calibLo, calibHi) CALIBRATION ... IS Of the three consumers competing for those bars, recency is worth least to the ranking: it is an ORDERING of confidence tiers, far less regime-sensitive than an absolute win rate, while the gate's power and the operator's read of the chart both want the newest data. The slice keeps every property that made it worth carving - never graded, never selected on, never seen by the gate, purged on both sides - so the backfilled rows are still honestly out-of-sample. RankSliceHiIndex is replaced by RankSliceLoIndex + OosScoreHiIndex; pass 3 now excludes the slice at the TOP of its walk and descends to 2 as it always did. The backfill walks [RankSliceLoIndex, oosCutoff) via a new m_dbBackfillStopIndex, clamped at both ends so a degenerate slice yields an empty walk rather than one that wanders into graded bars. Verified no reference to the old helper survives. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
AnimateDread
|
1eeed3ac06 |
revert(ui): restore the unconditional era-end arrow repaint
|
||
|
AnimateDread
|
b19799910d |
fix(ui): chart arrows follow the BEST checkpoint, not the latest era
User report: "as soon as the next era training begins the chart signals are
erased, they should persist for as long as they are accurate."
Cause: PruneDirectionalClusters ran unconditionally at the end of pass 3, and it
DELETEs the arrow on any bar the CURRENT era scored Neutral. A model exploring
away from its best therefore wipes the chart every era even though the best
checkpoint still calls those turns. On the run that prompted this the model sat
at OOS recall Neutral:100% for 40+ consecutive eras, so essentially every arrow
was deleted at every era boundary.
The render is now deferred to the era-end block - the first point that knows
whether the era beat the best checkpoint - and only a new best repaints. On eras
that did not improve, the previous best's arrows stay untouched. Two exceptions
keep the chart from ever showing nothing: before the first checkpoint exists
there is no best to preserve, so early eras still paint; and a finishing run
repaints unconditionally, because FinalizeTrainRun is about to restore the
deployed weights and the chart must describe THOSE.
Recorded for ensemble members too. A member's own best era is not the deployable
one (the joint checkpoint decides that), but it is still the most accurate thing
that member has drawn, and the alternative is a chart that empties itself.
Also verified against the log, since two other symptoms were reported alongside:
era cadence PAI-b6b5 (before these changes) 3.76 s/era
PAI-17ae (after) 3.53 s/era
topology both "2 dense from 16 units | input 800 (16 bars x 50)"
calibration both fitted on 1699 held-out bars
So training speed is unchanged - the ~6% is the ranking slice removing 20% of
pass 3's bars. It only FEELS fast because this is a single PAI chart taking the
whole 120 ms budget, not four ensemble members sharing it behind an era barrier.
The Neutral collapse is also pre-existing, not new: b6b5 ran at Neutral 94-100%
with 1-5% directional calls for all 723 of its eras, before any of this work.
That is the known neutral-collapse/recall-gate failure mode, and it is what
"barely drawing signals" actually is. Worth watching, separately: b6b5 reached
best-bal 34.2% by era 723 while 17ae is at 13.6% after 77 - too early to read,
but it is the number to check once 17ae has run comparable eras.
Compile-verified: 0 errors, 0 warnings.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
||
|
AnimateDread
|
7caf2f626e |
feat: derived taper restored; DB ranking reads a reserved slice, shrunk
TOPOLOGY - reverts the two constants and drops CausalHiddenLayerFloor. The MQL5 article's 30%-per-layer cut and floor of 20 are load-bearing on ITS first-layer width of 1000 (1000->300->90->27 needs a floor to stop). This codebase MEASURES that width, and on the live SP500 H4 config it is 16 units - already floored, with the budget printing "11360 estimated in-sample bars cannot support a 800-wide input ... roughly 1.1 weights per training bar - expect overfitting". At 16 units a floor of 20 makes lastHidden >= m_initialNeuronsCount, so ComputeHiddenLayerCount returns on its first branch and the width taper - the only part derived from this symbol's data - became dead code on all four ensemble members, with depth (2 -> 4) set entirely by counting feature domains. ComputeLayerWidths had already rejected this exact pair of constants in its own comment. The causal floor's premise does not hold either: layers are not inference steps. The "1 layer linear / 2 nonlinear / 3 multi-connected" result is Lippmann 1987 and is about hard-threshold units; with sigmoid/ReLU, Cybenko 1989 and Hornik 1991 give universal approximation from a single hidden layer. Depth buys parameter efficiency for compositional functions, not reasoning hops. ForceHiddenLayers remains for measuring depth directly. RANKING SLICE - the backfill no longer reads the window it is judged on. The deployed checkpoint is CHOSEN as the best-scoring era on the OOS window, so win rates measured back over it are selection-inflated, and the backfill was writing exactly those into the table filter weights rank on: the selection set consumed twice, beside a deploy gate that applies a Sidak correction for that effect. The newest RANK_SLICE_PCT_OF_OOS (20%) of the OOS window, plus a label-horizon purge, is now reserved and graded by nothing - not pass 3, not checkpoint selection, not the gate. The backfill reads only that. The gate keeps ~80% of its measurement (power goes as the square root, so ~10% of a sigma), and the slice is the newest data, which is the regime about to be traded. RankSliceBars returns 0 when no honest slice fits and the backfill then REFUSES and says so, rather than falling back to the scoring window and looking like a success. SHRINKAGE - per-tier win rates are shrunk toward the filter's own pooled rate by MIN_TRADES_FOR_WIN_RATE pseudo-trades before becoming weights. The raw ratio at the minimum sample count carries a ~15pp standard error, so a tier that went 8-2 was handed weight 80 and outranked a tier measured over hundreds of calls at 55 - the ranking was being driven by which small tier got lucky. Opt-in per call site (priorWeight 0 keeps the raw behaviour). Compile-verified: 0 errors, 0 warnings. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
AnimateDread
|
b6736fd40b |
fix: the DB backfill could never run, and HEAD did not compile
Four defects in 64c5dd5/1a05e63, found by review + a baseline compile. Goals 1-8 of that session are unchanged; this makes 6 and 8 actually reachable. 1. HEAD DID NOT COMPILE - 6 errors. CControlPanel::Minimize/Maximize were declared `virtual bool ... override`, but CAppDialog declares both as `virtual void` (Controls\Dialog.mqh). errors 265 + 404 on each, plus 151 on `bool ok = CAppDialog::Minimize()`. Return type is void now; there was never a success flag to forward. Verified: 0 errors, 0 warnings. 2. THE BACKFILL COULD NEVER ADVANCE, and neither could the OOS continual simulation (that one has been dead since it was written). Both are armed at the instant convergence is declared, and both advance only from inside Train(), one chunk per call. But ScheduleTrainingIfNeeded's only per-tick ArmStudyEvent site sits in the `else` of a branch taken whenever m_trainingComplete is set and m_trainRunActive is clear - which is exactly the state FinalizeTrainRun() leaves behind one line before they are armed. Train() was never called again, so the walks sat at their start index forever: no "simulation complete" line, and not one row written to the DB this feature exists to fill. Only a manual Resume/Retrain unstuck them. Both flags now keep the model schedulable. 3. IN AI_HYBRID - the mode this ships in - the backfill was never even armed. Ensemble members deploy at Train() ENTRY and return immediately (so no era is wasted), which skips the era-end block the backfill was started from. All four members were a no-op for a second, independent reason. Armed on the ensemble deploy path too, from m_resumeBars/m_resumeOosCutoff. 4. RE-RUNS DUPLICATED ROWS. RegisterSignal inserts unconditionally - no key, no duplicate check - and m_dbBackfillDone is in-memory, so every later attach that retrained to convergence wrote a second full set of rows for the same bars. The ranking would count one bar once per model that ever deployed, weighting superseded opinions as heavily as the live one. A .dbfill marker stamps the deployed era; written only on completion (an interrupted walk redoes itself rather than ranking a partial window) and deleted with the other sidecars on reset-weights. Also: WarmBlocking's timeout was silent, which restored the exact silent pin failure it was added to prevent - it now says so in the journal, and returns true for "no reference pairs to wait for" so the warning stays rare enough to be read. Not addressed, needs a decision: the backfill scores the OOS window with the checkpoint that was SELECTED as best on that same window, then writes those win rates into the table filter weights rank on - the selection set consumed twice, undiscounted, while the deploy gate right next to it applies a family-wise correction for exactly that effect. The rows are also simulated triple-barrier outcomes at today's spread sharing a table with realised fills. The completion log line now states both plainly. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
AnimateDread
|
64c5dd55d3 | feat: implement one-shot pattern-database backfill and enhance accuracy tracking for ensemble models | ||
|
AnimateDread
|
e049b624ba |
feat(ensemble): deploy gate on the COMBINED VOTE, with a joint checkpoint
The unit of evaluation in ensemble mode becomes the vote, because the
vote is what trades (user: "at the end of the day they will vote
together during live trading so that would make sense").
Four decisions move from the member to the ensemble:
* which era is "best" -> the era whose COMBINED VOTE scored best
* what is checkpointed -> a JOINT snapshot: every member's weights
at that one era
* when the run gives up -> one shared plateau ladder
* whether it may deploy -> family-wise gate on the vote
WHY THE JOINT CHECKPOINT IS THE POINT: per-member selection picks each
net's own best era, and those eras differ. The resulting quartet was
never measured together at any instant, so the vote it casts live is a
configuration no OOS number ever described. Capturing all four at the
era whose vote won makes the deployed ensemble exactly the measured one.
Correct because of the era barrier (
|
||
|
AnimateDread
|
b77e7b4766 |
fix(ensemble): responsive panel + synchronized eras + combined-vote accuracy
Four user-reported/requested items, one root cause chain:
1) DEAD CONTROL PANEL in AI_HYBRID mode. All members posted custom event
id 1 and handled id 1001, and CExpertCustom broadcasts every chart
event to every filter - so each posted event ran a train chunk in ALL
N members (N*N chunks per round) and the chart thread never idled
long enough to deliver clicks/drags. profiling.csv: 99.45% of time in
OnChartEventHandler. Fix: per-instance study-event ids
(STUDY_EVENT_ID_BASE + construction order, offset above the Controls
library's ON_* codes - id 1 was also ON_DBL_CLICK, so panel
double-clicks fired training chunks). ArmStudyEvent() is the single
post site; lost-event watchdog replaces the accidental
sibling-clears-my-flag rescue.
2) WARM-UP DUPLICATION. The auto-tune sweep is deterministic over
identical features/labels, and it ends in the full MI diagnostic
suite, which the MI-share gate never intercepted on the sweep path -
four members ran four identical ~36s sweep+report blocks. First
member publishes outcome (g_ensembleChartTuneDone/Installed/Settings);
the rest apply it and skip both.
3) DEINIT STRANDED PANEL+ARROWS (user repro 18:52). Root cause from the
log: the 4,500ms budget runs from MetaTrader's stop REQUEST - a heavy
autosave in flight ate it, OnDeinit got ~430ms and died in the first
member's arrow persist ("Abnormal termination" 432ms in). Fix: early
visible-UI sweep (native prefix deletes for status/panel/dialog)
right after ClearStatusLabel, and a fast path for still-training
models - their arrows are re-rendered every era, so they get one bulk
purge instead of scan+atomic-write in the death window.
4) ENSEMBLE FEATURES (user requests): era BARRIER - members advance era
by era together; a member ahead of the slowest still-training member
declines Train() calls and its chunk budget is donated
(TRAIN_TIME_BUDGET_MS = 120/activeTrainers, UI headroom constant).
COMBINED-VOTE OOS SCORE - each member's pass-3 scan contributes its
adjusted per-bar decision (0.0 on abstain) to a shared row buffer;
the last member to finish the era scores the averaged vote vs the
mirrored Min_Vote_Open against the same target-before-stop outcomes
members grade themselves on, publishing an "Ensemble vote" line on
the aggregated panel. Member headlines now carry their lifetime win
rate with break-even.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
|
||
|
AnimateDread
|
7dff532c70 |
fix(ensemble): shared MI diagnostics + divided chunk budget - warm-up and panel responsiveness
Two user-reported ensemble regressions, one cause each: - "getting ready is very long": every member ran the full MI diagnostic suite (headline MI, positive control, alignment, lag profile, geometry scan + winner test - ~200 permuted draws per line) on IDENTICAL features and labels, reporting the same numbers four times. First member runs it, the rest adopt with one log line. Documented caveat: if the geometry scan ever ADOPTS a winner under its gate (it never has), the adoption becomes donor-only and the gate must be revisited. - "panel not responsive": four members chunks queue back-to-back on the one chart thread - 4 x 120ms = 480ms worst-case click latency, the exact regime the 200ms note in Training.mqh already documents as broken. Ensemble members now use a 30ms chunk budget, restoring solo UI latency at slightly higher dispatch overhead. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> |
||
|
AnimateDread
|
0adaea48b6 |
fix(resume): model reload stalled training - three hardenings on the resume path
A resumed META model hot-looped pass 1 (0->100% scan oscillation, silent for
3 minutes until the stall reporter fired) because EVERY window failed at the
first AD/Wyckoff feature: the init-time param adoption called
ReInitADIndicators unconditionally, destroying five freshly-calculating
indicator instances to recreate them with BYTE-IDENTICAL params (verified by
parsing the .nnw header - the MI tuner had kept the configured settings), at
process start, on a box with 1 GB free of 31. The replacements sat cold for
6+ minutes while full-history resweeps starved the indicator threads harder.
- AdoptIndicatorParams: installs a loaded param set into the tuner and
rebuilds handles ONLY when the set actually differs from what the live
indicators run. Both call sites (resume init + panel reload) use it.
- Resumed models get the same 3 warm-up passes as fresh ones. The skip was
the shared root cause of the cold-ATR (
|
||
|
AnimateDread
|
df48c37f65 |
feat: per-family x per-side OOS breakdown in the META era report
350-era S2 verdict on SP500 H1: the meta head carries REAL ranking skill (+1.0-1.3pp mean over base, 101/350 eras clear their own 2-sigma bar, traded subset wins 66.1% at <30% coverage vs 64.5% base) but 0/350 eras produced a positive cov x (p - BE): the candidate stream sits 3pp under the derived geometry's 67.5% break-even and ~2.6pp of recovered skill cannot bridge it. Skill plateaued by mid-run (1.28pp -> 1.05pp), so more eras only buy multiplicity, and the deploy gate correctly shipped nothing. The aggregate can hide a deployable subset (one family/side clearing BE blended with junk), so the META era line now decomposes the SAME traded population into MA/RSI/MACD/Ichimoku x LONG/SHORT cells, each as traded/candidates base->traded win rate. 32 cells is a best-of-N search by construction - any candidate cell faces the family-wise rule before belief. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> |
||
|
AnimateDread
|
444909d0a3 |
feat: S2 meta-labeling head - binary trade-quality model over the classic-candidate corpus
The NN now has a target that is not per-bar direction (closed, best-of-999 p=1.0000): P(win | this journaled candidate, at the EA's own SL/TP, net of cost). One net for all 52 pattern-sides, AIType=AI_META. - NetForward.mqh: the host-side softmax+CE gradient generalized total==3 -> 2||3 on both backprop paths; a 2-class softmax IS a logistic head, and no compute backend changes. - SignalMETA.mqh (new): corpus loaded read-only from the LARGEST signal DB on disk (decoupled from the config fingerprint that burned four S1 runs); the GMT->server offset is measured PER ROW against entryPrice vs bar open (DST-immune, histogram logged); a window-span regime filter drops the pre-2017 daily-backfill rows; 31-feature setup descriptor appended at the input (26 one-hot + side + tanh netVote + SL/TP ATR + spread/ATR). - Training.mqh: candidate-queued pass 1, binary-target pass 2, per-candidate calibration (2.5) and OOS (3) walks. Counter mapping win->Buy / loss->Sell lets checkpoint selection, the edge floor, the plateau ladder and the family-wise deploy gate run UNCHANGED: precision reads as win rate among traded candidates, chance as the base win rate, recalls as sensitivity/ specificity. Era-end META line: coverage x (p - break-even) vs the null. - Labels are the side-conditional triple-barrier win caches - never the DB's stop-and-reverse outcome. Logit adjustment deliberately skipped (~40% base rate). Live inference + online learning guarded off until S3. - Fingerprint: conditional |TGT:META1; State\META\ folder + 2-output filename slot keep meta models fully separate from direction models. Compiles clean (0 errors, 0 warnings). S2 run = attach a chart with AIType=AI_META; S3 wires the votes via the per-side hooks. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> |
||
|
AnimateDread
|
0848c8a16c |
fix: live inference queried the 1-tick forming bar - a window training never built
RefreshLatestSignal ran at the first tick after a bar opens and built its window at r=0: series index 0 at that instant is a candle with one tick of data - (close-open)/atr ~ 0, high ~ low, degenerate volume, indicators on a 1-tick bar. Training never produces such a window (every labeled bar is fully closed, entry at that bar's CLOSE), so the deployed model's final timestep - the one the LSTM/HYBRID output is keyed to - was out-of-distribution on every live decision, and pass 3's deploy-gate OOS scores measured a different query than live executed. The parity index is r=1: the newest CLOSED bar, whose close IS the current price - the exact instant the label's hypothetical entry happens. Single backtests shared the old skew (same r=0), which is why the tester agreed with live while both disagreed with training. Bookkeeping split that the index change forces: m_lastBarTime/dtStudied stay anchored to the FORMING bar's open (they gate against SERIES_LASTBAR_DATE; anchoring at bar 1 would re-fire the refresh every tick), while bt - the arrow, its High/Low placement, and NMS declustering - anchors to the decision bar, now matching the rescan path's convention. Also: a failed refresh no longer trades the previous bar's signal for the whole bar. RefreshLatestSignal returns success, zeroes dPrevSignal on failure (no opinion beats a stale one), and RefreshConvergedSignal advances dtStudied only on success so the next tick retries - the tester path (m_lastBarTime) already worked this way; this is the live path catching up. FORCES RE-VALIDATION of deployed models: the effective live query distribution changes. Bundled with the backprop transpose fix's retrain. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> |
||
|
AnimateDread
|
c5acc5a7a8 |
perf: pass 1 forward-passed ~40% of bars that a later pass redid anyway
Pass 1 already skipped its feedForward on QUEUED bars, because pass 2 redoes them. The same argument covers two more bands it was still forwarding: OOS window (30% of bars) - pass 3 re-forwards every one of them calibration band (~10% of bars) - pass 2.5 re-forwards every one of them All three passes derive their bounds from the same helpers and apply the identical eligibility test, so the bar sets are equal by construction, not by coincidence. Only the two purge bands and the ineligible edge bars are visited in pass 1 and nowhere else - those keep their forward pass. The scan's copy was never the one that survived. Its arrow-cache write was overwritten by pass 3's (with the thresholded, post-training decision), its status-label paint was transient, and its predicted-class tally measured last era's weights. Those tallies move to pass 2.5 and pass 3, on the raw argmax exactly as pass 1 and pass 2 count it, so the population behind the panel's "Predicted -> Buy/Sell/Neutral" line is unchanged and stays comparable with the "Actual" line beside it, which pass 1 still accumulates over every labelled bar. Verified unaffected by the cut: dPrevSignal and m_lastBarTime are both written last by bars 0/1, which are label-ineligible and therefore still forwarded, so FinalizeTrainRun's `dtStudied = m_lastBarTime` and Lifecycle's newBarPending sentinel read the same values as before. Correctness, not just speed: batch norm is UNFROZEN during pass 1 (passes 2.5 and 3 freeze it deliberately), so every scan-time forward on a held-out bar was advancing the BN running mean/variance from data the model is graded on. Those running statistics are inference-time model state. It is the mild, unsupervised kind of leakage - feature statistics, not labels - but it fed the weights pass 3 then scored, and it is now gone. Cost: ~40% of all bars lose one forward pass per era, ~16% of net time once pass 2's backward pass is weighted in. Per-dispatch, so it lands on every backend. Both variants compile 0 errors / 0 warnings. Build tag scan-nofwd-v5. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
AnimateDread
|
e2c959331f |
perf: the excursion head cost 3.6x era time - cut its dispatches ~250x
Measured on exc-race-v3: LSTM era 300s -> 1087s (net 272->748s, "other" 30->337s). My estimate had been "single-digit percent". The cost is per-DISPATCH, not per-FLOP, and therefore hits EVERY backend: the head is 19k weights and ~2.4 GFLOP an era - seconds of arithmetic - but ~48k forward/backward calls x several layer submits each, and its 760-wide layer exceeds the CPU DLL's inline threshold so each one pays a real handoff. The classifier's own net time tripled too, from contention with a second pool on an already-full box. Three changes, all backend-neutral because they remove submits rather than tune threads: SCORE ONLY DISJOINT WINDOWS (~64x). Adjacent bars share all but one bar of their horizon, so 16k consecutive bars were always ~250 independent observations - the full-sample tally was never worth more than the disjoint one, it just quoted an n that was ~64x too large. Dropping it costs nothing statistically and removes 63 of every 64 forward passes. The two parallel tallies collapse into one, which is also less code. The trailing ring still advances on every bar: it needs the outcome SEQUENCE, and that is array lookups, not a forward pass. TRAIN ON EVERY 4th PRIMARY BAR (4x). The target is low-dimensional and strongly autocorrelated - neighbouring bars carry near-identical excursion information - so per-bar training buys resolution the target does not have. Strided on ATTEMPTS, not acceptances, so a stretch of unlabelled bars cannot silently change the spacing. OWN TIMING COLUMN. The head's passes were landing in the era line's "other" bucket, which is how a 3.6x regression read as an unexplained jump in the one column nobody attributes. A cost that cannot be seen in the timing line cannot be traded off against anything. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
AnimateDread
|
950b0fdab0 |
diag: name the cause when every feature window fails, and enforce the width contract
Era 0 stalls with "NOT ONE of 54681 scanned bars produced a usable feature window, windows ok=0 failed=54681" and nothing else. That line reads identically for a cold ATR, a conditionally-missing optional feature block and an out-of-range index, so it cannot be diagnosed without one restart per hypothesis. Two changes: 1. WIDTH CONTRACT in BufferTempData. Every enabled block must emit exactly m_neuronsCount values on EVERY bar. A block that emits its values on some bars and skips them on others (indicator, panel or series unavailable for that bar) does not merely shorten the window - it SHIFTS every feature after it into the wrong slot, and the net then trains on silently misaligned inputs that still look like a valid window to everything downstream. Now rejected, rolled back and reported once, naming the optional blocks (XA / SPR / swing context) as the ones carrying an availability test. Worth having independently of the current stall. 2. BuildFeatureWindow records WHICH lookback slot rejected and how much of the window was assembled, and the pass-1 stall report renders it: "slot 0 of 20 REJECTED (window had 0 of 760)" is an indicator warm-up or history-edge read; "every lookback bar ACCEPTED and the window was still short: 640 of 760" is a missing 6-value block. No behaviour change on a healthy run: the width check is an equality that already holds, and the diagnostics render only inside the total-failure branch. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
AnimateDread
|
2d28f6542b |
feat: excursion-size head (Stage 1, measurement only)
Direction is closed - normalised asymmetry fails on three instruments with a working positive control, and the classifier's own best-of-999 era-cap test agrees (+0.9pp = 1.48 sigma, family-wise p=1.0000). SIZE is a different question and RANGE clears at ~4x its null. Checked the denomination before building on that, since the source memo warns to: m_excUpCache holds (maxHigh - fill)/ATR, so "RANGE is predictable" is a claim about travel RELATIVE to current ATR, not a restatement of "ATR is autocorrelated". It is exactly the part a fixed multiple (stop 3.31*ATR, target 1.64*ATR) discards. A second small CNet, 760 -> 24 -> 32 sigmoid outputs = P(price reaches ladder rung k) upward and downward. Survival parameterisation rather than regressing the multiple, because it needs nothing new from CNet: sigmoid outputs and the per-neuron delta the `total != 3` branch already applies (a quantile head would need a linear activation and a pinball gradient in Network.mqh, Network.cl and the DirectML path, on a class four topologies share). Targets are free - m_ladderUpAt already records first-touch age per rung with 0 meaning never reached. Separate net, not extra outputs on the classifier: more outputs would change m_outputNeuronsCount, the .nnw shape and the fingerprint, and push the count off 3 - the exact condition backProp uses to select the joint softmax gradient the 3-class head depends on. The classifier is bit-for-bit unaffected and this is removable without trace. STAGE 1 PLACES NO ORDERS. It reports a Brier skill score against the constant per-rung base rate - the baseline a fixed ATR multiple already assumes - with both predictors fitted IS and evaluated OOS, so neither gets a look at the test set. Positive skill justifies Stage 2 (drive SL/TP and sizing off ExcursionQuantile, which is defined and deliberately uncalled). Zero or negative means ATR already carries everything and Stage 2 must not be built. Trains only on primary occurrences: the replay queue oversamples for CLASS balance, and a direction-balanced sample is a biased SIZE sample. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
AnimateDread
|
0c8b4dc30d |
fix: the deploy gate graded the un-thresholded model
coveragePct, dirPrecPct and the declustered TRADED tally were all computed from oPrevSignal - the RAW argmax - while the live order, the arrow and the panel all run on oDeploySignal, which is argmax AFTER the confidence threshold. The gate was certifying a strategy the EA does not trade. Invisible until now: the threshold sat at ~0.02, so the two populations were the same set. The held-out calibration slice ( |
||
|
AnimateDread
|
2189316c35 |
fix: the operating point was fitted on bars the net had memorized
FitDirConfThreshold harvested its margin histogram from pass 2's own
backprop samples. Pairing every fit against the same era's OOS result
shows what that measured:
PAI era 1 IS 25% cov @ 66.1% (-0.8pp) -> OOS 64% (-3pp) gap +2.1pp
PAI era 76 IS 90% cov @ 79.6% (+12.7pp) -> OOS 65% (-2pp) gap +14.6pp
LSTM era 9 IS 77% cov @ 81.6% (+14.6pp) -> OOS 63% (-4pp) gap +18.6pp
The gap grows monotonically while OOS stays flat, so within a handful of
eras the curve stops describing behaviour on unseen bars. That is fatal
here specifically, because the objective branches on the SIGN of
(p - break-even): the memorized curve reads +12pp at 95% coverage, so
coverage x (p - p0) correctly maximises coverage and returns ~0.02 - fire
on every bar. The "p < p0 -> get more selective" branch, which is the
actual regime and the entire point of
|
||
|
AnimateDread
|
d919a4aea2 |
feat: 10-bar decluster window + alternation on every signal consumer
SignalClusterWindow 3 -> 10 for all topologies. On H1 a 3-bar window
collapsed only the tightest runs and left visible clusters at every
turn; 10 bars is closer to the spacing of genuinely distinct setups.
ALTERNATION. Rule 1 only collapses a same-direction run INSIDE the
window; past it a second Buy is emitted with no Sell between, giving
Buy/Buy/Buy/Sell. With both directions tradeable that sequence is the
model re-entering a move it is already in rather than finding a new
one. The kept sequence must now alternate: the first signal passes,
and after that a direction passes only if the last KEPT signal was the
opposite one.
Added to ALL THREE consumers, with identical logic, because they must
agree:
- NmsLiveAccept -> the live trade
- pass 3's OOS replay -> the tally the deploy gate grades
- PruneDirectionalClusters -> the drawn history
A rule applied to only some of these certifies one strategy and trades
another - the same defect class as the geometry the gate certified
while OpenParams placed something else (
|
||
|
AnimateDread
|
ccfbc62561 |
fix: the recall gate was unsatisfiable and the LR decay was a spiral
Both made the run structurally unable to succeed, independently of any
signal in the data. Found by reading the 13:01 log.
RECALL GATE. m_objectiveMet required Buy, Sell AND Neutral OOS recall
each >= 40%. First-touch resolution (
|
||
|
AnimateDread
|
ba13eefecc |
fix: a resumed model cached a cold ATR as permanent, so it never trained
BufferTempData cached EVERY failure - m_featureCacheHasValue[idx]=true with m_featureCacheValid[idx]=false - and the cache never re-tries a miss. So a single feature read taken before the terminal had finished calculating the indicator buffers marked those bars unusable for the rest of the process, even though the data arrived milliseconds later. MT5 fills an indicator's buffers asynchronously after the handle is created, and a cold ATR returns 0 for EVERY index, not just its warm-up tail. BufferTempDataCompute rejects a bar with no ATR (correctly - the price features would be meaningless), so the whole window failed, and the whole cache was poisoned. Only resumed models were hit, because only they read features that early. Topology.mqh sets m_warmupPassesRemaining = netLoaded ? 0 : 3: a fresh start sits through three separately-scheduled Train() calls before anything touches a feature, which is exactly what those passes are for. A resumed one skips them and TuneIndicatorsAndTrain drives StartLabelCachePrebuild and the MI report from the first chart event. Its rationale - "a restart already has a proven-synced history" - holds for HISTORY and not for INDICATORS, which are recreated every process start. Downstream: BuildFeatureWindow failed on every bar of every era, so add_loop never went true, so pass 2, pass 3, the era counter and the checkpoint were all skipped and pass 1 swept 0->100% forever. The "0 samples" MI report line at startup was the same failure, four seconds earlier, already visible in the log. - a miss is now cached only when it is PERMANENT; the two "not ready yet" guards mark m_featureFailTransient and are recomputed on the next visit. Steady-state cost is ~ind_Periods bars per era, not 54k. - an era that discards itself now drops the feature cache before restarting, so any remaining cause of this state self-heals instead of looping. Deleting the .nnw "fixed" this only by turning the model back into a fresh one. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
AnimateDread
|
464a0fe19d |
diag: an era that discards itself now says so instead of scanning forever
add_loop is exactly "at least one bar produced a usable feature
window". When it stays false, pass 2, pass 3, the era counter, the
checkpoint and every log line in the era-end block are ALL skipped:
Train() returns having done nothing, m_eraResumePending is still false,
and the next call restarts the SAME era from bar 0. That is an
infinite 0->100% "scan" loop that prints absolutely nothing - the only
remaining silent restart path in Train(), and it matches the reported
symptom exactly.
Pass 1 now counts usable vs unusable windows and reports at the pass
boundary, which demonstrably executes:
- total failure routes through ReportTrainStall (already capped at
one line a minute, and carries the run-state flags) naming the
counts, the required window width and the bar count
- success prints how long the scan took and how many samples it
handed to pass 2, but only once the era has passed 10s - a fast
era stays as quiet as before, a slow one distinguishes "advancing"
from "sweeping the same bars forever"
A PARTIAL failure is normal and deliberately does not shout: pass 1
walks oldest-to-newest and the deepest bars predate the indicators'
warm-up, so those windows fail and are cached as misses.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
||
|
AnimateDread
|
1a5157befc |
fix: training could only advance one 120ms chunk per bar
ScheduleTrainingIfNeeded() armed the next Train() call only when dtStudied < lastBarDate. That watermark test is right for a CONVERGED model - one inference refresh per new bar - and wrong for a training run, because Train() is chunked: it does ~120ms of work and yields, needing thousands of calls to finish one era, and every one of those calls has to be armed from there. dtStudied is two incompatible things. Train() sets it to the training WINDOW START (~2008); FinalizeTrainRun() sets it to the last bar SCANNED (~now). So the moment any run finalized, the scheduler went silent until the next candle closed. On H1 that is one chunk per hour. The symptom was indistinguishable from a hang: no era lines, no heartbeats, not one of the six instrumented stall branches - because Train() was not being CALLED. The TRAIN STALL line that caught it reported runActive=Y only because m_trainRunActive had been set microseconds earlier in that same call, and eraResume=N proved no era was in flight. Two log bursts, 28 minutes apart, exactly one H1 bar. Before |
||
|
AnimateDread
|
3855d4666a |
diag: the heartbeat could be outrun by the condition it watched for
It fired only on 4096-item boundaries once an era had already run 60s. Those boundaries are all crossed in the first few chunks of pass 1, so an era that became slow AFTER them printed nothing at all - which is precisely what happened: 20 minutes, four pegged cores, zero heartbeats. I read that silence as "the era loop is never reached" and went looking for a wedge above it. The silence may simply have meant "past the last boundary". A diagnostic whose trigger can be outrun by the condition it watches for is worse than no diagnostic, because it produces confident wrong conclusions. Now time-gated: checked every 256 items (the mask only keeps GetTickCount off the hot path), prints when the era has run >60s and >30s since the last line, up to 12 per era. Progress/phase for the panel is still published on every call, before any gate. Both build variants compile 0 errors, 0 warnings. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> |
||
|
AnimateDread
|
b461844767 |
fix: prebuild and era sized different windows; diag: Train() names its branch
TWO things, one incident. 1) THE BUG I SHIPPED IN |
||
|
AnimateDread
|
783fd9e7a6 |
fix: the panel showed "100%" for the whole of pass 1
The simple panel derived its percentage from pass 2's counters: (m_isTrainCursor+1) / max(m_isTrainQueueCount,1). During pass 1 those are 0 and 0, so the expression is (0+1)/max(0,1) = 100%. An era spends its first pass scanning ~38k bars - minutes of work - and the panel reported that phase as finished the entire time. Observed by the user as "started learning at 100% of their era and are stuck there", and it actively misled the diagnosis: the one number on screen said the opposite of what was happening. The UI cannot fix this on its own - it can see pass 2's counters but has no way to know which pass owns them. So each pass now PUBLISHES its own progress and a short phase name through TrainHeartbeat (which every pass already calls per item), and the panel just displays them: "learning (era 45, scan 34%)". Published before the heartbeat's 4096-item journal gate, so the panel updates continuously while the journal stays quiet. Both build variants compile 0 errors, 0 warnings. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> |
||
|
AnimateDread
|
694b75686e |
diag: slow eras must explain themselves - heartbeat + era time split + pass-1 paint
The 23:42 restart left all four charts grinding ~25x slower than the 18:01 baseline (era lines in 86 seconds there; 20+ minutes of nothing here), and NOTHING could say why from outside: pass 1 logs nothing, its status paint sat inside the !wouldQueue branch so the IS sweep - 80% of the pass, processed FIRST - painted nothing either, the VPS has no debugger for a thread stack, and the hourly new-bar cache invalidation cancels and restarts an unfinished era, so a slow era can stay invisible FOREVER. Externals gave: four chart threads at ~95% pure user-mode compute, DLL pool idle, no file writes. That narrows it to "MQL5-side per-item work in the era passes" and no further. So training now explains itself: - TrainHeartbeat: one line per 4096 processed items, only after an era has already run 60s, at most 6 lines per era - a healthy era stays exactly as quiet as before. Reports position and the cumulative split: feature-window builds vs net forward/backprop vs everything else. Hooked into all three passes. - The era summary line gains "| ERA TOOK Ns (feature windows X, net fwd/back Y, other Z)" whenever an era exceeded 120s. - Pass 1 paints its progress for QUEUED bars too, not just the OOS slice, so the panel shows "learning (era N)" instead of sitting on the idle writer's "Getting ready..." for the entire IS sweep. The label is throttled internally; painting per bar costs nothing. Both build variants compile 0 errors, 0 warnings. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> |
||
|
AnimateDread
|
0c85c54a5b |
fix: a restart no longer loses the measured geometry or the training window
Terminal restart, 22:25: all four resumed models sat on empty windows with enum 2:6 barriers. Three interlocking causes, all visible in one log excerpt: 1) THE PRE-SCAN WINDOW WAS SIZED BY THE SAVED WATERMARK. A resumed model's dtStudied sits at its last studied bar, so Bars(dtStudied, now) ~ 0 and the resumed-model MI pre-scan built a zero-bar "complete" label cache - logged as "Buy: 0 | Sell: 0 | Neutral: 0". Train()'s own era start RESETS dtStudied to the training-window rule before computing its window; the pre-scan did not. The rule is now factored into TrainWindowStart() and both use it. The scan also refuses to arm before SERIES_SYNCHRONIZED (it ran in the same second as OnInit), and deployed models keep their watermark - for them it gates inference recency, not a training window. 2) THE HORIZON LATCHED ON AN INDICATOR WARM-UP. ComputeBarrierHorizonBars ran against a ZigZag with 0 calculated legs, fell back, and EnsureBarrierHorizon latched fallback(32) x slMult x tpMult = 384 for the process lifetime. A leg-starved horizon is now PROVISIONAL: re-resolved on the next rebuild, the label cache wiped if it moved (labels from two horizons answer different questions), and the geometry deriver refuses to run from it - a pair derived over a warm-up window would get PINNED. 3) THE DERIVED GEOMETRY WAS NEVER PERSISTED. The .cfg is written at model creation and at weights-reset - both BEFORE era 0 derives - so the measured pair lived only in memory: every restart read back zeros, adopted nothing, fell back to the enum barriers, and the era-0-only gate meant a resumed model could NEVER re-derive. A full day of training on 3.33/1.62 resumed as 2:6. Now: the settled pair is pinned to the .cfg the moment derivation completes (one-shot, atomic write), and the derive gate accepts any model with no pinned pair, not just era 0 - mid-run stability is carried by m_geometryDerived itself, which never allows a second derivation. Both build variants compile 0 errors, 0 warnings. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> |
||
|
AnimateDread
|
199726f651 |
fix: a one-sided era can no longer become the best checkpoint
Measured on HYBRID, era 29 of the first win-scored run: the model collapsed
to always-Buy and was crowned "new best selection score 67.1%". Under
win-based scoring that is not a coincidence - the always-call-the-drift-side
model IS the chance reference, so it scores exactly chance (P(winLong) ~ 67%
on SP500), while every honest two-sided era scores 63-66% because shorts win
less often against the drift. Raw score ranking therefore actively prefers
the degenerate model, every regression restores back to it, and live NMS
collapses its near-constant signal to ~25 trades per era - observed as
"hybrid barely trades".
bothSidesLive already blocked one-sided eras from DEPLOYING (tradeableOK,
|
||
|
AnimateDread
|
5cef0947f4 |
fix: the deploy gate was benchmarking a win rate against a label frequency
The gate rests on an invariant stated at ExpertSignalAIBase.mqh:199 - under a
driftless walk P(touch +k before -m) is m/(m+k), and break-even for a k:m trade
is ALSO m/(m+k), so "beats chance" and "is profitable" are the same test.
That invariant needs reward >= risk, and the measured geometry no longer
satisfies it. With target 1.62*ATR and stop 3.33*ATR, break-even is 67.3%, but
both-won bars were stripped out of Buy and Sell so the label base rate read
37.5%. chancePrecPct is max(BuyTotal,SellTotal)/bars, so the gate was clearing
models nearly 30pp short of break-even: 42% "directional precision" is +4 sigma
against 37.5% and loses money on every single trade. Live since
|
||
|
AnimateDread
|
19dfb91108 |
feat: fitted directional confidence threshold - selectivity gets a mechanism
The training loss and the selection metric wanted different things and only the second one knew it. Logit-adjusted cross-entropy has no term for "how often should I trade", so the head calls a direction on 87-91% of bars. The selection metric is precision x coverage credit, saturating at the coverage floor - above the floor extra calls earn NOTHING and only precision counts. So selection wanted few good calls, the loss produced many mediocre ones, and all selection could do was pick the least-bad era out of what it was handed. Nothing pushed the model toward selectivity. This gives the decision RULE the policy instead of distorting the loss (which is estimating class probabilities correctly, and a probability estimate should not be bent to encode a trading policy - Elkan 2001: estimate, then choose the operating point separately). AdjustedSignalFromSoftmax now abstains unless the winning direction's softmax margin over its best rival clears a fitted threshold. Margin, not the winning probability: the latter moves with overall calibration rather than with how close the decision actually was. Fitted on IS, applied to OOS and live. Pass 2 already forward-passes every IS sample, so the margin histogram is harvested there for free (primary occurrences only, so the oversampled replay queue cannot skew the operating point); the fit runs at the end of pass 2, BEFORE pass 3, so the deploy gate grades the thresholded model on bars the threshold never saw. Fitting on pass 3's own predictions would be choosing the operating point on the data being graded - the best-of-N error corrected in five other places here. Objective: maximise IS directional precision subject to still clearing the SAME coverage floor the deploy gate uses (base rate x 0.25, re-derived locally so the two cannot drift apart). Swept top-down in one pass; ties go to the LOWER threshold, since equal precision for less coverage is strictly worse. Under DIR_CONF_MIN_FIT_CALLS (200) it runs unthresholded rather than on a guess. The threshold is part of the MODEL, not the run: captured with Net.CaptureWeights(), restored with the weights at both restore sites, and appended to the .cfg under the same length-guard convention so a deployed model reloads at the operating point its gate actually cleared. A pre-2026-08-09 .cfg reads 0.0, which is exactly the behaviour it was trained under. Per-era line now prints "@margin>=X.XX" next to coverage, so a coverage drop can be attributed to the operating point rather than guessed at. Both build variants compile 0 errors / 0 warnings. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> |
||
|
AnimateDread
|
371f8aaecd |
fix: the Adam second moment was never Adam - all four tiers
Root cause of the B=32 regression, and it predates F4 entirely. Every Adam
kernel stored v already square-rooted and then fed that stored value back in
as if it were the variance:
v_new = sqrt(b2 * v_old + (1 - b2) * g^2)
That recursion has a fixed point at v ~= b2 = 0.999 for ANY gradient below
unit scale, so the denominator stops tracking the gradient and Adam degrades
into plain SGD with lr = lt. Measured against the shipped WarriorCPU.dll
(batch_accum_check.cpp, TestOptimizerScaleInvariance), 4000 steps of a
constant gradient: 3285x less displacement at |g|=1e-5 than at |g|=1, where
a scale-invariant optimizer gives the same distance for both. After the fix
all six magnitudes read 1.199 and v tracks |g| exactly.
It hit conv/LSTM specifically because they sit behind a batch-norm with
running variance ~2.6e+05, so their gradients arrive divided by ~500 - deep
in the degraded regime - while the dense stack near the loss stayed in the
working one. In situ on SP500 H1: lstm1 dW/W 2.62/10.0/7.14% -> 0.024/0.022/
0.003%, conv1 decaying to 0.000% by era 30. NeuronBatchNorm.mqh already
squared v back for gamma/beta and its comment named the kernels as wrong,
which is exactly why gamma/beta kept training while the stages behind froze.
Persisted .nnw needs no migration - v keeps its std-dev meaning.
Also, the two ways F4 exposed it, both mine:
- No LR compensation for B fewer steps per era. sqrt(B) for adaptive methods
(Krizhevsky 2014; Granziol et al. 2022), applied once in
InitialEtaForOptimizer(). Linear scaling (Goyal et al. 2017) is for SGD.
- Plateau patience denominated in eras, so raising B made the ladder 32x more
impatient in its only unit. PAI converged at era 41 on ~49k updates where
the same config had been finding new bests at era 1028.
TrainPlateauPatienceEras() stretches it by the same sqrt(B).
TRAIN_BATCH_SIZE 32 -> 8 so the patience stretch stays affordable (8 -> 23
eras per stage, not 8 -> 45). Both helpers are identities at B=1.
Deploy gate: DEPLOY_MIN_SIDE_RECALL_PCT (10%) folded into tradeableOK. The
perceptron reported Sell:0% recall in all 41 eras, cleared the floor on Buy
alone at 36.6% vs 34% chance, deployed, and sprayed buy arrows. Folded into
the ranking key rather than checked at deploy time so a one-sided era cannot
become best-so-far in the first place.
Deinit: the arrow purge now runs BEFORE ExtPanel.Destroy(), an unbounded
CAppDialog teardown that sat ahead of it - the same ordering inversion the
rule there exists to prevent. CONV was force-terminated 4.8 s into OnDeinit
(vs ~1.1 s for the three that finished) having reached none of its cleanup,
so its arrows stayed on the chart. Steps are now timed in the log.
PurgeChart's verification rescan filtered on OBJ_ARROW, the same blind spot
as the bulk delete, so "persisted 10 ... cleared 0" passed silently. It now
walks every object type and reports the object counts when both are zero.
Both build variants compile 0 errors / 0 warnings; both DLLs rebuilt.
FORCES A RETRAIN (already forced by N1) and both DLLs must ship with the .ex5.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
|
||
|
AnimateDread
|
0c01dc279b |
feat: mini-batch gradient accumulation (F4), front-end-aware capacity budget (F6), split Wyckoff categoricals (N1)
Completes the 2026-08-09 training audit. FORCES A RETRAIN of every
Wyckoff-enabled config (N1 re-keys the fingerprint), and BOTH DLLs must be
redeployed alongside the .ex5 - they carry new exports.
F4 - mini-batch accumulation, TRAIN_BATCH_SIZE=32. Training was pure online
SGD (one weight update per bar), which is the mechanical source of the
era-to-era whipsaw every downstream guard was built to cope with. The O(n^2)
outer product is native - AccumulateWeightGrad / AccumulateWeightGradConv /
AccumulateBufferInto in Network.cl, WarriorCPU and WarriorDML - while the
optimizer step is host-side MQL5 shared by all tiers (ApplyAccumToBlock), so
there is one Adam/SGD implementation instead of four that can drift.
- the LSTM needs no outer-product kernel (WeightsGradient already holds the
sample's full dW) but could NOT simply be left un-zeroed between samples:
CPU_LSTMSeqBackward/DML_LSTMSeqBackward memset it on entry. Hence a
separate accumulator plus an elementwise add.
- batch-norm gamma/beta accumulate in host arrays, not new BatchOptions
slots - BN_OPT_STRIDE is baked into every persisted .nnw.
- scoped to pass 2; online learning keeps immediate updates. Every save /
checkpoint / scoring boundary flushes, scaling by the real sample count.
- degrades to per-sample updates (one log line) on a tier that cannot
accumulate, so old devices and DLL-free builds are unaffected.
- verified offline: DirectML/batch_accum_check.cpp drives the real exports
against an independent reference; at B=1 the accumulator matches the
shipped unbatched kernel's own gradient to 1.1e-16. Math only - the
in-situ check remains the per-layer dW/W report on a real era.
F6 - ComputeFirstLayerWidth budgeted against the RAW input width even where a
conv/LSTM front end had already reduced it, so an LSTM's dense stack was
charged for 1,280 inputs when it receives 64. Confirmed from the deployed
.cfg files: CONV, LSTM and HYBRID were all pinned at the 16-unit floor. Now
budgeted against the front-end output and capped at it (never fan out), with
the derivation reordered so both stages settle first.
N1 - EventCode/EventPhase/StructuralPhase are signed categoricals packing
direction and Wyckoff stage into one scalar across a sign discontinuity. Split
into direction + [0,1] magnitude, the same convention the base OHLC block uses.
Information-preserving; 13 readings now occupy 16 inputs.
Compiled clean (0 errors, 0 warnings); both DLLs rebuilt.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
|
||
|
AnimateDread
|
274630f802 |
fix: training-stability audit fixes F1/F2/F3/F5 - unbiased shuffle, real plateau escapes, fresh optimizer state on restore, pure OOS metric
Four of the six findings from research/training_pipeline_audit_2026-08-09.md (F4 mini-batching and F6 feature re-encode deliberately deferred - see the report's implementation-status section for why): - F1: pass-2 Fisher-Yates (and AutoTune's MI block shuffle) used MathRand()%, which is 15-bit - provably non-uniform on every full-history era over 32,768 queued samples. New 30-bit ShuffleRandomIndex(). - F2: plateau warm restarts were a no-op whenever eta already sat at its ceiling (the normal state of a non-regressing plateau) - the ladder was just a 24-era countdown. Restarts now overshoot to 5x the ceiling (PLATEAU_RESTART_BOOST) and anneal geometrically back over the patience window, SGDR-style; ETA_MIN widened 1e-4 -> 1e-5 so the decay schedule has real range. - F3: checkpoint restores put weights back but kept the rejected trajectory's Adam moments, so the optimizer immediately pushed back toward the rolled-back state (the restore->regress->restore oscillation). CNet::ResetOptimizerState() zeroes moments/momentum/step counters (weights, BN statistics, gamma/beta untouched) on every mid-run restore, every boosted restart, and the deploy-time restore that online learning continues from. - F5: batch-norm running statistics now freeze for the pass-3 OOS scoring walk, so the selection metric the checkpoint ranking and deploy gate read is a pure function of the checkpoint instead of partly measuring BN drift. Defensive unfreeze in FinalizeTrainRun covers stop-mid-pass; live/online adaptation and the OOS continual-learning simulation stay adaptive by design. Compiled clean (0 errors, 0 warnings) via the staged-tree recipe. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> |
||
|
AnimateDread
|
ee48381cbd |
fix: NMS gates the TRADE, not just the arrow - one arrow is now one trade
NmsLiveAccept() appeared in exactly one place: wrapped around DrawObject(). It never touched dPrevSignal, and dPrevSignal is what LongCondition() / ShortCondition() / SignedAIConfidence() read. So a declustered bar lost its arrow and still opened a position. Measured on SP500 H1 2026-08-09: CONV called a direction on 64% of bars, so the ~500 bars visible on screen held ~320 decisions - and ~40 arrows were drawn. Roughly one arrow per eight positions the EA would take. And the survivors are not a random eighth. Rule 2 of the declustering keeps the HIGHER-CONFIDENCE side of a cluster, so the visible set is systematically the best member of each run. A chart showing the best of every eight decisions and hiding the rest reads far better than the model is - the same best-of-N selection error already corrected in the geometry scan, the indicator tuner, the lag profile and the deploy gate, this time on the display layer, where it is most likely to mislead the person deciding whether to trade. Fixed by neutralising dPrevSignal when NMS rejects, rather than adding a "may trade" flag consulted at each read site: that leaves exactly ONE definition of what the model decided this bar, so the arrow, the panel's "Current signal", the confidence feeding sizing/SL/TP/trailing, the refresh tally and the order itself cannot drift apart again. Also reports the consequence instead of hiding it. Every OOS counter on the era line still scores every directional call - a population ~8x larger than what now trades - so the line carries a second figure: | TRADED (declustered) NN% on N calls (edge +Npp) replaying the identical rule over pass 3 (which walks OOS bars oldest to newest, the same order the live sweep sees). Its cursors are separate members from the live ones so a training pass can never disturb the live chart's declustering. Deliberately NOT switched into selectionScore yet. Declustering cuts coverage from ~64% of bars to ~8%, well under MIN_COVERAGE_FRACTION_OF_BASE_RATE, which would make every checkpoint undeployable overnight - the minRR collision and the recall-floor catch-22 twice over. The floor gets re-derived from these measurements first. Compiles clean: 0 errors, 0 warnings. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> |
||
|
AnimateDread
|
1df305431d |
feat: gate deployment on the null of the MAXIMUM, not the per-era null
EDGE_MIN_SIGMAS is a PER-ERA test and the deployed model is the MAXIMUM
over every era a run ranks. A 2-sigma one-sided test passes on noise with
probability 0.0228 per era, so over N eras the chance at least one clears
it is 1-(1-0.0228)^N: 34% by era 18, 80% by era 70, 93% by era 112. The
gate was near-certain to open on a long run whatever the data held.
It did. HYBRID deployed 2026-08-08 at dir-precision 35.5% vs 34% chance -
+1.5pp, best of 112 eras whose per-era values wandered 30%..35.5%. At the
call counts these runs produce that is p_family 0.92..0.9999.
Every OTHER best-of-N decision here already carries this correction, and
every one REJECTS on this data: the barrier-geometry winner (null of the
maximum over 6, p=0.3902), the indicator tuner (Sidak, p=1.0000), the MI
lag profile (null of the maximum over 21 lags). The one decision that
ships a model to a live account had none.
BestCheckpointSurvivesSelection() re-tests the checkpoint that is about to
deploy:
z = (precision - chance)/SE, SE = sqrt(p0(1-p0)/n)
p_single = P(Z >= z)
p_family = 1 - (1-p_single)^N
against DEPLOY_FAMILY_WISE_ALPHA. It uses the checkpoint's OWN
snapshotted precision/chance/call-count, not the latest era's, because
the model that ships is the one that has to clear the bar.
N counts CANDIDATE eras (coverage measurable, at least one directional
call) - an era that called nothing directional could never have become
the best, so counting it would make the gate stricter than the search
that actually happened.
Conservative on purpose: consecutive eras share OOS bars and differ by
one gradient step, so they are nowhere near N independent draws and the
true family-wise error is below this bound. This gate decides what trades
real money and the house posture is reject-unless-demonstrated.
Effect at 2900 directional calls / N=112: required edge goes 1.76pp ->
2.92pp. A real edge clears it; +1.5pp does not.
Applied to BOTH automatic paths - the plateau ladder's stage-3 deploy and
the m_trainingComplete assignment - which must stay identical or the flag
persisted into the .nnw disagrees with the decision to stop, and a reload
runs inference on a model the ladder refused.
NOT applied to the two operator paths (era-cap deploy, panel Deploy
button). Those stay the operator's call; ReportSelectionGateVerdict()
logs the verdict beside them so an authorised deploy can never later be
misread as a validated one.
NormalUpperTail() is A&S 26.2.17 (|err| < 7.5e-8), self-contained rather
than pulling in Math\Stat. Verified against reference values to 6dp:
Q(1.645)=0.049985, Q(1.96)=0.024998, Q(3.0)=0.001350. Its locals are
ntB1..ntB5 because AI\Network.mqh line 79 does "#define b1 AdamBeta1".
Compiles clean: 0 errors, 0 warnings.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
|
||
|
AnimateDread
|
bfc1da9de1 |
fix: the sequence models were reading the window backwards
BuildFeatureWindow() replaces eight hand-rolled copies of the same loop
and feeds the window OLDEST BAR FIRST. Every copy fed it newest-first,
because MQL5 timeseries indices run backwards and `r + b` with b ascending
walks into the past.
Harmless for PAI and CONV - a dense layer learns a weight per position
either way, a conv learns time-mirrored kernels. Not harmless for the
recurrent stacks:
- LSTM_SeqStepForward reads `inputs + t*Iw`, so step t is block t.
- It writes output[] only when t == steps-1: the visible output IS the
last hidden state.
- c_t = f*c_{t-1} + i*g decays toward the start of the sequence.
lstm_seq_flowcheck.cpp measured block 0's influence on the output at
1.2e-2 of block T-1's, at the shipped forget bias of 1.0.
So the bar being PREDICTED sat at the far end of the decay and the output
was handed to the OLDEST bar in the window - the exact inverse of what the
window is for. ~80x backwards on LSTM and HYBRID, on all three tiers
(OpenCL kernel, CPU DLL, pure-MQL5 inference), which is why it never
surfaced as a backend discrepancy.
This does not create edge - the MI diagnostics read at the noise floor
(p=0.4975) with a working positive control. It makes the one hypothesis
those diagnostics explicitly do NOT cover testable: they are marginal and
per-bar, and state they "cannot rule out one that only exists in
combination or across time". The sequence model is the instrument for
across-time structure and it has been crippled, so that hypothesis has
never been honestly tested.
Fingerprint gets an unconditional |WIN:2 - the vector keeps its shape and
its features, so a stale .nnw would load cleanly and run a model fitted to
one ordering against the other, silently. Re-keying every config is the
point, not collateral damage. FORCES A FULL RETRAIN.
Also: the now-relative bar caches are re-keyed on the two live paths.
EnsureBarCachesCapacity() was only ever called from training paths, but
once m_trainingComplete is set ScheduleTrainingIfNeeded() routes every bar
to RefreshConvergedSignal() and Train() is never re-entered - so nothing
cleared the feature cache again for the life of the process. A chart that
trained to convergence kept replaying the rows computed for the last
training era's bar grid: the live signal froze at its convergence-time
value, and OnlineLearnStep() backpropped those stale features against
freshly resolved labels. Backtests were never affected (an inference-only
process never allocates the arrays, so every read recomputes).
Compiles clean: 0 errors, 0 warnings.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
|
||
|
AnimateDread
|
5f647ba5db |
fix: improve error messages and suppress false sharing-violation logs
- BufferDouble: replace hardcoded "DirectML/CPU-DLL" with dynamic backend name and add buffer index/element count to all error prints for easier debugging. - NetPersistence: distinguish missing file from transient lock by probing FileIsExist before logging, eliminating false "sharing violation" warnings when no saved model exists on first run. |
||
|
AnimateDread
|
ceb6342dfd |
feat(ai): spread as a volatility-regime feature, and fix a stale-index cache in both new blocks
Adds spread/ATR and the spread change ratio as network inputs (EnableSpreadFeature,
default on). Spread is the one microstructure channel that is both FX-available and
genuinely historical in the Strategy Tester - "during testing, the spread is not modeled
but is taken from historical data" - so unlike swap, signed tick flow or depth of market it
is something a backtest can honestly validate.
What it encodes, stated precisely because the raw measurement overstates it.
research/test_spread.py found spr/atr the strongest single feature in this codebase, on 5
of 8 instrument/geometry cells at 2-4x any volume feature. But the barrier LABEL charges
the spread inside its own barriers, so a wide-spread bar is mechanically likelier to
resolve as a loss and the feature would partly be predicting its own cost model. Relabelling
at zero cost and re-measuring the identical feature showed 20-40% of it WAS that tautology
and the majority was not (XAUUSD retained 97%). What survives is a volatility-regime
reading: spread is near-fixed while ATR is not, so the ratio runs high exactly when
realised volatility is below its own ATR estimate, which genuinely predicts whether
ATR-scaled barriers get reached. It is UNSIGNED - Neutral-vs-directional only, never a side.
Also fixes a stale-index bug I introduced with the cross-asset panel and had just repeated
in the spread series. Both cached on length alone:
if(m_crossAsset.Bars() >= bars) return true;
MQL5 series indices are relative to NOW, so one new closed candle shifts every index by
one. Keyed only on length, the panel keeps serving its index 0 as a bar that is no longer
the newest, and every cross-asset value is read one bar out of step with the price features
sitting beside it in the same vector - silently, with no error and no shape change. This is
the same class of defect as the dtStudied watermark behind the zero-direction backtests.
Both now carry a datetime anchor on m_Time.GetData(0), the same invalidation key the
label/feature bar caches already use.
And a performance fix that fell out of it: with correct invalidation the panel rebuilds on
every new bar, and RefreshConvergedSignal runs per bar - which in the tester would mean one
full multi-symbol resample per simulated bar at training depth. Inference only reads bars
0..m_historyBars-1 plus the panel's own slow window, so it now requests exactly that. The
cache check is >=, so a deeper panel left from training still satisfies it.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
|
||
|
AnimateDread
|
8710240cd5 |
fix(signals): revive a dead MA model, and demote Sanyaku from state to event
Two defects surfaced by research/test_classic.py, both verified fixed by re-running the
transcription against 178k bars of EURUSD H1.
CSignalMA model 1 could never fire. For any recursive average - and MA_TYPE_EMA is the
shipped default - MA(i) = a*Close(i) + (1-a)*MA(i+1), so
DiffMA(i) = a * (Close(i) - MA(i+1))
DiffCloseMA(i) = (1-a) * (Close(i) - MA(i+1))
are positive multiples of one quantity and always share a sign. Model 1 asks for a close
BELOW a RISING average, which is precisely the combination that identity forbids: 0.000%
of bars, either direction, any symbol. The MQL5 standard library this was ported from
defaults to MODE_SMA, where the two are merely correlated - the bug arrived with the EMA
default, not with the port. Reading the slope one bar back (DiffMAPrev) breaks the tie for
every MA type while keeping the model's stated meaning. Now fires on 7.92% of bars.
CSignalIchimoku model 11 fired on 27% of bars at weight 100. Sanyaku is three standing
STATES conjoined with no transition term, so it held across long stretches - and being
last in the if-chain at the top weight, the module's highest-conviction reading was also
its most common one, overwriting all eight event models below it on a quarter of all bars.
The old comment rejected an event form because "demanding all three flip on the same bar
would fire almost never" - true, but that is not the alternative. Kouten is the TURN: the
ALIGNMENT transitions, and only one role need change for it to. Testing !Sanyaku(idx+1)
fires once per aligned stretch. Now 2.17%, in line with Kumo breakout (2.4%) and the
strong TK cross (1.1%). DataReady() extended one bar deeper to cover the lookback.
Neither pattern showed edge before or after; this is about the models meaning what they
say and the vote not being dominated by a constant.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
|
||
|
AnimateDread
|
a77ff64b13 |
fix(deinit): a full model write was running ahead of the cheap cleanup
"Abnormal termination" is back, and this time it is not the arrows. The timing names the culprit exactly: 16:02:31.547 OnDeinit: shutting down 16:02:36.003 Abnormal termination <- 4.46 s, MetaTrader gave up 16:02:36.226 chart signals - persisted <- cleanup finished 0.2 s LATE OnDeinit called StopTraining() BEFORE the chart cleanup. StopTraining() finalises an in-flight run, and FinalizeTrainRun() restores the best checkpoint and then persists it - a full ~1MB model write per signal. So the expensive step ran ahead of the cheap bounded one, which is precisely the inversion the shutdown ordering exists to prevent. The previous fix put PersistWeightsOnShutdown last and missed that StopTraining smuggles a second save in at the front. Two changes: Cleanup now runs FIRST, then StopTraining, then the weight save. The visible teardown is cheap and bounded, so it always completes even when everything after it is killed. And the deploy-persist inside FinalizeTrainRun is suppressed during shutdown. RestoreWeights() is an in-MEMORY swap, so the best checkpoint is already the live net by that line, and PersistWeightsOnShutdown writes exactly those weights moments later. The old path wrote the same model twice per signal - eight full writes across four charts - for no benefit. A user-pressed Stop still persists immediately, because nothing else would. Compiles 0 errors / 0 warnings. Build tag deinit-order-v2. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> |
||
|
AnimateDread
|
018afb1ba9 |
fix(autotune): MI scorer read an array nobody filled; add the permutation floor
THE TUNER WAS A SILENT NO-OP. Every chart logged auto-tune complete - 17 candidate settings scored in ~139s, feature/label mutual information 0.0000 -> 0.0000 nats (no improvement) 0.0000 is not a weak result, it is a broken measurement: finite-sample MI is biased UPWARD, so even pure noise scores above zero. Cause: ScoreCurrentParamsByMI called BufferTempDataCompute(), which APPENDS the bar's features to TempData and never touches m_featureCache - only the caching wrapper BufferTempData() writes that array. It then read m_featureCache, which ReInitADIndicators had just invalidated. Every column came back constant, FeatureColumnMI returned 0 for all of them, and all 17 candidates tied at exactly zero. 139 s per chart to return the settings it started with. Now reads the values back out of TempData, where they actually land. And an exactly-zero best score is called out as a fault rather than reported as "no improvement", because that is what it is. ADDED: a PERMUTATION BASELINE, which is the diagnostic this project has been missing. MI's finite-sample bias is ~(bins-1)(classes-1)/(2n) nats - at these sample sizes the same order as any real edge in this domain - so a raw MI figure is uninterpretable on its own. Shuffling the labels destroys every genuine association while leaving sample size, binning and class proportions intact, so the score it produces IS this dataset's noise floor, measured rather than approximated. The log now reads feature/label information - X nats against a shuffled-label floor of Y and says outright whether the features carry usable information about the target. It needs no training, no topology and no convergence, so unlike every accuracy number in this codebase it cannot be confounded by an optimizer or an objective. If the score sits on the floor, no change of architecture can help - which is the question the last three days of zero-edge results have been circling. DEPLOY FLOOR: `dirPrecPct > chancePrecPct` passed anything above chance by any amount. At ~11,000 directional calls the standard error of the precision estimate is ~0.4pp, so that gate was accepting sub-one-sigma noise - the perceptron deployed at edge +0pp on 2026-08-01. Now requires EDGE_MIN_SIGMAS (2.0) standard errors above chance, computed from the actual call count, so the bar scales with the evidence instead of needing a hand-picked constant. Recorded with it, because it is why chance is the right reference at all: under a driftless random walk P(touch +k*ATR before -m*ATR) = m/(m+k), and the break-even win rate for a k:m reward:risk trade is ALSO m/(m+k). The label's own base rate IS the break-even rate, at every SL/TP setting. So "beats chance" and "is profitable" are the same test, and no choice of SL/TP can manufacture an edge - only prediction can. Both builds compile 0 errors / 0 warnings. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> |
||
|
AnimateDread
|
d7eea325fb |
refactor(ai): extract Layer.mqh and deduplicate AI config
- Moves CLayer neuron construction to AI/Impl/Layer.mqh to keep Network.mqh clean - Unifies four previously duplicated architecture initialisation blocks (MLP/CONV/LSTM/HYBRID) into a single shared function - Eliminates risk of behavioural drift where one architecture missed a setter, causing mismatched feature sets or targets |