2026-08-01 11:27:28 -04:00
|
|
|
//+------------------------------------------------------------------+
|
|
|
|
|
//| Lifecycle.mqh |
|
|
|
|
|
//| |
|
|
|
|
|
//| Construction/destruction, the CExpertSignal vote API |
|
|
|
|
|
//| (LongCondition/ShortCondition/ConfidenceTier/pattern weights), |
|
|
|
|
|
//| tick + chart-event dispatch, and the per-config chart lock. |
|
|
|
|
|
//| |
|
|
|
|
|
//| PARTIAL IMPLEMENTATION FILE - not standalone. |
|
|
|
|
|
//| CExpertSignalAIBase method BODIES only. The class declaration |
|
|
|
|
|
//| lives in Expert\ExpertSignalAIBase.mqh, which includes this file |
|
|
|
|
|
//| at the bottom, after the declaration. Do not include it |
|
|
|
|
|
//| anywhere else and do not compile it on its own. |
|
|
|
|
|
//+------------------------------------------------------------------+
|
|
|
|
|
#ifndef WARRIOR_AIBASE_LIFECYCLE_MQH
|
|
|
|
|
#define WARRIOR_AIBASE_LIFECYCLE_MQH
|
|
|
|
|
//+------------------------------------------------------------------+
|
|
|
|
|
//| Constructor |
|
|
|
|
|
//+------------------------------------------------------------------+
|
|
|
|
|
//--- These are only fallback defaults for a fresh object before the EA's OnInit() applies the
|
|
|
|
|
//--- active input values via the public setters in Warrior_EA.mq5. The input-driven values are the
|
|
|
|
|
//--- source of truth for the actual run configuration.
|
|
|
|
|
CExpertSignalAIBase::CExpertSignalAIBase(void) :
|
|
|
|
|
ID("NULL"),
|
|
|
|
|
m_neuronsCount(0),
|
|
|
|
|
m_minTrainYear(1970),
|
|
|
|
|
m_optimizationAlgo(TrainingOptimizer), // see the member declaration comment
|
|
|
|
|
//--- Placeholder only; InitNeuralNetwork() replaces it with ComputeFirstLayerWidth() before anything
|
|
|
|
|
//--- reads it. Deliberately the floor rather than 0, so a hypothetical path that built a topology
|
|
|
|
|
//--- without going through init would produce a small usable net instead of a zero-width layer.
|
|
|
|
|
m_initialNeuronsCount(FIRST_LAYER_MIN_WIDTH),
|
|
|
|
|
m_outputNeuronsCount(OUTPUT_CLASSIFICATION),
|
|
|
|
|
//--- Frozen. Nothing reads these to build a topology any more - the taper derives its own endpoints
|
|
|
|
|
//--- (BuildFreshTopology) - but they still occupy positional slots in the .cfg sidecar and the weights
|
|
|
|
|
//--- fingerprint. Held at their historical defaults so both stay byte-stable; changing either value
|
|
|
|
|
//--- would re-key every model on disk for no behavioural reason whatsoever.
|
|
|
|
|
m_minNeuronsCount(MIN_NEURONS_20),
|
|
|
|
|
m_neuronsReduction(RF_70),
|
|
|
|
|
m_hiddenLayersCount(3),
|
|
|
|
|
m_lstmHiddenSize(32),
|
|
|
|
|
m_convFilterCount(16),
|
|
|
|
|
m_historyBars(14),
|
|
|
|
|
m_fractalPeriods(5),
|
|
|
|
|
m_pattern_0(25),
|
|
|
|
|
m_pattern_1(50),
|
|
|
|
|
m_pattern_2(75),
|
|
|
|
|
m_pattern_3(100),
|
feat: S2 meta-labeling head - binary trade-quality model over the classic-candidate corpus
The NN now has a target that is not per-bar direction (closed, best-of-999
p=1.0000): P(win | this journaled candidate, at the EA's own SL/TP, net of
cost). One net for all 52 pattern-sides, AIType=AI_META.
- NetForward.mqh: the host-side softmax+CE gradient generalized total==3 ->
2||3 on both backprop paths; a 2-class softmax IS a logistic head, and no
compute backend changes.
- SignalMETA.mqh (new): corpus loaded read-only from the LARGEST signal DB on
disk (decoupled from the config fingerprint that burned four S1 runs); the
GMT->server offset is measured PER ROW against entryPrice vs bar open
(DST-immune, histogram logged); a window-span regime filter drops the
pre-2017 daily-backfill rows; 31-feature setup descriptor appended at the
input (26 one-hot + side + tanh netVote + SL/TP ATR + spread/ATR).
- Training.mqh: candidate-queued pass 1, binary-target pass 2, per-candidate
calibration (2.5) and OOS (3) walks. Counter mapping win->Buy / loss->Sell
lets checkpoint selection, the edge floor, the plateau ladder and the
family-wise deploy gate run UNCHANGED: precision reads as win rate among
traded candidates, chance as the base win rate, recalls as sensitivity/
specificity. Era-end META line: coverage x (p - break-even) vs the null.
- Labels are the side-conditional triple-barrier win caches - never the DB's
stop-and-reverse outcome. Logit adjustment deliberately skipped (~40% base
rate). Live inference + online learning guarded off until S3.
- Fingerprint: conditional |TGT:META1; State\META\ folder + 2-output filename
slot keep meta models fully separate from direction models.
Compiles clean (0 errors, 0 warnings). S2 run = attach a chart with
AIType=AI_META; S3 wires the votes via the per-side hooks.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-13 06:52:31 -04:00
|
|
|
m_trainTarget(0),
|
2026-08-15 16:50:36 -04:00
|
|
|
m_ensembleMember(false),
|
2026-08-15 16:54:43 -04:00
|
|
|
m_ensemblePanelSlot(-1),
|
2026-08-15 19:00:40 -04:00
|
|
|
m_fracLegCount(0),
|
feat: S2 meta-labeling head - binary trade-quality model over the classic-candidate corpus
The NN now has a target that is not per-bar direction (closed, best-of-999
p=1.0000): P(win | this journaled candidate, at the EA's own SL/TP, net of
cost). One net for all 52 pattern-sides, AIType=AI_META.
- NetForward.mqh: the host-side softmax+CE gradient generalized total==3 ->
2||3 on both backprop paths; a 2-class softmax IS a logistic head, and no
compute backend changes.
- SignalMETA.mqh (new): corpus loaded read-only from the LARGEST signal DB on
disk (decoupled from the config fingerprint that burned four S1 runs); the
GMT->server offset is measured PER ROW against entryPrice vs bar open
(DST-immune, histogram logged); a window-span regime filter drops the
pre-2017 daily-backfill rows; 31-feature setup descriptor appended at the
input (26 one-hot + side + tanh netVote + SL/TP ATR + spread/ATR).
- Training.mqh: candidate-queued pass 1, binary-target pass 2, per-candidate
calibration (2.5) and OOS (3) walks. Counter mapping win->Buy / loss->Sell
lets checkpoint selection, the edge floor, the plateau ladder and the
family-wise deploy gate run UNCHANGED: precision reads as win rate among
traded candidates, chance as the base win rate, recalls as sensitivity/
specificity. Era-end META line: coverage x (p - break-even) vs the null.
- Labels are the side-conditional triple-barrier win caches - never the DB's
stop-and-reverse outcome. Logit adjustment deliberately skipped (~40% base
rate). Live inference + online learning guarded off until S3.
- Fingerprint: conditional |TGT:META1; State\META\ folder + 2-output filename
slot keep meta models fully separate from direction models.
Compiles clean (0 errors, 0 warnings). S2 run = attach a chart with
AIType=AI_META; S3 wires the votes via the per-side hooks.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-13 06:52:31 -04:00
|
|
|
m_metaCandCount(0),
|
2026-08-01 11:27:28 -04:00
|
|
|
m_useVolumes(true),
|
|
|
|
|
m_useTime(true),
|
|
|
|
|
m_useATR(true),
|
|
|
|
|
m_useMA(false),
|
|
|
|
|
m_useRSI(false),
|
|
|
|
|
m_useMACD(false),
|
|
|
|
|
m_useIchimoku(false),
|
|
|
|
|
m_useSwingContext(false),
|
|
|
|
|
m_useNews(false),
|
fix(signals): revive a dead MA model, and demote Sanyaku from state to event
Two defects surfaced by research/test_classic.py, both verified fixed by re-running the
transcription against 178k bars of EURUSD H1.
CSignalMA model 1 could never fire. For any recursive average - and MA_TYPE_EMA is the
shipped default - MA(i) = a*Close(i) + (1-a)*MA(i+1), so
DiffMA(i) = a * (Close(i) - MA(i+1))
DiffCloseMA(i) = (1-a) * (Close(i) - MA(i+1))
are positive multiples of one quantity and always share a sign. Model 1 asks for a close
BELOW a RISING average, which is precisely the combination that identity forbids: 0.000%
of bars, either direction, any symbol. The MQL5 standard library this was ported from
defaults to MODE_SMA, where the two are merely correlated - the bug arrived with the EMA
default, not with the port. Reading the slope one bar back (DiffMAPrev) breaks the tie for
every MA type while keeping the model's stated meaning. Now fires on 7.92% of bars.
CSignalIchimoku model 11 fired on 27% of bars at weight 100. Sanyaku is three standing
STATES conjoined with no transition term, so it held across long stretches - and being
last in the if-chain at the top weight, the module's highest-conviction reading was also
its most common one, overwriting all eight event models below it on a quarter of all bars.
The old comment rejected an event form because "demanding all three flip on the same bar
would fire almost never" - true, but that is not the alternative. Kouten is the TURN: the
ALIGNMENT transitions, and only one role need change for it to. Testing !Sanyaku(idx+1)
fires once per aligned stretch. Now 2.17%, in line with Kumo breakout (2.4%) and the
strong TK cross (1.1%). DataReady() extended one bar deeper to cover the lookback.
Neither pattern showed edge before or after; this is about the models meaning what they
say and the vote not being dominated by a constant.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-01 17:14:34 -04:00
|
|
|
m_useCrossAsset(false),
|
feat(ai): spread as a volatility-regime feature, and fix a stale-index cache in both new blocks
Adds spread/ATR and the spread change ratio as network inputs (EnableSpreadFeature,
default on). Spread is the one microstructure channel that is both FX-available and
genuinely historical in the Strategy Tester - "during testing, the spread is not modeled
but is taken from historical data" - so unlike swap, signed tick flow or depth of market it
is something a backtest can honestly validate.
What it encodes, stated precisely because the raw measurement overstates it.
research/test_spread.py found spr/atr the strongest single feature in this codebase, on 5
of 8 instrument/geometry cells at 2-4x any volume feature. But the barrier LABEL charges
the spread inside its own barriers, so a wide-spread bar is mechanically likelier to
resolve as a loss and the feature would partly be predicting its own cost model. Relabelling
at zero cost and re-measuring the identical feature showed 20-40% of it WAS that tautology
and the majority was not (XAUUSD retained 97%). What survives is a volatility-regime
reading: spread is near-fixed while ATR is not, so the ratio runs high exactly when
realised volatility is below its own ATR estimate, which genuinely predicts whether
ATR-scaled barriers get reached. It is UNSIGNED - Neutral-vs-directional only, never a side.
Also fixes a stale-index bug I introduced with the cross-asset panel and had just repeated
in the spread series. Both cached on length alone:
if(m_crossAsset.Bars() >= bars) return true;
MQL5 series indices are relative to NOW, so one new closed candle shifts every index by
one. Keyed only on length, the panel keeps serving its index 0 as a bar that is no longer
the newest, and every cross-asset value is read one bar out of step with the price features
sitting beside it in the same vector - silently, with no error and no shape change. This is
the same class of defect as the dtStudied watermark behind the zero-direction backtests.
Both now carry a datetime anchor on m_Time.GetData(0), the same invalidation key the
label/feature bar caches already use.
And a performance fix that fell out of it: with correct invalidation the panel rebuilds on
every new bar, and RefreshConvergedSignal runs per bar - which in the tester would mean one
full multi-symbol resample per simulated bar at training depth. Inference only reads bars
0..m_historyBars-1 plus the panel's own slow window, so it now requests exactly that. The
cache check is >=, so a deeper panel left from training still satisfies it.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-01 17:42:40 -04:00
|
|
|
m_useSpreadFeature(false),
|
|
|
|
|
m_spreadSeriesBars(0),
|
|
|
|
|
m_spreadSeriesAnchor(0),
|
|
|
|
|
m_crossAssetAnchor(0),
|
2026-08-11 21:29:14 -04:00
|
|
|
m_crossAssetPairsPinned(""),
|
|
|
|
|
m_crossAssetCfgSaved(false),
|
2026-08-16 13:39:00 -04:00
|
|
|
m_useAltData(false),
|
2026-08-16 15:12:54 -04:00
|
|
|
m_altDataEnabled(true),
|
2026-08-16 20:04:13 -04:00
|
|
|
m_altDataLateWarned(false),
|
2026-08-16 13:39:00 -04:00
|
|
|
m_altDataNamesPinned(""),
|
2026-08-01 11:27:28 -04:00
|
|
|
m_newsFeatureWindowMinutes(60),
|
|
|
|
|
m_useADCumulativeDelta(false),
|
|
|
|
|
m_useADShorteningOfThrust(false),
|
|
|
|
|
m_useADWyckoffEventStream(false),
|
|
|
|
|
m_useADWyckoffFailedStructure(false),
|
|
|
|
|
m_useADWyckoffSignificantBarInversion(false),
|
|
|
|
|
m_autoTuneIndicators(false),
|
|
|
|
|
m_indicatorsPtr(NULL),
|
|
|
|
|
Net(NULL),
|
|
|
|
|
m_shadowNet(NULL),
|
|
|
|
|
TempData(NULL),
|
|
|
|
|
dError(-1),
|
|
|
|
|
dUndefine(0),
|
|
|
|
|
dForecast(0),
|
|
|
|
|
dPrevSignal(0),
|
|
|
|
|
m_refreshOk(0),
|
|
|
|
|
m_refreshFailFeatures(0),
|
|
|
|
|
m_refreshFailShort(0),
|
|
|
|
|
m_refreshBuy(0),
|
|
|
|
|
m_refreshSell(0),
|
|
|
|
|
m_refreshNeutral(0),
|
|
|
|
|
m_voteGateBlocked(0),
|
|
|
|
|
m_voteGatePassed(0),
|
|
|
|
|
m_voteGateCompleteAtFirst(-1),
|
|
|
|
|
m_voteGateLoadedAtFirst(-1),
|
|
|
|
|
m_signalClusterWindow(6),
|
|
|
|
|
m_nmsLiveBuyTime(0),
|
|
|
|
|
m_nmsLiveSellTime(0),
|
|
|
|
|
m_nmsLiveBuyAccept(false),
|
|
|
|
|
m_nmsLiveSellAccept(false),
|
|
|
|
|
m_nmsLiveKeptTime(0),
|
|
|
|
|
m_nmsLiveKeptDir(Neutral),
|
|
|
|
|
m_nmsLiveKeptConf(0),
|
|
|
|
|
dtStudied(0),
|
|
|
|
|
m_eraCount(0),
|
|
|
|
|
m_trainingComplete(false),
|
|
|
|
|
m_inferenceOnly(false),
|
|
|
|
|
m_modelLoadedFromDisk(false),
|
|
|
|
|
m_topologySuperseded(false),
|
|
|
|
|
m_mqlInferenceValidated(false),
|
|
|
|
|
m_shadowBootstrapAttempted(false),
|
|
|
|
|
m_enableOnlineLearning(true),
|
|
|
|
|
m_freezePriorCalibration(false),
|
|
|
|
|
m_onlineLearnedUpToTime(0),
|
|
|
|
|
m_onlineRollingAcc(-1.0),
|
|
|
|
|
m_onlineSamples(0),
|
|
|
|
|
m_onlineBarsSincePersist(0),
|
|
|
|
|
m_onlineBlendFrozen(false),
|
|
|
|
|
bEventStudy(false),
|
|
|
|
|
m_oosSplitPct(30),
|
|
|
|
|
dOosError(-1),
|
|
|
|
|
dOosForecast(0),
|
|
|
|
|
m_oosSamples(0),
|
|
|
|
|
m_cumIsCorrect(0),
|
|
|
|
|
m_cumIsTotal(0),
|
|
|
|
|
m_cumOosCorrect(0),
|
|
|
|
|
m_cumOosTotal(0),
|
|
|
|
|
m_oosOutSpreadSum(0),
|
|
|
|
|
m_oosOutCount(0),
|
fix(geometry): the target was small BY CONSTRUCTION - ratio is now policy, scale is measured, ladder ceiling removed
The derivation read the stop from q75 of ADVERSE travel and the target from q50
of FAVOURABLE travel. Over one horizon those distributions are broadly the same
shape, so q75 > q50 MECHANICALLY - the target came out smaller than the stop no
matter what the market did. SP500 H4 shipped stop 3.07 / target 1.70: a 0.55:1
payoff needing 64.3%. That was never a measurement, it was two mismatched
constants.
The reachability line printed beside it - "target on 50.0% of bars, stop on
25.0%" - is exactly 1-q50 and 1-q75. Tautological. It cannot disconfirm
anything, and it read as validation.
WIDTH AND RATIO ARE INDEPENDENT AND ONLY ONE PAYS. EV = edge x width;
ratio is EV-neutral (a driftless walk reaches +m before -k with probability
k/(k+m), which IS break-even). Width is what buys cost efficiency: the spread
is a fixed 0.047*ATR here, so the shipped 4.77*ATR width paid it 21 times per
unit of travel. So:
RATIO = policy. BARRIER_TARGET_RR = 2.0 (user's 1:2). Break-even 33.3%.
SCALE = measured. The stop quantile is chosen from a ladder, WIDEST FIRST,
taking the first rung whose implied 2x target is still reached often
enough to be a trainable class.
That last clause is the difference from the min-reward:risk raise removed in
2026-08-09, which forced target = 2 x stop with NO reachability test, landed on
6.66*ATR reachable on 3.3% of bars, and trained the model to predict something
that essentially never happened. Same ratio; the scale now retreats until the
data says the target is attainable. Every rung is logged.
LADDER CEILING REMOVED. BARRIER_LADDER stopped at 5.00 and the expectancy scan's
"best resolvable pair on width alone" came back as stop 5.05 / target 4.95 - it
pinned to the top rung. A recommendation landing exactly on the edge of its own
search space is a boundary, not a finding: it cannot tell "5 ATR is optimal"
from "5 ATR is all we allowed". Extended to 20*ATR (8 -> 14 rungs). Nothing else
needs editing - every consumer is parameterised by BARRIER_LADDER_COUNT - and
the horizon constraints (decided >= 60%, reachability floor) now bind instead of
a constant.
THE SCAN COULD NOT SEE THE SHIPPED GEOMETRY. ReportBarrierGeometryScan looked
the configured pair up in its integer grid, and DeriveBarrierGeometry produces
CONTINUOUS multiples (3.07/1.70) that can never equal a grid point - so
cfgExcess stayed at its -1.0 sentinel and the report printed "configured 3:2
scores -1.00000", which reads as a catastrophic score and actually means "never
evaluated". Worse, the grid skipped target<stop entirely because it "inverts the
trade's whole premise" - while the derivation was shipping exactly that. The
incumbent is now always scored as a peer (never crowned; it is already in force
and is not an enum pairing the scan could adopt).
BREAK-EVEN NOW INCLUDES THE SPREAD. Every report quoted the frictionless
SL/(SL+TP). On SP500 H4 that read 64.3% while the MEASURED zero-skill rate was
62.1% - a 2.2pp gap that IS the cost, and that made every model look 2.2pp
better than it was. CostAdjustedBreakEvenPct() prices a win at (TP - spread) and
a loss at (SL + spread), matching the expectancy scan's convention exactly so
the two reports cannot disagree.
It also feeds FitDirConfThreshold, which is the correctness half: the operating
point subtracts break-even from precision, so the frictionless figure made every
candidate threshold look better by the width of the spread - 2.2pp against a
measured edge of 2.3pp, i.e. very nearly all of it.
Era line now carries both: "break-even 64.3% frictionless, 66.6% AFTER SPREAD".
Forces a full relabel and retrain. Requested.
NOT COMPILED - user compiles.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-17 00:34:32 -04:00
|
|
|
m_spreadAtr(0.0),
|
feat(diagnostics): split a reported "Neutral" into CHOSE vs TIED - they need opposite fixes
ApplyClassificationSoftmax() requires a STRICT majority over both rivals and
sends every tie, 2-way or 3-way, to Neutral. So "OOS recall Neutral:100%" is
two completely different events sharing one label:
CHOSE - the net genuinely ranks Neutral highest. A class-prior/label problem.
TIED - the top two are EXACTLY equal, so the net expressed no preference and
the tie-break reported Neutral. A SATURATION problem: the head is
SIGMOID, and a saturated sigmoid returns exactly 0.0f or 1.0f in the
DLL's float32, so two classes pinned to the same rail compare equal
and the bar is silently discarded.
Nothing in the logs could tell them apart, and the fixes point opposite ways.
Eras 1-25 of the 2026-08-17 solo PAI run read "Neutral 100%" at spread avg 0.99
- fully saturated - and broke out at era 27 as the spread fell to 0.75. That is
consistent with EITHER story. The user reports the Neutral phase on most runs,
so it is worth four longs to stop guessing.
Four per-era counters on the pass 3 OOS walk, reported as:
| Neutral CHOSE 12.4% / TIED 38.1% (of which B=S 1204) | rail 61.2%
m_oosNeutralStrict - Neutral strictly highest
m_oosNeutralTie - no strict winner; the tie-break produced Neutral
m_oosTieBuySell - the costly subset: Buy and Sell tied AT the top, i.e. a
DIRECTIONAL reading thrown away by float equality
m_oosRailBars - any raw output sitting on a sigmoid asymptote, the
saturation that makes exact ties possible at all
Read on the RAW logits, before ApplyClassificationSoftmax() overwrites TempData
in place. Legitimate because softmax is strictly monotone: it cannot change the
ordering and cannot break a tie either, so the raw reading and the decision
always agree. Placed alongside the existing min/max/spread capture so all the
output diagnostics describe the same values.
Measurement only - no decision path reads these.
NOT COMPILED - user compiles.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-17 00:06:54 -04:00
|
|
|
m_oosNeutralStrict(0),
|
|
|
|
|
m_oosNeutralTie(0),
|
|
|
|
|
m_oosTieBuySell(0),
|
|
|
|
|
m_oosRailBars(0),
|
2026-08-01 11:27:28 -04:00
|
|
|
m_countBuySignals(0),
|
|
|
|
|
m_countSellSignals(0),
|
|
|
|
|
m_countNeutralSignals(0),
|
|
|
|
|
m_trueBuyCount(0),
|
|
|
|
|
m_trueSellCount(0),
|
|
|
|
|
m_trueNeutralCount(0),
|
|
|
|
|
m_logitAdjustTau(1.0),
|
|
|
|
|
m_logitAdjustLogged(false),
|
|
|
|
|
m_logitAdjustSkipWarned(false),
|
|
|
|
|
m_prevEraTrueBuyCount(0),
|
|
|
|
|
m_prevEraTrueSellCount(0),
|
|
|
|
|
m_prevEraTrueNeutralCount(0),
|
|
|
|
|
m_oosBuyHits(0),
|
|
|
|
|
m_oosBuyTotal(0),
|
|
|
|
|
m_oosSellHits(0),
|
|
|
|
|
m_oosSellTotal(0),
|
|
|
|
|
m_oosNeutralHits(0),
|
|
|
|
|
m_oosNeutralTotal(0),
|
|
|
|
|
m_oosBuyPredicted(0),
|
|
|
|
|
m_oosBuyPredictedHits(0),
|
|
|
|
|
m_oosSellPredicted(0),
|
|
|
|
|
m_oosSellPredictedHits(0),
|
|
|
|
|
m_oosNeutralPredicted(0),
|
|
|
|
|
m_oosNeutralPredictedHits(0),
|
fix: the deploy gate was benchmarking a win rate against a label frequency
The gate rests on an invariant stated at ExpertSignalAIBase.mqh:199 - under a
driftless walk P(touch +k before -m) is m/(m+k), and break-even for a k:m trade
is ALSO m/(m+k), so "beats chance" and "is profitable" are the same test.
That invariant needs reward >= risk, and the measured geometry no longer
satisfies it. With target 1.62*ATR and stop 3.33*ATR, break-even is 67.3%, but
both-won bars were stripped out of Buy and Sell so the label base rate read
37.5%. chancePrecPct is max(BuyTotal,SellTotal)/bars, so the gate was clearing
models nearly 30pp short of break-even: 42% "directional precision" is +4 sigma
against 37.5% and loses money on every single trade. Live since 217b9bc.
Root cause is that label agreement stopped being the same question as trade
profitability. Buy implies winLong, but the converse fails on every both-won
bar, and the label can only name one of two directions that both pay.
So stop asking the model whether it matched a label and start asking whether
its trade paid:
- cache winLong/winShort per bar beside the label, under the same validity
flag; published from the barrier walk before the collapse to 3 classes
- dirPrecPct now counts wins on the side actually called
- chancePrecPct is max(P(winLong), P(winShort)), MEASURED - the textbook
m/(m+k) would credit SP500's drift to the model
- the NMS "what would I have made" pair, the live-fired precision, and the
IS/OOS cumulative win rates all move to the same test. IS and OOS are read
side by side as the overfitting signal, so measuring one in wins and the
other in agreement would put a fixed gap between them that has nothing to do
with generalization
- the confidence threshold is FITTED on wins too, so the operating point
maximises what the gate grades
- per-class label-agreement precision is still computed and logged; it is the
right diagnostic for class separation, just not for a deploy decision
- era line renamed dir-precision -> win-rate, chance -> chance=break-even
Both build variants compile 0 errors, 0 warnings.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-09 16:00:51 -04:00
|
|
|
m_oosBuyPredictedWins(0),
|
|
|
|
|
m_oosSellPredictedWins(0),
|
|
|
|
|
m_oosWinLongTotal(0),
|
|
|
|
|
m_oosWinShortTotal(0),
|
2026-08-01 11:27:28 -04:00
|
|
|
m_oosConfidenceSum(0),
|
|
|
|
|
m_confidenceCalScale(1.0),
|
|
|
|
|
m_minDirectionalRecallPct(40),
|
feat(gate): grade OOS calls on the exit policy actually in force, and move vote combining out of the members and into the orchestrator
Option (a) from the exit-policy question: the certified number must be the traded
number. Plus the modularity correction the user called for on 778b6c0.
1. VOTE COMBINING BELONGS TO THE ORCHESTRATOR, NOT TO A MEMBER.
778b6c0 fixed the last-writer-wins bug on g_LiveAISignedConfidence by having a
member average its siblings through g_warriorEnsemble. That trades a scheduling bug
for a coupling bug, and it is the wrong shape for this EA: every signal runs in its
own instance, minds its own state, and VOTES to the orchestrator, which is the only
thing allowed to combine opinions.
Replaced with a publish/aggregate pair in Variables\ConfidenceBridge.mqh whose split
is enforced by shape rather than by convention:
- PublishAIVote(slot, conf) - a member writes ONLY its own slot, reads nobody's;
- AggregateAIVotes() - called by CExpertSignalCustom::LiveSignedConfidence.
CExpertSignalAIBase::EnsembleLiveSignedConfidence is gone. The orchestrator also
republishes the aggregate into g_LiveAISignedConfidence, because the intelligent
trailing reads that global directly and must act on the same number the exit route
does rather than on a leftover from whichever member ticked last. A solo AI signal
owns slot 0, so the non-ensemble path is unchanged.
2. THE GATE NOW REPLAYS THE REAL EXIT RULE.
SimulateTradeOutcome() walks the same price series with the same fill/barrier/spread
convention as ComputeLabelForBar - deliberately by copy, so a disagreement between
the two can only be a policy effect and never a discrepancy between two pieces of our
own arithmetic - and terminates at the FIRST of stop / target / vote reversal /
horizon. Barriers are tested before the vote on the same bar: intrabar we cannot know
which came first, and the barrier is what the broker executes automatically, so
checking the vote first would credit the exit policy with escapes a real stop would
have taken out of its hands.
It runs AFTER pass 3, not inside it. A vote-flip exit for a trade entered at bar r is
decided by the model's output at bars r-1, r-2, ... - NEWER bars - and pass 3 walks
oldest-to-newest, so at the moment r is graded its own exit does not exist yet. Only
once m_oosDecisionSeries is complete over the whole OOS window can the replay run.
In ensemble mode that series carries the member's adjusted decision and the live exit
reads the ensemble aggregate, which is the coupling the user identified: an LSTM entry
really can be closed by the ensemble turning against it.
3. THE STATISTIC HAS TO CHANGE WITH THE POLICY, AND THAT IS THE REAL FINDING.
A barrier exit pays a fixed R. A vote exit pays whatever the close happens to be. So
the moment vote exits are enabled the payoff is CONTINUOUS, and "win rate vs
break-even" stops being a meaningful test - there is no fixed break-even for a
variable payoff. SimulateTradeOutcome therefore returns R rather than a bool, and the
replay reports expectancy in R with its SE taken from the R distribution (overlap-
deflated on the same EffectiveSampleSize doctrine as every other SE here), not from a
binomial.
This is the same class of error as win-based scoring in 2026-08-09: measuring a
variable-payoff process with a fixed-payoff statistic. Naming it now, while vote exits
are still off, is much cheaper than discovering it after they go on.
4. WHY THIS IS SAFE TO SHIP TODAY.
Min_Vote_Close ships Disabled (101, unreachable on both scales it drives) and reaches
the AI signal through the new ExitPolicy() setter as 1.01, which the setter turns into
0.0 = no vote exit. Under that policy every replayed trade resolves at a barrier and
the simulation is arithmetically the same trade the deploy gate already certifies -
they cannot drift. The report says so explicitly, and prints ONCE per run in that
state; when vote exits are on it prints every era, because then the divergence is the
thing to watch. Nothing about today's numbers moves.
The gate switchover is wired but dormant by construction: it becomes exit-aware the
moment the input is enabled, which is exactly what "the certified number is the traded
number" has to mean.
KNOWN LIMIT, stated rather than hidden: only the AI early-exit route is replayed. The
rule-based averaged-vote close (m_threshold_close) depends on every other filter's
live vote, which pass 3 does not reproduce, so a position the classic filters would
have closed is held to its barrier here. The replay therefore holds LONGER than live
and overstates barrier-reached outcomes. Faithful only while the AI is the dominant
vote - which is the configuration this is being built for.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-17 16:25:57 -04:00
|
|
|
//--- No vote-driven exit until the inputs say otherwise - matches Min_Vote_Close's shipped Disabled.
|
|
|
|
|
m_exitVoteThreshold(0.0),
|
|
|
|
|
m_exitHoldToBarrier(false),
|
|
|
|
|
m_simRSum(0.0),
|
|
|
|
|
m_simRSumSq(0.0),
|
|
|
|
|
m_simTrades(0),
|
|
|
|
|
m_simVoteExits(0),
|
|
|
|
|
m_simBarrierWins(0),
|
|
|
|
|
m_exitReplayReported(false),
|
feat(gates): derive the recall floor instead of configuring it, publish what the configuration can PROVE, and stop an arbitrary member driving live exits
Three changes, all from the same principle: measure what is there before aiming
at it, and never certify a number you do not trade.
1. THE RECALL FLOOR IS DERIVED, AND IT MOVES BELOW CHANCE.
MinRecall=40 was a constant doing a statistical job. Its reference point is the
33.3% recall a zero-skill 3-class model gets on EVERY class, and against that the
constant was accidentally calibrated for exactly one sample size: on USDJPY CONV
(n_eff 195) 40% is chance + 2.0 SE; on SP500 PAI (n_eff 42) the same 40% is chance
+ 0.9 SE. One chart was being held to a bar twice as strict as the other, for no
reason anyone chose.
CollapseRecallFloorPct() computes it per class from that class's own effective
sample - EffectiveSampleSize(), so the overlap deflation the rest of the gates use
applies here too - as chance - EDGE_MIN_SIGMAS x SE. 26.5% at n_eff 195, 18.8% at
n_eff 42.
BELOW chance, deliberately, and this is the substantive change rather than the
arithmetic. This gate's only job is refusing to call a COLLAPSED model converged.
It is not a quality bar; the deploy gate is the quality bar and it is already
rigorous (chance + 2 SE on the deflated sample, Sidak over candidate eras, then the
cross-instrument pooled certificate). A convergence gate that ALSO demands
provably-above-chance recall on all three classes double-counts that job, and it
has failed that way twice here: MinRecall=60 blocked every SP500 H1 run in 2026-07,
and the 40 that replaced it made Neutral structurally unreachable once first-touch
resolution cut Neutral to a 0.65% residue. A floor nothing can reach does not make
a funded account safer, it stops the run converging at all.
Testing significantly BELOW chance instead catches what a fixed 40 was actually
catching - a model that has stopped emitting a class - and cannot become
unreachable by construction. It also fixes the direction the old constant scaled:
it now widens on a thin OOS window, where low recall genuinely cannot be told from
noise, and tightens on a rich one. Today's SP500 PAI (Buy 51 / Sell 18 / Neutral 30)
is still correctly blocked on Sell.
This also resolves a standing contradiction the code half-admitted at the
isBetterEra comment: selection ranks on coverage-weighted PRECISION while
convergence gated on RECALL, so a sparse high-precision abstainer - precisely the
model that could clear the deploy bar - was blocked by the floor.
The era line now PRINTS the derived floor. Anyone comparing these recalls against a
remembered "40" is reading the wrong bar.
2. DETECTABILITY: WHAT THIS CONFIGURATION COULD PROVE, BEFORE IT TRAINS.
The DEPLOY BAR line states the bar. It never said what reaching it would take, and
that is the actionable direction. ReportDetectability() inverts the same identity -
the gate passes when edge >= z x sqrt(p(1-p)/n_eff), so certifying an edge d needs
n_eff >= z^2 p(1-p)/d^2 independent calls, hence L times as many raw ones - and
prints a +2 / +5 / +10pp ladder as required independent calls, raw calls, and share
of the OOS window, marking any rung that needs more than the window holds IMPOSSIBLE.
Every term is a property of the CONFIGURATION - geometry via break-even, horizon via
mean label lifespan, window via oosCutoff - so no amount of training moves any of
them. It fires once, at the first healthy sweep, beside ReportFeatureHealth, for the
same reason: that is the first moment the bar grid, the measured geometry and the
lifespan are real numbers rather than defaults. It gates nothing.
This is the EdgeFinder discipline applied to our own gate: establish what the market
and the measurement design have to offer, then point the net at it - rather than
spending a thousand eras chasing something this OOS window could never certify.
3. AN ARBITRARY MEMBER WAS DRIVING LIVE EXITS AND TRAILING (user-identified).
Every ensemble member ran
g_LiveAISignedConfidence = SignedAIConfidence();
unconditionally, every tick. Last writer wins. Its consumers are the AI early-exit
route (CExpertSignalCustom::LiveSignedConfidence) and TrailingIntelligent - so on a
four-model chart an LSTM entry could be closed, and its stop moved, on the
Perceptron's opinion alone, decided by scheduling order. Not the vote, not a
weighted blend.
Now the mean across registered members, matching how the ensemble actually trades:
the open decision is the weighted-average vote, and an abstaining member contributes
0 and dilutes exactly as it does there. Members still training read 0, so a
half-trained ensemble reads WEAKER rather than louder - the safe direction for an
exit trigger. Deployed and paused members are included, which is the opposite of the
era barrier's exemption rule and correct for the opposite reason: that one asks who
must be waited for, this asks who has an opinion.
Latent today and staying that way for now by choice - Min_Vote_Close ships Disabled
(101, unreachable on both scales it drives) and TrailingStrategy is off, so live
exits are SL/TP only and the certified hold-to-barrier win rate is what actually
gets traded. Fixed now precisely because the plan is to enable vote exits once the
models are accurate, at which point a scheduling-order exit would be both harmful
and very hard to see.
STILL OPEN, and needs a decision before vote exits go on: the member gate and the
ensemble vote gate both grade hold-to-barrier, so enabling vote exits makes the
certified number stop describing the traded one. Warrior_EA.mq5 currently argues
barrier models may keep vote exits because "their label IS the vote's own horizon" -
that does not hold, since a vote flip at bar 5 of a 64-bar horizon is not the
target-before-stop outcome the gate measured. Either grade the OOS call on the real
exit rule (first of SL / TP / vote-flip / horizon) through the fill engine, or set
HoldToBarrier for ensemble members so the policy cannot drift from the certificate.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-17 16:15:08 -04:00
|
|
|
m_lastRecallFloorPct(0.0),
|
|
|
|
|
m_detectabilityReported(false),
|
2026-08-01 11:27:28 -04:00
|
|
|
m_priorBuy(0.0),
|
|
|
|
|
m_priorSell(0.0),
|
|
|
|
|
m_priorNeutral(0.0),
|
|
|
|
|
m_oosBuyFired(0),
|
|
|
|
|
m_oosBuyFiredHits(0),
|
|
|
|
|
m_oosSellFired(0),
|
|
|
|
|
m_oosSellFiredHits(0),
|
fix: NMS gates the TRADE, not just the arrow - one arrow is now one trade
NmsLiveAccept() appeared in exactly one place: wrapped around DrawObject().
It never touched dPrevSignal, and dPrevSignal is what LongCondition() /
ShortCondition() / SignedAIConfidence() read. So a declustered bar lost its
arrow and still opened a position.
Measured on SP500 H1 2026-08-09: CONV called a direction on 64% of bars,
so the ~500 bars visible on screen held ~320 decisions - and ~40 arrows
were drawn. Roughly one arrow per eight positions the EA would take.
And the survivors are not a random eighth. Rule 2 of the declustering
keeps the HIGHER-CONFIDENCE side of a cluster, so the visible set is
systematically the best member of each run. A chart showing the best of
every eight decisions and hiding the rest reads far better than the model
is - the same best-of-N selection error already corrected in the geometry
scan, the indicator tuner, the lag profile and the deploy gate, this time
on the display layer, where it is most likely to mislead the person
deciding whether to trade.
Fixed by neutralising dPrevSignal when NMS rejects, rather than adding a
"may trade" flag consulted at each read site: that leaves exactly ONE
definition of what the model decided this bar, so the arrow, the panel's
"Current signal", the confidence feeding sizing/SL/TP/trailing, the
refresh tally and the order itself cannot drift apart again.
Also reports the consequence instead of hiding it. Every OOS counter on
the era line still scores every directional call - a population ~8x larger
than what now trades - so the line carries a second figure:
| TRADED (declustered) NN% on N calls (edge +Npp)
replaying the identical rule over pass 3 (which walks OOS bars oldest to
newest, the same order the live sweep sees). Its cursors are separate
members from the live ones so a training pass can never disturb the live
chart's declustering.
Deliberately NOT switched into selectionScore yet. Declustering cuts
coverage from ~64% of bars to ~8%, well under
MIN_COVERAGE_FRACTION_OF_BASE_RATE, which would make every checkpoint
undeployable overnight - the minRR collision and the recall-floor catch-22
twice over. The floor gets re-derived from these measurements first.
Compiles clean: 0 errors, 0 warnings.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-09 10:22:31 -04:00
|
|
|
m_oosNmsFired(0),
|
|
|
|
|
m_oosNmsHits(0),
|
|
|
|
|
m_oosNmsLastBuyIdx(-1),
|
|
|
|
|
m_oosNmsLastSellIdx(-1),
|
|
|
|
|
m_oosNmsKeptIdx(-1),
|
|
|
|
|
m_oosNmsKeptConf(0.0),
|
|
|
|
|
m_oosNmsKeptDir(Neutral),
|
2026-08-01 11:27:28 -04:00
|
|
|
m_lastBuyFiredPrecPct(-1),
|
|
|
|
|
m_lastSellFiredPrecPct(-1),
|
|
|
|
|
m_lastBuyFired(0),
|
|
|
|
|
m_lastSellFired(0),
|
|
|
|
|
m_maxClassSampleWeight(1.5),
|
|
|
|
|
m_swingConfirmationBars(100),
|
|
|
|
|
m_barrierHorizonBars(BARRIER_HORIZON_FALLBACK),
|
|
|
|
|
m_barrierHorizonResolved(false),
|
|
|
|
|
m_barrierFallbackWarned(false),
|
|
|
|
|
m_lastBarrierTimedOut(false),
|
feat(labels): the scheduled close-all is now a vertical barrier in the label walk
User report: "I exit everything on Friday close to avoid weekend swap...
if the NN training thinks I hold over the weekend it could produce
inaccurate results" - it thought exactly that. TripleBarrierLabel walked
its full horizon (64 bars, mean lifespan ~18 H4 bars ~ 3 days) straight
through the scheduled flat, scoring trades the deployed EA is guaranteed
to have closed on Friday 23:45. SQX applies this rule when building
strategies; the EA's own labels did not.
NextScheduledCloseAll() mirrors CExpertCustom::OnTick's live check
exactly (same three inputs, same -1 disabled sentinels, same
CLOSE_EVERYDAY semantics, same server clock). The walk stops at the
first bar that does not END by the cutoff - OHLC cannot order the
tradable fraction of a partial bar, and ties go to the refusal, as
everywhere in this file. An unresolved trade at the cutoff times out to
Neutral, exactly as live would flatten it. Excursions, the first-passage
ladder and the label lifespan truncate with the walk, so the DERIVED
geometry is automatically sized to the tradable window - a target the
flat rule never lets price reach stops counting as reachable.
The prebuild census now splits timeouts: "horizon too short?" vs "ended
by the scheduled close-all" - different questions, different fixes.
Schedule disabled = no cutoff, exactly like live.
Models trained under weekend-blind labels are fitted to a different
target; charts with the close-all enabled (the default) should be reset
to retrain under the honest labels.
NOT COMPILED - user compiles in MetaEditor.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-19 11:17:14 -04:00
|
|
|
m_lastLabelWeekendCut(false),
|
feat(ensemble): per-NN inputs replace the preset selector - the meta head becomes the vote's gate
User design (2026-08-19): 'remove the enum menu that selects neural networks... individual
inputs for every NN just like classic signals... the META NN should be integrated into the
voting decision pipeline when enabled... as a bonus meta labelling is applied to enabled NNs.'
- AI_CHOICE is GONE (tombstoned per the stale-.set doctrine). Use_MLP/Use_CONV/Use_LSTM/
Use_CONVLSTM are ordinary bools like the classic votes; the ensemble arithmetic adapts to
any subset because the consensus divisor is the enabled capable weight. Two or more
enabled = ensemble (|ENS1 token + joint gate, exactly the old AI_HYBRID fingerprints, so
existing weight files keep loading); one = the old solo preset; none = classic-only.
- Use_MetaLabeling un-couples META from the direction NNs (the old selector made them
mutually exclusive). S3 ships: CSignalMETA::LiveMetaGate scores each vote-cleared entry
(shared window at bar 1 + proposal descriptor: side, net vote, live geometry, spread/ATR;
pattern one-hot ZEROED - ranking, not calibrated probability, documented in the body) and
vetoes below the cost-adjusted break-even. Entries only; fail-open everywhere, loudly.
- COEXISTENCE HAZARDS closed: VoteCapableWeight()=0 and ProspectiveVote()=false for the
meta target - solo-only until today, a trained META would otherwise sit in the consensus
divisor as a permanent abstainer and shrink every vote by its module weight.
- CERTIFIED == TRADED: the ensemble era verdict replays the identical veto through the same
g_warriorMetaGate pointer over its OOS fired bars (bar re-resolved from the row's own
time; fail-open counted as fires and reported: 'metaGate: N approved, M vetoed, K
unscored'). The overlay deliberately does NOT replay it (veto-filter-in-replay class,
calendar-cliff precedent) - documented at the sweep site. Solo charts' own gate does not
model the veto - the standing solo-gate caveat, documented at the input.
- DB continuity: the pattern/journal DB fingerprint's first slot was (int)AIType;
DbLegacyAiSlot() maps every legacy-expressible config to its OLD value (new 2-3 member
subsets get 100+bitmask, outside the legacy range) so no existing database re-keys.
filterID becomes the enabled roster via one EnabledNNSummary().
- HUD: the meta line shows the gate (armed/(trn), last P vs BE, ok/veto tally); the
armed/disarmed announcement fires on state change via one latch (MetaGateArmedNow), not
only when an entry happens to be proposed.
NOT COMPILED - user compiles in MetaEditor.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-19 13:01:02 -04:00
|
|
|
m_metaGateArmed(false),
|
|
|
|
|
m_metaGateLastP(-1.0),
|
|
|
|
|
m_metaGateLastBe(-1.0),
|
|
|
|
|
m_metaGateApproved(0),
|
|
|
|
|
m_metaGateVetoed(0),
|
fix: a restart no longer loses the measured geometry or the training window
Terminal restart, 22:25: all four resumed models sat on empty windows with
enum 2:6 barriers. Three interlocking causes, all visible in one log excerpt:
1) THE PRE-SCAN WINDOW WAS SIZED BY THE SAVED WATERMARK. A resumed model's
dtStudied sits at its last studied bar, so Bars(dtStudied, now) ~ 0 and the
resumed-model MI pre-scan built a zero-bar "complete" label cache - logged as
"Buy: 0 | Sell: 0 | Neutral: 0". Train()'s own era start RESETS dtStudied to
the training-window rule before computing its window; the pre-scan did not.
The rule is now factored into TrainWindowStart() and both use it. The scan
also refuses to arm before SERIES_SYNCHRONIZED (it ran in the same second as
OnInit), and deployed models keep their watermark - for them it gates
inference recency, not a training window.
2) THE HORIZON LATCHED ON AN INDICATOR WARM-UP. ComputeBarrierHorizonBars ran
against a ZigZag with 0 calculated legs, fell back, and EnsureBarrierHorizon
latched fallback(32) x slMult x tpMult = 384 for the process lifetime. A
leg-starved horizon is now PROVISIONAL: re-resolved on the next rebuild, the
label cache wiped if it moved (labels from two horizons answer different
questions), and the geometry deriver refuses to run from it - a pair derived
over a warm-up window would get PINNED.
3) THE DERIVED GEOMETRY WAS NEVER PERSISTED. The .cfg is written at model
creation and at weights-reset - both BEFORE era 0 derives - so the measured
pair lived only in memory: every restart read back zeros, adopted nothing,
fell back to the enum barriers, and the era-0-only gate meant a resumed model
could NEVER re-derive. A full day of training on 3.33/1.62 resumed as 2:6.
Now: the settled pair is pinned to the .cfg the moment derivation completes
(one-shot, atomic write), and the derive gate accepts any model with no
pinned pair, not just era 0 - mid-run stability is carried by
m_geometryDerived itself, which never allows a second derivation.
Both build variants compile 0 errors, 0 warnings.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-09 22:40:43 -04:00
|
|
|
m_barrierHorizonLegStarved(false),
|
|
|
|
|
m_horizonStarvedWarned(false),
|
|
|
|
|
m_geometryCfgSaved(false),
|
fix(consistency): one geometry authority, one exit authority, and the MI screen finally gets a veto
Consistency pass before a fresh deployment. Three places where two systems were
choosing the same thing and one of them silently lost.
1. THE GEOMETRY SCAN'S DECISION WAS INERT - measured, not suspected.
USDJPY, 2026-08-17:
14:24:12.844 adopting barrier geometry 2:8 ... Relabelling and training on it.
14:24:12.979 triple-barrier labels - stop 1.61*ATR, target 3.21*ATR ... this is what trains
It adopted 2:8 and trained on 1.61:3.21. ReportBarrierGeometryScan wrote only
m_sl_mode/m_tp_mode, and BarrierMultiples ranks the DERIVED pair ABOVE those ints -
so on any model carrying a derived pair (every model with a .cfg, including a fresh
one whose weights are gone but whose sidecar survived) the adoption changed nothing.
Worse, had it changed something it would have been undone immediately: the adoption
sets m_labelCachePrebuilt = false, and that prebuild re-runs DeriveBarrierGeometry at
era 0, which overwrites m_derivedSl/TpMult from the excursion quantiles.
ONE AUTHORITY: the derived pair, because it is what the labels read, what the deploy
gate certifies, what g_Derived*AtrMult places on the live order, and what the .cfg
pins across restarts. The scan now writes THAT (floored by MIN_SL_ATR_MULTIPLIER, the
same floor DeriveBarrierGeometry applies so the live stop can never be wider than the
labelled one), republishes to the bridge immediately rather than at the next era end,
and forces the sidecar to be rewritten. m_geometryAdopted latches it so the derive
pass the adoption itself triggers cannot overwrite it.
The scan outranks the derive for an evidential reason, not an architectural one: its
winner cleared a permutation test against the null of the MAXIMUM over every eligible
pairing, and it scores the incumbent derived pair as a peer in that same field. The
derive is a descriptive quantile read with no significance test attached.
BEHAVIOURAL CHANGE, and the reason to flag it before a fresh test: barrier geometry
will now actually move when the scan says so. Until today it never did.
2. ONE EXIT AUTHORITY, tied to whose certificate the trade was placed under.
CheckClosePosition had two routes. The AI early-exit reads the AI vote undiluted and
is exactly what the new exit replay reproduces. The blended route thresholds
m_direction, the average over EVERY filter including classic ones whose live votes
pass 3 never computes - so it can close a position the certificate never modelled,
and no replay can ever check it.
When g_DerivedSlAtrMult > 0 the AI's measured geometry is on the order, which means
the deploy gate's certificate is the reason the trade exists. In that state the AI now
governs the exit and the blended route is suppressed. Classic-only configurations are
untouched: there the blended route is the only exit opinion and stays exactly as it
was. Nothing moves at the shipped defaults either way (Min_Vote_Close = Disabled).
3. EDGEFINDER, SECOND HALF: THE MEASUREMENT NOW STEERS.
The MI suite has always printed its verdicts and then trained the direction target
regardless of what they said. That gap IS the difference between this and the
EdgeFinder discipline: measure what the market offers, THEN aim.
m_dirEvidence is set when EITHER the feature/label mutual information OR the
normalised excursion asymmetry clears its block-permuted null - an OR, because the two
look for the same thing by different routes and requiring both would reject on the
weaker of two independent measurements. Normalised asymmetry specifically, never the
raw one, which is the volatility confound.
Deploy - solo AND ensemble - now requires it. A run without it still trains, and keeps
its checkpoint: the research value is real and the measurement can be wrong. It simply
may not go live. Reported separately from the statistical gate because the remedy is
different: a failed selection test says train differently, this says look somewhere
else. Excursion SIZE keeps clearing where direction does not, and that is a
risk-control head rather than an entry signal.
For the ensemble the check is per-chart by construction - the MI suite runs once and
shares its outcome across members - which is the honest treatment: four models finding
nothing between them is not four chances at an edge, it is four fits to the same
absent information.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-17 16:43:20 -04:00
|
|
|
m_geometryAdopted(false),
|
|
|
|
|
m_dirEvidence(false),
|
|
|
|
|
m_dirEvidenceWhy("not measured yet"),
|
fix: both-won bars were labelled "do not trade" - resolve by first touch
Removing the min-reward:risk raise let the MEASURED geometry come back with
the target NEARER than the stop (SP500 H1: target 1.62*ATR at q50 of
favourable, stop 3.33*ATR at q75 of adverse). That reopened a branch the
code called unreachable: price can reach +target and -target inside one
horizon, winning in BOTH directions, and those bars fell through to Neutral.
Neutral has only three producers, both-lost is unreachable (you cannot touch
-3.33 without crossing -1.62 first, which wins the short), and timeouts logged
at 1.0% of Neutral - so ~27% of ALL bars were being handed to the model as the
abstain class when a trade either way would have collected its target. The
cleanest positives in the sample, labelled "do not trade", while the fitted
confidence threshold was being asked to find selectivity in what was left.
Resolved by FIRST TOUCH: the target reached earlier is the trade that would
have closed first. Same forward window, no extra lookahead. Same-bar ties stay
Neutral - OHLC cannot order two touches, and unlike an intrabar stop tie there
is no pessimistic side to fall to, so a guess would inject a coin-flip
direction into the target.
Also:
- count both-won and its same-bar tie subset in the prebuild line, so the
share is measured rather than inferred from arithmetic on a log line
- scope the timeout counter to IS, matching the tally it is reported as a
percentage OF; it was incremented over the whole scan and divided by an
in-sample denominator
- clear m_lastBarrierTimedOut at the top of the walk with the excursions, not
at the bottom - the two early returns published the previous bar's verdict
- mark the pass-1 label line PROVISIONAL. It prints the enum fallback because
geometry can only be derived from excursions that do not exist yet, and it
reads exactly like a config change that failed to take effect
FORCES RETRAIN. Both build variants compile 0 errors, 0 warnings.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-09 15:35:42 -04:00
|
|
|
m_lastBarrierBothWon(false),
|
|
|
|
|
m_lastBarrierBothWonTied(false),
|
2026-08-01 11:27:28 -04:00
|
|
|
m_labelPrebuildTimeoutCount(0),
|
feat(labels): the scheduled close-all is now a vertical barrier in the label walk
User report: "I exit everything on Friday close to avoid weekend swap...
if the NN training thinks I hold over the weekend it could produce
inaccurate results" - it thought exactly that. TripleBarrierLabel walked
its full horizon (64 bars, mean lifespan ~18 H4 bars ~ 3 days) straight
through the scheduled flat, scoring trades the deployed EA is guaranteed
to have closed on Friday 23:45. SQX applies this rule when building
strategies; the EA's own labels did not.
NextScheduledCloseAll() mirrors CExpertCustom::OnTick's live check
exactly (same three inputs, same -1 disabled sentinels, same
CLOSE_EVERYDAY semantics, same server clock). The walk stops at the
first bar that does not END by the cutoff - OHLC cannot order the
tradable fraction of a partial bar, and ties go to the refusal, as
everywhere in this file. An unresolved trade at the cutoff times out to
Neutral, exactly as live would flatten it. Excursions, the first-passage
ladder and the label lifespan truncate with the walk, so the DERIVED
geometry is automatically sized to the tradable window - a target the
flat rule never lets price reach stops counting as reachable.
The prebuild census now splits timeouts: "horizon too short?" vs "ended
by the scheduled close-all" - different questions, different fixes.
Schedule disabled = no cutoff, exactly like live.
Models trained under weekend-blind labels are fitted to a different
target; charts with the close-all enabled (the default) should be reset
to retrain under the honest labels.
NOT COMPILED - user compiles in MetaEditor.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-19 11:17:14 -04:00
|
|
|
m_labelPrebuildWeekendCutCount(0),
|
fix: both-won bars were labelled "do not trade" - resolve by first touch
Removing the min-reward:risk raise let the MEASURED geometry come back with
the target NEARER than the stop (SP500 H1: target 1.62*ATR at q50 of
favourable, stop 3.33*ATR at q75 of adverse). That reopened a branch the
code called unreachable: price can reach +target and -target inside one
horizon, winning in BOTH directions, and those bars fell through to Neutral.
Neutral has only three producers, both-lost is unreachable (you cannot touch
-3.33 without crossing -1.62 first, which wins the short), and timeouts logged
at 1.0% of Neutral - so ~27% of ALL bars were being handed to the model as the
abstain class when a trade either way would have collected its target. The
cleanest positives in the sample, labelled "do not trade", while the fitted
confidence threshold was being asked to find selectivity in what was left.
Resolved by FIRST TOUCH: the target reached earlier is the trade that would
have closed first. Same forward window, no extra lookahead. Same-bar ties stay
Neutral - OHLC cannot order two touches, and unlike an intrabar stop tie there
is no pessimistic side to fall to, so a guess would inject a coin-flip
direction into the target.
Also:
- count both-won and its same-bar tie subset in the prebuild line, so the
share is measured rather than inferred from arithmetic on a log line
- scope the timeout counter to IS, matching the tally it is reported as a
percentage OF; it was incremented over the whole scan and divided by an
in-sample denominator
- clear m_lastBarrierTimedOut at the top of the walk with the excursions, not
at the bottom - the two early returns published the previous bar's verdict
- mark the pass-1 label line PROVISIONAL. It prints the enum fallback because
geometry can only be derived from excursions that do not exist yet, and it
reads exactly like a config change that failed to take effect
FORCES RETRAIN. Both build variants compile 0 errors, 0 warnings.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-09 15:35:42 -04:00
|
|
|
m_labelPrebuildBothWonCount(0),
|
|
|
|
|
m_labelPrebuildBothWonTieCount(0),
|
2026-08-01 11:27:28 -04:00
|
|
|
m_maxErasPerRun(300),
|
|
|
|
|
m_arrowRestoreIndex(0),
|
|
|
|
|
m_arrowRestorePending(false),
|
|
|
|
|
m_arrowRestoreStartMs(0),
|
|
|
|
|
m_rescanIndex(0),
|
|
|
|
|
m_rescanHi(0),
|
|
|
|
|
m_rescanBarsNow(0),
|
|
|
|
|
m_rescanPending(false),
|
|
|
|
|
m_rescanStartMs(0),
|
|
|
|
|
m_rescanRawBuy(0),
|
|
|
|
|
m_rescanRawSell(0),
|
|
|
|
|
m_rescanRawNeutral(0),
|
|
|
|
|
m_trainRunActive(false),
|
|
|
|
|
m_eraResumePending(false),
|
|
|
|
|
m_resumeBars(0),
|
|
|
|
|
m_resumeTotalIter(0),
|
|
|
|
|
m_resumeOosCutoff(0),
|
|
|
|
|
m_resumeBarIndex(0),
|
|
|
|
|
m_resumeAddLoop(false),
|
|
|
|
|
m_isTrainQueueCount(0),
|
|
|
|
|
m_isTrainCursor(0),
|
|
|
|
|
m_isPass2Active(false),
|
|
|
|
|
m_isPass2Done(false),
|
|
|
|
|
m_isPass3Active(false),
|
|
|
|
|
m_oosScoreIndex(0),
|
|
|
|
|
m_oosScoreStartIndex(0),
|
fix: the operating point was fitted on bars the net had memorized
FitDirConfThreshold harvested its margin histogram from pass 2's own
backprop samples. Pairing every fit against the same era's OOS result
shows what that measured:
PAI era 1 IS 25% cov @ 66.1% (-0.8pp) -> OOS 64% (-3pp) gap +2.1pp
PAI era 76 IS 90% cov @ 79.6% (+12.7pp) -> OOS 65% (-2pp) gap +14.6pp
LSTM era 9 IS 77% cov @ 81.6% (+14.6pp) -> OOS 63% (-4pp) gap +18.6pp
The gap grows monotonically while OOS stays flat, so within a handful of
eras the curve stops describing behaviour on unseen bars. That is fatal
here specifically, because the objective branches on the SIGN of
(p - break-even): the memorized curve reads +12pp at 95% coverage, so
coverage x (p - p0) correctly maximises coverage and returns ~0.02 - fire
on every bar. The "p < p0 -> get more selective" branch, which is the
actual regime and the entire point of 983a6a3, could never fire because IS
never showed p < p0.
Carve a calibration slice out of the IS span - DIR_CONF_CALIB_PCT_OF_IS,
purged from backprop by one label horizon on BOTH sides (the far-side
purge is not optional: without it the newest training bars carry labels
partly decided by price action inside the slice, putting the memorization
straight back into the curve). Score it in a new chunked pass 2.5, after
pass 2 has trained and before pass 3 grades - the only position where the
histogram is simultaneously not-trained-on, not-graded, and current with
the weights it will be applied to.
Costs 15% of the training data. Worth it beyond honesty: the deploy gate
needs dirPrecPct > chance + EDGE_MIN_SIGMAS*SE, and a threshold pinned
near zero dilutes any edge concentrated in the confident bars across every
bar the model calls, driving dirPrecPct toward chance by construction. A
threshold that can be selective is the only mechanism by which a small,
concentrated edge could ever clear that gate.
Also: a sparse histogram now KEEPS the previous threshold instead of
resetting to 0.0. A failed measurement must not decay to the most exposed
setting in the range.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-10 15:58:18 -04:00
|
|
|
m_isCalibActive(false),
|
|
|
|
|
m_isCalibDone(false),
|
|
|
|
|
m_calibIndex(0),
|
|
|
|
|
m_calibStartIndex(0),
|
feat: excursion-size head (Stage 1, measurement only)
Direction is closed - normalised asymmetry fails on three instruments
with a working positive control, and the classifier's own best-of-999
era-cap test agrees (+0.9pp = 1.48 sigma, family-wise p=1.0000). SIZE is
a different question and RANGE clears at ~4x its null.
Checked the denomination before building on that, since the source memo
warns to: m_excUpCache holds (maxHigh - fill)/ATR, so "RANGE is
predictable" is a claim about travel RELATIVE to current ATR, not a
restatement of "ATR is autocorrelated". It is exactly the part a fixed
multiple (stop 3.31*ATR, target 1.64*ATR) discards.
A second small CNet, 760 -> 24 -> 32 sigmoid outputs = P(price reaches
ladder rung k) upward and downward. Survival parameterisation rather than
regressing the multiple, because it needs nothing new from CNet: sigmoid
outputs and the per-neuron delta the `total != 3` branch already applies
(a quantile head would need a linear activation and a pinball gradient in
Network.mqh, Network.cl and the DirectML path, on a class four topologies
share). Targets are free - m_ladderUpAt already records first-touch age
per rung with 0 meaning never reached.
Separate net, not extra outputs on the classifier: more outputs would
change m_outputNeuronsCount, the .nnw shape and the fingerprint, and push
the count off 3 - the exact condition backProp uses to select the joint
softmax gradient the 3-class head depends on. The classifier is
bit-for-bit unaffected and this is removable without trace.
STAGE 1 PLACES NO ORDERS. It reports a Brier skill score against the
constant per-rung base rate - the baseline a fixed ATR multiple already
assumes - with both predictors fitted IS and evaluated OOS, so neither
gets a look at the test set. Positive skill justifies Stage 2 (drive
SL/TP and sizing off ExcursionQuantile, which is defined and deliberately
uncalled). Zero or negative means ATR already carries everything and
Stage 2 must not be built.
Trains only on primary occurrences: the replay queue oversamples for
CLASS balance, and a direction-balanced sample is a biased SIZE sample.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-11 07:40:01 -04:00
|
|
|
m_excNet(NULL),
|
|
|
|
|
m_excTgt(NULL),
|
|
|
|
|
m_excOut(NULL),
|
|
|
|
|
m_excHeadFailed(false),
|
|
|
|
|
m_excBaseTotal(0),
|
|
|
|
|
m_excScored(0),
|
fix: the excursion gate would have passed Stage 2 on an artifact I made
Second-opinion review killed the +4.2% far-rung result, correctly, and
the mechanism is my own bug. A head trained toward {0.05,0.9} converges
to 0.05+0.85p, so its bias is 0.05-0.15p: negative where p is near 1,
POSITIVE where p < 1/3, growing monotonically as the rung gets farther.
Against a baseline frozen at the IS rate, an upward-biased head scores
positive Brier skill whenever the OOS rate merely sits above the IS rate.
Predicted signature: huge negatives near, ~zero at p=1/3, growing
positives far. Observed: -82% ... -0.6% ... +1.2/+2.7/+4.2. The far rungs
were not the clean end of a distorted measurement, they were the other
face of the same artifact. Everything before 25aca83 is void.
The gate was a bare `skill >= 2%` point estimate over 8 rungs x 4
topologies x N eras, reported per era - a best-of-~300 with no interval
and no multiplicity control, which is the shape of the four traps already
documented here. It now needs FOUR things at once:
DECISION RUNGS only the rungs ExcursionQuantile actually reads at the
live geometry (target 1.62, stop 3.31 ATR), fixed
before looking. Skill at 5 ATR is skill about a
distance no order is placed at - and the TARGET side
currently interpolates 1.5/2.0, which measured -2.2%
and -1.3%.
DISJOINT SAMPLE one bar per horizon. Adjacent bars share 63 of 64
horizon bars, so ~16k scored bars is ~250 independent
ones and every SE over the full set is ~8x understated.
VS ORACLE the best constant achievable ON THE SCORED BLOCK,
closed form from H and n (Brier = H*(1-H/n)). A head
that learned only a LEVEL nearer the OOS rate than the
frozen IS constant scores positive against the old
baseline and <= 0 here. This is the control that
separates per-bar skill from base-rate drift.
MONOTONE CURVE P(reach k) must be non-increasing in k. Nothing
constrained 8 independent sigmoids to obey that, and
ExcursionQuantile returns the FIRST crossing - so a
tangled curve is misread exactly where the head is
least sure. Counted and reported, not silently used.
The pass message now also states what a pass would and would not buy:
expectancy is -costs at zero directional edge whatever the stop distance,
and under prop DD limits LOWER variance also lowers P(reach target before
limit), so "better drawdown" is a choice of failure mode, not a win.
Still owed before any Stage 2: a race against a trailing-quantile
incumbent and a vol-feature logistic. Beating a frozen global constant is
the weakest admissible bar for replacing a global constant.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-11 15:49:32 -04:00
|
|
|
m_excScoredD(0),
|
|
|
|
|
m_excMonoViol(0),
|
perf: the excursion head cost 3.6x era time - cut its dispatches ~250x
Measured on exc-race-v3: LSTM era 300s -> 1087s (net 272->748s, "other"
30->337s). My estimate had been "single-digit percent". The cost is
per-DISPATCH, not per-FLOP, and therefore hits EVERY backend: the head is
19k weights and ~2.4 GFLOP an era - seconds of arithmetic - but ~48k
forward/backward calls x several layer submits each, and its 760-wide
layer exceeds the CPU DLL's inline threshold so each one pays a real
handoff. The classifier's own net time tripled too, from contention with
a second pool on an already-full box.
Three changes, all backend-neutral because they remove submits rather
than tune threads:
SCORE ONLY DISJOINT WINDOWS (~64x). Adjacent bars share all but one bar
of their horizon, so 16k consecutive bars were always ~250 independent
observations - the full-sample tally was never worth more than the
disjoint one, it just quoted an n that was ~64x too large. Dropping it
costs nothing statistically and removes 63 of every 64 forward passes.
The two parallel tallies collapse into one, which is also less code.
The trailing ring still advances on every bar: it needs the outcome
SEQUENCE, and that is array lookups, not a forward pass.
TRAIN ON EVERY 4th PRIMARY BAR (4x). The target is low-dimensional and
strongly autocorrelated - neighbouring bars carry near-identical
excursion information - so per-bar training buys resolution the target
does not have. Strided on ATTEMPTS, not acceptances, so a stretch of
unlabelled bars cannot silently change the spacing.
OWN TIMING COLUMN. The head's passes were landing in the era line's
"other" bucket, which is how a 3.6x regression read as an unexplained
jump in the one column nobody attributes. A cost that cannot be seen in
the timing line cannot be traded off against anything.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-11 16:38:29 -04:00
|
|
|
m_excTrainTick(0),
|
|
|
|
|
m_excUs(0),
|
feat: race the excursion head against a trailing-quantile incumbent
Beating a frozen global constant is the weakest admissible bar for
replacing a global constant. The honest incumbent is a rolling rung
frequency: it adapts to the volatility regime - exactly what the head
claims to predict - and needs no model, no 760 inputs and no training.
Implemented as a ring of per-bar outcome bitmasks (32 rungs fit one
ulong), sized horizon + EXCURSION_TRAIL_WINDOW. The newest `horizon`
entries are held back UNRESOLVED: a bar's rung outcomes are only known
one horizon later, so using them would be lookahead and would flatter the
incumbent into an opponent the head could never fairly beat. Pass 3 walks
oldest-to-newest, so "pushed more than horizon bars ago" is exactly
"resolved by now". Each push is O(rungs), not O(window).
The head's decision-rung Brier is pro-rated to the trailing estimate's
coverage before the ratio, since the incumbent only scores bars where its
window is warm.
This line is worth reading on its own, independently of the head: if the
trailing quantile beats the global constant, that is a cheap risk-control
win available with no machine learning at all - and it is the same number
either way, so the run answers both questions in one pass.
The ring is deliberately NOT reset per era - it estimates the market, not
the era, and re-warming 500 bars every era would leave the incumbent
unusable over the first chunk of every scoring pass, handing the head a
free win on exactly those bars.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-11 15:57:11 -04:00
|
|
|
m_excTrailHead(0),
|
|
|
|
|
m_excTrailCount(0),
|
|
|
|
|
m_excTrailN(0),
|
|
|
|
|
m_excTrailScored(0),
|
2026-08-01 11:27:28 -04:00
|
|
|
m_lastStatusLabelUpdateTick(0),
|
|
|
|
|
m_lastBuyRecallPct(-1),
|
|
|
|
|
m_lastSellRecallPct(-1),
|
|
|
|
|
m_lastDisplayNeuron0(0),
|
|
|
|
|
m_lastDisplayNeuron1(0),
|
|
|
|
|
m_lastDisplayNeuron2(0),
|
|
|
|
|
m_lastDisplaySignal(0),
|
|
|
|
|
m_lastBarTime(0),
|
|
|
|
|
m_modelEta(InitialEtaForOptimizer()),
|
|
|
|
|
m_etaCeiling(InitialEtaForOptimizer()),
|
|
|
|
|
m_erasSinceCooldown(0),
|
|
|
|
|
m_bestOosForecast(-1),
|
|
|
|
|
m_bestBalancedOos(-1),
|
|
|
|
|
m_bestPassedRecall(false),
|
fix: a one-sided era can no longer become the best checkpoint
Measured on HYBRID, era 29 of the first win-scored run: the model collapsed
to always-Buy and was crowned "new best selection score 67.1%". Under
win-based scoring that is not a coincidence - the always-call-the-drift-side
model IS the chance reference, so it scores exactly chance (P(winLong) ~ 67%
on SP500), while every honest two-sided era scores 63-66% because shorts win
less often against the drift. Raw score ranking therefore actively prefers
the degenerate model, every regression restores back to it, and live NMS
collapses its near-constant signal to ~25 trades per era - observed as
"hybrid barely trades".
bothSidesLive already blocked one-sided eras from DEPLOYING (tradeableOK,
371f8aa), but among not-yet-deployable eras the score alone ranked - the same
early phase the coverage credit was added for, failing the same way through a
different door.
The ranking key is now three lexicographic tiers: deployable > two-sided >
score. A one-sided era cannot displace a two-sided best regardless of score -
by construction its score is a property of the data's drift, not the model -
and a two-sided era displaces a one-sided best no matter how much lower it
scores. m_bestBothSidesLive is snapshotted with the checkpoint and reset with
the rest of the best-tracking state.
Both build variants compile 0 errors, 0 warnings.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-09 21:19:44 -04:00
|
|
|
m_bestBothSidesLive(false),
|
diag: slow eras must explain themselves - heartbeat + era time split + pass-1 paint
The 23:42 restart left all four charts grinding ~25x slower than the 18:01
baseline (era lines in 86 seconds there; 20+ minutes of nothing here), and
NOTHING could say why from outside: pass 1 logs nothing, its status paint sat
inside the !wouldQueue branch so the IS sweep - 80% of the pass, processed
FIRST - painted nothing either, the VPS has no debugger for a thread stack,
and the hourly new-bar cache invalidation cancels and restarts an unfinished
era, so a slow era can stay invisible FOREVER. Externals gave: four chart
threads at ~95% pure user-mode compute, DLL pool idle, no file writes. That
narrows it to "MQL5-side per-item work in the era passes" and no further.
So training now explains itself:
- TrainHeartbeat: one line per 4096 processed items, only after an era has
already run 60s, at most 6 lines per era - a healthy era stays exactly as
quiet as before. Reports position and the cumulative split: feature-window
builds vs net forward/backprop vs everything else. Hooked into all three
passes.
- The era summary line gains "| ERA TOOK Ns (feature windows X, net fwd/back
Y, other Z)" whenever an era exceeded 120s.
- Pass 1 paints its progress for QUEUED bars too, not just the OOS slice, so
the panel shows "learning (era N)" instead of sitting on the idle writer's
"Getting ready..." for the entire IS sweep. The label is throttled
internally; painting per bar costs nothing.
Both build variants compile 0 errors, 0 warnings.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-10 00:10:45 -04:00
|
|
|
m_eraStartTick(0),
|
|
|
|
|
m_passFeatUs(0),
|
|
|
|
|
m_passNetUs(0),
|
|
|
|
|
m_passHeartbeatPrints(0),
|
diag: an era that discards itself now says so instead of scanning forever
add_loop is exactly "at least one bar produced a usable feature
window". When it stays false, pass 2, pass 3, the era counter, the
checkpoint and every log line in the era-end block are ALL skipped:
Train() returns having done nothing, m_eraResumePending is still false,
and the next call restarts the SAME era from bar 0. That is an
infinite 0->100% "scan" loop that prints absolutely nothing - the only
remaining silent restart path in Train(), and it matches the reported
symptom exactly.
Pass 1 now counts usable vs unusable windows and reports at the pass
boundary, which demonstrably executes:
- total failure routes through ReportTrainStall (already capped at
one line a minute, and carries the run-state flags) naming the
counts, the required window width and the bar count
- success prints how long the scan took and how many samples it
handed to pass 2, but only once the era has passed 10s - a fast
era stays as quiet as before, a slow one distinguishes "advancing"
from "sweeping the same bars forever"
A PARTIAL failure is normal and deliberately does not shout: pass 1
walks oldest-to-newest and the deepest bars predate the indicators'
warm-up, so those windows fail and are cached as misses.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-10 11:29:33 -04:00
|
|
|
m_passWindowOk(0),
|
|
|
|
|
m_passWindowFail(0),
|
fix: the recall gate was unsatisfiable and the LR decay was a spiral
Both made the run structurally unable to succeed, independently of any
signal in the data. Found by reading the 13:01 log.
RECALL GATE. m_objectiveMet required Buy, Sell AND Neutral OOS recall
each >= 40%. First-touch resolution (ce52654) collapsed Neutral from
the ~94% majority it was under exact-pivot labels to a same-bar-tie
residue - 250 of 38,261 bars, 0.65% - so the floor was asking the model
to identify 40% of coin-flip ties before it could converge. Measured:
CONV, LSTM and HYBRID all logged "Neutral:0% (need >=40% each)" on
every era. No model could ever satisfy it; every run was destined for
the plateau ladder or the era cap.
Only the DIRECTIONAL floors are load-bearing for the anti-collapse job
the gate exists to do: an all-Neutral model shows Buy and Sell recall
at 0% and is blocked by them. Neutral's own floor guarded the mirror
bias (over-calling Buy/Sell at Neutral's expense), which was real at
94% prevalence and is not at 0.65% - there, almost never calling
Neutral is correct rather than biased.
Prevalence-guarded rather than hardcoded off, so it returns by itself
if a future label rule makes Neutral substantial again. Deliberately
NOT extended to Buy/Sell: exempting a thin directional class reopens
the era-44-46 hole, which directionalRecallMeasured only half-covers -
it checks those classes were MEASURED, not that they passed.
ETA DECAY. A regressing era restored the checkpoint, reset the
optimizer and cut eta - all on the FIRST regression. The next era then
started from an identical state with a smaller step, regressed again,
and got the same treatment. The loop is self-sustaining and cannot
discover anything, because rolling the weights back is exactly what
removes the exploration that would end it.
Measured on PAI: eras 2-11 every one a regression against era 1, eta
0.000594 -> 0.000024, dW/W 0.000%/0.000% from era 2 onward. Ten eras,
~45s each, reproducing era 1 exactly and unable to do anything else.
Now requires ETA_DECAY_PATIENCE_ERAS consecutive regressions - the
standard ReduceLROnPlateau formulation. A single bad era is noise, and
an improving era clears the counter so alternating runs never
accumulate into a decay.
Build tag -> gate-patience-v3. It had not moved in six commits, which
is why the running binary could not be identified from its own log.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-10 13:28:58 -04:00
|
|
|
m_consecutiveRegressions(0),
|
fix: a resumed model cached a cold ATR as permanent, so it never trained
BufferTempData cached EVERY failure - m_featureCacheHasValue[idx]=true
with m_featureCacheValid[idx]=false - and the cache never re-tries a
miss. So a single feature read taken before the terminal had finished
calculating the indicator buffers marked those bars unusable for the
rest of the process, even though the data arrived milliseconds later.
MT5 fills an indicator's buffers asynchronously after the handle is
created, and a cold ATR returns 0 for EVERY index, not just its warm-up
tail. BufferTempDataCompute rejects a bar with no ATR (correctly - the
price features would be meaningless), so the whole window failed, and
the whole cache was poisoned.
Only resumed models were hit, because only they read features that
early. Topology.mqh sets m_warmupPassesRemaining = netLoaded ? 0 : 3:
a fresh start sits through three separately-scheduled Train() calls
before anything touches a feature, which is exactly what those passes
are for. A resumed one skips them and TuneIndicatorsAndTrain drives
StartLabelCachePrebuild and the MI report from the first chart event.
Its rationale - "a restart already has a proven-synced history" - holds
for HISTORY and not for INDICATORS, which are recreated every process
start.
Downstream: BuildFeatureWindow failed on every bar of every era, so
add_loop never went true, so pass 2, pass 3, the era counter and the
checkpoint were all skipped and pass 1 swept 0->100% forever. The
"0 samples" MI report line at startup was the same failure, four
seconds earlier, already visible in the log.
- a miss is now cached only when it is PERMANENT; the two "not ready
yet" guards mark m_featureFailTransient and are recomputed on the
next visit. Steady-state cost is ~ind_Periods bars per era, not 54k.
- an era that discards itself now drops the feature cache before
restarting, so any remaining cause of this state self-heals instead
of looping.
Deleting the .nnw "fixed" this only by turning the model back into a
fresh one.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-10 11:41:31 -04:00
|
|
|
m_featureFailTransient(false),
|
fix(ensemble+depth): the barrier had no liveness escape, and the depth gate could not report the one state the evidence pointed at
Two charts (USDJPY 50,179 bars / XAUUSD 33,982) sat at era 0 for 38 minutes with
four of their eight members completely silent. Nothing in this commit guesses at
why the sweep fails - the last five guesses were all wrong. It makes the failure
say what it is, and stops one broken member taking its whole chart down with it.
WHAT THE LOG ACTUALLY SAYS, before any of this.
- The running build IS d9f834d (pulled 14:18, compiled 14:19:01, 0 errors), so
every depth instrument from 1dda479/7e63a8b/45c9e21 was live.
- It printed NOTHING. Zero "PRIMING", zero "CAPPED", zero "Per-indicator depth"
in 27 MB of journal. The instrument built to find the depth shortfall returned
"not this".
- On USDJPY at 14:24, CONV-cad8 completed eras 0 AND 1 across all 50,179 bars -
same chart, same 832-value window, same indicators, byte-identical fingerprint -
while LSTM-cad8 and HYB-cad8 reported ok=0 failed=50163. So it is not the
symbol, the history, the bar count or the indicator depth. It is per-member.
- ok=0 means the NEWEST anchors failed too, and a short indicator cannot do that.
The depth reading in project_silent_block_failures is therefore retired by its own
instrumentation. THE ROOT CAUSE IS STILL UNKNOWN and this commit does not claim one.
1. THE DEPTH GATE'S SILENT PATH WAS THE STATE IT WAS HUNTING.
ServableBars() read `if(servable <= 0 || servable >= want) return want;` - one
branch over three unrelated states, silent in all of them:
enabled == 0 -> nothing tunable is on. No cap. Healthy.
enabled > 0, servable == -1 -> a handle answered INVALID.
enabled > 0, servable == 0 -> created, never calculated.
BarsCalculated() returns -1 for a dead handle, so a dead MA is indistinguishable
from "no tunable indicators enabled" - and both returned `want` without printing a
character. That is exactly the state a per-member, every-index, depth-independent
failure produces, and it is the single reason a build carrying full depth
instrumentation logged nothing through the whole outage.
TunableBarsCalculated() now also reports HOW MANY indicators it consulted, and the
dead-handle case is reported (latched, with per-handle depths). The RETURN is
deliberately unchanged - what to do about a dead handle is not yet known, and
changing control flow on an unproven cause is how the last four fixes here went
wrong. SettledBars() routes its three pass-through states via ServableBars() so the
report is reachable from the training sweep, which is the only caller that hits it.
2. THE STALL REPORT NAMED A SLOT, NEVER A BLOCK.
"lookback slot 0 REJECTED (window had 24 of 832 values)" plus a guess ("an
indicator warm-up or a history-edge read"). Which guard fired was INFERRED by
counting 4+5+4+4+6+1 = 24 and concluding feature 25 must be the MA. The arithmetic
was right; every conclusion drawn from it was wrong, because a value count names a
POSITION and a position cannot tell cold from capped from invalid from off-the-end.
Every guard that can reject a bar now records itself - m_featureFailBlock - and the
report carries it, the series index, IndicatorDepthReport()'s per-handle depths,
and for each indicator whether the NEWEST bar reads. That last field is the whole
diagnosis in one word: newest-also-EMPTY means the buffer is unreadable everywhere
(cold or dead handle), newest-reads means a genuine history edge. Instrumented:
open, ATR, MA, RSI, MACD, Ichimoku, and all five AD blocks via ADIndicatorCold().
3. THE TOTAL-FAILURE BACKOFF WAS GATED ON THE WRONG QUESTION.
It armed only when m_featureFailTransient was set. Keeping that flag correct across
every guard is a list that has to stay right forever - the same shape of fix the
feature cache abandoned for the same reason - and the gate is pointless anyway: a
sweep where ZERO of 50,163 bars produced a window will produce zero again if it
restarts a millisecond later, transient or not. Doing that at full speed is what
starved six indicator threads on a six-core box. The backoff is now unconditional
on a total failure. The flag keeps its real job, deciding whether a MISS may be
cached, which is a per-bar question and not a scheduling one.
4. THE ERA BARRIER DEADLOCKED, AND SILENCED THE MEMBERS IT FROZE.
EnsembleMinTrainingEra() exempted deployed, stopped and paused members and its
comment concluded "so nothing deadlocks". Those three are all VOLUNTARY. A member
that simply CANNOT finish an era is none of them, so it pinned the minimum at its
own era with no time limit - and the hold branch's only action was
`m_lastEraCompleteTick = GetTickCount()`, which silences the stall watchdog. So on
USDJPY the two members that could not train reported, and the two healthy members
frozen behind them wrote nothing anywhere. The outage was visible only through the
members that were not suffering it.
- BarrierEraHeartbeat() stamps a clock on real era CHANGE, kept separate from
m_lastEraCompleteTick precisely because the barrier resets that one. Only a
member AT the minimum can be a blocker; a member ahead is idle by design and is
never counted as stuck.
- After ENSEMBLE_BARRIER_STUCK_MS (12 min) a non-advancing member is dropped from
the barrier minimum. It keeps training and rejoins the instant it completes an
era - at which point, being behind, it legitimately becomes the minimum again,
which is the documented resumed-laggard behaviour.
- Both transitions say so loudly, and the release states plainly that the
combined-vote score cannot be computed while the ensemble is desynchronised.
- A held member now writes a rate-limited journal line naming WHICH members it is
waiting on, so the blocker is read off one line.
5. THE PANEL FLICKER.
OnTickHandler gates its terse writer on !m_trainRunActive, and a barrier-held member
returns from Train() before ever setting it - so both writers thought they were the
only one updating the label and fought every tick. That is the reported "Getting
ready..." <-> "Waiting at era N for slower ensemble members" oscillation, and it hit
Perceptron but not Convolutional purely because Convolutional had a run active from
a completed era and Perceptron, resumed from disk, never did. Train()'s message is
the specific one, so it wins.
NEXT STEP once this is running: the stall line now ends in "REJECTED BY: ..." and
the per-handle depths. Read it. Do not reason around it.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-17 15:21:15 -04:00
|
|
|
m_featureFailBlock(""),
|
|
|
|
|
m_featureFailIdx(-1),
|
diag: name the cause when every feature window fails, and enforce the width contract
Era 0 stalls with "NOT ONE of 54681 scanned bars produced a usable
feature window, windows ok=0 failed=54681" and nothing else. That line
reads identically for a cold ATR, a conditionally-missing optional
feature block and an out-of-range index, so it cannot be diagnosed
without one restart per hypothesis.
Two changes:
1. WIDTH CONTRACT in BufferTempData. Every enabled block must emit
exactly m_neuronsCount values on EVERY bar. A block that emits its
values on some bars and skips them on others (indicator, panel or
series unavailable for that bar) does not merely shorten the window -
it SHIFTS every feature after it into the wrong slot, and the net
then trains on silently misaligned inputs that still look like a
valid window to everything downstream. Now rejected, rolled back and
reported once, naming the optional blocks (XA / SPR / swing context)
as the ones carrying an availability test. Worth having independently
of the current stall.
2. BuildFeatureWindow records WHICH lookback slot rejected and how much
of the window was assembled, and the pass-1 stall report renders it:
"slot 0 of 20 REJECTED (window had 0 of 760)" is an indicator warm-up
or history-edge read; "every lookback bar ACCEPTED and the window was
still short: 640 of 760" is a missing 6-value block.
No behaviour change on a healthy run: the width check is an equality
that already holds, and the diagnostics render only inside the
total-failure branch.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-11 10:31:56 -04:00
|
|
|
m_windowFailSlot(-2),
|
|
|
|
|
m_windowFailTotal(0),
|
|
|
|
|
m_featureWidthWarned(false),
|
fix(altdata): median-fill instead of zero-fill, and a one-shot feature-vector autopsy
ALT-DATA AUDIT. The files themselves are healthy - all six symbols, 6,073 daily
rows, 2010-01-01 to 2026-08-17, no constant or degenerate columns, sane tails
(mac_cpi/mac_unemp flat ~47d is monthly data behaving correctly). The problem is
not the data, it is what happens where the data ISN'T.
CAltDataPanel::Features() returned an all-ZERO vector for any bar older than the
file's first row, and left blank cells at 0 too. Both were deliberate ('the block
is additive context and must degrade, never reject the bar') and that reasoning
holds for the CHANGE columns - but half these features are LEVELS: vix, ivol,
mac_y10, mac_cpi, mac_unemp, eia_util. For a level, 0 is not a missing reading,
it is an impossible one far outside the series' range. VIX does not visit zero.
And the spike lands in exactly the wrong place. Every alt file starts 2010-01-01
while the charts run far deeper - USDJPY H4 reaches ~1994, roughly HALF its
history - so 'alt block is all zeros' is precisely the predicate 'this bar is
older than 2010'. The IS/OOS split is chronological, so that predicate covers
~half of IS and none of OOS: an in-sample feature guaranteed to be useless
out-of-sample, and a bimodal input for the first BatchNorm to normalise. Not a
lookahead leak - a distribution corruption, which is quieter and was never
reported anywhere.
Now filled with the column MEDIAN over the covered range. A constant cannot leak
whatever its source - it takes the same value on every pre-coverage bar, so it
carries no information about which of those bars won - which is what makes a
median computed over later data legitimate here. Median not mean because the
series are skewed. Blank cells get the same treatment (eia_stk_idx1y alone has
181 blanks in 6,073 rows) and the count is now logged at load.
THE BACKOFF WAS ALREADY THERE AND WAS DEAD. Training.mqh arms m_coldSweepTick on
m_featureFailTransient, but only the open/ATR guards ever set that flag, so
f0cf659's cold ADMovingAverage looked PERMANENT and the sweep re-ran at full
speed forever. Setting the flag in the indicator guards revives the mechanism
that was already designed for this; no second backoff was needed and the one I
first wrote has been removed in favour of it.
SELF-HEALING, as asked. ReportFeatureHealth() runs once, the first time pass 1
produces usable windows, samples ~400 bars spread across the whole training range
and names every feature slot that is CONSTANT or mostly-zero, tagging alt-block
slots as alt[i]. Both of today's failures were the same shape - a block silently
produces nothing while every downstream number stays plausible - and neither an
accuracy figure nor a model can tell 'this feature is always 0' from 'this
feature is genuinely 0 here'. Evenly spaced sampling so a block that dies only in
deep history is caught as surely as one dead everywhere. A report, not a gate:
a rare-flag feature can be legitimately constant, and refusing to train would
turn a diagnostic into an outage.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-17 12:53:51 -04:00
|
|
|
m_featureHealthReported(false),
|
2026-08-10 07:44:03 -04:00
|
|
|
m_lastHeartbeatTick(0),
|
2026-08-10 07:22:29 -04:00
|
|
|
m_passProgressPct(0),
|
|
|
|
|
m_passLabel("starting"),
|
fix: prebuild and era sized different windows; diag: Train() names its branch
TWO things, one incident.
1) THE BUG I SHIPPED IN 0c85c54. m_tuneStartTrainBar is declared, initialised
to 0, and NEVER ASSIGNED - the assignment existed before the God-class split
and the split dropped it, leaving a dead member. Harmless while nothing read
it; a real defect the moment 0c85c54 made StartLabelCachePrebuild() reset
dtStudied from it. Train() then computed the window as
max(StartTrainBar, floor) while the prebuild computed max(0, floor), where
StartTrainBar is the non-zero datetime OnChartEventHandler passes through from
the "New Bar" event. The two therefore disagreed about `bars`, so
EnsureBarCachesCapacity() saw a changed size at era start, wiped the caches,
and re-armed a full 38k-bar prebuild - instead of training. Restored the
assignment so both sides evaluate the identical expression.
2) THE REASON IT TOOK ALL NIGHT TO FIND. Train() is a state machine with six
early-return branches above the era loop and every one of them is silent. Four
charts burned a core each for 15 minutes with an empty journal: the pass
heartbeats (694b756) proved the era loop was never reached, no prebuild
completion line appeared either, and nothing external can see inside a single
MQL5 thread - per-thread CPU says "busy", file writes say nothing, and the VPS
has no debugger. That is an undiagnosable state, and it is the thing to fix,
not just the bug of the day.
ReportTrainStall() now names the branch Train() is taking whenever no era has
completed for 3 minutes, at most once a minute per signal, with the state that
decides the branch: run/prebuild/simOos/resume flags, era, dtStudied, and -
for the cache-invalidation branch specifically - BOTH bar counts, since two
sizings disagreeing is exactly what re-arms the prebuild forever. Silent on a
healthy run: an era completing resets the clock.
Both build variants compile 0 errors, 0 warnings.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-10 07:32:08 -04:00
|
|
|
m_lastEraCompleteTick(0),
|
|
|
|
|
m_lastStallReportTick(0),
|
2026-08-01 11:27:28 -04:00
|
|
|
m_haveOosCheckpoint(false),
|
feat: gate deployment on the null of the MAXIMUM, not the per-era null
EDGE_MIN_SIGMAS is a PER-ERA test and the deployed model is the MAXIMUM
over every era a run ranks. A 2-sigma one-sided test passes on noise with
probability 0.0228 per era, so over N eras the chance at least one clears
it is 1-(1-0.0228)^N: 34% by era 18, 80% by era 70, 93% by era 112. The
gate was near-certain to open on a long run whatever the data held.
It did. HYBRID deployed 2026-08-08 at dir-precision 35.5% vs 34% chance -
+1.5pp, best of 112 eras whose per-era values wandered 30%..35.5%. At the
call counts these runs produce that is p_family 0.92..0.9999.
Every OTHER best-of-N decision here already carries this correction, and
every one REJECTS on this data: the barrier-geometry winner (null of the
maximum over 6, p=0.3902), the indicator tuner (Sidak, p=1.0000), the MI
lag profile (null of the maximum over 21 lags). The one decision that
ships a model to a live account had none.
BestCheckpointSurvivesSelection() re-tests the checkpoint that is about to
deploy:
z = (precision - chance)/SE, SE = sqrt(p0(1-p0)/n)
p_single = P(Z >= z)
p_family = 1 - (1-p_single)^N
against DEPLOY_FAMILY_WISE_ALPHA. It uses the checkpoint's OWN
snapshotted precision/chance/call-count, not the latest era's, because
the model that ships is the one that has to clear the bar.
N counts CANDIDATE eras (coverage measurable, at least one directional
call) - an era that called nothing directional could never have become
the best, so counting it would make the gate stricter than the search
that actually happened.
Conservative on purpose: consecutive eras share OOS bars and differ by
one gradient step, so they are nowhere near N independent draws and the
true family-wise error is below this bound. This gate decides what trades
real money and the house posture is reject-unless-demonstrated.
Effect at 2900 directional calls / N=112: required edge goes 1.76pp ->
2.92pp. A real edge clears it; +1.5pp does not.
Applied to BOTH automatic paths - the plateau ladder's stage-3 deploy and
the m_trainingComplete assignment - which must stay identical or the flag
persisted into the .nnw disagrees with the decision to stop, and a reload
runs inference on a model the ladder refused.
NOT applied to the two operator paths (era-cap deploy, panel Deploy
button). Those stay the operator's call; ReportSelectionGateVerdict()
logs the verdict beside them so an authorised deploy can never later be
misread as a validated one.
NormalUpperTail() is A&S 26.2.17 (|err| < 7.5e-8), self-contained rather
than pulling in Math\Stat. Verified against reference values to 6dp:
Q(1.645)=0.049985, Q(1.96)=0.024998, Q(3.0)=0.001350. Its locals are
ntB1..ntB5 because AI\Network.mqh line 79 does "#define b1 AdamBeta1".
Compiles clean: 0 errors, 0 warnings.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-08 18:01:04 -04:00
|
|
|
m_bestDirPrecPct(-1.0),
|
|
|
|
|
m_bestChancePrecPct(-1.0),
|
|
|
|
|
m_bestDirCalls(0),
|
|
|
|
|
m_deployCandidateEras(0),
|
2026-08-01 11:27:28 -04:00
|
|
|
m_oosStable(false),
|
|
|
|
|
m_objectiveMet(false),
|
|
|
|
|
m_erasSinceBestBalanced(0),
|
|
|
|
|
m_plateauStage(0),
|
feat(search): stop on the IN-SAMPLE plateau, and shrink every best-of-K effect before quoting it
Points 3 and 4 of the four-point plan.
1. IN-SAMPLE EARLY STOP - and the reason it is worth having is not compute.
The plateau ladder stops on the OOS SELECTION score. That is a peek: by the time it
fires, every one of those eras has been evaluated out of sample, so all of them sit
in the family the deploy gate corrects over (g_ensCandidateEras, Sidak). Training
longer therefore does not merely cost time - it RAISES the bar the eventual winner
has to clear.
The new stop reads the TRAINING error, which the gate never looks at. When the
optimiser has stopped improving on data it can see, more eras will not find a better
model; they will only enlarge the OOS family. Ending there shrinks the correction,
and the shrinkage is legitimate precisely BECAUSE the stopping rule never consulted
an out-of-sample number.
That distinction is the whole point and it is the one this project has got wrong four
times: stop on IS and the family really is smaller; stop on OOS and those eras were
searched and still count. Both stops now exist; only this one buys a lower bar.
Deliberately more patient than the OOS ladder (IS_ERROR_PATIENCE_MULT = 3x): training
error is noisy per era - mini-batch order alone moves it - and ending a run that is
still learning costs far more than a few wasted eras. Improvement is RELATIVE
(IS_ERROR_IMPROVE_FRAC = 1%), so it does not depend on the loss's absolute scale, and
it only acts when a checkpoint exists, since otherwise it would end a run with
nothing to deploy. Reset per RUN alongside the ladder, so a resumed run cannot
early-stop on its first era against a previous run's best.
2. WINNER'S-CURSE SHRINKAGE ON THE BARRIER-GEOMETRY WINNER.
The family-wise permutation gate already establishes that the RANKING is not noise.
It says nothing about the SIZE of the winner's effect - and a best-of-K maximum is
biased upward by construction, being the largest of K noisy draws. The adoption
message quotes that raw maximum and compares it against the incumbent, so the number
a reader plans on is the inflated one.
The penalty is now measured, not assumed: the same permutation draws that produce the
p-value also produce, per draw, the MAXIMUM excess across all candidates under pure
noise. The mean of those maxima is exactly what a best-of-K selection is expected to
report when there is nothing there. This is the empirical form of the sqrt(2 ln K) x SE
penalty the SQX EdgeFinder plugin applies to every maximum it reports (Stats.java:79-88),
and it needs no normality assumption because the draws ARE the null distribution.
Applied in James-Stein form - effect x max(0, 1 - penalty^2/effect^2) - so a large
effect is nearly untouched and a marginal one collapses toward zero.
Reported, not gated. The adoption decision still turns on the permutation p-value,
which is the right test for "is the ranking real"; the shrunk number is there so the
magnitude quoted beside it is one worth planning on. Closes the first of the two
EdgeFinder ports identified on 2026-08-12.
NOTE on the second EdgeFinder port, deliberately not done here: "let the measurement
steer the target" is already true where it matters most - ReportGeometryExpectancyScan
ADOPTS the winning barrier geometry under the family-wise gate rather than advising
it, and the MI excursion suite publishes a verdict per instrument per config. What is
still missing is steering the TRAINING TARGET itself (direction vs excursion) off
those verdicts, and that is a design change rather than a surgical one - direction is
a closed verdict while excursion SIZE keeps clearing, so the honest version of that
change is a target-selection policy, not a flag.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-17 16:29:00 -04:00
|
|
|
m_bestIsError(-1.0),
|
|
|
|
|
m_erasSinceBestIsError(0),
|
fix(plateau): the IS-error early stop was inert for every ensemble member
15 hours of training, and the stop that exists to END a run announced itself
1,299 consecutive times without ending anything:
SP500 ConvLSTM IN-SAMPLE ERROR PLATEAU - not improved in 1297 / 1298 / 1299
eras (best 0.2689, now 0.3269) ... era 1396, 1397, 1398
SP500 LSTM 536 eras SP500 CONV 442 eras SP500 PAI 150 eras
XAUUSD HYB 478 eras XAUUSD LSTM 296 eras XAUUSD CONV 366 eras
CAUSE: it wrote its decision into m_plateauStage, and EnsembleEraVerdict mirrors
the shared ladder onto every member - `mm.m_plateauStage = g_ensPlateauStage` -
on EVERY era, purely so each member's status line shows the collective stage. A
display mirror was silently overwriting a decision, so the stop re-armed and
re-fired the next era, forever.
This is the worst possible direction for this particular bug. Every one of those
1,299 eras was scored out of sample and joined the family the deploy gate
corrects over (Sidak, g_ensCandidateEras). The stop's entire purpose is to make
that family SMALLER; instead the run spent fifteen hours raising its own bar.
- m_isErrorPlateaued: a one-way per-member latch, cleared only by a fresh run.
Nothing in the ladder may reset it. The stop condition and the two solo deploy
conditions read the latch, not the mirrored stage.
- The orchestrator combines: EnsembleEraVerdict requires UNANIMITY across
participating members (same participation test the era barrier uses, so an
excluded or finished member cannot veto). One member still learning can still
move the combined vote, and the vote is what the gate certifies.
- Fed in as `dueStage = PLATEAU_STAGE_DEPLOY`, NOT written to g_ensPlateauStage.
The block that actually ends the run sits under `dueStage > g_ensPlateauStage`,
so assigning the stage directly makes that test false and the deploy never
happens - the same inert-write shape as the bug being fixed. Caught before
committing; raising dueStage carries it through the ladder's own path (warm
restarts skipped, family-wise vote test, measurement screen, joint checkpoint)
unchanged.
- g_ensIsPlateauAnnounced: announce once per run, not once per era.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-18 13:09:26 -04:00
|
|
|
m_isErrorPlateaued(false),
|
fix: training-stability audit fixes F1/F2/F3/F5 - unbiased shuffle, real plateau escapes, fresh optimizer state on restore, pure OOS metric
Four of the six findings from research/training_pipeline_audit_2026-08-09.md
(F4 mini-batching and F6 feature re-encode deliberately deferred - see the
report's implementation-status section for why):
- F1: pass-2 Fisher-Yates (and AutoTune's MI block shuffle) used MathRand()%,
which is 15-bit - provably non-uniform on every full-history era over 32,768
queued samples. New 30-bit ShuffleRandomIndex().
- F2: plateau warm restarts were a no-op whenever eta already sat at its
ceiling (the normal state of a non-regressing plateau) - the ladder was just
a 24-era countdown. Restarts now overshoot to 5x the ceiling
(PLATEAU_RESTART_BOOST) and anneal geometrically back over the patience
window, SGDR-style; ETA_MIN widened 1e-4 -> 1e-5 so the decay schedule has
real range.
- F3: checkpoint restores put weights back but kept the rejected trajectory's
Adam moments, so the optimizer immediately pushed back toward the rolled-back
state (the restore->regress->restore oscillation). CNet::ResetOptimizerState()
zeroes moments/momentum/step counters (weights, BN statistics, gamma/beta
untouched) on every mid-run restore, every boosted restart, and the
deploy-time restore that online learning continues from.
- F5: batch-norm running statistics now freeze for the pass-3 OOS scoring walk,
so the selection metric the checkpoint ranking and deploy gate read is a pure
function of the checkpoint instead of partly measuring BN drift. Defensive
unfreeze in FinalizeTrainRun covers stop-mid-pass; live/online adaptation and
the OOS continual-learning simulation stay adaptive by design.
Compiled clean (0 errors, 0 warnings) via the staged-tree recipe.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-09 10:54:09 -04:00
|
|
|
m_restartBoostErasLeft(0),
|
2026-08-01 11:27:28 -04:00
|
|
|
m_syncWaitStartTick(0),
|
|
|
|
|
m_warmupPassesRemaining(0),
|
2026-08-13 10:23:11 -04:00
|
|
|
m_coldSweepTick(0),
|
fix(train): clamp the sweep to indicator-servable depth - the scan wall was CopyBuffer, not a cold indicator
Symptom: on a 3-chart run with contention ruled out (SP500 sitting at era 2552),
USDJPY and XAUUSD produced 0 usable windows out of 50,163 and 33,966 - forever,
re-sweeping on every discard, which is the panel oscillating 0->100%.
Bars() is the PRICE series depth. A CUSTOM indicator's is not: MT5 calculates it
in its own context bounded by "Max bars in chart" (TERMINAL_MAXBARS), and
CopyBuffer past that limit does not short-read, it FAILS - so CDoubleBuffer keeps
nothing and EVERY index answers EMPTY_VALUE. ADMovingAverage is the only custom
indicator whose feature block REJECTS on EMPTY_VALUE (ADZigZag, also CiCustom,
neutral-fills; RSI/MACD/Ichimoku/ATR are built-ins served at any depth), so the
sweep died on feature 25 of every bar while the 24 price features under it were
fine. That is exactly the "window had 24 of 832 values" the stall report named.
Perfectly depth-correlated, measured 2026-08-17:
SP500 16,234 bars -> era 2552 XAUUSD 33,982 -> 0 windows
XTIUSD 16,611 bars -> era 71 USDJPY 50,179 -> 0 windows
This RETIRES the 2026-08-17 cold-indicator reading of the same stall. f0cf659 was
right that the rejection must be transient and that the dead backoff had to arm -
the branch did change to 'cold-indicator backoff' - but waiting cannot fix a depth
the terminal will never grant. So Train() now clamps to TunableBarsCalculated()
(which existed and was only ever used for a tuner printout) and trains on the
history that IS available, naming TERMINAL_MAXBARS in the log so the cause is
readable next time. m_coldSweepTick still owns the genuinely transient case: that
reads back as -1, not a positive short count. Recomputed per era, so the clamp
lifts by itself if the setting is raised.
Also fixes a real off-by-one it was hiding: the MA block reads GetData(idx) AND
GetData(idx + 1) for its bar-over-bar change, but ResizeBuffers sized m_MA to
barIndex exactly - so the deepest bar of every sweep read one past the end and was
rejected as cold. Same shape as the +ichiKijun the Ichimoku/close pair already has.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-17 13:27:14 -04:00
|
|
|
m_indicatorDepthCapBars(0),
|
fix(ensemble+depth): the barrier had no liveness escape, and the depth gate could not report the one state the evidence pointed at
Two charts (USDJPY 50,179 bars / XAUUSD 33,982) sat at era 0 for 38 minutes with
four of their eight members completely silent. Nothing in this commit guesses at
why the sweep fails - the last five guesses were all wrong. It makes the failure
say what it is, and stops one broken member taking its whole chart down with it.
WHAT THE LOG ACTUALLY SAYS, before any of this.
- The running build IS d9f834d (pulled 14:18, compiled 14:19:01, 0 errors), so
every depth instrument from 1dda479/7e63a8b/45c9e21 was live.
- It printed NOTHING. Zero "PRIMING", zero "CAPPED", zero "Per-indicator depth"
in 27 MB of journal. The instrument built to find the depth shortfall returned
"not this".
- On USDJPY at 14:24, CONV-cad8 completed eras 0 AND 1 across all 50,179 bars -
same chart, same 832-value window, same indicators, byte-identical fingerprint -
while LSTM-cad8 and HYB-cad8 reported ok=0 failed=50163. So it is not the
symbol, the history, the bar count or the indicator depth. It is per-member.
- ok=0 means the NEWEST anchors failed too, and a short indicator cannot do that.
The depth reading in project_silent_block_failures is therefore retired by its own
instrumentation. THE ROOT CAUSE IS STILL UNKNOWN and this commit does not claim one.
1. THE DEPTH GATE'S SILENT PATH WAS THE STATE IT WAS HUNTING.
ServableBars() read `if(servable <= 0 || servable >= want) return want;` - one
branch over three unrelated states, silent in all of them:
enabled == 0 -> nothing tunable is on. No cap. Healthy.
enabled > 0, servable == -1 -> a handle answered INVALID.
enabled > 0, servable == 0 -> created, never calculated.
BarsCalculated() returns -1 for a dead handle, so a dead MA is indistinguishable
from "no tunable indicators enabled" - and both returned `want` without printing a
character. That is exactly the state a per-member, every-index, depth-independent
failure produces, and it is the single reason a build carrying full depth
instrumentation logged nothing through the whole outage.
TunableBarsCalculated() now also reports HOW MANY indicators it consulted, and the
dead-handle case is reported (latched, with per-handle depths). The RETURN is
deliberately unchanged - what to do about a dead handle is not yet known, and
changing control flow on an unproven cause is how the last four fixes here went
wrong. SettledBars() routes its three pass-through states via ServableBars() so the
report is reachable from the training sweep, which is the only caller that hits it.
2. THE STALL REPORT NAMED A SLOT, NEVER A BLOCK.
"lookback slot 0 REJECTED (window had 24 of 832 values)" plus a guess ("an
indicator warm-up or a history-edge read"). Which guard fired was INFERRED by
counting 4+5+4+4+6+1 = 24 and concluding feature 25 must be the MA. The arithmetic
was right; every conclusion drawn from it was wrong, because a value count names a
POSITION and a position cannot tell cold from capped from invalid from off-the-end.
Every guard that can reject a bar now records itself - m_featureFailBlock - and the
report carries it, the series index, IndicatorDepthReport()'s per-handle depths,
and for each indicator whether the NEWEST bar reads. That last field is the whole
diagnosis in one word: newest-also-EMPTY means the buffer is unreadable everywhere
(cold or dead handle), newest-reads means a genuine history edge. Instrumented:
open, ATR, MA, RSI, MACD, Ichimoku, and all five AD blocks via ADIndicatorCold().
3. THE TOTAL-FAILURE BACKOFF WAS GATED ON THE WRONG QUESTION.
It armed only when m_featureFailTransient was set. Keeping that flag correct across
every guard is a list that has to stay right forever - the same shape of fix the
feature cache abandoned for the same reason - and the gate is pointless anyway: a
sweep where ZERO of 50,163 bars produced a window will produce zero again if it
restarts a millisecond later, transient or not. Doing that at full speed is what
starved six indicator threads on a six-core box. The backoff is now unconditional
on a total failure. The flag keeps its real job, deciding whether a MISS may be
cached, which is a per-bar question and not a scheduling one.
4. THE ERA BARRIER DEADLOCKED, AND SILENCED THE MEMBERS IT FROZE.
EnsembleMinTrainingEra() exempted deployed, stopped and paused members and its
comment concluded "so nothing deadlocks". Those three are all VOLUNTARY. A member
that simply CANNOT finish an era is none of them, so it pinned the minimum at its
own era with no time limit - and the hold branch's only action was
`m_lastEraCompleteTick = GetTickCount()`, which silences the stall watchdog. So on
USDJPY the two members that could not train reported, and the two healthy members
frozen behind them wrote nothing anywhere. The outage was visible only through the
members that were not suffering it.
- BarrierEraHeartbeat() stamps a clock on real era CHANGE, kept separate from
m_lastEraCompleteTick precisely because the barrier resets that one. Only a
member AT the minimum can be a blocker; a member ahead is idle by design and is
never counted as stuck.
- After ENSEMBLE_BARRIER_STUCK_MS (12 min) a non-advancing member is dropped from
the barrier minimum. It keeps training and rejoins the instant it completes an
era - at which point, being behind, it legitimately becomes the minimum again,
which is the documented resumed-laggard behaviour.
- Both transitions say so loudly, and the release states plainly that the
combined-vote score cannot be computed while the ensemble is desynchronised.
- A held member now writes a rate-limited journal line naming WHICH members it is
waiting on, so the blocker is read off one line.
5. THE PANEL FLICKER.
OnTickHandler gates its terse writer on !m_trainRunActive, and a barrier-held member
returns from Train() before ever setting it - so both writers thought they were the
only one updating the label and fought every tick. That is the reported "Getting
ready..." <-> "Waiting at era N for slower ensemble members" oscillation, and it hit
Perceptron but not Convolutional purely because Convolutional had a run active from
a completed era and Perceptron, resumed from disk, never did. Train()'s message is
the specific one, so it wins.
NEXT STEP once this is running: the stall line now ends in "REJECTED BY: ..." and
the per-handle depths. Read it. Do not reason around it.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-17 15:21:15 -04:00
|
|
|
m_indicatorDepthDeadWarned(false),
|
fix(indicators+panel): the dead handle is MEASURED now - recreate it; and order the ensemble panel by member, not by who published first
THE ANSWER, off the instrumentation added in be39674, first run:
ConvLSTM [HYB-2484]: TUNABLE INDICATOR REPORTS NO CALCULATED BARS - 1 tunable
indicator(s) enabled and the least-ready answers BarsCalculated()=-1 ...
Per-indicator depth: price=33982 MA=-1 ZigZag=33982 ATR=33982
MA=-1 with price, ZigZag and ATR all at full depth. **The handle is INVALID, not
short.** Same line on USDJPY (price=50179 MA=-1). Depth was never the problem;
the previous session's five theories were all answering the wrong question.
And it is per-member, not per-chart: LSTM-2484 ran the 34-candidate auto-tune on
that same chart at 15:24:18 and went on to train normally (feature health, 51
features, excursion head) reading the same indicator. Only ConvLSTM's handle -
the last member constructed - was dead. WHY is still not established. All four
members request ADMovingAverage with identical params, so MT5 hands them the SAME
refcounted handle, and the tuner's inner loop is Create-then-IndicatorRelease over
exactly that shared handle; that is the obvious suspect and it is NOT yet proven,
so this commit does not act on it.
1. IndicatorDepthReport() NOW PRINTS HANDLE NUMBERS, not just depths.
"MA=-1" says the handle is dead. "MA=-1(h12)" against another member's "MA=33982
(h12)" says it is the SAME handle and someone released it; "(h-1)" says it was
never created. That is the difference between a refcount bug and a creation
failure and it is one field. This is the measurement the shared-handle suspicion
needs before anyone acts on it.
2. RECREATE A DEAD HANDLE INSTEAD OF SWEEPING AGAINST IT.
A member that cannot read its own indicator must rebuild it. RepairDeadIndicatorHandles()
re-Creates only the ENABLED tunables reporting BarsCalculated() < 0 - a merely COLD
indicator (valid handle, 0 bars) is left alone to warm up the normal way. It does
NOT release first: -1 means the terminal no longer knows the handle, so there is
nothing to give back, and MT5 recycles handle VALUES so releasing a stale one could
decrement whatever now owns that number. 30s cooldown, because every ServableBars()
consumer reaches it including live inference on every tick. The feature cache is
dropped with it, and the log names before/after depths.
Cause-agnostic on purpose. Whatever is killing the handle, sweeping 50,163 bars
against a buffer that answers EMPTY_VALUE at every index - then discarding the era
and doing it again - is not a recovery.
3. THE SWEEP NOW HOLDS ON A DEAD HANDLE.
ServableBars() keeps answering `want` (its contract; live inference and online
learning have their own refusal paths and a 0 there reads as "no history at all").
SettledBars() - the training sweep's entry, the one caller that can afford to wait -
returns 0 instead, so Train() holds and reports rather than burning a full-history
pass it is guaranteed to throw away. A recreated handle is cold, so it primes
through the existing settle path on the next call. If the repair fails the member
holds indefinitely and says so every minute, and be39674's barrier liveness escape
releases the rest of the ensemble after 12 minutes - which is the correct
degradation and is exactly what the log shows happening.
4. THE PANEL ROWS WERE ORDERED BY WHO PUBLISHED FIRST.
Reported on XAUUSD: LSTM, ConvLSTM, Perceptron, Convolutional instead of
Perceptron, Convolutional, LSTM, ConvLSTM. ClaimEnsemblePanelSlot() handed out the
next free row on each member's FIRST PublishStatus() call, so the order was a race -
the members busy sweeping published before the ones sitting idle at the era barrier,
and be39674 sharpened it by (correctly) making a held member stop writing the terse
line. Rows are now keyed to m_ensembleIndex, the registration/construction order,
which is fixed for the life of the chart. Claimed on every publish rather than once,
so it is idempotent and refreshes the tag for a member whose ID was not final when it
first published (the config-tag suffix is appended during InitIndicators, after
EnsembleMember() registers). Unclaimed rows are skipped by the render and excluded
from the model count, so a member that has not published yet leaves no gap and shifts
nobody.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-17 15:40:00 -04:00
|
|
|
m_handleRepairTick(0),
|
fix(ensemble+depth): the barrier had no liveness escape, and the depth gate could not report the one state the evidence pointed at
Two charts (USDJPY 50,179 bars / XAUUSD 33,982) sat at era 0 for 38 minutes with
four of their eight members completely silent. Nothing in this commit guesses at
why the sweep fails - the last five guesses were all wrong. It makes the failure
say what it is, and stops one broken member taking its whole chart down with it.
WHAT THE LOG ACTUALLY SAYS, before any of this.
- The running build IS d9f834d (pulled 14:18, compiled 14:19:01, 0 errors), so
every depth instrument from 1dda479/7e63a8b/45c9e21 was live.
- It printed NOTHING. Zero "PRIMING", zero "CAPPED", zero "Per-indicator depth"
in 27 MB of journal. The instrument built to find the depth shortfall returned
"not this".
- On USDJPY at 14:24, CONV-cad8 completed eras 0 AND 1 across all 50,179 bars -
same chart, same 832-value window, same indicators, byte-identical fingerprint -
while LSTM-cad8 and HYB-cad8 reported ok=0 failed=50163. So it is not the
symbol, the history, the bar count or the indicator depth. It is per-member.
- ok=0 means the NEWEST anchors failed too, and a short indicator cannot do that.
The depth reading in project_silent_block_failures is therefore retired by its own
instrumentation. THE ROOT CAUSE IS STILL UNKNOWN and this commit does not claim one.
1. THE DEPTH GATE'S SILENT PATH WAS THE STATE IT WAS HUNTING.
ServableBars() read `if(servable <= 0 || servable >= want) return want;` - one
branch over three unrelated states, silent in all of them:
enabled == 0 -> nothing tunable is on. No cap. Healthy.
enabled > 0, servable == -1 -> a handle answered INVALID.
enabled > 0, servable == 0 -> created, never calculated.
BarsCalculated() returns -1 for a dead handle, so a dead MA is indistinguishable
from "no tunable indicators enabled" - and both returned `want` without printing a
character. That is exactly the state a per-member, every-index, depth-independent
failure produces, and it is the single reason a build carrying full depth
instrumentation logged nothing through the whole outage.
TunableBarsCalculated() now also reports HOW MANY indicators it consulted, and the
dead-handle case is reported (latched, with per-handle depths). The RETURN is
deliberately unchanged - what to do about a dead handle is not yet known, and
changing control flow on an unproven cause is how the last four fixes here went
wrong. SettledBars() routes its three pass-through states via ServableBars() so the
report is reachable from the training sweep, which is the only caller that hits it.
2. THE STALL REPORT NAMED A SLOT, NEVER A BLOCK.
"lookback slot 0 REJECTED (window had 24 of 832 values)" plus a guess ("an
indicator warm-up or a history-edge read"). Which guard fired was INFERRED by
counting 4+5+4+4+6+1 = 24 and concluding feature 25 must be the MA. The arithmetic
was right; every conclusion drawn from it was wrong, because a value count names a
POSITION and a position cannot tell cold from capped from invalid from off-the-end.
Every guard that can reject a bar now records itself - m_featureFailBlock - and the
report carries it, the series index, IndicatorDepthReport()'s per-handle depths,
and for each indicator whether the NEWEST bar reads. That last field is the whole
diagnosis in one word: newest-also-EMPTY means the buffer is unreadable everywhere
(cold or dead handle), newest-reads means a genuine history edge. Instrumented:
open, ATR, MA, RSI, MACD, Ichimoku, and all five AD blocks via ADIndicatorCold().
3. THE TOTAL-FAILURE BACKOFF WAS GATED ON THE WRONG QUESTION.
It armed only when m_featureFailTransient was set. Keeping that flag correct across
every guard is a list that has to stay right forever - the same shape of fix the
feature cache abandoned for the same reason - and the gate is pointless anyway: a
sweep where ZERO of 50,163 bars produced a window will produce zero again if it
restarts a millisecond later, transient or not. Doing that at full speed is what
starved six indicator threads on a six-core box. The backoff is now unconditional
on a total failure. The flag keeps its real job, deciding whether a MISS may be
cached, which is a per-bar question and not a scheduling one.
4. THE ERA BARRIER DEADLOCKED, AND SILENCED THE MEMBERS IT FROZE.
EnsembleMinTrainingEra() exempted deployed, stopped and paused members and its
comment concluded "so nothing deadlocks". Those three are all VOLUNTARY. A member
that simply CANNOT finish an era is none of them, so it pinned the minimum at its
own era with no time limit - and the hold branch's only action was
`m_lastEraCompleteTick = GetTickCount()`, which silences the stall watchdog. So on
USDJPY the two members that could not train reported, and the two healthy members
frozen behind them wrote nothing anywhere. The outage was visible only through the
members that were not suffering it.
- BarrierEraHeartbeat() stamps a clock on real era CHANGE, kept separate from
m_lastEraCompleteTick precisely because the barrier resets that one. Only a
member AT the minimum can be a blocker; a member ahead is idle by design and is
never counted as stuck.
- After ENSEMBLE_BARRIER_STUCK_MS (12 min) a non-advancing member is dropped from
the barrier minimum. It keeps training and rejoins the instant it completes an
era - at which point, being behind, it legitimately becomes the minimum again,
which is the documented resumed-laggard behaviour.
- Both transitions say so loudly, and the release states plainly that the
combined-vote score cannot be computed while the ensemble is desynchronised.
- A held member now writes a rate-limited journal line naming WHICH members it is
waiting on, so the blocker is read off one line.
5. THE PANEL FLICKER.
OnTickHandler gates its terse writer on !m_trainRunActive, and a barrier-held member
returns from Train() before ever setting it - so both writers thought they were the
only one updating the label and fought every tick. That is the reported "Getting
ready..." <-> "Waiting at era N for slower ensemble members" oscillation, and it hit
Perceptron but not Convolutional purely because Convolutional had a run active from
a completed era and Perceptron, resumed from disk, never did. Train()'s message is
the specific one, so it wins.
NEXT STEP once this is running: the stall line now ends in "REJECTED BY: ..." and
the per-handle depths. Read it. Do not reason around it.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-17 15:21:15 -04:00
|
|
|
m_barrierEraSeen(-1),
|
|
|
|
|
m_barrierEraTick(0),
|
|
|
|
|
m_barrierExcluded(false),
|
|
|
|
|
m_barrierHoldReportTick(0),
|
fix(depth): route EVERY ResizeBuffers call site through one indicator-depth gate
1dda479 clamped the training sweep. It left five other paths asking the indicators
for a depth they cannot serve, and on a live account the quiet ones are worse than
the stall was - a stalled chart is visible, a chart trading on a degraded feature
window is not.
ServableBars(want, context) is now the single gate, and all six go through it:
training sweep clamp, floored at TRAIN_MIN_CLAMPED_BARS (below that a small
positive BarsCalculated is warm-up, which m_coldSweepTick owns)
label prebuild clamp - labels come from price/ADZigZag and would survive a
capped MA, but ResizeBuffers sizes EVERY buffer and a failed
CopyBuffer leaves m_MA EMPTY for the next reader, so this path
could silently re-break the block Train()'s clamp just fixed
live inference HOLD. Below `need` the swing block takes its degraded path and
inference runs on a different feature distribution than the model
was fitted on. This EA sizes real positions off that output, so
no signal beats a mismatched one
online learning HOLD, same reason and worse - this path WRITES to a live trading
model, so a mismatched (features, label) pair is not a wrong arrow,
it is a wrong weight update that compounds every bar
chart rescan clamp - SIGNAL_RESCAN_LOOKBACK_BARS is 5000 and MT5's smallest
"Max bars in chart" is also 5000, so this one is genuinely
reachable; uncapped it repaints the window all-Neutral
research export clamp before the emptiness test, so a capped symbol exports the
depth it has rather than writing a CSV with a dead feature block -
an artefact that looks complete and is silently wrong
Both HOLDs are insurance, not expected states: `need` tops out near 1,152 bars
(16 + 750 + 384 + 2) against a 5,000 floor on the terminal setting. They exist so
the failure mode is unreachable rather than merely unlikely.
Not changed: a genuinely SHORT price history still takes the old degraded path at
every site. That is pre-existing behaviour and narrowing it would mute charts that
trade today, so it stays a separate decision rather than a side effect of this fix.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-17 13:33:16 -04:00
|
|
|
m_inferenceDepthRefusalWarned(false),
|
fix(buffers): revert the MA +1 - it asked for a bar that does not exist and stopped every chart
REGRESSION I INTRODUCED IN 1dda479, live for ~15 minutes.
CSeries::BufferResize -> CheckLoadHistory -> CheckTerminalHistory succeeds only when
Bars() >= size. Train() calls ResizeBuffers with barIndex == Bars(), so sizing the MA
buffer to barIndex + 1 asks for one bar more than the symbol has and fails the WHOLE
ResizeBuffers call. The log named it exactly:
failed to get 50180 bars for USDJPY,PERIOD_H4 (Bars() = 50,179)
failed to get 33983 bars for XAUUSD,PERIOD_H4 (Bars() = 33,982)
StartLabelCachePrebuild() then bailed on the false return and stayed silent, so the
only symptom was Train() reporting "arming the first label-cache prebuild" forever
with labelCacheBars=0 - the panel's "getting ready".
The premise was wrong, not just the arithmetic. The MA block reads GetData(idx) AND
GetData(idx + 1), and at the OLDEST bar that second read is SUPPOSED to fail - there
is no older bar to difference against. Rejecting that one bar is correct behaviour;
buying it cost the entire history.
Two more things, since the same defect had a second instance and no alarm:
- The Ichimoku pair (closeBars and m_Ichimoku, both barIndex + ichiKijun) is the same
bug with a far larger constant, latent only because the feature is off. Both are now
clamped to Bars(). The oldest ichiKijun bars then have no cloud, which that block's
EMPTY_VALUE guard already handles per-bar - the right outcome.
- The prebuild's bare `return` on a false ResizeBuffers now says so once, naming the
depth and Bars(). MQL5's own "failed to get N bars" was in the log the entire time,
from a stack frame nothing connected to the prebuild.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-17 14:12:47 -04:00
|
|
|
m_prebuildBlockWarned(false),
|
feat(depth): prime -> settle -> sweep, and name which handle is short
"Max bars in chart" is set to Unlimited, so the static-terminal-limit reading in
1dda479 was wrong. Two other candidate causes are falsified too: the price series
is fully downloaded and flat (USDJPY 50,162 -> 50,163 over 80 minutes, i.e. one new
H4 bar), and the handles have been stable since 13:07 with zero windows for the 17
minutes after, so it is not download-in-progress and not handle churn. The MA period
tops out at 200 (ADIndicatorTuner MA_PERIOD_PRESETS) against ~50k bars, so it is not
indicator cost either.
What IS verified stays verified: CopyBuffer past the calculated depth fails outright
rather than short-reading, so the buffer holds nothing and every index reads
EMPTY_VALUE; m_MA is the only CiCustom whose block REJECTS on that (m_ADZigZag
neutral-fills, RSI/MACD/Ichimoku/ATR are built-ins); the wall is therefore feature 25
of every bar, exactly as the "24 of 832" stall lines said. And it is depth-correlated:
16k-bar charts train, 34k/50k get zero windows forever.
So the WHY is still open, and this fix does not depend on it. Per the user's protocol:
the request itself is the primer, so prime at full depth, then poll TunableBarsCalculated()
every 3s and hold the sweep until it stops changing (3 steady probes), then use whatever
it settled at. Bounded at 10 min, and a give-up is logged as a give-up so an abandoned
depth is never mistaken for a settled one. This supersedes 1dda479's clamp on the two
training paths, which snapshotted a value that may still have been climbing; the clamp
remains for the paths that cannot wait (inference/online/rescan/export, see 7e63a8b).
The load-bearing part is what does NOT happen while waiting: no sweep. A 50k-bar feature
scan starves the indicator threads the request just woke, which is how the failure
sustained itself for 40 minutes at a time - discard era, re-sweep, discard, which is the
0->100% oscillation on the panel.
Also adds IndicatorDepthReport(): per-handle BarsCalculated() on the priming, cap and
stall lines. The logs proved WHICH FEATURE died but never WHICH HANDLE was short, so the
cause had to be inferred - and was guessed wrong twice. The next occurrence reads it off.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-17 13:44:27 -04:00
|
|
|
m_depthSettleStart(0),
|
|
|
|
|
m_depthProbeTick(0),
|
|
|
|
|
m_depthProbeLast(0),
|
|
|
|
|
m_depthProbeStable(0),
|
2026-08-01 11:27:28 -04:00
|
|
|
m_labelCacheBars(0),
|
|
|
|
|
m_labelCacheAnchorTime(0),
|
|
|
|
|
m_labelCachePrebuilt(false),
|
feat: derive the ATR multiples from measured excursions - no hardcoded geometry
The barrier was still two constants. SL_Mode/TP_Mode left the Inputs tab in
3482b6c, but the fallback was a hardcoded 2:6 and the geometry scan only ever
chose from a hardcoded grid {2,3} x {2,3,4,6,8,10}. Picking the least-bad of
eleven guesses is not deriving anything.
WHY THE SCAN WAS THE WRONG INSTRUMENT, now measurable rather than argued. It
ranks pairings by how predictable their OUTCOME is - a question about direction.
The excursion test (2c78f3b) ran on SP500 H1 and direction is the one thing
absent: ASYMMETRY p=0.0846, against RANGE/UP/DOWN all at p=0.0050, with RANGE
scoring 0.01345 vs a 0.00343 null - 4x, where the barrier label sits at 1.01x.
Hence the scan failing its own gate on every run, and its "winner" wandering
2:8 -> 3:8 -> 2:8 -> 2:4 across four runs of the same data. Excursion SIZE is
strongly measurable, so derive the geometry from that instead.
stop = q25 of measured ADVERSE travel (ordinary noise does not reach it)
target = q50 of measured FAVOURABLE travel (reached ~half the time, by
construction, inside the horizon)
Continuous, in ATR units, superseding the enum multiples. Reachability ("target
on X% of bars, stop on Y%") and the implied break-even are printed so the choice
is auditable rather than trusted.
FIXED-POINT ITERATION, not one-shot. ComputeBarrierHorizonBars scales the
horizon with the target (first-passage time grows with the band) and the
excursions are measured OVER the horizon, so target -> horizon -> excursions ->
target is a real loop - deriving once sizes the target from travel measured
under the PREVIOUS horizon. Re-measures until the multiples move <5%, capped at
3 passes, and says so if it does not settle.
Does NOT create expectancy, and the log says as much: chance precision equals
break-even at every geometry (m/(m+k) on both sides). It buys a target the
market reaches and a stop that survives noise. Where Min_Risk_Reward_Ratio
forces a target the market rarely reaches, it WARNS rather than overriding -
the ratio is the user's risk policy, so the honest move is to state its cost.
That is the collision that once rejected 100% of setups.
Pinned in the .cfg as doubles appended AFTER this morning's two ints, so .cfg
files written earlier today still load (their length guard finds no doubles) and
a model that carries them was trained on them and never re-derives.
Also fixes a message from e5ceed6 that claimed "this model resumed from disk"
unconditionally - it printed above a "seeding era 0" line on a brand-new model,
because the branch fires whenever the cache is not built, which is equally true
before a fresh model's first prebuild. A diagnostic that misreports its own
trigger is worse than one that says nothing: it gets quoted back as evidence.
FORCES A FULL RETRAIN (labels change).
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-07 12:06:25 -04:00
|
|
|
m_lastExcUp(0.0),
|
|
|
|
|
m_lastExcDown(0.0),
|
fix(labels): overlapping-label sample correction + horizon cap on the scale ladder
Three defects, all surfaced by the 2026-08-17 SP500 H4 run that shipped
stop 4.86 / target 9.71 (width 14.57*ATR, horizon 384).
1. EVERY STANDARD ERROR ASSUMED INDEPENDENT SAMPLES. Triple-barrier labels
started one per bar overlap by the label's lifespan, so n calls are worth
~n/L independent observations (Lopez de Prado, AFML ch. 4 - sample
uniqueness). All three sqrt(p(1-p)/n) sites divided by the RAW count.
The tell: the operating point's null-of-the-maximum gate is family-wise and
should fire on ~5% of eras under the null. Measured fire rates - PAI 47/73
(64%), ConvLSTM 9/24, LSTM 8/21 (38%), CONV 4/62 (6%). CONV, the only model
whose margin distribution admits few bins, sat on the null; the rest cleared
a bar that was too low by ~sqrt(L). PAI's deployed threshold consequently
alternated between the ENDS of its own range era to era (0.10 -> 0.88 ->
0.86 -> 0.66; coverage 16% <-> 73%).
TripleBarrierLabel now records when each label became KNOWABLE - the first
winning touch, or both stops, or the timeout - and the prebuild accumulates
the mean. EffectiveSampleSize() feeds the operating point, the member deploy
gate and the ensemble vote gate. Conservative by construction (n/L is an
upper bound on the damage); gates get harder, never easier.
2. THE SCALE LADDER RAN AWAY, again. Horizon scales as swingMedian*sl*tp, and
since 4d8cb08 reachability is measured OVER that horizon - so a wider rung
buys itself the time that makes it look reachable. Same target -> horizon ->
reach -> target loop the excursion window is kept short to avoid; fixing the
window confusion reopened it through the other door. It walked 128 -> 256 ->
384 bars and stopped at q90, the widest rung there is, with every rung
reading 39-48% against a 20% floor. A floor nothing fails selects nothing.
Rungs whose required horizon exceeds BARRIER_HORIZON_MAX are now rejected -
the same rule ReportGeometryExpectancyScan already applied. It was printing
the shipped pair as CLAMPED and disqualified ('h384!') two lines under the
deriver that chose it: two subsystems, one geometry, opposite verdicts.
3. THE RUNG SNAP DESTROYED THE RATIO IT WAS COMPARING. Both legs snapped
independently to the coarse first-passage grid, re-rating each candidate:
q90 4.86/9.71 -> 5.00/10.00 (2.00), q85 4.07/8.14 -> 5.00/10.00 (IDENTICAL
measurement), q75 3.07/6.13 -> 4.00/6.50 (1.63 - a nearer target). So the
ladder compared win shares taken at ratios from 1.63 to 2.17 and read the
differences as scale. It is why the reach column came out non-monotone in
width (q75 48.5% above q90 42.9%). The stop now snaps to its nearest rung in
log space and the target follows the ratio off it; the pair actually measured
is returned and logged, so a collision reads as a collision.
Also: LadderWinShare guarded against the conditional (fractal) geometry path,
which fills n from m_fracLegCount while leaving idxList empty - a latent
out-of-bounds on a currently-dead path.
New log lines: mean label lifespan and effective n on the label-cache line, the
required-vs-available horizon per rung, and the grid pair the reconciliation
actually measured (its tolerance now scales with the grid skew instead of a flat
5pp).
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-17 01:12:05 -04:00
|
|
|
//--- 0 samples => MeanLabelLifespan() returns 1.0 => EffectiveSampleSize() is the identity, so an
|
|
|
|
|
//--- un-prebuilt model behaves exactly as it did before the overlap correction existed rather than
|
|
|
|
|
//--- shrinking its own samples on a guess. See m_lastLabelLifespan.
|
|
|
|
|
m_lastLabelLifespan(0),
|
|
|
|
|
m_labelLifespanSum(0.0),
|
|
|
|
|
m_labelLifespanCount(0),
|
feat(measurement): fix zero-skill denominator, publish the deploy bar, measure lifespan per rung, add a MEASURE scale objective
The last run could not have demonstrated an edge either way, and nothing in the
log said so. Four changes so it does.
1. THE ZERO-SKILL LINE DIVIDED BY THE WRONG DENOMINATOR. m_oosWinLongTotal resets
every era; m_oosSamples only resets on a full model reset. So 'always-long %'
decayed as ~1/era: a run whose true rate is 37% printed 1.2% at era 33 and
0.0% at era 2219. This is the SAME bug already found and fixed for
logBuyPredPct thirty lines above ('era-15 Buy:2% that was really ~30%'), left
in the one line whose whole job is to be the reference every other number is
read against. Correct at era 1, wrong everywhere after - including the '62%
zero-skill' figure in the 2026-08-16 notes. Now per-era, and always-short is
finally readable.
2. THE DEPLOY GATE STATES ITS OWN BAR. 'edge -1pp' era after era cannot separate
'short by a hair' from 'short by an amount no strategy could cover'. The era
line now prints the required win rate, the SE, the effective n and the
lifespan it was deflated by; above 100% it says UNREACHABLE. At 4,738 OOS bars
and L=75.6 there are ~63 independent observations, putting the bar near 66% at
typical coverage.
3. LIFESPAN MEASURED PER RUNG. The first-passage cache already stores touch ages
at every ladder level, so each candidate geometry's resolution time is
readable without training on it - L-vs-width becomes a measurement across the
whole ladder in ONE run rather than a second chart. Each rung reports L,
n_eff, min provable edge and min provable EV.
4. SCALE OBJECTIVE IS PHASE-AWARE, defaulting to MEASURE. Width and detectability
are opposed: labels overlap by L, L grows like m*k = width^2 at fixed ratio,
so min provable EV ~ width^2 while the cost saving from width is only linear.
Doubling width quadruples the smallest EV you can prove. DEPLOY (widest that
clears reachability) is right once an edge is known; MEASURE (narrowest that
keeps round-trip spread under BARRIER_MAX_COST_FRACTION_PCT) is right while it
still has to be shown. The direction does not depend on the exponent, and
item 3 makes the exponent checkable.
Fixed in review: m_lastRungLifespan is cleared on every LadderWinShare entry or a
rejected rung reports the previous rung's lifespan as its own; per-rung
detectability is labelled IS-sample based (the deriver may not see the holdout),
so absolute figures are optimistic while the ranking is unaffected.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-17 10:37:05 -04:00
|
|
|
//--- -1 = the deploy gate has not run its arithmetic yet this era; the era line then omits the bar
|
|
|
|
|
//--- rather than printing a stale one from a previous era.
|
|
|
|
|
m_lastEdgeFloorPct(-1.0),
|
|
|
|
|
m_lastPrecSE(-1.0),
|
|
|
|
|
m_lastEffN(-1.0),
|
|
|
|
|
m_lastRungLifespan(0.0),
|
feat(gate): cross-instrument pooled certification
The deploy bottleneck is CERTIFICATION, not training. A 4,738-bar OOS window at
L=75.6 holds ~63 independent observations; certifying a 3pp edge at 2 sigma needs
~1,036. More bars of the same symbol barely help - they overlap. Other symbols do
not.
WHAT POOLS. Not win rates: symbols have different derived geometries, different
break-evens and different drifts, so averaging raw rates across them is
meaningless. What pools is each symbol's EXCESS OVER ITS OWN CHANCE RATE,
combined by inverse-variance weighting (fixed-effects meta-analysis). Each symbol
keeps its own model, geometry and chance rate; only the evidence is combined.
THE CORRELATION PROBLEM, bracketed rather than assumed away. SP500 and NAS100 are
~0.9 correlated and pooling them as independent inflates the evidence. Nothing
here can measure that without sharing return series, so instead of guessing a
correction the gate reports both ends:
SE_INDEP = sqrt(1/SUM(1/var_i)) all members independent
SE_CORR = SUM(w_i * sqrt(var_i)) all members perfectly correlated
The truth is always between. THE GATE USES SE_CORR, so a pass cannot be an
artifact of correlated instruments - that bound already assumes the worst. The
ratio is logged as the diversification credit the gate declines to claim, so the
cost of that conservatism is visible instead of hidden.
SCOPE, deliberately limited: the pooled result is REPORTED, never folded into
tradeableOK. The local gate certifies the model that actually trades this symbol;
the pool answers the different question of whether the strategy has an edge at
all. Letting a cross-symbol result license a local deploy would ship a model that
never cleared its own bar - so it cannot.
Mechanics: one file per instrument (no concurrent-write path to get wrong), every
FileOpen carrying FILE_SHARE_READ|FILE_SHARE_WRITE, records skipped rather than
reinterpreted on a version mismatch, 12h staleness cutoff so a stopped chart
cannot vote, and pooling refused below 3 instruments. Poolability requires
matching timeframe and ratio; differing SYMBOL is the entire point. Publishing is
unconditional - a pool that only hears from winners is a selection effect, not a
meta-analysis.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-17 10:45:37 -04:00
|
|
|
m_poolWriteWarned(false),
|
|
|
|
|
m_lastPoolPasses(false),
|
|
|
|
|
m_lastPoolReport(""),
|
feat: derive the ATR multiples from measured excursions - no hardcoded geometry
The barrier was still two constants. SL_Mode/TP_Mode left the Inputs tab in
3482b6c, but the fallback was a hardcoded 2:6 and the geometry scan only ever
chose from a hardcoded grid {2,3} x {2,3,4,6,8,10}. Picking the least-bad of
eleven guesses is not deriving anything.
WHY THE SCAN WAS THE WRONG INSTRUMENT, now measurable rather than argued. It
ranks pairings by how predictable their OUTCOME is - a question about direction.
The excursion test (2c78f3b) ran on SP500 H1 and direction is the one thing
absent: ASYMMETRY p=0.0846, against RANGE/UP/DOWN all at p=0.0050, with RANGE
scoring 0.01345 vs a 0.00343 null - 4x, where the barrier label sits at 1.01x.
Hence the scan failing its own gate on every run, and its "winner" wandering
2:8 -> 3:8 -> 2:8 -> 2:4 across four runs of the same data. Excursion SIZE is
strongly measurable, so derive the geometry from that instead.
stop = q25 of measured ADVERSE travel (ordinary noise does not reach it)
target = q50 of measured FAVOURABLE travel (reached ~half the time, by
construction, inside the horizon)
Continuous, in ATR units, superseding the enum multiples. Reachability ("target
on X% of bars, stop on Y%") and the implied break-even are printed so the choice
is auditable rather than trusted.
FIXED-POINT ITERATION, not one-shot. ComputeBarrierHorizonBars scales the
horizon with the target (first-passage time grows with the band) and the
excursions are measured OVER the horizon, so target -> horizon -> excursions ->
target is a real loop - deriving once sizes the target from travel measured
under the PREVIOUS horizon. Re-measures until the multiples move <5%, capped at
3 passes, and says so if it does not settle.
Does NOT create expectancy, and the log says as much: chance precision equals
break-even at every geometry (m/(m+k) on both sides). It buys a target the
market reaches and a stop that survives noise. Where Min_Risk_Reward_Ratio
forces a target the market rarely reaches, it WARNS rather than overriding -
the ratio is the user's risk policy, so the honest move is to state its cost.
That is the collision that once rejected 100% of setups.
Pinned in the .cfg as doubles appended AFTER this morning's two ints, so .cfg
files written earlier today still load (their length guard finds no doubles) and
a model that carries them was trained on them and never re-derives.
Also fixes a message from e5ceed6 that claimed "this model resumed from disk"
unconditionally - it printed above a "seeding era 0" line on a brand-new model,
because the branch fires whenever the cache is not built, which is equally true
before a fresh model's first prebuild. A diagnostic that misreports its own
trigger is worse than one that says nothing: it gets quoted back as evidence.
FORCES A FULL RETRAIN (labels change).
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-07 12:06:25 -04:00
|
|
|
m_derivedSlMult(0.0),
|
|
|
|
|
m_derivedTpMult(0.0),
|
feat: fitted directional confidence threshold - selectivity gets a mechanism
The training loss and the selection metric wanted different things and only
the second one knew it. Logit-adjusted cross-entropy has no term for "how
often should I trade", so the head calls a direction on 87-91% of bars. The
selection metric is precision x coverage credit, saturating at the coverage
floor - above the floor extra calls earn NOTHING and only precision counts.
So selection wanted few good calls, the loss produced many mediocre ones, and
all selection could do was pick the least-bad era out of what it was handed.
Nothing pushed the model toward selectivity.
This gives the decision RULE the policy instead of distorting the loss (which
is estimating class probabilities correctly, and a probability estimate should
not be bent to encode a trading policy - Elkan 2001: estimate, then choose the
operating point separately). AdjustedSignalFromSoftmax now abstains unless the
winning direction's softmax margin over its best rival clears a fitted
threshold. Margin, not the winning probability: the latter moves with overall
calibration rather than with how close the decision actually was.
Fitted on IS, applied to OOS and live. Pass 2 already forward-passes every IS
sample, so the margin histogram is harvested there for free (primary
occurrences only, so the oversampled replay queue cannot skew the operating
point); the fit runs at the end of pass 2, BEFORE pass 3, so the deploy gate
grades the thresholded model on bars the threshold never saw. Fitting on
pass 3's own predictions would be choosing the operating point on the data
being graded - the best-of-N error corrected in five other places here.
Objective: maximise IS directional precision subject to still clearing the
SAME coverage floor the deploy gate uses (base rate x 0.25, re-derived
locally so the two cannot drift apart). Swept top-down in one pass; ties go
to the LOWER threshold, since equal precision for less coverage is strictly
worse. Under DIR_CONF_MIN_FIT_CALLS (200) it runs unthresholded rather than
on a guess.
The threshold is part of the MODEL, not the run: captured with
Net.CaptureWeights(), restored with the weights at both restore sites, and
appended to the .cfg under the same length-guard convention so a deployed
model reloads at the operating point its gate actually cleared. A pre-2026-08-09
.cfg reads 0.0, which is exactly the behaviour it was trained under.
Per-era line now prints "@margin>=X.XX" next to coverage, so a coverage drop
can be attributed to the operating point rather than guessed at.
Both build variants compile 0 errors / 0 warnings.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-09 15:04:37 -04:00
|
|
|
//--- 0.0 = unthresholded until the first pass 2 fits it (see DIR_CONF_THRESHOLD_BINS). Deliberately
|
|
|
|
|
//--- the permissive value: a model that has not measured its own operating point must not silently
|
|
|
|
|
//--- abstain on everything.
|
|
|
|
|
m_dirConfThreshold(0.0),
|
|
|
|
|
m_bestDirConfThreshold(0.0),
|
|
|
|
|
m_dirConfPrimaryBars(0),
|
|
|
|
|
m_dirConfSparseWarned(false),
|
feat: derive the ATR multiples from measured excursions - no hardcoded geometry
The barrier was still two constants. SL_Mode/TP_Mode left the Inputs tab in
3482b6c, but the fallback was a hardcoded 2:6 and the geometry scan only ever
chose from a hardcoded grid {2,3} x {2,3,4,6,8,10}. Picking the least-bad of
eleven guesses is not deriving anything.
WHY THE SCAN WAS THE WRONG INSTRUMENT, now measurable rather than argued. It
ranks pairings by how predictable their OUTCOME is - a question about direction.
The excursion test (2c78f3b) ran on SP500 H1 and direction is the one thing
absent: ASYMMETRY p=0.0846, against RANGE/UP/DOWN all at p=0.0050, with RANGE
scoring 0.01345 vs a 0.00343 null - 4x, where the barrier label sits at 1.01x.
Hence the scan failing its own gate on every run, and its "winner" wandering
2:8 -> 3:8 -> 2:8 -> 2:4 across four runs of the same data. Excursion SIZE is
strongly measurable, so derive the geometry from that instead.
stop = q25 of measured ADVERSE travel (ordinary noise does not reach it)
target = q50 of measured FAVOURABLE travel (reached ~half the time, by
construction, inside the horizon)
Continuous, in ATR units, superseding the enum multiples. Reachability ("target
on X% of bars, stop on Y%") and the implied break-even are printed so the choice
is auditable rather than trusted.
FIXED-POINT ITERATION, not one-shot. ComputeBarrierHorizonBars scales the
horizon with the target (first-passage time grows with the band) and the
excursions are measured OVER the horizon, so target -> horizon -> excursions ->
target is a real loop - deriving once sizes the target from travel measured
under the PREVIOUS horizon. Re-measures until the multiples move <5%, capped at
3 passes, and says so if it does not settle.
Does NOT create expectancy, and the log says as much: chance precision equals
break-even at every geometry (m/(m+k) on both sides). It buys a target the
market reaches and a stop that survives noise. Where Min_Risk_Reward_Ratio
forces a target the market rarely reaches, it WARNS rather than overriding -
the ratio is the user's risk policy, so the honest move is to state its cost.
That is the collision that once rejected 100% of setups.
Pinned in the .cfg as doubles appended AFTER this morning's two ints, so .cfg
files written earlier today still load (their length guard finds no doubles) and
a model that carries them was trained on them and never re-derives.
Also fixes a message from e5ceed6 that claimed "this model resumed from disk"
unconditionally - it printed above a "seeding era 0" line on a brand-new model,
because the branch fires whenever the cache is not built, which is equally true
before a fresh model's first prebuild. A diagnostic that misreports its own
trigger is worse than one that says nothing: it gets quoted back as evidence.
FORCES A FULL RETRAIN (labels change).
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-07 12:06:25 -04:00
|
|
|
m_geometryDerived(false),
|
|
|
|
|
m_geometryDerivePasses(0),
|
fix: excursion window must not depend on the barrier it sizes
DIRECTION IS NOT THERE, and this run is what establishes it. Three symbols:
raw ASYMMETRY clears on all three (p=0.0199 / 0.0050 / 0.0050)
norm ASYMMETRY collapses on all three (p=0.3433 / 0.5075 / 0.2736),
USDCAD landing BELOW its own null
RANGE control strengthens to 3-5x its null everywhere
Divide sigma out and the apparent directional signal vanishes entirely. What
cleared was volatility leaking through an unnormalised difference. Note this
would have passed any replication test: three instruments at p=0.005 is exactly
the evidence one would accept before committing to a rebuild, and the confound
reproduces perfectly. Replication was never going to catch it - only the
normalisation could.
Two defects of mine, both surfaced by the same run.
1. THE GEOMETRY DERIVATION WAS DIVERGING, NOT CONVERGING. It produced a
14.57*ATR stop and a 29.14*ATR target that only 5.7% of bars ever reach.
Excursions were measured over the barrier horizon; the horizon scales with
the target; the target is a quantile of the excursions - so target ->
horizon -> excursions -> target ran away, and "settled" only because the
horizon ladder caps at 384 bars. A saturated runaway, which the iteration
guard could not catch because it watches for OSCILLATION.
Fixed at the root: excursions now accumulate only over m_swingMedianBars -
the UNSCALED median ZigZag leg, a property of the instrument that owes
nothing to the barrier. The barrier walk still runs the full horizon,
because that is how long the trade is held; only the MEASUREMENT used to
size the barrier is confined to a geometry-independent window.
(The Min_Risk_Reward_Ratio warning fired correctly and is what flagged it -
the diagnostic worked while the derivation behind it did not.)
2. THE CONFOUND VERDICT WAS UNREACHABLE. `sizeCleared && !asymCleared` was
tested first and is true whenever size clears - i.e. always - so the branch
that NAMES the volatility confound never printed; all three symbols showed
the generic size-not-direction message instead. Verdict chain rewritten with
the specific case first, and the dangling elses my first patch introduced
removed.
FORCES A FULL RETRAIN (the excursion window changes every derived barrier).
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-07 13:57:23 -04:00
|
|
|
m_swingMedianBars(0),
|
2026-08-01 11:27:28 -04:00
|
|
|
m_labelPrebuildActive(false),
|
|
|
|
|
m_prebuildSeedPending(false),
|
|
|
|
|
m_labelPrebuildBars(0),
|
|
|
|
|
m_labelPrebuildOosCutoff(0),
|
|
|
|
|
m_labelPrebuildIndex(-1),
|
|
|
|
|
m_labelPrebuildBuyCount(0),
|
|
|
|
|
m_labelPrebuildSellCount(0),
|
|
|
|
|
m_labelPrebuildNeutralCount(0),
|
|
|
|
|
m_simOosNet(NULL),
|
|
|
|
|
m_simOosRunActive(false),
|
|
|
|
|
m_simOosCutoff(0),
|
|
|
|
|
m_simOosBarIndex(-1),
|
|
|
|
|
m_simOosForecast(0),
|
|
|
|
|
m_simOosSamples(0),
|
2026-08-16 21:08:41 -04:00
|
|
|
m_dbBackfillActive(false),
|
|
|
|
|
m_dbBackfillDone(false),
|
|
|
|
|
m_dbBackfillIndex(0),
|
|
|
|
|
m_dbBackfillStartIndex(0),
|
fix(gate): move the ranking slice to the OLD end - it walled off the recent chart
NOT COMPILED - user compiles.
User: "there is quite some trading going on, but absolutely nothing on the recent
area of the chart, like there is a hard wall starting around november 2025."
That wall is 7caf2f6's ranking slice, and it was placed at the wrong end. Chart
arrows are only ever drawn on bars pass 3 GRADES, and the slice reserved the
NEWEST 20% of the OOS window plus a label-horizon purge. At the live sizing -
~4,860 OOS bars, 128-bar horizon - that is ~1,100 H4 bars withheld from grading,
about ten months back from today, exactly where the wall appears.
The invisible cost was worse than the visible one: it handed the deploy gate the
OLDEST 80% of the OOS window and withheld the most recent regime from the single
decision that has to generalise forward.
Both fixed by putting the reserve at the oldest end instead:
[0, oosScoreHi) OOS - graded by pass 3 (NEWEST, arrows restored)
[oosScoreHi, rankLo) purge - one label horizon
[rankLo, oosCutoff) RANKING - backfill only, graded by nobody
[oosCutoff, calibLo) purge
[calibLo, calibHi) CALIBRATION
... IS
Of the three consumers competing for those bars, recency is worth least to the
ranking: it is an ORDERING of confidence tiers, far less regime-sensitive than an
absolute win rate, while the gate's power and the operator's read of the chart
both want the newest data. The slice keeps every property that made it worth
carving - never graded, never selected on, never seen by the gate, purged on both
sides - so the backfilled rows are still honestly out-of-sample.
RankSliceHiIndex is replaced by RankSliceLoIndex + OosScoreHiIndex; pass 3 now
excludes the slice at the TOP of its walk and descends to 2 as it always did.
The backfill walks [RankSliceLoIndex, oosCutoff) via a new m_dbBackfillStopIndex,
clamped at both ends so a degenerate slice yields an empty walk rather than one
that wanders into graded bars. Verified no reference to the old helper survives.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-16 22:58:55 -04:00
|
|
|
m_dbBackfillStopIndex(2),
|
2026-08-16 21:08:41 -04:00
|
|
|
m_dbBackfillBars(0),
|
|
|
|
|
m_dbBackfillFired(0),
|
fix: the DB backfill could never run, and HEAD did not compile
Four defects in 64c5dd5/1a05e63, found by review + a baseline compile.
Goals 1-8 of that session are unchanged; this makes 6 and 8 actually reachable.
1. HEAD DID NOT COMPILE - 6 errors. CControlPanel::Minimize/Maximize were
declared `virtual bool ... override`, but CAppDialog declares both as
`virtual void` (Controls\Dialog.mqh). errors 265 + 404 on each, plus 151
on `bool ok = CAppDialog::Minimize()`. Return type is void now; there was
never a success flag to forward. Verified: 0 errors, 0 warnings.
2. THE BACKFILL COULD NEVER ADVANCE, and neither could the OOS continual
simulation (that one has been dead since it was written). Both are armed
at the instant convergence is declared, and both advance only from inside
Train(), one chunk per call. But ScheduleTrainingIfNeeded's only per-tick
ArmStudyEvent site sits in the `else` of a branch taken whenever
m_trainingComplete is set and m_trainRunActive is clear - which is exactly
the state FinalizeTrainRun() leaves behind one line before they are armed.
Train() was never called again, so the walks sat at their start index
forever: no "simulation complete" line, and not one row written to the DB
this feature exists to fill. Only a manual Resume/Retrain unstuck them.
Both flags now keep the model schedulable.
3. IN AI_HYBRID - the mode this ships in - the backfill was never even armed.
Ensemble members deploy at Train() ENTRY and return immediately (so no era
is wasted), which skips the era-end block the backfill was started from.
All four members were a no-op for a second, independent reason. Armed on
the ensemble deploy path too, from m_resumeBars/m_resumeOosCutoff.
4. RE-RUNS DUPLICATED ROWS. RegisterSignal inserts unconditionally - no key,
no duplicate check - and m_dbBackfillDone is in-memory, so every later
attach that retrained to convergence wrote a second full set of rows for
the same bars. The ranking would count one bar once per model that ever
deployed, weighting superseded opinions as heavily as the live one. A
.dbfill marker stamps the deployed era; written only on completion (an
interrupted walk redoes itself rather than ranking a partial window) and
deleted with the other sidecars on reset-weights.
Also: WarmBlocking's timeout was silent, which restored the exact silent
pin failure it was added to prevent - it now says so in the journal, and
returns true for "no reference pairs to wait for" so the warning stays rare
enough to be read.
Not addressed, needs a decision: the backfill scores the OOS window with the
checkpoint that was SELECTED as best on that same window, then writes those
win rates into the table filter weights rank on - the selection set consumed
twice, undiscounted, while the deploy gate right next to it applies a
family-wise correction for exactly that effect. The rows are also simulated
triple-barrier outcomes at today's spread sharing a table with realised
fills. The completion log line now states both plainly.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-16 21:25:51 -04:00
|
|
|
m_dbBackfillEra(-1),
|
2026-08-01 11:27:28 -04:00
|
|
|
m_tuneTrialIndex(-1),
|
|
|
|
|
m_tuneBestOosForecast(-1),
|
|
|
|
|
m_tuneLastTrialWasWin(true),
|
|
|
|
|
m_tuneHaveBestCheckpoint(false),
|
|
|
|
|
m_tuneStartTrainBar(0),
|
|
|
|
|
m_tuneFilterDone(false),
|
|
|
|
|
m_trainingPaused(false),
|
|
|
|
|
m_trainingStopRequested(false),
|
|
|
|
|
m_activeFileCommon(true),
|
|
|
|
|
m_configLockName(""),
|
|
|
|
|
m_isInitialized(false),
|
fix(deinit): a full model write was running ahead of the cheap cleanup
"Abnormal termination" is back, and this time it is not the arrows. The
timing names the culprit exactly:
16:02:31.547 OnDeinit: shutting down
16:02:36.003 Abnormal termination <- 4.46 s, MetaTrader gave up
16:02:36.226 chart signals - persisted <- cleanup finished 0.2 s LATE
OnDeinit called StopTraining() BEFORE the chart cleanup. StopTraining()
finalises an in-flight run, and FinalizeTrainRun() restores the best
checkpoint and then persists it - a full ~1MB model write per signal. So
the expensive step ran ahead of the cheap bounded one, which is precisely
the inversion the shutdown ordering exists to prevent. The previous fix
put PersistWeightsOnShutdown last and missed that StopTraining smuggles a
second save in at the front.
Two changes:
Cleanup now runs FIRST, then StopTraining, then the weight save. The
visible teardown is cheap and bounded, so it always completes even when
everything after it is killed.
And the deploy-persist inside FinalizeTrainRun is suppressed during
shutdown. RestoreWeights() is an in-MEMORY swap, so the best checkpoint
is already the live net by that line, and PersistWeightsOnShutdown writes
exactly those weights moments later. The old path wrote the same model
twice per signal - eight full writes across four charts - for no benefit.
A user-pressed Stop still persists immediately, because nothing else
would.
Compiles 0 errors / 0 warnings. Build tag deinit-order-v2.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-01 16:06:40 -04:00
|
|
|
m_shutdownInProgress(false),
|
diag(autotune): one label shuffle cannot settle the no-edge question
The permutation baseline added in 018afb1 came back on all four charts as
0.00401 nats against floors of 0.00267 / 0.00298 / 0.00318 - three draws
whose spread is as wide as the excess being judged, because one shuffle
is one sample from the null, not the null. That is not enough to retire a
topology on.
Now MI_NOISE_PERMUTATIONS draws, reported as mean +/- sd with a z-score,
plus two numbers the mean over 26 columns cannot express:
- the STRONGEST single feature's MI, against its own shuffled value.
One informative column among 25 useless ones is precisely the case
the mean hides, and precisely the case worth finding.
- the excess as a percentage of H(Y). At these sample sizes a z-score
can be comfortably significant while the effect is worthless, so
"is it real" and "is it big enough to matter" are asked separately
and answered separately.
The verdict line also now states the measure's limit every time rather
than only when the news is bad: this is a MARGINAL, PER-BAR statistic and
the network reads m_historyBars bars jointly, so it can prove signal
exists but never that it does not. It rules out a per-feature edge - and
therefore any indicator retuning - not an edge that lives in a
combination or across time.
Compiles 0 errors / 0 warnings, standard and Market.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-01 13:32:12 -04:00
|
|
|
m_lastArrowsSaved(0),
|
fix: the Adam second moment was never Adam - all four tiers
Root cause of the B=32 regression, and it predates F4 entirely. Every Adam
kernel stored v already square-rooted and then fed that stored value back in
as if it were the variance:
v_new = sqrt(b2 * v_old + (1 - b2) * g^2)
That recursion has a fixed point at v ~= b2 = 0.999 for ANY gradient below
unit scale, so the denominator stops tracking the gradient and Adam degrades
into plain SGD with lr = lt. Measured against the shipped WarriorCPU.dll
(batch_accum_check.cpp, TestOptimizerScaleInvariance), 4000 steps of a
constant gradient: 3285x less displacement at |g|=1e-5 than at |g|=1, where
a scale-invariant optimizer gives the same distance for both. After the fix
all six magnitudes read 1.199 and v tracks |g| exactly.
It hit conv/LSTM specifically because they sit behind a batch-norm with
running variance ~2.6e+05, so their gradients arrive divided by ~500 - deep
in the degraded regime - while the dense stack near the loss stayed in the
working one. In situ on SP500 H1: lstm1 dW/W 2.62/10.0/7.14% -> 0.024/0.022/
0.003%, conv1 decaying to 0.000% by era 30. NeuronBatchNorm.mqh already
squared v back for gamma/beta and its comment named the kernels as wrong,
which is exactly why gamma/beta kept training while the stages behind froze.
Persisted .nnw needs no migration - v keeps its std-dev meaning.
Also, the two ways F4 exposed it, both mine:
- No LR compensation for B fewer steps per era. sqrt(B) for adaptive methods
(Krizhevsky 2014; Granziol et al. 2022), applied once in
InitialEtaForOptimizer(). Linear scaling (Goyal et al. 2017) is for SGD.
- Plateau patience denominated in eras, so raising B made the ladder 32x more
impatient in its only unit. PAI converged at era 41 on ~49k updates where
the same config had been finding new bests at era 1028.
TrainPlateauPatienceEras() stretches it by the same sqrt(B).
TRAIN_BATCH_SIZE 32 -> 8 so the patience stretch stays affordable (8 -> 23
eras per stage, not 8 -> 45). Both helpers are identities at B=1.
Deploy gate: DEPLOY_MIN_SIDE_RECALL_PCT (10%) folded into tradeableOK. The
perceptron reported Sell:0% recall in all 41 eras, cleared the floor on Buy
alone at 36.6% vs 34% chance, deployed, and sprayed buy arrows. Folded into
the ranking key rather than checked at deploy time so a one-sided era cannot
become best-so-far in the first place.
Deinit: the arrow purge now runs BEFORE ExtPanel.Destroy(), an unbounded
CAppDialog teardown that sat ahead of it - the same ordering inversion the
rule there exists to prevent. CONV was force-terminated 4.8 s into OnDeinit
(vs ~1.1 s for the three that finished) having reached none of its cleanup,
so its arrows stayed on the chart. Steps are now timed in the log.
PurgeChart's verification rescan filtered on OBJ_ARROW, the same blind spot
as the bulk delete, so "persisted 10 ... cleared 0" passed silently. It now
walks every object type and reports the object counts when both are zero.
Both build variants compile 0 errors / 0 warnings; both DLLs rebuilt.
FORCES A RETRAIN (already forced by N1) and both DLLs must ship with the .ex5.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-09 14:02:35 -04:00
|
|
|
m_purgeMismatchWarned(false),
|
diag(autotune): one label shuffle cannot settle the no-edge question
The permutation baseline added in 018afb1 came back on all four charts as
0.00401 nats against floors of 0.00267 / 0.00298 / 0.00318 - three draws
whose spread is as wide as the excess being judged, because one shuffle
is one sample from the null, not the null. That is not enough to retire a
topology on.
Now MI_NOISE_PERMUTATIONS draws, reported as mean +/- sd with a z-score,
plus two numbers the mean over 26 columns cannot express:
- the STRONGEST single feature's MI, against its own shuffled value.
One informative column among 25 useless ones is precisely the case
the mean hides, and precisely the case worth finding.
- the excess as a percentage of H(Y). At these sample sizes a z-score
can be comfortably significant while the effect is worthless, so
"is it real" and "is it big enough to matter" are asked separately
and answered separately.
The verdict line also now states the measure's limit every time rather
than only when the news is bad: this is a MARGINAL, PER-BAR statistic and
the network reads m_historyBars bars jointly, so it can prove signal
exists but never that it does not. It rules out a per-feature edge - and
therefore any indicator retuning - not an edge that lives in a
combination or across time.
Compiles 0 errors / 0 warnings, standard and Market.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-01 13:32:12 -04:00
|
|
|
m_miBestColumn(0.0),
|
2026-08-01 14:01:32 -04:00
|
|
|
m_miLabelEntropy(0.0),
|
diag(autotune): a positive control, and a scan that separates "no signal"
from "signal knocked out of step"
Four architecturally different networks landed on the same precision -
Buy 23-25% against a 25.4% base rate, Sell 19-22% against 22.0% - while
making completely different calls (HYBRID votes Sell on 69% of bars, PAI
on 41%). Precision equal to the base rate is what INDEPENDENCE looks
like, and precision under independence is fixed by the label
distribution, not by the architecture, so all four converging on it is
arithmetic rather than coincidence. Accuracy meanwhile tracks coverage
exactly as independence predicts (31.1/30.3/25.0 predicted vs
31.8/28.9/24.6 observed for PAI/CONV/HYB).
But "no information in the data" and "information destroyed upstream of
every topology" produce that identical picture, and the MI test alone
cannot tell them apart either. Two additions:
POSITIVE CONTROL. Three "measurements" in this codebase have turned out
to be silent no-ops that produced plausible numbers - the MI scorer
reading an array nobody filled, the eval-mode guard that switched off the
imbalance correction, the alternation gate whose premise was never true.
So the estimator now has to prove it responds to a signal known to be
present before any floor reading is believed: the label of a neighbouring
sample row, ~19 bars away and far inside the 128-bar barrier horizon, so
the two outcome windows overlap heavily and MUST be associated. Same
binning, same estimator. Near the floor => every MI figure is void.
ALIGNMENT SCAN. Re-scores against the label taken from bar i+k for k in
-5..+5. A peak at k != 0 is a feature/label misalignment - an off-by-one
in the label index, a horizon applied to the wrong bar, a feature window
that lags what it claims - which would destroy the information before any
topology saw it and would look identical in every accuracy number this EA
prints. A flat profile says the features simply do not carry this target.
The sampled range is trimmed by |k| at both ends so a shift is measured
rather than an edge effect, and both bars must carry a real label.
Also: BuildMiSample publishes its stride instead of the report
recomputing that arithmetic (it would drift), and the control sizes its
buffers from its own sample count rather than the caller's.
Compiles 0 errors / 0 warnings, standard and Market.
Build tag mi-control-align-v1. Redeploy only - no retrain, no model
deletion; the diagnostic runs on resumed models.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-01 14:12:10 -04:00
|
|
|
m_miStrideBars(0),
|
fix(diag): the symbol sweep was measuring its own sampling, not the market
Twelve cells came back with higher-timeframe "signal" 5-9x anything on
H1, at p=0.005. It was an artifact, and the sweep's own columns gave it
away: excess tracked the sampling STRIDE almost monotonically, and the
three D1 cells - stride collapsed to 1-5 bars against a 128-bar horizon,
i.e. ~99% window overlap - were the three highest. Three flaws, all the
same family: comparing numbers without the spread that belongs to them.
1. THE NULL ASSUMED INDEPENDENCE THE LABELS DO NOT HAVE. Triple-barrier
labels overlap; two rows less than one horizon apart share most of their
outcome window. A free Fisher-Yates shuffle destroys that dependence
along with the association, making the null far narrower than the truth
and handing out significance that isn't there - Lopez de Prado ch. 4
arriving through the back door of the significance test. Now permutes
contiguous BLOCKS of at least one horizon, so the null keeps the
autocorrelation and the p-value means what it says. It degrades honestly:
severe overlap leaves few blocks, the null widens, nothing reaches
significance. The block count is now printed, because THAT - not the row
count - is the sample size a p-value rests on, and a warning fires under
30 blocks so "not significant" is not misread as "no signal" when it
means "not enough independent history to tell".
2. THE POSITIVE CONTROL'S STRENGTH DEPENDED ON THE DATASET. It paired
each row's label with the NEXT SAMPLE ROW's, whose distance is the
stride - so on M5, where stride ran 160-717 bars against a 128-bar
horizon, it was pairing two windows that never overlap. All three M5
cells duly reported a FAILED estimator and voided their own results with
nothing wrong. A control whose strength varies with the cell cannot
certify the cell. Now pinned to a quarter of the horizon, where ~75%
overlap is guaranteed by construction.
3. THE LOOKAHEAD VERDICT HAD NO MARGIN. It flagged 7 of 12 cells on gaps
of 0.00008-0.00040 nats against a measured null sd of ~0.00030 - noise,
every one. Now requires 3 sd, the same discipline the deploy floor
applies to precision.
Compiles 0 errors / 0 warnings, standard and Market. Build tag
blockperm-v1. Supersedes every number from the sweep.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-01 15:11:40 -04:00
|
|
|
m_miNullBlocks(0),
|
feat(labels): measure which barrier is predictable at entry, don't guess
The alignment scan settled the shape of the problem: 4.7x more is
knowable 5 bars into a 128-bar window than at the entry the model
actually trades. A 6xATR target reached over 128 bars is decided
overwhelmingly by what happens DURING the window, so whatever the entry
state knows is buried under 128 bars of later noise. That is a property
of the TARGET, and it is why four different architectures all landed on
precision exactly equal to the base rate - no topology can undo it.
So measure the target. For each SL/TP pairing a user can actually select,
relabel the same sampled bars and score how much the SAME features say
about THAT outcome at entry. Seconds, no training, no topology, and it
runs on the diagnostic path that already exists.
Ranked on excess over its OWN null as a share of its OWN H(Y), never on
raw nats: each geometry has a different class balance, hence a different
finite-sample bias and a different amount of information there to find,
so raw MI would rank the most BALANCED label rather than the most
PREDICTABLE one. The break-even win rate m/(m+k) is printed beside each
so the ranking is read next to the bar the model must clear.
Stated in the output because it is the easy thing to get wrong: chance
precision EQUALS break-even at every geometry, so a tighter target does
not hand you expectancy. It buys predictability - less noise piled on top
of what the entry state knows - which is the one thing changing topology
cannot do.
Read-only by construction: it relabels a sampled copy via
TripleBarrierLabel(), never writes the label cache (which belongs to the
configured geometry), and restores the horizon and overrides it borrowed.
The overrides apply only when BOTH are positive, so a half-set pair can
never silently relabel a live run.
Compiles 0 errors / 0 warnings, standard and Market. Build tag
geometry-scan-v1. Redeploy only - no retrain to READ the ranking.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-01 14:32:04 -04:00
|
|
|
m_miReportDone(false),
|
feat(rank): AI models rank their own confidence tiers from held-out outcomes
Closes the caveat 4858507 shipped with: the vote is a confidence percentage,
but only to the extent the pattern weights are measured. AI tier weights sat
at their designed defaults (25/50/75/100) because AI rows only ever arrive
from LIVE journaling, of which a training run produces almost none.
AND A STALE-TIER BUG THAT MADE THE EVIDENCE MEANINGLESS. The OOS scan bucketed
every scanned bar by ConfidenceTier(), which reads dPrevSignal - and
dPrevSignal is assigned in PASS 1 only, never anywhere in the OOS scan. So an
entire era's fires were bucketed by one stale, unrelated bar's confidence and
landed in a SINGLE tier. That is the "tier prec T0:72%(828) T1:n/a(0)
T2:n/a(0) T3:n/a(0)" symptom recorded on 2026-08-16 and attributed to the
calibration clamp. The clamp was real and was fixed then; this is a second,
independent cause of the identical output that survived that fix untouched -
which is why the log kept reading the same afterwards. Two causes, one symptom.
Now ConfidenceTierFor(adjSig): the bar this iteration actually scored.
WHY THIS DOES NOT WRITE ROWS TO THE SIGNAL DB, which was the obvious reading of
"fill the database during training". The user's own observation is the reason:
a classic Pattern_2 is a fixed geometric condition, so its win rate is
legitimately accumulated over years, but an AI Pattern_2 means "confidence
landed in tier 2" and tier 2 under era 100's weights is a different statement
from tier 2 under era 500's. The DB's value is ACCUMULATION, and accumulation
is exactly what is wrong here - it would average together models that no
longer exist, while colliding with the per-table row cap and mixing
measured-on-holdout outcomes into the live ledger's own tables. What the DB
actually supplies is a measured win rate per pattern, and pass 3 already
computes that on held-out bars, thousands at a time. So the model ranks itself
once per era, REPLACING rather than accumulating, which makes the weights
describe the current weights by construction.
ESTIMATOR. Not WinRateFromCounts(): it returns NO_DATA below 100 raw trades
BEFORE shrinking, which here would fire on every tier every era and hand all
four the pooled rate - the tiers could never separate and the mechanism would
be inert. Shrinkage is the answer to a small sample; a floor in front of it
means the shrinkage never runs. Instead: a Beta prior of TIER_PRIOR_EFF_N
pseudo-observations centred on the model's pooled holdout rate, counted in
EFFECTIVE observations, because overlapping triple-barrier labels mean 800 raw
fires can be worth ~12 independent ones. Rounded to the integer, not to the
decade NormalizeWinRate() uses, which would collapse the shrunk tiers back
into one number.
NO SAME-ERA CIRCULARITY, and it falls out of the ordering rather than a guard:
weights are computed at the END of era N, so the vote scored during era N was
cast with era N-1's weights. The deploy gate never grades a vote whose weights
were fitted on the bars it is scoring. Residual leakage remains - the same OOS
bars each era under a different model - and is stated in the code rather than
papered over.
Both DB clobber paths are closed: ApplyPatternWeight() declines once
self-ranked, and UpdateSignalsWeights()' filter.Weight() call is guarded by
SelfRanked() - guarding only the tiers would have let the hourly ranking pass
undo half the self-ranking.
NOT COMPILED - user compiles in MetaEditor.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-18 16:00:32 -04:00
|
|
|
m_tiersSelfRanked(false),
|
fix(chart): display now reads era-end SNAPSHOTS - the live cache is wiped mid-era
Full-pipeline analysis after "threshold 30, attained often, nothing drawn,
still glued to buy". The log falsified the premise before any code did:
21:40:43 swept 4999, 794 voters, drew 491. Strongest 43.0% vs 30.0%
21:42:07 swept 4999, 0 voters, drew 0
21:51:30 swept 4999, 0 voters, drew 0
21:56:30 swept 4999, 922 voters, drew 382. Strongest 44.0% vs 30.0%
The arrows WERE drawn - 491 of them, then 382 - and then erased. ONE root
cause, three symptoms: every display path read m_arrowSignalCache, which is
wiped to sentinel at each era start and only complete again when pass 3
finishes. With eras at ~30s and a sweep at ~17s:
* ARROW FLICKER: a sweep landing mid-era found no voters anywhere, and its
else-branch deleted the arrow on every voteless bar - erasing the previous
sweep's entire output. The chart cycled populated -> blank -> populated;
the user kept catching the blank phase.
* READOUT GLUE: the newest-cache walk found only sentinel for ~90% of every
era and fell through to dPrevSignal - the frozen purge-band edge bar that
reads Buy. 659638e fixed which bar was frozen, not the freezing.
* VOTER FLAP: 1299 -> 257 -> 1113 across back-to-back sweeps - each saw a
different fraction of half-rebuilt caches.
THE FIX, structural rather than another patch:
1. Era-end snapshots. RankTiersFromOos() runs at pass-3 completion - the one
moment the cache is complete - and now copies it (raw signals, newest
LOOKBACK+16 bars) into member-owned snapshot state, unconditionally,
BEFORE its early return: an all-Neutral era is a snapshot worth showing,
not an absence of one. Raw signals rather than votes, so a tier re-rank
between eras reprices them at read time via LiveVoteContribution for free.
2. The sweep (SnapshotVoteAt) and the prospective readout both read
snapshots; the readout's fallback chain is live-cache -> snapshot ->
dPrevSignal, and the snapshot leg is the one that fires most of the time.
3. NO DATA IS NOT A VERDICT: a den==0 bar no longer deletes - only an actual
sub-threshold vote takes an arrow down. This alone ends the wipe half of
the flicker even where snapshots are missing (before the first era).
4. Arming moved from an era-counter diff (which fires at era BOUNDARIES,
i.e. precisely when caches are about to be wiped) to
g_warriorOverlayArmRequest, set by each RankTiersFromOos - "a member's
snapshot just got fresher", the only event a redraw can act on. 60s rate
limit collapses the four members' burst into one sweep. Classic-only
charts arm once at start.
5. Census now reports the direction split - "922 had a voter (610 buy / 312
sell)" - so "the vote leans buy" is checkable from the log instead of
inferred from arrow colours.
Also visible in the log and worth knowing: the threshold flip-flopped
30 -> 40 -> 30 across the evening's re-inits (census lines at 21:42-21:51
ran at 40), so part of the observed blankness was configuration, not code.
NOT COMPILED - user compiles in MetaEditor.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-18 22:03:20 -04:00
|
|
|
m_overlaySnapBars(0),
|
|
|
|
|
m_prospectiveSigSnap(-2.0),
|
feat(hud): per-member neuron lines + a vote label that moves as the nets learn
Both 2026-08-19 reports were the same staleness: every source behind the
label was an ERA artifact (live cache refills at pass-3 completion, the
snapshot copies once per era, dPrevSignal is the frozen purge-band edge
bar) - so the readout stepped at era cadence at best, stayed glued to
one direction, and lagged the era counter.
DisplayInference(): throttled (4s, 1s across an era boundary),
SIDE-EFFECT-FREE forward of the current decision bar (window ending on
bar 1, same question the live path asks) through the LEARNER net.
Batch-norm running stats are bracketed frozen/RESTORED via the new
CNet::GetBatchNormFrozen() + CNeuronBatchNormOCL::StatsFrozen() - restore,
not unfreeze, because a display tick can land between pass-3 chunks whose
whole scan holds them frozen. Writes nothing a trading or training path
reads (dPrevSignal, NMS state, tallies, watermarks all untouched;
RefreshLatestSignal is not reusable here precisely because it writes all
of them). LSTM safe by construction: h/c zeroed per forward.
ProspectiveVote() reads the fresh forward as its FIRST source; the
era-artifact chain becomes the fallback (meta head, warm-up, window
holes).
DisplayHudLine(): the reference library's training label, per ensemble
member - name, output activations (softmax probs or raw scalar), the
decision, its weighted vote (the exact consensus numerator term), era,
recent average error, "(trn)" while not vote-capable. Rendered under the
vote line in RefreshVoteReadout BEFORE the live-vote defer (member lines
are telemetry, not tradable readings), coloured by the member's own
direction in muted tones - the vote line's strict
green-only-when-it-would-trade rule is untouched.
NOT COMPILED - user compiles in MetaEditor.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-19 08:48:34 -04:00
|
|
|
m_dispSignal(0.0),
|
|
|
|
|
m_dispValid(false),
|
|
|
|
|
m_dispStamp(0),
|
|
|
|
|
m_dispEra(-1),
|
feat(logs): throttle the settled per-era diagnostics - measured 22MB/9.5h of confirmed-working systems
Measured from the journal (2026-08-19): the era deep-dive line (~2KB) plus
the excursion verdict, tier re-rank, calibration move, barrier hold and
selection-regressed note each printed EVERY era for EVERY member - ~940
eras/member/day - long after the systems they watch were confirmed
working. Yesterday's file was 1.3GB (70% of it the news-filter calendar
spam the sweep fix already removed).
VerboseMode returns as an INPUT (demoted 2026-08-01 for the marketplace;
that track is dead since the 2026-08-16 pivot) and gains a second job:
false throttles each settled per-era print to eras 0-3 plus every
TRAIN_LOG_EVERY_ERAS-th (25 ~= one deep-dive per ~15min per member);
true restores the per-era firehose, flippable live.
Never throttled: anything that marks a CHANGE - new bests, restores +
eta decays, plateau stage transitions, deploy approvals, warnings,
errors, the label-cache/adoption one-shots, and the combined-vote gate
line (the active system's primary telemetry, still every era).
Semantic fixes over blanket gating:
- barrier hold now ARMS silently and prints only when the hold outlasts
the 2-min report interval - a brief hold every era is the design, the
long hold is the watchdog case the line exists for;
- the ensemble deploy REFUSAL prints immediately when its reason
changes (that is a finding), on cadence when unchanged;
- the filtered-view census prints when its RESULT moves (drawn count,
or strongest vote by >=2pp) and at least every 10th sweep.
NOT COMPILED - user compiles in MetaEditor.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-19 09:36:15 -04:00
|
|
|
m_lastEnsRefusalKey(0),
|
2026-08-02 12:25:20 -04:00
|
|
|
m_miReportDeferrals(0),
|
feat(labels): measure which barrier is predictable at entry, don't guess
The alignment scan settled the shape of the problem: 4.7x more is
knowable 5 bars into a 128-bar window than at the entry the model
actually trades. A 6xATR target reached over 128 bars is decided
overwhelmingly by what happens DURING the window, so whatever the entry
state knows is buried under 128 bars of later noise. That is a property
of the TARGET, and it is why four different architectures all landed on
precision exactly equal to the base rate - no topology can undo it.
So measure the target. For each SL/TP pairing a user can actually select,
relabel the same sampled bars and score how much the SAME features say
about THAT outcome at entry. Seconds, no training, no topology, and it
runs on the diagnostic path that already exists.
Ranked on excess over its OWN null as a share of its OWN H(Y), never on
raw nats: each geometry has a different class balance, hence a different
finite-sample bias and a different amount of information there to find,
so raw MI would rank the most BALANCED label rather than the most
PREDICTABLE one. The break-even win rate m/(m+k) is printed beside each
so the ranking is read next to the bar the model must clear.
Stated in the output because it is the easy thing to get wrong: chance
precision EQUALS break-even at every geometry, so a tighter target does
not hand you expectancy. It buys predictability - less noise piled on top
of what the entry state knows - which is the one thing changing topology
cannot do.
Read-only by construction: it relabels a sampled copy via
TripleBarrierLabel(), never writes the label cache (which belongs to the
configured geometry), and restores the horizon and overrides it borrowed.
The overrides apply only when BOTH are positive, so a half-set pair can
never silently relabel a live run.
Compiles 0 errors / 0 warnings, standard and Market. Build tag
geometry-scan-v1. Redeploy only - no retrain to READ the ranking.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-01 14:32:04 -04:00
|
|
|
m_barrierScanSlMult(0.0),
|
|
|
|
|
m_barrierScanTpMult(0.0),
|
fix(labels): the geometry scan rewarded the labels it should reject
First run named 3:10 on all four charts, at 2.3x the configured 2:6. That
answer was wrong and the fault was the ranking statistic.
3:10 wants a horizon of ~swingMedian*30 (~320 bars) and gets
BARRIER_HORIZON_MAX. Clamped, most trades never resolve, the unresolved
remainder all lands in Neutral, and H(Y) collapses. The old statistic
divided the excess BY H(Y) - so a collapsing denominator made the most
degenerate label look like the most predictable one. Every geometry from
2:6 upward was already showing the clamped h128, and the two widest
scored highest, which is the fingerprint of the artefact rather than of
signal.
Two fixes:
Rank on the raw excess in nats. Subtracting each geometry's OWN measured
null already removes the class-balance bias, which is the only thing the
normalisation was ever needed for.
Disqualify clamped geometries outright rather than ranking them down. The
deployed EA holds until SL or TP with no bar limit, so a truncated label
trains the model on a question the strategy never asks. They are still
printed, marked '!', so the disqualification is visible instead of a
silent omission - and the scan now says so explicitly when nothing
eligible is left, because "the limit is the feature set, not the target"
is itself the finding in that case.
The scan also reports each geometry's directional share and timeout share
now. A label nobody can trade is not a candidate however well it scores,
and that has to be visible in the same line as the score.
Compiles 0 errors / 0 warnings. Build tag geometry-scan-v2.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-01 14:46:25 -04:00
|
|
|
m_barrierScanLiveLabels(false),
|
|
|
|
|
m_barrierScanTimeouts(0),
|
|
|
|
|
m_barrierHorizonClamped(false)
|
2026-08-01 11:27:28 -04:00
|
|
|
{
|
fix(ensemble): responsive panel + synchronized eras + combined-vote accuracy
Four user-reported/requested items, one root cause chain:
1) DEAD CONTROL PANEL in AI_HYBRID mode. All members posted custom event
id 1 and handled id 1001, and CExpertCustom broadcasts every chart
event to every filter - so each posted event ran a train chunk in ALL
N members (N*N chunks per round) and the chart thread never idled
long enough to deliver clicks/drags. profiling.csv: 99.45% of time in
OnChartEventHandler. Fix: per-instance study-event ids
(STUDY_EVENT_ID_BASE + construction order, offset above the Controls
library's ON_* codes - id 1 was also ON_DBL_CLICK, so panel
double-clicks fired training chunks). ArmStudyEvent() is the single
post site; lost-event watchdog replaces the accidental
sibling-clears-my-flag rescue.
2) WARM-UP DUPLICATION. The auto-tune sweep is deterministic over
identical features/labels, and it ends in the full MI diagnostic
suite, which the MI-share gate never intercepted on the sweep path -
four members ran four identical ~36s sweep+report blocks. First
member publishes outcome (g_ensembleChartTuneDone/Installed/Settings);
the rest apply it and skip both.
3) DEINIT STRANDED PANEL+ARROWS (user repro 18:52). Root cause from the
log: the 4,500ms budget runs from MetaTrader's stop REQUEST - a heavy
autosave in flight ate it, OnDeinit got ~430ms and died in the first
member's arrow persist ("Abnormal termination" 432ms in). Fix: early
visible-UI sweep (native prefix deletes for status/panel/dialog)
right after ClearStatusLabel, and a fast path for still-training
models - their arrows are re-rendered every era, so they get one bulk
purge instead of scan+atomic-write in the death window.
4) ENSEMBLE FEATURES (user requests): era BARRIER - members advance era
by era together; a member ahead of the slowest still-training member
declines Train() calls and its chunk budget is donated
(TRAIN_TIME_BUDGET_MS = 120/activeTrainers, UI headroom constant).
COMBINED-VOTE OOS SCORE - each member's pass-3 scan contributes its
adjusted per-bar decision (0.0 on abstain) to a shared row buffer;
the last member to finish the era scores the averaged vote vs the
mirrored Min_Vote_Open against the same target-before-stop outcomes
members grade themselves on, publishing an "Ensemble vote" line on
the aggregated panel. Member headlines now carry their lifetime win
rate with break-even.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-16 19:06:04 -04:00
|
|
|
//--- Claim this instance's study-event id - see STUDY_EVENT_ID_BASE (ExpertSignalAIBase.mqh) for why
|
|
|
|
|
//--- these are per instance and offset above the Controls library's event codes.
|
|
|
|
|
m_studyEventId = (ushort)(STUDY_EVENT_ID_BASE + g_warriorStudyEventSeq++);
|
|
|
|
|
m_studyArmedTick = 0;
|
|
|
|
|
m_ensembleIndex = -1; // not an ensemble member until EnsembleMember(true) registers one (the flag itself is in the init list)
|
feat(ensemble): deploy gate on the COMBINED VOTE, with a joint checkpoint
The unit of evaluation in ensemble mode becomes the vote, because the
vote is what trades (user: "at the end of the day they will vote
together during live trading so that would make sense").
Four decisions move from the member to the ensemble:
* which era is "best" -> the era whose COMBINED VOTE scored best
* what is checkpointed -> a JOINT snapshot: every member's weights
at that one era
* when the run gives up -> one shared plateau ladder
* whether it may deploy -> family-wise gate on the vote
WHY THE JOINT CHECKPOINT IS THE POINT: per-member selection picks each
net's own best era, and those eras differ. The resulting quartet was
never measured together at any instant, so the vote it casts live is a
configuration no OOS number ever described. Capturing all four at the
era whose vote won makes the deployed ensemble exactly the measured one.
Correct because of the era barrier (b77e7b4): Train() runs at most one
era per call and a member that finished era N is held until every member
reaches N, so when the last member scores the vote no member's weights
have advanced past end-of-era-N. That makes the deferred simultaneous
capture a guarantee rather than a race. Each snapshot is era-STAMPED and
deploy requires every stamp to equal the winning era - otherwise a member
whose capture failed would still hold an older snapshot and the deployed
quartet would again be one nothing measured. Partial capture rolls the
era back out of "best" so the search continues instead of freezing
behind a checkpoint that does not exist.
Statistics mirror the per-member gate one for one - same coverage floor
(MIN_COVERAGE_FRACTION_OF_BASE_RATE), same always-call-one-direction
chance reference, same EDGE_MIN_SIGMAS margin, same Sidak correction over
the eras ranked (DEPLOY_FAMILY_WISE_ALPHA). Only the population differs:
the bars the VOTE fired on, at Min_Vote_Open, rather than the bars one
member called. Two-sidedness is required of the vote itself - a vote that
never goes short IS the always-long model the chance reference prices in.
Members keep their own per-era statistics and their own learning-rate
dynamics (regression restore, eta decay); those are per-net training
mechanics, not deployment decisions. The shared ladder is mirrored onto
each member so per-era log lines report the state that actually governs
them. Solo charts are untouched on every path.
Verified: full MetaEditor compile, 0 errors 0 warnings.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-16 19:25:25 -04:00
|
|
|
//--- per-era stash the ensemble verdict reads (see EnsembleStashEraStats); -1/false = "no era yet"
|
|
|
|
|
m_eraStatPrecPct = -1.0;
|
|
|
|
|
m_eraStatChancePct = -1.0;
|
|
|
|
|
m_eraStatCalls = 0;
|
|
|
|
|
m_eraStatTradeable = false;
|
|
|
|
|
m_eraStatTwoSided = false;
|
|
|
|
|
m_eraStatScore = 0.0;
|
|
|
|
|
m_eraStatBlended = 0.0;
|
|
|
|
|
m_eraStatThreshold = 0.0;
|
|
|
|
|
m_checkpointEra = -1;
|
2026-08-01 11:27:28 -04:00
|
|
|
//--- indicator tuning defaults live in CADIndicatorTuner's own constructor (Expert\ADIndicatorTuner.mqh),
|
|
|
|
|
//--- which runs automatically for the m_indicatorTuner member above.
|
|
|
|
|
}
|
|
|
|
|
//+------------------------------------------------------------------+
|
|
|
|
|
//| Destructor |
|
|
|
|
|
//+------------------------------------------------------------------+
|
|
|
|
|
CExpertSignalAIBase::~CExpertSignalAIBase(void)
|
|
|
|
|
{
|
|
|
|
|
//--- deliberately NOT calling PersistOnShutdown() here: OnDeinit() (Warrior_EA.mq5) already calls
|
|
|
|
|
//--- it explicitly for every signal, one call stack frame shallower, BEFORE Expert.Deinit() tears
|
|
|
|
|
//--- these objects down. Doing it again here nested inside that same teardown cascade doubled the
|
|
|
|
|
//--- stack depth of an already-deep recursive save (layers -> neurons -> connections) right at the
|
|
|
|
|
//--- point in MT5's lifecycle (EA recompile while attached) that has the least stack headroom, and
|
|
|
|
|
//--- reliably crashed the terminal with a stack overflow. Keep this destructor cheap.
|
|
|
|
|
if(CheckPointer(Net) != POINTER_INVALID)
|
|
|
|
|
delete Net;
|
|
|
|
|
if(CheckPointer(m_shadowNet) != POINTER_INVALID)
|
|
|
|
|
delete m_shadowNet;
|
|
|
|
|
if(CheckPointer(TempData) != POINTER_INVALID)
|
|
|
|
|
delete TempData;
|
|
|
|
|
if(CheckPointer(m_simOosNet) != POINTER_INVALID)
|
|
|
|
|
delete m_simOosNet;
|
feat: excursion-size head (Stage 1, measurement only)
Direction is closed - normalised asymmetry fails on three instruments
with a working positive control, and the classifier's own best-of-999
era-cap test agrees (+0.9pp = 1.48 sigma, family-wise p=1.0000). SIZE is
a different question and RANGE clears at ~4x its null.
Checked the denomination before building on that, since the source memo
warns to: m_excUpCache holds (maxHigh - fill)/ATR, so "RANGE is
predictable" is a claim about travel RELATIVE to current ATR, not a
restatement of "ATR is autocorrelated". It is exactly the part a fixed
multiple (stop 3.31*ATR, target 1.64*ATR) discards.
A second small CNet, 760 -> 24 -> 32 sigmoid outputs = P(price reaches
ladder rung k) upward and downward. Survival parameterisation rather than
regressing the multiple, because it needs nothing new from CNet: sigmoid
outputs and the per-neuron delta the `total != 3` branch already applies
(a quantile head would need a linear activation and a pinball gradient in
Network.mqh, Network.cl and the DirectML path, on a class four topologies
share). Targets are free - m_ladderUpAt already records first-touch age
per rung with 0 meaning never reached.
Separate net, not extra outputs on the classifier: more outputs would
change m_outputNeuronsCount, the .nnw shape and the fingerprint, and push
the count off 3 - the exact condition backProp uses to select the joint
softmax gradient the 3-class head depends on. The classifier is
bit-for-bit unaffected and this is removable without trace.
STAGE 1 PLACES NO ORDERS. It reports a Brier skill score against the
constant per-rung base rate - the baseline a fixed ATR multiple already
assumes - with both predictors fitted IS and evaluated OOS, so neither
gets a look at the test set. Positive skill justifies Stage 2 (drive
SL/TP and sizing off ExcursionQuantile, which is defined and deliberately
uncalled). Zero or negative means ATR already carries everything and
Stage 2 must not be built.
Trains only on primary occurrences: the replay queue oversamples for
CLASS balance, and a direction-balanced sample is a biased SIZE sample.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-11 07:40:01 -04:00
|
|
|
//--- Excursion head: never persisted (Stage 1 is a measurement), so teardown is the whole lifecycle.
|
|
|
|
|
if(CheckPointer(m_excNet) != POINTER_INVALID)
|
|
|
|
|
delete m_excNet;
|
|
|
|
|
if(CheckPointer(m_excTgt) != POINTER_INVALID)
|
|
|
|
|
delete m_excTgt;
|
|
|
|
|
if(CheckPointer(m_excOut) != POINTER_INVALID)
|
|
|
|
|
delete m_excOut;
|
fix(chart): arrows survived the EA that drew them - persist, then clear
Reported: on deinit the panel and status label go, the signal arrows stay.
Two independent causes, both fixed here.
1. It was partly deliberate. ShutdownChartCleanup carried a second
behaviour selected by a `preserveChartArrows` flag derived from the
deinit reason: on RECOMPILE / PARAMETERS / CHARTCHANGE / TEMPLATE the
arrows were left on the chart on purpose, to avoid a reload flicker.
That branch IS the reported symptom, an operator cannot tell it apart
from a cleanup that failed, and it was outright wrong whenever the
reload changed the config - REASON_PARAMETERS means exactly that, and
the preserved arrows then belonged to a model the chart no longer
runs, with nothing marking them stale. It is gone, along with the flag
and m_purgeChartOnDestruct. One path now: persist, clear, restore on
the next attach.
2. Whatever remains was unfalsifiable. PurgeChart was a single
ObjectsDeleteAll(prefix) whose return value was discarded, with no
caller ever looking at the chart again - so "the arrows are still
there" and "the arrows were never there" produced identical evidence,
which is why the report survived three sessions. It now verifies:
after the bulk delete it walks the OBJ_ARROW-typed list (a handful of
objects, not the whole chart), deletes any surviving WarSig_ by name,
and says so. Costs one typed scan when the bulk delete works, which is
the normal case; names the root cause when it does not.
Every failure mode of SaveChartSignals was also silent - it returned void
and had three bare early returns. It returns bool now, logs the open
error with the filename, and the shutdown purge is CONDITIONAL on it: for
a converged model the chart objects are the only copy of its signal
history (nothing redraws them - the renderer runs per training era and a
deployed model has none left), so a chart left littered because the disk
write failed beats a clean chart bought by destroying the history. Either
way the log now says which happened.
Also states the user's rule once, where arrows come back rather than
across InitNeuralNetwork's several exits: no weights loaded for this
config => clear the sidecar and start visually clean. A fresh run must
not inherit calls it never made, and the first save would otherwise adopt
them (the sidecar is rebuilt by scanning the chart).
Compiles 0 errors / 0 warnings, standard and Market. Needs redeploy.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-01 13:28:34 -04:00
|
|
|
//--- Unconditional now that the warm-reload "leave the arrows up" branch is gone (see
|
|
|
|
|
//--- ShutdownChartCleanup). Cheap and idempotent: OnDeinit already purged, so this normally deletes
|
|
|
|
|
//--- nothing - it exists for the teardown paths that never reach OnDeinit (a failed OnInit).
|
|
|
|
|
PurgeChart();
|
2026-08-01 11:27:28 -04:00
|
|
|
//--- Last, and cheap by design (one global-variable delete): the claim must outlive every save above
|
|
|
|
|
//--- it, or a chart re-attaching during this teardown could start writing the same files mid-save.
|
|
|
|
|
ReleaseConfigLock();
|
|
|
|
|
}
|
|
|
|
|
//+------------------------------------------------------------------+
|
|
|
|
|
//| Sets the file/id identity a subclass constructor would otherwise |
|
|
|
|
|
//| repeat verbatim (ID, m_id, m_folderPath, m_fileName, pattern count)|
|
|
|
|
|
//+------------------------------------------------------------------+
|
|
|
|
|
void CExpertSignalAIBase::SetIdentity(string id, string shortId, int patternCount = 4)
|
|
|
|
|
{
|
|
|
|
|
ID = id;
|
|
|
|
|
m_id = shortId;
|
|
|
|
|
m_folderPath = eaName + "\\" + "Neural Networks" + "\\" + "State" + "\\" + m_id + "\\";
|
|
|
|
|
m_fileName = m_folderPath + _Symbol + "_" + IntegerToString(_Period);
|
|
|
|
|
m_pattern_count = patternCount;
|
|
|
|
|
}
|
|
|
|
|
//+------------------------------------------------------------------+
|
|
|
|
|
//| "Voting" that price will grow. |
|
|
|
|
|
//+------------------------------------------------------------------+
|
|
|
|
|
int CExpertSignalAIBase::LongCondition(void)
|
|
|
|
|
{
|
|
|
|
|
int result = 0;
|
|
|
|
|
//--- Readiness gate: live trading still requires a converged model, but an inference-only tester run
|
|
|
|
|
//--- may replay a model that was ACTUALLY loaded from disk even if its persisted trainingComplete flag
|
|
|
|
|
//--- is false. Without that exception the tester could seed/refresh dPrevSignal from the deployed model
|
|
|
|
|
//--- and draw chart arrows from those weights, yet this gate would still hard-zero the trading vote.
|
|
|
|
|
//--- A fresh random topology still cannot trade in the tester because m_modelLoadedFromDisk stays false.
|
|
|
|
|
//--- Census: this gate is invisible to the refresh-path counters and is a live candidate for the
|
|
|
|
|
//--- all-bars-zero-direction backtest - see m_voteGateBlocked.
|
|
|
|
|
NoteVoteGate(DoubleToSignal(dPrevSignal) == Buy);
|
|
|
|
|
if(!m_trainingComplete && !(m_inferenceOnly && m_modelLoadedFromDisk))
|
|
|
|
|
return 0;
|
|
|
|
|
//--- No alternation gate any more - see the removal note at m_voteGateBlocked's declaration. Under
|
|
|
|
|
//--- triple-barrier labels consecutive same-direction setups are ordinary and correct.
|
|
|
|
|
//--- "not yet studied" sentinel - dPrevSignal == -2 is not a real Sell. Its MAGNITUDE is 2, so it
|
|
|
|
|
//--- passed straight through the confidence floor that used to sit here (|-2| exceeds any 0..1
|
|
|
|
|
//--- threshold); only the m_trainingComplete gate above was keeping it out. Checked explicitly now
|
|
|
|
|
//--- that the floor is gone, rather than left resting on that.
|
|
|
|
|
if(dPrevSignal == -2)
|
|
|
|
|
return 0;
|
|
|
|
|
//--- NO confidence floor here, by design - see m_minSignalConfidence's former declaration site. A
|
|
|
|
|
//--- weak call is not blocked at the AI's own boundary; it votes at its tier weight (as low as
|
|
|
|
|
//--- m_pattern_0) and is then filtered by Min vote to open, exactly like a weak classic vote.
|
|
|
|
|
if(DoubleToSignal(dPrevSignal) == Buy)
|
|
|
|
|
{
|
|
|
|
|
int tier = ConfidenceTier();
|
|
|
|
|
result = PatternWeightForTier(tier);
|
|
|
|
|
m_active_pattern = "Pattern_" + IntegerToString(tier);
|
|
|
|
|
m_active_direction = "Buy";
|
|
|
|
|
}
|
|
|
|
|
return(result);
|
|
|
|
|
}
|
|
|
|
|
//+------------------------------------------------------------------+
|
|
|
|
|
//| "Voting" that price will fall. |
|
|
|
|
|
//+------------------------------------------------------------------+
|
|
|
|
|
int CExpertSignalAIBase::ShortCondition(void)
|
|
|
|
|
{
|
|
|
|
|
int result = 0;
|
|
|
|
|
//--- Readiness gate - see LongCondition's matching comment.
|
|
|
|
|
NoteVoteGate(DoubleToSignal(dPrevSignal) == Sell);
|
|
|
|
|
if(!m_trainingComplete && !(m_inferenceOnly && m_modelLoadedFromDisk))
|
|
|
|
|
return 0;
|
|
|
|
|
//--- "not yet studied" sentinel, and no confidence floor - see LongCondition's matching comments.
|
|
|
|
|
if(dPrevSignal == -2)
|
|
|
|
|
return 0;
|
|
|
|
|
if(DoubleToSignal(dPrevSignal) == Sell)
|
|
|
|
|
{
|
|
|
|
|
int tier = ConfidenceTier();
|
|
|
|
|
result = PatternWeightForTier(tier);
|
|
|
|
|
m_active_pattern = "Pattern_" + IntegerToString(tier);
|
|
|
|
|
m_active_direction = "Sell";
|
|
|
|
|
}
|
|
|
|
|
return result;
|
|
|
|
|
}
|
|
|
|
|
//+------------------------------------------------------------------+
|
fix(gate): the ensemble gate certified a vote the EA never casts
g_ensembleVoteThreshold's comment claims the combined-vote scorer "fires on
the same criterion the live trade does". It did not. Two independent
mismatches, both silent:
CURRENCY. Each member contributed its raw signed confidence x100 - a 33..100
number straight off the softmax head. Live contributes m_weight x the tier's
pattern weight, and BOTH of those are rewritten from the signal DB by
UpdateSignalsWeights(). A head output and a DB-ranked win-rate weight share
an axis and nothing relates them, so the same bar was one number to the gate
and a different one to the order path. Same shape as the 2026-08-09 geometry
incident: certified on one game, paid on another.
DENOMINATOR. The gate divided by the member count, so an abstaining member
pulled the average toward zero. CExpertSignalCustom::Direction() skips a zero
contribution in BOTH the sum and the count (`if(direction == 0) continue;`
before `number++`) - live is a mean over VOTERS. The gate was therefore
scoring a strictly more agreement-heavy set of bars than the EA trades. The
contribution hook's own comment asserted the opposite ("abstentions dilute
the average exactly as they do in the live vote"), while the AI_CHOICE enum
20 lines away correctly documented union semantics.
LiveVoteContribution() is now the single definition of "what this member
votes", called from the gate; the live path reaches the same arithmetic
through LongCondition/ShortCondition. g_ensVoteVoterMask records who actually
voted, separately from who evaluated the bar, because those are the divisor
and the shared-population test respectively.
ConfidenceTier() is split into ConfidenceTierFor(signal) plus a thin live-bar
wrapper - the OOS scan holds the scanned bar's decision in a local, and
dPrevSignal is a different bar.
NOT changed, deliberately: the gate still does not model live NMS
declustering, and the per-member solo gate still scores every directional
call rather than threshold-clearing ones. Both are selection-metric changes
and this codebase has twice been bitten by switching one blind.
Also corrects a stale paragraph in m_pattern_0's declaration block quoting
80/87/93/100 as the tier defaults. The constructor is 25/50/75/100 and has
been since the confidence floor and alternation gate were removed; the block
carried both tables at once, and the dead one was quoted back as fact.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-18 15:23:33 -04:00
|
|
|
//| Buckets a confidence magnitude into one of 4 equal bands between |
|
|
|
|
|
//| the head's own structural floor and 1.0 - see m_pattern_0's |
|
2026-08-01 11:27:28 -04:00
|
|
|
//| declaration comment for the resulting tier/weight table. Neither |
|
|
|
|
|
//| head has a configurable floor: the boundary is 1/3 for the 3-class|
|
|
|
|
|
//| softmax and 0.5 for regression (DoubleToSignal's own threshold), |
|
|
|
|
|
//| both arithmetic properties of the head rather than settings. |
|
|
|
|
|
//+------------------------------------------------------------------+
|
fix(gate): the ensemble gate certified a vote the EA never casts
g_ensembleVoteThreshold's comment claims the combined-vote scorer "fires on
the same criterion the live trade does". It did not. Two independent
mismatches, both silent:
CURRENCY. Each member contributed its raw signed confidence x100 - a 33..100
number straight off the softmax head. Live contributes m_weight x the tier's
pattern weight, and BOTH of those are rewritten from the signal DB by
UpdateSignalsWeights(). A head output and a DB-ranked win-rate weight share
an axis and nothing relates them, so the same bar was one number to the gate
and a different one to the order path. Same shape as the 2026-08-09 geometry
incident: certified on one game, paid on another.
DENOMINATOR. The gate divided by the member count, so an abstaining member
pulled the average toward zero. CExpertSignalCustom::Direction() skips a zero
contribution in BOTH the sum and the count (`if(direction == 0) continue;`
before `number++`) - live is a mean over VOTERS. The gate was therefore
scoring a strictly more agreement-heavy set of bars than the EA trades. The
contribution hook's own comment asserted the opposite ("abstentions dilute
the average exactly as they do in the live vote"), while the AI_CHOICE enum
20 lines away correctly documented union semantics.
LiveVoteContribution() is now the single definition of "what this member
votes", called from the gate; the live path reaches the same arithmetic
through LongCondition/ShortCondition. g_ensVoteVoterMask records who actually
voted, separately from who evaluated the bar, because those are the divisor
and the shared-population test respectively.
ConfidenceTier() is split into ConfidenceTierFor(signal) plus a thin live-bar
wrapper - the OOS scan holds the scanned bar's decision in a local, and
dPrevSignal is a different bar.
NOT changed, deliberately: the gate still does not model live NMS
declustering, and the per-member solo gate still scores every directional
call rather than threshold-clearing ones. Both are selection-metric changes
and this codebase has twice been bitten by switching one blind.
Also corrects a stale paragraph in m_pattern_0's declaration block quoting
80/87/93/100 as the tier defaults. The constructor is 25/50/75/100 and has
been since the confidence floor and alternation gate were removed; the block
carried both tables at once, and the dead one was quoted back as fact.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-18 15:23:33 -04:00
|
|
|
int CExpertSignalAIBase::ConfidenceTierFor(const double signal)
|
2026-08-01 11:27:28 -04:00
|
|
|
{
|
|
|
|
|
//--- The head's own STRUCTURAL decision boundary - the lowest confidence magnitude that head can
|
|
|
|
|
//--- possibly report for a directional call - not a user setting:
|
|
|
|
|
//--- - 3-class classification: the winning class of a 3-way softmax is arithmetically >= 1/3, since
|
|
|
|
|
//--- three probabilities summing to 1 cannot all be below it. Nothing can ever be read below this.
|
|
|
|
|
//--- - single-neuron regression: 0.5, DoubleToSignal()'s own decision boundary.
|
|
|
|
|
//--- Quartiling from HERE (rather than from an input, as the classification branch used to) is what
|
|
|
|
|
//--- makes the tier boundaries a fixed property of the model instead of something that silently moves
|
|
|
|
|
//--- whenever the trader adjusts an unrelated vote threshold - and it is what lets the same tier
|
|
|
|
|
//--- weights mean the same thing on both heads. See m_pattern_0's declaration comment for the weights.
|
|
|
|
|
double floorConf = (m_outputNeuronsCount == 3) ? (1.0 / 3.0) : 0.5;
|
|
|
|
|
double span = MathMax(1.0 - floorConf, 0.0001);
|
fix: drop the ranking slice for the calibration band; un-collapse the tiers
NOT COMPILED - user compiles.
(1) THE RANKING SLICE IS GONE. It reserved 20% of the OOS window so the
pattern-DB backfill would read bars the deployed checkpoint was not SELECTED on.
That objection stands; carving a new region to answer it did not. The calibration
band already has every property the slice was buying:
never trained on | never graded by pass 3 (which walks [0, oosCutoff) and so
never reaches it) | never seen by the deploy gate | purged by a full label
horizon on BOTH sides | and larger besides - 1,684 bars vs the ~970 carved
So the backfill now walks [calibLo, calibHi) and pass 3 goes back to grading the
entire OOS window, exactly as before any of this. The gate gets its full sample
back (~10% of a sigma), the split loses a region, and the failure mode found an
hour ago - a reserved region silently blanking ~10 months of chart arrows,
because arrows are only drawn on bars pass 3 grades - becomes impossible.
One impurity, stated in the completion log rather than hidden:
m_dirConfThreshold is FITTED on that band and the walk applies it to decide which
bars fired, so coverage there is mildly optimistic. One scalar under a coverage
floor, against checkpoint selection over hundreds of eras.
This backfill IS the deploy-time warm-up: it runs right after FinalizeTrainRun()
restores the deployed weights, so it scores with exactly what is about to trade.
(2) EVERY CALL WAS TIER 0, AND IT WAS ARITHMETIC. ConfidenceTier() quartiles
[floorConf, 1] where floorConf = 1/3 - the lowest magnitude a 3-way softmax
winner can hold. But it was fed CalibratedConfidenceMagnitude(), which multiplies
by m_confidenceCalScale, clamped to [0.3, 1.5]. That lower clamp is BELOW 1/3.
Whenever calibration bottoms out, t goes negative and MathMax(0, ...) pins every
call to tier 0.
Which is what the live run does. m_confidenceCalScale is EMA'd toward
empiricalAccuracy / avgClaimedConfidence; with the model over-calling Neutral,
3-class agreement sits near 10% against a claimed confidence near 0.9, so the
ratio is ~0.11 and clamps to 0.3 every era. Logged:
tier prec T0:72%(828) T1:n/a(0) T2:n/a(0) T3:n/a(0)
828 calls, one bucket - the four tier weights and the entire per-tier pattern-DB
ranking reduced to a single number. The backfill was feeding a mechanism that
structurally could not rank.
Tiering now reads the RAW head magnitude, which genuinely lives on the
[1/3, 1] range these bounds were written for. Calibration keeps its real jobs -
AIConfidence() for MM sizing and SignedAIConfidence() for the vote are unchanged.
STILL OPEN, deliberately not touched here: the calibration TARGET itself.
empiricalAccuracy is 3-class agreement, which is the wrong quantity to scale a
DIRECTIONAL confidence against - it counts a Neutral class that is 0.19% of
labels. The honest target is the win rate on the calls the confidence describes
(directional precision), with the claimed-confidence average taken over those
same called bars. That needs a new accumulator and it interacts with the Neutral
over-calling being fixed elsewhere, so it wants one clean run first.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-16 23:06:11 -04:00
|
|
|
//--- RAW magnitude, not the calibrated one (fixed 2026-08-16). The bounds above describe the range
|
|
|
|
|
//--- the HEAD can emit - a 3-way softmax winner is arithmetically >= 1/3 - but
|
|
|
|
|
//--- CalibratedConfidenceMagnitude() multiplies by m_confidenceCalScale, which is clamped to
|
|
|
|
|
//--- [0.3, 1.5]. That lower clamp sits BELOW this floor of 1/3, so the moment calibration bottoms
|
|
|
|
|
//--- out the quartiling is fed values it considers impossible: t goes negative, MathFloor takes it
|
|
|
|
|
//--- further negative, and MathMax(0, ...) pins EVERY call to tier 0.
|
|
|
|
|
//---
|
|
|
|
|
//--- Not hypothetical - it is what the live SP500 H4 run does. m_confidenceCalScale is EMA'd toward
|
|
|
|
|
//--- empiricalAccuracy / avgClaimedConfidence (see its update in Training.mqh); with the model
|
|
|
|
|
//--- over-calling Neutral, 3-class agreement accuracy sits near 10% against a claimed confidence
|
|
|
|
|
//--- near 0.9, so the ratio is ~0.11 and clamps to the 0.3 floor every era. Logged result:
|
|
|
|
|
//--- "tier prec T0:72%(828) T1:n/a(0) T2:n/a(0) T3:n/a(0)" - 828 calls, one bucket, and the four
|
|
|
|
|
//--- tier weights plus the whole per-tier pattern-DB ranking reduced to a single number.
|
|
|
|
|
//---
|
|
|
|
|
//--- Tiering wants RELATIVE confidence bucketing, which is exactly what the raw head output gives,
|
|
|
|
|
//--- on precisely the [floorConf, 1] range these bounds were written for. Calibration is still the
|
|
|
|
|
//--- right thing for consumers that need an absolute probability - AIConfidence() for MM sizing and
|
|
|
|
|
//--- SignedAIConfidence() for the vote both keep using it, unchanged.
|
fix(gate): the ensemble gate certified a vote the EA never casts
g_ensembleVoteThreshold's comment claims the combined-vote scorer "fires on
the same criterion the live trade does". It did not. Two independent
mismatches, both silent:
CURRENCY. Each member contributed its raw signed confidence x100 - a 33..100
number straight off the softmax head. Live contributes m_weight x the tier's
pattern weight, and BOTH of those are rewritten from the signal DB by
UpdateSignalsWeights(). A head output and a DB-ranked win-rate weight share
an axis and nothing relates them, so the same bar was one number to the gate
and a different one to the order path. Same shape as the 2026-08-09 geometry
incident: certified on one game, paid on another.
DENOMINATOR. The gate divided by the member count, so an abstaining member
pulled the average toward zero. CExpertSignalCustom::Direction() skips a zero
contribution in BOTH the sum and the count (`if(direction == 0) continue;`
before `number++`) - live is a mean over VOTERS. The gate was therefore
scoring a strictly more agreement-heavy set of bars than the EA trades. The
contribution hook's own comment asserted the opposite ("abstentions dilute
the average exactly as they do in the live vote"), while the AI_CHOICE enum
20 lines away correctly documented union semantics.
LiveVoteContribution() is now the single definition of "what this member
votes", called from the gate; the live path reaches the same arithmetic
through LongCondition/ShortCondition. g_ensVoteVoterMask records who actually
voted, separately from who evaluated the bar, because those are the divisor
and the shared-population test respectively.
ConfidenceTier() is split into ConfidenceTierFor(signal) plus a thin live-bar
wrapper - the OOS scan holds the scanned bar's decision in a local, and
dPrevSignal is a different bar.
NOT changed, deliberately: the gate still does not model live NMS
declustering, and the per-member solo gate still scores every directional
call rather than threshold-clearing ones. Both are selection-metric changes
and this codebase has twice been bitten by switching one blind.
Also corrects a stale paragraph in m_pattern_0's declaration block quoting
80/87/93/100 as the tier defaults. The constructor is 25/50/75/100 and has
been since the confidence floor and alternation gate were removed; the block
carried both tables at once, and the dead one was quoted back as fact.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-18 15:23:33 -04:00
|
|
|
double rawMag = MathAbs(signal);
|
fix: drop the ranking slice for the calibration band; un-collapse the tiers
NOT COMPILED - user compiles.
(1) THE RANKING SLICE IS GONE. It reserved 20% of the OOS window so the
pattern-DB backfill would read bars the deployed checkpoint was not SELECTED on.
That objection stands; carving a new region to answer it did not. The calibration
band already has every property the slice was buying:
never trained on | never graded by pass 3 (which walks [0, oosCutoff) and so
never reaches it) | never seen by the deploy gate | purged by a full label
horizon on BOTH sides | and larger besides - 1,684 bars vs the ~970 carved
So the backfill now walks [calibLo, calibHi) and pass 3 goes back to grading the
entire OOS window, exactly as before any of this. The gate gets its full sample
back (~10% of a sigma), the split loses a region, and the failure mode found an
hour ago - a reserved region silently blanking ~10 months of chart arrows,
because arrows are only drawn on bars pass 3 grades - becomes impossible.
One impurity, stated in the completion log rather than hidden:
m_dirConfThreshold is FITTED on that band and the walk applies it to decide which
bars fired, so coverage there is mildly optimistic. One scalar under a coverage
floor, against checkpoint selection over hundreds of eras.
This backfill IS the deploy-time warm-up: it runs right after FinalizeTrainRun()
restores the deployed weights, so it scores with exactly what is about to trade.
(2) EVERY CALL WAS TIER 0, AND IT WAS ARITHMETIC. ConfidenceTier() quartiles
[floorConf, 1] where floorConf = 1/3 - the lowest magnitude a 3-way softmax
winner can hold. But it was fed CalibratedConfidenceMagnitude(), which multiplies
by m_confidenceCalScale, clamped to [0.3, 1.5]. That lower clamp is BELOW 1/3.
Whenever calibration bottoms out, t goes negative and MathMax(0, ...) pins every
call to tier 0.
Which is what the live run does. m_confidenceCalScale is EMA'd toward
empiricalAccuracy / avgClaimedConfidence; with the model over-calling Neutral,
3-class agreement sits near 10% against a claimed confidence near 0.9, so the
ratio is ~0.11 and clamps to 0.3 every era. Logged:
tier prec T0:72%(828) T1:n/a(0) T2:n/a(0) T3:n/a(0)
828 calls, one bucket - the four tier weights and the entire per-tier pattern-DB
ranking reduced to a single number. The backfill was feeding a mechanism that
structurally could not rank.
Tiering now reads the RAW head magnitude, which genuinely lives on the
[1/3, 1] range these bounds were written for. Calibration keeps its real jobs -
AIConfidence() for MM sizing and SignedAIConfidence() for the vote are unchanged.
STILL OPEN, deliberately not touched here: the calibration TARGET itself.
empiricalAccuracy is 3-class agreement, which is the wrong quantity to scale a
DIRECTIONAL confidence against - it counts a Neutral class that is 0.19% of
labels. The honest target is the win rate on the calls the confidence describes
(directional precision), with the claimed-confidence average taken over those
same called bars. That needs a new accumulator and it interacts with the Neutral
over-calling being fixed elsewhere, so it wants one clean run first.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-16 23:06:11 -04:00
|
|
|
if(!MathIsValidNumber(rawMag))
|
|
|
|
|
return 0;
|
|
|
|
|
double t = (MathMin(1.0, rawMag) - floorConf) / span;
|
2026-08-01 11:27:28 -04:00
|
|
|
int tier = (int)MathFloor(t * 4.0);
|
|
|
|
|
return MathMax(0, MathMin(tier, 3));
|
|
|
|
|
}
|
|
|
|
|
//+------------------------------------------------------------------+
|
fix(gate): the ensemble gate certified a vote the EA never casts
g_ensembleVoteThreshold's comment claims the combined-vote scorer "fires on
the same criterion the live trade does". It did not. Two independent
mismatches, both silent:
CURRENCY. Each member contributed its raw signed confidence x100 - a 33..100
number straight off the softmax head. Live contributes m_weight x the tier's
pattern weight, and BOTH of those are rewritten from the signal DB by
UpdateSignalsWeights(). A head output and a DB-ranked win-rate weight share
an axis and nothing relates them, so the same bar was one number to the gate
and a different one to the order path. Same shape as the 2026-08-09 geometry
incident: certified on one game, paid on another.
DENOMINATOR. The gate divided by the member count, so an abstaining member
pulled the average toward zero. CExpertSignalCustom::Direction() skips a zero
contribution in BOTH the sum and the count (`if(direction == 0) continue;`
before `number++`) - live is a mean over VOTERS. The gate was therefore
scoring a strictly more agreement-heavy set of bars than the EA trades. The
contribution hook's own comment asserted the opposite ("abstentions dilute
the average exactly as they do in the live vote"), while the AI_CHOICE enum
20 lines away correctly documented union semantics.
LiveVoteContribution() is now the single definition of "what this member
votes", called from the gate; the live path reaches the same arithmetic
through LongCondition/ShortCondition. g_ensVoteVoterMask records who actually
voted, separately from who evaluated the bar, because those are the divisor
and the shared-population test respectively.
ConfidenceTier() is split into ConfidenceTierFor(signal) plus a thin live-bar
wrapper - the OOS scan holds the scanned bar's decision in a local, and
dPrevSignal is a different bar.
NOT changed, deliberately: the gate still does not model live NMS
declustering, and the per-member solo gate still scores every directional
call rather than threshold-clearing ones. Both are selection-metric changes
and this codebase has twice been bitten by switching one blind.
Also corrects a stale paragraph in m_pattern_0's declaration block quoting
80/87/93/100 as the tier defaults. The constructor is 25/50/75/100 and has
been since the confidence floor and alternation gate were removed; the block
carried both tables at once, and the dead one was quoted back as fact.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-18 15:23:33 -04:00
|
|
|
//| The live bar's tier - the only caller shape that existed before |
|
|
|
|
|
//| ConfidenceTierFor() was split out, kept so LongCondition()/ |
|
|
|
|
|
//| ShortCondition() read exactly as they did. |
|
|
|
|
|
//+------------------------------------------------------------------+
|
|
|
|
|
int CExpertSignalAIBase::ConfidenceTier(void)
|
|
|
|
|
{
|
|
|
|
|
return ConfidenceTierFor(dPrevSignal);
|
|
|
|
|
}
|
|
|
|
|
//+------------------------------------------------------------------+
|
|
|
|
|
//| The signed vote this member would cast for a given decision, in |
|
|
|
|
|
//| the units CExpertSignalCustom::Direction() sums - see the |
|
|
|
|
|
//| declaration comment for why this is a shared function and not two |
|
|
|
|
|
//| parallel expressions. |
|
|
|
|
|
//| |
|
|
|
|
|
//| Mirrors the live path line for line: |
|
|
|
|
|
//| LongCondition/ShortCondition -> PatternWeightForTier(tier) |
|
|
|
|
|
//| CExpertSignalCustom::Direction() -> m_weight * (long - short) |
|
|
|
|
|
//| Both m_weight and the four tier weights are rewritten from the |
|
|
|
|
|
//| signal DB by UpdateSignalsWeights(), so this number MOVES as |
|
|
|
|
|
//| ranking lands. That is deliberate and it is the point: the gate |
|
|
|
|
|
//| has to score the vote the EA would actually have cast, not a |
|
|
|
|
|
//| frozen idealisation of it. |
|
|
|
|
|
//+------------------------------------------------------------------+
|
|
|
|
|
double CExpertSignalAIBase::LiveVoteContribution(const double signal)
|
|
|
|
|
{
|
|
|
|
|
ENUM_SIGNAL s = DoubleToSignal(signal);
|
|
|
|
|
if(s != Buy && s != Sell)
|
|
|
|
|
return 0.0; // abstention - live drops it from the sum AND the divisor
|
|
|
|
|
double w = (double)PatternWeightForTier(ConfidenceTierFor(signal));
|
|
|
|
|
if(!MathIsValidNumber(w) || w <= 0.0)
|
|
|
|
|
return 0.0; // a pattern ranked to weight 0 contributes nothing and is not a voter
|
|
|
|
|
double contribution = m_weight * w;
|
|
|
|
|
if(!MathIsValidNumber(contribution))
|
|
|
|
|
return 0.0;
|
|
|
|
|
return (s == Buy) ? contribution : -contribution;
|
|
|
|
|
}
|
|
|
|
|
//+------------------------------------------------------------------+
|
2026-08-01 11:27:28 -04:00
|
|
|
//| Returns the given tier's current pattern weight (0-100) |
|
|
|
|
|
//+------------------------------------------------------------------+
|
|
|
|
|
int CExpertSignalAIBase::PatternWeightForTier(int tier)
|
|
|
|
|
{
|
|
|
|
|
switch(tier)
|
|
|
|
|
{
|
|
|
|
|
case 0:
|
|
|
|
|
return m_pattern_0;
|
|
|
|
|
case 1:
|
|
|
|
|
return m_pattern_1;
|
|
|
|
|
case 2:
|
|
|
|
|
return m_pattern_2;
|
|
|
|
|
default:
|
|
|
|
|
return m_pattern_3;
|
|
|
|
|
}
|
|
|
|
|
}
|
|
|
|
|
//+------------------------------------------------------------------+
|
feat(rank): AI models rank their own confidence tiers from held-out outcomes
Closes the caveat 4858507 shipped with: the vote is a confidence percentage,
but only to the extent the pattern weights are measured. AI tier weights sat
at their designed defaults (25/50/75/100) because AI rows only ever arrive
from LIVE journaling, of which a training run produces almost none.
AND A STALE-TIER BUG THAT MADE THE EVIDENCE MEANINGLESS. The OOS scan bucketed
every scanned bar by ConfidenceTier(), which reads dPrevSignal - and
dPrevSignal is assigned in PASS 1 only, never anywhere in the OOS scan. So an
entire era's fires were bucketed by one stale, unrelated bar's confidence and
landed in a SINGLE tier. That is the "tier prec T0:72%(828) T1:n/a(0)
T2:n/a(0) T3:n/a(0)" symptom recorded on 2026-08-16 and attributed to the
calibration clamp. The clamp was real and was fixed then; this is a second,
independent cause of the identical output that survived that fix untouched -
which is why the log kept reading the same afterwards. Two causes, one symptom.
Now ConfidenceTierFor(adjSig): the bar this iteration actually scored.
WHY THIS DOES NOT WRITE ROWS TO THE SIGNAL DB, which was the obvious reading of
"fill the database during training". The user's own observation is the reason:
a classic Pattern_2 is a fixed geometric condition, so its win rate is
legitimately accumulated over years, but an AI Pattern_2 means "confidence
landed in tier 2" and tier 2 under era 100's weights is a different statement
from tier 2 under era 500's. The DB's value is ACCUMULATION, and accumulation
is exactly what is wrong here - it would average together models that no
longer exist, while colliding with the per-table row cap and mixing
measured-on-holdout outcomes into the live ledger's own tables. What the DB
actually supplies is a measured win rate per pattern, and pass 3 already
computes that on held-out bars, thousands at a time. So the model ranks itself
once per era, REPLACING rather than accumulating, which makes the weights
describe the current weights by construction.
ESTIMATOR. Not WinRateFromCounts(): it returns NO_DATA below 100 raw trades
BEFORE shrinking, which here would fire on every tier every era and hand all
four the pooled rate - the tiers could never separate and the mechanism would
be inert. Shrinkage is the answer to a small sample; a floor in front of it
means the shrinkage never runs. Instead: a Beta prior of TIER_PRIOR_EFF_N
pseudo-observations centred on the model's pooled holdout rate, counted in
EFFECTIVE observations, because overlapping triple-barrier labels mean 800 raw
fires can be worth ~12 independent ones. Rounded to the integer, not to the
decade NormalizeWinRate() uses, which would collapse the shrunk tiers back
into one number.
NO SAME-ERA CIRCULARITY, and it falls out of the ordering rather than a guard:
weights are computed at the END of era N, so the vote scored during era N was
cast with era N-1's weights. The deploy gate never grades a vote whose weights
were fitted on the bars it is scoring. Residual leakage remains - the same OOS
bars each era under a different model - and is stated in the code rather than
papered over.
Both DB clobber paths are closed: ApplyPatternWeight() declines once
self-ranked, and UpdateSignalsWeights()' filter.Weight() call is guarded by
SelfRanked() - guarding only the tiers would have let the hourly ranking pass
undo half the self-ranking.
NOT COMPILED - user compiles in MetaEditor.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-18 16:00:32 -04:00
|
|
|
//| TURN THIS ERA'S HELD-OUT OUTCOMES INTO THE VOTE WEIGHTS. |
|
|
|
|
|
//| |
|
|
|
|
|
//| The problem this solves, in the user's own words (2026-08-18): |
|
|
|
|
|
//| "NNs are different from classic signals - 0.33 could change |
|
|
|
|
|
//| meaning as the neurons weights change". Exactly so. A classic |
|
|
|
|
|
//| Pattern_2 is a fixed geometric condition, so win rates for it can |
|
|
|
|
|
//| be accumulated over years and stay meaningful. An AI Pattern_2 |
|
|
|
|
|
//| means "confidence landed in tier 2", and tier 2 under era 100's |
|
|
|
|
|
//| weights is a different statement from tier 2 under era 500's. Any |
|
|
|
|
|
//| accumulated ledger of AI rows therefore describes models that no |
|
|
|
|
|
//| longer exist, and averages them together. |
|
|
|
|
|
//| |
|
|
|
|
|
//| WHY THIS DOES NOT WRITE TO THE SIGNAL DB, which was the obvious |
|
|
|
|
|
//| reading of "fill the database during training": the DB's value is |
|
|
|
|
|
//| accumulation, and accumulation is precisely what is wrong here. |
|
|
|
|
|
//| Synthetic rows would also collide with the per-table row cap and |
|
|
|
|
|
//| mix measured-on-holdout outcomes into the same tables the LIVE |
|
|
|
|
|
//| ledger uses. What the DB actually supplies is a measured win rate |
|
|
|
|
|
//| per pattern - and pass 3 already computes exactly that, on held- |
|
|
|
|
|
//| out bars, thousands at a time instead of a handful of live trades.|
|
|
|
|
|
//| So the AI ranks itself from that, once per era, replacing rather |
|
|
|
|
|
//| than accumulating - which makes the weights describe the CURRENT |
|
|
|
|
|
//| weights by construction. |
|
|
|
|
|
//| |
|
|
|
|
|
//| SHRINKAGE IS NOT OPTIONAL HERE. The OOS window holds ~63 |
|
|
|
|
|
//| EFFECTIVE observations (overlapping triple-barrier labels - see |
|
|
|
|
|
//| EffectiveSampleSize), so four tiers hold ~8 apiece and a raw |
|
|
|
|
|
//| per-tier ratio would be noise dressed as a probability. Each tier |
|
|
|
|
|
//| is shrunk toward this model's POOLED holdout win rate with |
|
|
|
|
|
//| MIN_TRADES_FOR_WIN_RATE pseudo-trades - the same estimator, and |
|
|
|
|
|
//| the same prior strength, UpdateSignalsWeights() applies to the |
|
|
|
|
|
//| classic ladders. A thin tier therefore reads as the model's own |
|
|
|
|
|
//| overall win rate and the tiers separate only as evidence earns it.|
|
|
|
|
|
//| That is the correct behaviour, not a failure to differentiate. |
|
|
|
|
|
//| |
|
|
|
|
|
//| NO SAME-ERA CIRCULARITY, and it falls out of the ordering rather |
|
|
|
|
|
//| than needing a guard: weights are computed at the END of era N, |
|
|
|
|
|
//| so the vote scored during era N was cast with era N-1's weights. |
|
|
|
|
|
//| The deploy gate never grades a vote whose weights were fitted on |
|
|
|
|
|
//| the very bars it is scoring. Residual leakage remains and is not |
|
|
|
|
|
//| papered over - it is the same OOS bars each era, under a |
|
|
|
|
|
//| different model - which is why this feeds the VOTE and not the |
|
|
|
|
|
//| gate's own pass/fail arithmetic. |
|
|
|
|
|
//+------------------------------------------------------------------+
|
|
|
|
|
void CExpertSignalAIBase::RankTiersFromOos(void)
|
|
|
|
|
{
|
fix(chart): display now reads era-end SNAPSHOTS - the live cache is wiped mid-era
Full-pipeline analysis after "threshold 30, attained often, nothing drawn,
still glued to buy". The log falsified the premise before any code did:
21:40:43 swept 4999, 794 voters, drew 491. Strongest 43.0% vs 30.0%
21:42:07 swept 4999, 0 voters, drew 0
21:51:30 swept 4999, 0 voters, drew 0
21:56:30 swept 4999, 922 voters, drew 382. Strongest 44.0% vs 30.0%
The arrows WERE drawn - 491 of them, then 382 - and then erased. ONE root
cause, three symptoms: every display path read m_arrowSignalCache, which is
wiped to sentinel at each era start and only complete again when pass 3
finishes. With eras at ~30s and a sweep at ~17s:
* ARROW FLICKER: a sweep landing mid-era found no voters anywhere, and its
else-branch deleted the arrow on every voteless bar - erasing the previous
sweep's entire output. The chart cycled populated -> blank -> populated;
the user kept catching the blank phase.
* READOUT GLUE: the newest-cache walk found only sentinel for ~90% of every
era and fell through to dPrevSignal - the frozen purge-band edge bar that
reads Buy. 659638e fixed which bar was frozen, not the freezing.
* VOTER FLAP: 1299 -> 257 -> 1113 across back-to-back sweeps - each saw a
different fraction of half-rebuilt caches.
THE FIX, structural rather than another patch:
1. Era-end snapshots. RankTiersFromOos() runs at pass-3 completion - the one
moment the cache is complete - and now copies it (raw signals, newest
LOOKBACK+16 bars) into member-owned snapshot state, unconditionally,
BEFORE its early return: an all-Neutral era is a snapshot worth showing,
not an absence of one. Raw signals rather than votes, so a tier re-rank
between eras reprices them at read time via LiveVoteContribution for free.
2. The sweep (SnapshotVoteAt) and the prospective readout both read
snapshots; the readout's fallback chain is live-cache -> snapshot ->
dPrevSignal, and the snapshot leg is the one that fires most of the time.
3. NO DATA IS NOT A VERDICT: a den==0 bar no longer deletes - only an actual
sub-threshold vote takes an arrow down. This alone ends the wipe half of
the flicker even where snapshots are missing (before the first era).
4. Arming moved from an era-counter diff (which fires at era BOUNDARIES,
i.e. precisely when caches are about to be wiped) to
g_warriorOverlayArmRequest, set by each RankTiersFromOos - "a member's
snapshot just got fresher", the only event a redraw can act on. 60s rate
limit collapses the four members' burst into one sweep. Classic-only
charts arm once at start.
5. Census now reports the direction split - "922 had a voter (610 buy / 312
sell)" - so "the vote leans buy" is checkable from the log instead of
inferred from arrow colours.
Also visible in the log and worth knowing: the threshold flip-flopped
30 -> 40 -> 30 across the evening's re-inits (census lines at 21:42-21:51
ran at 40), so part of the observed blankness was configuration, not code.
NOT COMPILED - user compiles in MetaEditor.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-18 22:03:20 -04:00
|
|
|
//--- SNAPSHOT FIRST, unconditionally - this runs at pass-3 completion, the single moment the
|
|
|
|
|
//--- arrow cache is complete for the era, and everything display-side (the overlay sweep, the
|
|
|
|
|
//--- prospective readout) reads the snapshot instead of the cache precisely because the cache is
|
|
|
|
|
//--- about to be wiped when the next era starts. Deliberately BEFORE the pooledFired early
|
|
|
|
|
//--- return: an all-Neutral era still scored every bar, and "the model says Neutral everywhere"
|
|
|
|
|
//--- is a snapshot worth showing, not an absence of one. Raw signals, not votes: a tier re-rank
|
|
|
|
|
//--- between eras then reprices them at read time (LiveVoteContribution) for free.
|
|
|
|
|
int snapN = MathMin(ArraySize(m_arrowSignalCache), SIGNAL_RESCAN_LOOKBACK_BARS + 16);
|
|
|
|
|
if(snapN > 0)
|
|
|
|
|
{
|
|
|
|
|
ArrayResize(m_overlaySigSnap, snapN);
|
|
|
|
|
ArrayCopy(m_overlaySigSnap, m_arrowSignalCache, 0, 0, snapN);
|
|
|
|
|
}
|
|
|
|
|
m_overlaySnapBars = snapN;
|
|
|
|
|
m_prospectiveSigSnap = -2.0;
|
|
|
|
|
for(int pi = 1; pi <= 16 && pi < snapN; pi++)
|
|
|
|
|
if(m_overlaySigSnap[pi] != -2.0 && MathIsValidNumber(m_overlaySigSnap[pi]))
|
|
|
|
|
{
|
|
|
|
|
m_prospectiveSigSnap = m_overlaySigSnap[pi];
|
|
|
|
|
break;
|
|
|
|
|
}
|
|
|
|
|
//--- ...and tell the EA a fresh snapshot exists, so the overlay redraws from it. Every member
|
|
|
|
|
//--- sets this each era; the EA's rate limit collapses the burst into one sweep.
|
|
|
|
|
g_warriorOverlayArmRequest = true;
|
feat(rank): AI models rank their own confidence tiers from held-out outcomes
Closes the caveat 4858507 shipped with: the vote is a confidence percentage,
but only to the extent the pattern weights are measured. AI tier weights sat
at their designed defaults (25/50/75/100) because AI rows only ever arrive
from LIVE journaling, of which a training run produces almost none.
AND A STALE-TIER BUG THAT MADE THE EVIDENCE MEANINGLESS. The OOS scan bucketed
every scanned bar by ConfidenceTier(), which reads dPrevSignal - and
dPrevSignal is assigned in PASS 1 only, never anywhere in the OOS scan. So an
entire era's fires were bucketed by one stale, unrelated bar's confidence and
landed in a SINGLE tier. That is the "tier prec T0:72%(828) T1:n/a(0)
T2:n/a(0) T3:n/a(0)" symptom recorded on 2026-08-16 and attributed to the
calibration clamp. The clamp was real and was fixed then; this is a second,
independent cause of the identical output that survived that fix untouched -
which is why the log kept reading the same afterwards. Two causes, one symptom.
Now ConfidenceTierFor(adjSig): the bar this iteration actually scored.
WHY THIS DOES NOT WRITE ROWS TO THE SIGNAL DB, which was the obvious reading of
"fill the database during training". The user's own observation is the reason:
a classic Pattern_2 is a fixed geometric condition, so its win rate is
legitimately accumulated over years, but an AI Pattern_2 means "confidence
landed in tier 2" and tier 2 under era 100's weights is a different statement
from tier 2 under era 500's. The DB's value is ACCUMULATION, and accumulation
is exactly what is wrong here - it would average together models that no
longer exist, while colliding with the per-table row cap and mixing
measured-on-holdout outcomes into the live ledger's own tables. What the DB
actually supplies is a measured win rate per pattern, and pass 3 already
computes that on held-out bars, thousands at a time. So the model ranks itself
once per era, REPLACING rather than accumulating, which makes the weights
describe the current weights by construction.
ESTIMATOR. Not WinRateFromCounts(): it returns NO_DATA below 100 raw trades
BEFORE shrinking, which here would fire on every tier every era and hand all
four the pooled rate - the tiers could never separate and the mechanism would
be inert. Shrinkage is the answer to a small sample; a floor in front of it
means the shrinkage never runs. Instead: a Beta prior of TIER_PRIOR_EFF_N
pseudo-observations centred on the model's pooled holdout rate, counted in
EFFECTIVE observations, because overlapping triple-barrier labels mean 800 raw
fires can be worth ~12 independent ones. Rounded to the integer, not to the
decade NormalizeWinRate() uses, which would collapse the shrunk tiers back
into one number.
NO SAME-ERA CIRCULARITY, and it falls out of the ordering rather than a guard:
weights are computed at the END of era N, so the vote scored during era N was
cast with era N-1's weights. The deploy gate never grades a vote whose weights
were fitted on the bars it is scoring. Residual leakage remains - the same OOS
bars each era under a different model - and is stated in the code rather than
papered over.
Both DB clobber paths are closed: ApplyPatternWeight() declines once
self-ranked, and UpdateSignalsWeights()' filter.Weight() call is guarded by
SelfRanked() - guarding only the tiers would have let the hourly ranking pass
undo half the self-ranking.
NOT COMPILED - user compiles in MetaEditor.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-18 16:00:32 -04:00
|
|
|
int pooledFired = 0, pooledHits = 0;
|
|
|
|
|
for(int t = 0; t < 4; t++)
|
|
|
|
|
{
|
|
|
|
|
pooledFired += m_oosTierFired[t];
|
|
|
|
|
pooledHits += m_oosTierHits[t];
|
|
|
|
|
}
|
|
|
|
|
//--- Nothing fired this era (all-Neutral, or a stopped/cap-hit era): leave the previous era's
|
|
|
|
|
//--- weights standing rather than collapsing every tier to a prior built on no evidence at all.
|
|
|
|
|
if(pooledFired <= 0)
|
|
|
|
|
return;
|
|
|
|
|
double pooledPct = 100.0 * pooledHits / pooledFired;
|
|
|
|
|
//--- WHY NOT WinRateFromCounts(), which is the estimator the classic ladders use: it returns
|
|
|
|
|
//--- NO_DATA_WIN_RATE for anything under MIN_TRADES_FOR_WIN_RATE raw trades, BEFORE it shrinks. That
|
|
|
|
|
//--- floor is right there - a classic pattern with 12 live rows should be left at its default, not
|
|
|
|
|
//--- re-weighted - but here it would fire on every tier every era and hand all four the pooled rate,
|
|
|
|
|
//--- so the tiers could never separate and the whole mechanism would be inert. Shrinkage is the
|
|
|
|
|
//--- answer to a small sample; a floor in front of it means the shrinkage never runs.
|
|
|
|
|
//---
|
|
|
|
|
//--- ...and the sample is measured EFFECTIVE, not raw. Triple-barrier labels overlap, so N fires
|
|
|
|
|
//--- resolving over a mean lifespan of L bars are worth about N/L independent observations (see
|
|
|
|
|
//--- EffectiveSampleSize / [[project_label_overlap_effective_n]]). A tier showing 800 raw fires may
|
|
|
|
|
//--- carry ~12 real ones. Shrinking on the raw count would treat that as overwhelming evidence and
|
|
|
|
|
//--- reproduce exactly the "clears by N sigma" error that overlap invalidated everywhere else.
|
|
|
|
|
string line = "";
|
|
|
|
|
for(int t = 0; t < 4; t++)
|
|
|
|
|
{
|
|
|
|
|
int fired = m_oosTierFired[t];
|
|
|
|
|
int hits = m_oosTierHits[t];
|
|
|
|
|
double w = pooledPct;
|
|
|
|
|
if(fired > 0)
|
|
|
|
|
{
|
|
|
|
|
double effN = MathMax(0.0, EffectiveSampleSize((double)fired));
|
|
|
|
|
double effHits = effN * ((double)hits / fired);
|
|
|
|
|
//--- Beta prior of TIER_PRIOR_EFF_N pseudo-observations centred on this model's pooled
|
|
|
|
|
//--- holdout rate. Deliberately small: the prior is counted in the same EFFECTIVE units as
|
|
|
|
|
//--- the evidence, and a tier holds ~8-15 of those, so a prior of 100 (the classic path's)
|
|
|
|
|
//--- would drown every tier in the pool. At 10 a tier with ~10 effective observations sits
|
|
|
|
|
//--- half on its own evidence and half on the pool, which is an honest reading of what one
|
|
|
|
|
//--- era's holdout can support.
|
|
|
|
|
w = (effHits + TIER_PRIOR_EFF_N * (pooledPct / 100.0)) * 100.0 / (effN + TIER_PRIOR_EFF_N);
|
|
|
|
|
}
|
|
|
|
|
//--- Rounded to the nearest INTEGER, not to the nearest 10 as NormalizeWinRate() does. After
|
|
|
|
|
//--- shrinkage the tiers legitimately sit within a few points of each other, and decade rounding
|
|
|
|
|
//--- would collapse them back into one number - undoing the separation this exists to produce.
|
|
|
|
|
int wi = (int)MathMax(0, MathMin(100, MathRound(w)));
|
|
|
|
|
ApplyTierWeight(t, wi);
|
|
|
|
|
line += StringFormat(" T%d=%d(%d fires, %.1f eff)", t, wi, fired,
|
|
|
|
|
(fired > 0 ? EffectiveSampleSize((double)fired) : 0.0));
|
|
|
|
|
}
|
|
|
|
|
//--- MODULE WEIGHT = how much this model's opinion COUNTS in the weighted mean, which since the
|
|
|
|
|
//--- 2026-08-18 currency change is a trust weight and no longer a discount on the estimate. The
|
|
|
|
|
//--- pooled holdout win rate is the honest measure of that trust.
|
|
|
|
|
Weight(MathMax(0.0, MathMin(1.0, pooledPct / 100.0)));
|
|
|
|
|
m_tiersSelfRanked = true;
|
feat(logs): throttle the settled per-era diagnostics - measured 22MB/9.5h of confirmed-working systems
Measured from the journal (2026-08-19): the era deep-dive line (~2KB) plus
the excursion verdict, tier re-rank, calibration move, barrier hold and
selection-regressed note each printed EVERY era for EVERY member - ~940
eras/member/day - long after the systems they watch were confirmed
working. Yesterday's file was 1.3GB (70% of it the news-filter calendar
spam the sweep fix already removed).
VerboseMode returns as an INPUT (demoted 2026-08-01 for the marketplace;
that track is dead since the 2026-08-16 pivot) and gains a second job:
false throttles each settled per-era print to eras 0-3 plus every
TRAIN_LOG_EVERY_ERAS-th (25 ~= one deep-dive per ~15min per member);
true restores the per-era firehose, flippable live.
Never throttled: anything that marks a CHANGE - new bests, restores +
eta decays, plateau stage transitions, deploy approvals, warnings,
errors, the label-cache/adoption one-shots, and the combined-vote gate
line (the active system's primary telemetry, still every era).
Semantic fixes over blanket gating:
- barrier hold now ARMS silently and prints only when the hold outlasts
the 2-min report interval - a brief hold every era is the design, the
long hold is the watchdog case the line exists for;
- the ensemble deploy REFUSAL prints immediately when its reason
changes (that is a finding), on cadence when unchanged;
- the filtered-view census prints when its RESULT moves (drawn count,
or strongest vote by >=2pp) and at least every 10th sweep.
NOT COMPILED - user compiles in MetaEditor.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-19 09:36:15 -04:00
|
|
|
//--- THROTTLED (2026-08-19): the re-rank happens (and must happen) every era, but saying so
|
|
|
|
|
//--- every era was ~950 near-identical lines/member/day once the system was confirmed working.
|
|
|
|
|
//--- The weights it prints are visible live on the member HUD lines anyway.
|
|
|
|
|
if(TrainLogDue())
|
|
|
|
|
Print(ID + StringFormat(": tier weights re-ranked from %d held-out fires (pooled %.1f%%) ->%s"
|
feat(rank): AI models rank their own confidence tiers from held-out outcomes
Closes the caveat 4858507 shipped with: the vote is a confidence percentage,
but only to the extent the pattern weights are measured. AI tier weights sat
at their designed defaults (25/50/75/100) because AI rows only ever arrive
from LIVE journaling, of which a training run produces almost none.
AND A STALE-TIER BUG THAT MADE THE EVIDENCE MEANINGLESS. The OOS scan bucketed
every scanned bar by ConfidenceTier(), which reads dPrevSignal - and
dPrevSignal is assigned in PASS 1 only, never anywhere in the OOS scan. So an
entire era's fires were bucketed by one stale, unrelated bar's confidence and
landed in a SINGLE tier. That is the "tier prec T0:72%(828) T1:n/a(0)
T2:n/a(0) T3:n/a(0)" symptom recorded on 2026-08-16 and attributed to the
calibration clamp. The clamp was real and was fixed then; this is a second,
independent cause of the identical output that survived that fix untouched -
which is why the log kept reading the same afterwards. Two causes, one symptom.
Now ConfidenceTierFor(adjSig): the bar this iteration actually scored.
WHY THIS DOES NOT WRITE ROWS TO THE SIGNAL DB, which was the obvious reading of
"fill the database during training". The user's own observation is the reason:
a classic Pattern_2 is a fixed geometric condition, so its win rate is
legitimately accumulated over years, but an AI Pattern_2 means "confidence
landed in tier 2" and tier 2 under era 100's weights is a different statement
from tier 2 under era 500's. The DB's value is ACCUMULATION, and accumulation
is exactly what is wrong here - it would average together models that no
longer exist, while colliding with the per-table row cap and mixing
measured-on-holdout outcomes into the live ledger's own tables. What the DB
actually supplies is a measured win rate per pattern, and pass 3 already
computes that on held-out bars, thousands at a time. So the model ranks itself
once per era, REPLACING rather than accumulating, which makes the weights
describe the current weights by construction.
ESTIMATOR. Not WinRateFromCounts(): it returns NO_DATA below 100 raw trades
BEFORE shrinking, which here would fire on every tier every era and hand all
four the pooled rate - the tiers could never separate and the mechanism would
be inert. Shrinkage is the answer to a small sample; a floor in front of it
means the shrinkage never runs. Instead: a Beta prior of TIER_PRIOR_EFF_N
pseudo-observations centred on the model's pooled holdout rate, counted in
EFFECTIVE observations, because overlapping triple-barrier labels mean 800 raw
fires can be worth ~12 independent ones. Rounded to the integer, not to the
decade NormalizeWinRate() uses, which would collapse the shrunk tiers back
into one number.
NO SAME-ERA CIRCULARITY, and it falls out of the ordering rather than a guard:
weights are computed at the END of era N, so the vote scored during era N was
cast with era N-1's weights. The deploy gate never grades a vote whose weights
were fitted on the bars it is scoring. Residual leakage remains - the same OOS
bars each era under a different model - and is stated in the code rather than
papered over.
Both DB clobber paths are closed: ApplyPatternWeight() declines once
self-ranked, and UpdateSignalsWeights()' filter.Weight() call is guarded by
SelfRanked() - guarding only the tiers would have let the hourly ranking pass
undo half the self-ranking.
NOT COMPILED - user compiles in MetaEditor.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-18 16:00:32 -04:00
|
|
|
" | module weight %.2f. These are the weights the NEXT era votes with.",
|
|
|
|
|
pooledFired, pooledPct, line, ModuleWeight()));
|
|
|
|
|
}
|
|
|
|
|
//+------------------------------------------------------------------+
|
2026-08-01 11:27:28 -04:00
|
|
|
//| Set the specified pattern's weight to the specified value |
|
|
|
|
|
//+------------------------------------------------------------------+
|
|
|
|
|
void CExpertSignalAIBase::ApplyPatternWeight(int patternNumber, int weight)
|
|
|
|
|
{
|
feat(rank): AI models rank their own confidence tiers from held-out outcomes
Closes the caveat 4858507 shipped with: the vote is a confidence percentage,
but only to the extent the pattern weights are measured. AI tier weights sat
at their designed defaults (25/50/75/100) because AI rows only ever arrive
from LIVE journaling, of which a training run produces almost none.
AND A STALE-TIER BUG THAT MADE THE EVIDENCE MEANINGLESS. The OOS scan bucketed
every scanned bar by ConfidenceTier(), which reads dPrevSignal - and
dPrevSignal is assigned in PASS 1 only, never anywhere in the OOS scan. So an
entire era's fires were bucketed by one stale, unrelated bar's confidence and
landed in a SINGLE tier. That is the "tier prec T0:72%(828) T1:n/a(0)
T2:n/a(0) T3:n/a(0)" symptom recorded on 2026-08-16 and attributed to the
calibration clamp. The clamp was real and was fixed then; this is a second,
independent cause of the identical output that survived that fix untouched -
which is why the log kept reading the same afterwards. Two causes, one symptom.
Now ConfidenceTierFor(adjSig): the bar this iteration actually scored.
WHY THIS DOES NOT WRITE ROWS TO THE SIGNAL DB, which was the obvious reading of
"fill the database during training". The user's own observation is the reason:
a classic Pattern_2 is a fixed geometric condition, so its win rate is
legitimately accumulated over years, but an AI Pattern_2 means "confidence
landed in tier 2" and tier 2 under era 100's weights is a different statement
from tier 2 under era 500's. The DB's value is ACCUMULATION, and accumulation
is exactly what is wrong here - it would average together models that no
longer exist, while colliding with the per-table row cap and mixing
measured-on-holdout outcomes into the live ledger's own tables. What the DB
actually supplies is a measured win rate per pattern, and pass 3 already
computes that on held-out bars, thousands at a time. So the model ranks itself
once per era, REPLACING rather than accumulating, which makes the weights
describe the current weights by construction.
ESTIMATOR. Not WinRateFromCounts(): it returns NO_DATA below 100 raw trades
BEFORE shrinking, which here would fire on every tier every era and hand all
four the pooled rate - the tiers could never separate and the mechanism would
be inert. Shrinkage is the answer to a small sample; a floor in front of it
means the shrinkage never runs. Instead: a Beta prior of TIER_PRIOR_EFF_N
pseudo-observations centred on the model's pooled holdout rate, counted in
EFFECTIVE observations, because overlapping triple-barrier labels mean 800 raw
fires can be worth ~12 independent ones. Rounded to the integer, not to the
decade NormalizeWinRate() uses, which would collapse the shrunk tiers back
into one number.
NO SAME-ERA CIRCULARITY, and it falls out of the ordering rather than a guard:
weights are computed at the END of era N, so the vote scored during era N was
cast with era N-1's weights. The deploy gate never grades a vote whose weights
were fitted on the bars it is scoring. Residual leakage remains - the same OOS
bars each era under a different model - and is stated in the code rather than
papered over.
Both DB clobber paths are closed: ApplyPatternWeight() declines once
self-ranked, and UpdateSignalsWeights()' filter.Weight() call is guarded by
SelfRanked() - guarding only the tiers would have let the hourly ranking pass
undo half the self-ranking.
NOT COMPILED - user compiles in MetaEditor.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-18 16:00:32 -04:00
|
|
|
//--- THE SIGNAL DB DOES NOT OUTRANK THE HOLDOUT. UpdateSignalsWeights() calls this hourly for every
|
|
|
|
|
//--- filter it can find rows for, and for an AI filter those rows are LIVE-journaled fires
|
|
|
|
|
//--- accumulated across eras - i.e. across models. Once this model has measured its own tiers on
|
|
|
|
|
//--- held-out bars (RankTiersFromOos), that measurement describes the weights actually loaded and
|
|
|
|
|
//--- the DB's does not, so the DB call is declined rather than allowed to overwrite it.
|
|
|
|
|
//--- Without this the ranking pass would silently undo every era's self-ranking within the hour.
|
|
|
|
|
if(m_tiersSelfRanked)
|
|
|
|
|
return;
|
2026-08-01 11:27:28 -04:00
|
|
|
switch(patternNumber)
|
|
|
|
|
{
|
|
|
|
|
case 0:
|
|
|
|
|
Pattern_0(weight);
|
|
|
|
|
break;
|
|
|
|
|
case 1:
|
|
|
|
|
Pattern_1(weight);
|
|
|
|
|
break;
|
|
|
|
|
case 2:
|
|
|
|
|
Pattern_2(weight);
|
|
|
|
|
break;
|
|
|
|
|
case 3:
|
|
|
|
|
Pattern_3(weight);
|
|
|
|
|
break;
|
|
|
|
|
default:
|
|
|
|
|
break;
|
|
|
|
|
}
|
|
|
|
|
}
|
|
|
|
|
//+------------------------------------------------------------------+
|
|
|
|
|
//| OnTick function |
|
|
|
|
|
//+------------------------------------------------------------------+
|
|
|
|
|
void CExpertSignalAIBase::OnTickHandler(void)
|
|
|
|
|
{
|
|
|
|
|
ScheduleTrainingIfNeeded();
|
|
|
|
|
}
|
|
|
|
|
//+------------------------------------------------------------------+
|
|
|
|
|
//| Schedules the next training pass (if one is due) and refreshes |
|
|
|
|
|
//| the per-tick status label. Factored out of OnTickHandler() so |
|
|
|
|
|
//| Warrior_EA.mq5's always-on timer (see PollTraining()) can drive |
|
|
|
|
|
//| this on a fixed wall-clock schedule too - training must not stall |
|
|
|
|
|
//| just because the market is closed and no ticks are arriving. |
|
|
|
|
|
//+------------------------------------------------------------------+
|
|
|
|
|
void CExpertSignalAIBase::ScheduleTrainingIfNeeded(void)
|
|
|
|
|
{
|
|
|
|
|
//--- stopped: no new training passes get scheduled at all (StartTraining() re-arms this).
|
|
|
|
|
//--- paused: still schedule so bEventStudy/dtStudied bookkeeping stays current, but Train() itself
|
|
|
|
|
//--- blocks at the next era boundary until resumed - keeps in-memory state coherent either way.
|
|
|
|
|
//--- complete: training already converged - a plain new bar must NOT re-enter Train()'s full era
|
|
|
|
|
//--- loop, which would otherwise reset the best-checkpoint/eta-decay tracking and run real
|
|
|
|
|
//--- Net.backProp() passes again, forever, once per bar, on an already-converged model (see
|
|
|
|
|
//--- RefreshConvergedSignal()'s declaration comment). Just keep the live signal current instead.
|
|
|
|
|
//--- publish this signal's current signed confidence for the intelligent trailing (and any other
|
|
|
|
|
//--- live-confidence consumer) - see g_LiveAISignedConfidence in Variables\ConfidenceBridge.mqh.
|
|
|
|
|
//--- Cheap: SignedAIConfidence() just reads the already-computed dPrevSignal.
|
feat(gates): derive the recall floor instead of configuring it, publish what the configuration can PROVE, and stop an arbitrary member driving live exits
Three changes, all from the same principle: measure what is there before aiming
at it, and never certify a number you do not trade.
1. THE RECALL FLOOR IS DERIVED, AND IT MOVES BELOW CHANCE.
MinRecall=40 was a constant doing a statistical job. Its reference point is the
33.3% recall a zero-skill 3-class model gets on EVERY class, and against that the
constant was accidentally calibrated for exactly one sample size: on USDJPY CONV
(n_eff 195) 40% is chance + 2.0 SE; on SP500 PAI (n_eff 42) the same 40% is chance
+ 0.9 SE. One chart was being held to a bar twice as strict as the other, for no
reason anyone chose.
CollapseRecallFloorPct() computes it per class from that class's own effective
sample - EffectiveSampleSize(), so the overlap deflation the rest of the gates use
applies here too - as chance - EDGE_MIN_SIGMAS x SE. 26.5% at n_eff 195, 18.8% at
n_eff 42.
BELOW chance, deliberately, and this is the substantive change rather than the
arithmetic. This gate's only job is refusing to call a COLLAPSED model converged.
It is not a quality bar; the deploy gate is the quality bar and it is already
rigorous (chance + 2 SE on the deflated sample, Sidak over candidate eras, then the
cross-instrument pooled certificate). A convergence gate that ALSO demands
provably-above-chance recall on all three classes double-counts that job, and it
has failed that way twice here: MinRecall=60 blocked every SP500 H1 run in 2026-07,
and the 40 that replaced it made Neutral structurally unreachable once first-touch
resolution cut Neutral to a 0.65% residue. A floor nothing can reach does not make
a funded account safer, it stops the run converging at all.
Testing significantly BELOW chance instead catches what a fixed 40 was actually
catching - a model that has stopped emitting a class - and cannot become
unreachable by construction. It also fixes the direction the old constant scaled:
it now widens on a thin OOS window, where low recall genuinely cannot be told from
noise, and tightens on a rich one. Today's SP500 PAI (Buy 51 / Sell 18 / Neutral 30)
is still correctly blocked on Sell.
This also resolves a standing contradiction the code half-admitted at the
isBetterEra comment: selection ranks on coverage-weighted PRECISION while
convergence gated on RECALL, so a sparse high-precision abstainer - precisely the
model that could clear the deploy bar - was blocked by the floor.
The era line now PRINTS the derived floor. Anyone comparing these recalls against a
remembered "40" is reading the wrong bar.
2. DETECTABILITY: WHAT THIS CONFIGURATION COULD PROVE, BEFORE IT TRAINS.
The DEPLOY BAR line states the bar. It never said what reaching it would take, and
that is the actionable direction. ReportDetectability() inverts the same identity -
the gate passes when edge >= z x sqrt(p(1-p)/n_eff), so certifying an edge d needs
n_eff >= z^2 p(1-p)/d^2 independent calls, hence L times as many raw ones - and
prints a +2 / +5 / +10pp ladder as required independent calls, raw calls, and share
of the OOS window, marking any rung that needs more than the window holds IMPOSSIBLE.
Every term is a property of the CONFIGURATION - geometry via break-even, horizon via
mean label lifespan, window via oosCutoff - so no amount of training moves any of
them. It fires once, at the first healthy sweep, beside ReportFeatureHealth, for the
same reason: that is the first moment the bar grid, the measured geometry and the
lifespan are real numbers rather than defaults. It gates nothing.
This is the EdgeFinder discipline applied to our own gate: establish what the market
and the measurement design have to offer, then point the net at it - rather than
spending a thousand eras chasing something this OOS window could never certify.
3. AN ARBITRARY MEMBER WAS DRIVING LIVE EXITS AND TRAILING (user-identified).
Every ensemble member ran
g_LiveAISignedConfidence = SignedAIConfidence();
unconditionally, every tick. Last writer wins. Its consumers are the AI early-exit
route (CExpertSignalCustom::LiveSignedConfidence) and TrailingIntelligent - so on a
four-model chart an LSTM entry could be closed, and its stop moved, on the
Perceptron's opinion alone, decided by scheduling order. Not the vote, not a
weighted blend.
Now the mean across registered members, matching how the ensemble actually trades:
the open decision is the weighted-average vote, and an abstaining member contributes
0 and dilutes exactly as it does there. Members still training read 0, so a
half-trained ensemble reads WEAKER rather than louder - the safe direction for an
exit trigger. Deployed and paused members are included, which is the opposite of the
era barrier's exemption rule and correct for the opposite reason: that one asks who
must be waited for, this asks who has an opinion.
Latent today and staying that way for now by choice - Min_Vote_Close ships Disabled
(101, unreachable on both scales it drives) and TrailingStrategy is off, so live
exits are SL/TP only and the certified hold-to-barrier win rate is what actually
gets traded. Fixed now precisely because the plan is to enable vote exits once the
models are accurate, at which point a scheduling-order exit would be both harmful
and very hard to see.
STILL OPEN, and needs a decision before vote exits go on: the member gate and the
ensemble vote gate both grade hold-to-barrier, so enabling vote exits makes the
certified number stop describing the traded one. Warrior_EA.mq5 currently argues
barrier models may keep vote exits because "their label IS the vote's own horizon" -
that does not hold, since a vote flip at bar 5 of a 64-bar horizon is not the
target-before-stop outcome the gate measured. Either grade the OOS call on the real
exit rule (first of SL / TP / vote-flip / horizon) through the fill engine, or set
HoldToBarrier for ensemble members so the policy cannot drift from the certificate.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-17 16:15:08 -04:00
|
|
|
//--- ENSEMBLE: the AVERAGE across members, not this member's own reading. Every member ran this line
|
|
|
|
|
//--- unconditionally, every tick, so the global was simply whichever member's OnTick happened to run
|
|
|
|
|
//--- last - and its two consumers are the AI early-exit route (CExpertSignalCustom::LiveSignedConfidence)
|
|
|
|
|
//--- and the intelligent trailing stop. So on a four-model chart an LSTM entry could be closed, and its
|
|
|
|
|
//--- stop moved, on the Perceptron's opinion alone, purely by scheduling order. Not the vote, not a
|
|
|
|
|
//--- weighted blend - an arbitrary member. (User-identified 2026-08-17.)
|
|
|
|
|
//--- Averaging matches how the ensemble actually trades: the open decision is the weighted-average
|
|
|
|
|
//--- vote, and a member that abstains contributes 0 and dilutes, exactly as it does there. Members
|
|
|
|
|
//--- still training read 0 from SignedAIConfidence(), so a half-trained ensemble reads WEAKER rather
|
|
|
|
|
//--- than louder, which is the safe direction for an exit trigger.
|
|
|
|
|
//--- Currently latent, and worth keeping that way deliberately: Min_Vote_Close ships Disabled (101,
|
|
|
|
|
//--- unreachable on both scales it drives) and TrailingStrategy is off, so neither consumer fires
|
|
|
|
|
//--- today. This is fixed now precisely because the plan is to enable vote exits once the models are
|
|
|
|
|
//--- accurate - at which point a scheduling-order exit would be actively harmful and very hard to see.
|
feat(gate): grade OOS calls on the exit policy actually in force, and move vote combining out of the members and into the orchestrator
Option (a) from the exit-policy question: the certified number must be the traded
number. Plus the modularity correction the user called for on 778b6c0.
1. VOTE COMBINING BELONGS TO THE ORCHESTRATOR, NOT TO A MEMBER.
778b6c0 fixed the last-writer-wins bug on g_LiveAISignedConfidence by having a
member average its siblings through g_warriorEnsemble. That trades a scheduling bug
for a coupling bug, and it is the wrong shape for this EA: every signal runs in its
own instance, minds its own state, and VOTES to the orchestrator, which is the only
thing allowed to combine opinions.
Replaced with a publish/aggregate pair in Variables\ConfidenceBridge.mqh whose split
is enforced by shape rather than by convention:
- PublishAIVote(slot, conf) - a member writes ONLY its own slot, reads nobody's;
- AggregateAIVotes() - called by CExpertSignalCustom::LiveSignedConfidence.
CExpertSignalAIBase::EnsembleLiveSignedConfidence is gone. The orchestrator also
republishes the aggregate into g_LiveAISignedConfidence, because the intelligent
trailing reads that global directly and must act on the same number the exit route
does rather than on a leftover from whichever member ticked last. A solo AI signal
owns slot 0, so the non-ensemble path is unchanged.
2. THE GATE NOW REPLAYS THE REAL EXIT RULE.
SimulateTradeOutcome() walks the same price series with the same fill/barrier/spread
convention as ComputeLabelForBar - deliberately by copy, so a disagreement between
the two can only be a policy effect and never a discrepancy between two pieces of our
own arithmetic - and terminates at the FIRST of stop / target / vote reversal /
horizon. Barriers are tested before the vote on the same bar: intrabar we cannot know
which came first, and the barrier is what the broker executes automatically, so
checking the vote first would credit the exit policy with escapes a real stop would
have taken out of its hands.
It runs AFTER pass 3, not inside it. A vote-flip exit for a trade entered at bar r is
decided by the model's output at bars r-1, r-2, ... - NEWER bars - and pass 3 walks
oldest-to-newest, so at the moment r is graded its own exit does not exist yet. Only
once m_oosDecisionSeries is complete over the whole OOS window can the replay run.
In ensemble mode that series carries the member's adjusted decision and the live exit
reads the ensemble aggregate, which is the coupling the user identified: an LSTM entry
really can be closed by the ensemble turning against it.
3. THE STATISTIC HAS TO CHANGE WITH THE POLICY, AND THAT IS THE REAL FINDING.
A barrier exit pays a fixed R. A vote exit pays whatever the close happens to be. So
the moment vote exits are enabled the payoff is CONTINUOUS, and "win rate vs
break-even" stops being a meaningful test - there is no fixed break-even for a
variable payoff. SimulateTradeOutcome therefore returns R rather than a bool, and the
replay reports expectancy in R with its SE taken from the R distribution (overlap-
deflated on the same EffectiveSampleSize doctrine as every other SE here), not from a
binomial.
This is the same class of error as win-based scoring in 2026-08-09: measuring a
variable-payoff process with a fixed-payoff statistic. Naming it now, while vote exits
are still off, is much cheaper than discovering it after they go on.
4. WHY THIS IS SAFE TO SHIP TODAY.
Min_Vote_Close ships Disabled (101, unreachable on both scales it drives) and reaches
the AI signal through the new ExitPolicy() setter as 1.01, which the setter turns into
0.0 = no vote exit. Under that policy every replayed trade resolves at a barrier and
the simulation is arithmetically the same trade the deploy gate already certifies -
they cannot drift. The report says so explicitly, and prints ONCE per run in that
state; when vote exits are on it prints every era, because then the divergence is the
thing to watch. Nothing about today's numbers moves.
The gate switchover is wired but dormant by construction: it becomes exit-aware the
moment the input is enabled, which is exactly what "the certified number is the traded
number" has to mean.
KNOWN LIMIT, stated rather than hidden: only the AI early-exit route is replayed. The
rule-based averaged-vote close (m_threshold_close) depends on every other filter's
live vote, which pass 3 does not reproduce, so a position the classic filters would
have closed is held to its barrier here. The replay therefore holds LONGER than live
and overstates barrier-reached outcomes. Faithful only while the AI is the dominant
vote - which is the configuration this is being built for.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-17 16:25:57 -04:00
|
|
|
//--- PUBLISH ONLY THIS SIGNAL'S OWN VOTE. Combining is the orchestrator's job, never a member's - see
|
|
|
|
|
//--- the vote board in Variables\ConfidenceBridge.mqh for the scheduling bug this replaces and for why
|
|
|
|
|
//--- "have a member average its siblings" was the wrong shape of fix in a codebase whose whole point is
|
|
|
|
|
//--- that signals do not reach into each other. A solo AI signal owns slot 0 and the aggregate is then
|
|
|
|
|
//--- just its own value, so the non-ensemble path is unchanged.
|
|
|
|
|
PublishAIVote(m_ensembleMember ? m_ensembleIndex : 0, SignedAIConfidence());
|
2026-08-01 11:27:28 -04:00
|
|
|
datetime lastBarDate = (datetime)SeriesInfoInteger(m_symbol.Name(), m_period, SERIES_LASTBAR_DATE);
|
|
|
|
|
// A failed lookup (0) must not silently read as "dtStudied is already caught up, nothing pending" -
|
|
|
|
|
// that would freeze this function into never re-triggering training/signal refresh again until some
|
|
|
|
|
// other path happens to bump dtStudied. Treat a failed lookup as pending instead (same >0-guard
|
|
|
|
|
// philosophy as the SERIES_FIRSTDATE lookup elsewhere in this class) so a transient history-sync
|
|
|
|
|
// hiccup costs one extra harmless check, not a silent stall.
|
|
|
|
|
bool newBarPending = (dPrevSignal == -2 || lastBarDate <= 0 || ((m_inferenceOnly ? m_lastBarTime : dtStudied) < lastBarDate));
|
fix: training could only advance one 120ms chunk per bar
ScheduleTrainingIfNeeded() armed the next Train() call only when
dtStudied < lastBarDate. That watermark test is right for a CONVERGED
model - one inference refresh per new bar - and wrong for a training
run, because Train() is chunked: it does ~120ms of work and yields,
needing thousands of calls to finish one era, and every one of those
calls has to be armed from there.
dtStudied is two incompatible things. Train() sets it to the training
WINDOW START (~2008); FinalizeTrainRun() sets it to the last bar
SCANNED (~now). So the moment any run finalized, the scheduler went
silent until the next candle closed. On H1 that is one chunk per hour.
The symptom was indistinguishable from a hang: no era lines, no
heartbeats, not one of the six instrumented stall branches - because
Train() was not being CALLED. The TRAIN STALL line that caught it
reported runActive=Y only because m_trainRunActive had been set
microseconds earlier in that same call, and eraResume=N proved no era
was in flight. Two log bursts, 28 minutes apart, exactly one H1 bar.
Before 0c85c54 this was survivable rather than correct: the saved
watermark left almost no bars eligible per era, so eras were nearly
free and one call per bar still looked like progress.
An unconverged model is now always pending. Pause/stop are handled by
m_trainingPaused/m_trainingStopRequested, which Train() checks itself.
Also: the one Train() exit that tears down the whole run on a buffer
failure was completely silent - it now says so. And the build tag moves
to train-dispatch-v2; it had not moved since ce52654, which is why the
running binary could not be identified from its own log.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-10 10:06:49 -04:00
|
|
|
//--- A MODEL THAT HAS NOT FINISHED TRAINING IS ALWAYS PENDING. The watermark test above answers
|
|
|
|
|
//--- "has a new bar closed since the last one we processed", which is the right question for a
|
|
|
|
|
//--- CONVERGED model (one inference refresh per bar, see the branch below) and the wrong one for a
|
|
|
|
|
//--- training run: Train() is chunked - it does ~TRAIN_TIME_BUDGET_MS of work and yields, needing
|
|
|
|
|
//--- thousands of calls to finish a single era - yet every one of those calls has to be armed from
|
|
|
|
|
//--- here. Gating them on the watermark meant training could only advance when a new BAR closed.
|
|
|
|
|
//--- On H1 that is one 120ms chunk per hour.
|
|
|
|
|
//--- The trap is that dtStudied is two different things: Train() sets it to the training WINDOW
|
|
|
|
|
//--- START (TrainWindowStart, ~2008) while FinalizeTrainRun() sets it to the last bar SCANNED
|
|
|
|
|
//--- (~now). So the moment any run finalized, dtStudied >= lastBarDate and this function went
|
|
|
|
|
//--- silent until the next candle - no era lines, no heartbeats, no stall branch, nothing, because
|
|
|
|
|
//--- Train() was not being CALLED at all. Observed 2026-08-10: four charts, 28 minutes of silence
|
|
|
|
|
//--- between two bursts exactly one H1 bar apart, and the TRAIN STALL line that finally caught it
|
|
|
|
|
//--- reported runActive=Y only because m_trainRunActive had been set microseconds earlier in that
|
|
|
|
|
//--- same call. Before 0c85c54 this was survivable rather than correct: eras were nearly free (the
|
|
|
|
|
//--- saved watermark left almost no bars eligible), so one call per bar still looked like progress.
|
|
|
|
|
//--- There is never a reason to withhold a Train() call from an unconverged model - pause/stop are
|
|
|
|
|
//--- handled by m_trainingPaused/m_trainingStopRequested, which Train() checks for itself.
|
fix: the DB backfill could never run, and HEAD did not compile
Four defects in 64c5dd5/1a05e63, found by review + a baseline compile.
Goals 1-8 of that session are unchanged; this makes 6 and 8 actually reachable.
1. HEAD DID NOT COMPILE - 6 errors. CControlPanel::Minimize/Maximize were
declared `virtual bool ... override`, but CAppDialog declares both as
`virtual void` (Controls\Dialog.mqh). errors 265 + 404 on each, plus 151
on `bool ok = CAppDialog::Minimize()`. Return type is void now; there was
never a success flag to forward. Verified: 0 errors, 0 warnings.
2. THE BACKFILL COULD NEVER ADVANCE, and neither could the OOS continual
simulation (that one has been dead since it was written). Both are armed
at the instant convergence is declared, and both advance only from inside
Train(), one chunk per call. But ScheduleTrainingIfNeeded's only per-tick
ArmStudyEvent site sits in the `else` of a branch taken whenever
m_trainingComplete is set and m_trainRunActive is clear - which is exactly
the state FinalizeTrainRun() leaves behind one line before they are armed.
Train() was never called again, so the walks sat at their start index
forever: no "simulation complete" line, and not one row written to the DB
this feature exists to fill. Only a manual Resume/Retrain unstuck them.
Both flags now keep the model schedulable.
3. IN AI_HYBRID - the mode this ships in - the backfill was never even armed.
Ensemble members deploy at Train() ENTRY and return immediately (so no era
is wasted), which skips the era-end block the backfill was started from.
All four members were a no-op for a second, independent reason. Armed on
the ensemble deploy path too, from m_resumeBars/m_resumeOosCutoff.
4. RE-RUNS DUPLICATED ROWS. RegisterSignal inserts unconditionally - no key,
no duplicate check - and m_dbBackfillDone is in-memory, so every later
attach that retrained to convergence wrote a second full set of rows for
the same bars. The ranking would count one bar once per model that ever
deployed, weighting superseded opinions as heavily as the live one. A
.dbfill marker stamps the deployed era; written only on completion (an
interrupted walk redoes itself rather than ranking a partial window) and
deleted with the other sidecars on reset-weights.
Also: WarmBlocking's timeout was silent, which restored the exact silent
pin failure it was added to prevent - it now says so in the journal, and
returns true for "no reference pairs to wait for" so the warning stays rare
enough to be read.
Not addressed, needs a decision: the backfill scores the OOS window with the
checkpoint that was SELECTED as best on that same window, then writes those
win rates into the table filter weights rank on - the selection set consumed
twice, undiscounted, while the deploy gate right next to it applies a
family-wise correction for exactly that effect. The rows are also simulated
triple-barrier outcomes at today's spread sharing a table with realised
fills. The completion log line now states both plainly.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-16 21:25:51 -04:00
|
|
|
//--- A POST-TRAINING WALK IS ALSO PENDING. Both of these are armed at the moment convergence is
|
|
|
|
|
//--- declared (Training.mqh's era-end block: StartOosContinualSimulation + StartPatternDatabaseBackfill)
|
|
|
|
|
//--- and both advance ONLY from inside Train(), one time-boxed chunk per call - so they need calls
|
|
|
|
|
//--- armed from here exactly like a training run does. Without this term they never got any: the
|
|
|
|
|
//--- convergence that arms them also sets m_trainingComplete, and FinalizeTrainRun() clears
|
|
|
|
|
//--- m_trainRunActive one line earlier, so the branch below took the converged path from that instant
|
|
|
|
|
//--- on and ArmStudyEvent() - the only per-tick arming site in the EA - was never reached again.
|
|
|
|
|
//--- Train() was simply never called, so both walks sat at their start index forever: the
|
|
|
|
|
//--- continual-learning OOS simulation never produced its "simulation complete" line, and the
|
|
|
|
|
//--- pattern-database backfill never wrote a row (the DB it exists to fill stayed empty, which is
|
|
|
|
|
//--- indistinguishable from the feature being absent). Only a manual Resume/Retrain click - which
|
|
|
|
|
//--- arms an event through a different path - could ever unstick them. Found 2026-08-16.
|
|
|
|
|
bool postTrainWalkPending = (m_simOosRunActive || m_dbBackfillActive);
|
|
|
|
|
bool trainingPending = !(m_trainingComplete || m_inferenceOnly) || postTrainWalkPending;
|
2026-08-01 11:27:28 -04:00
|
|
|
//--- m_inferenceOnly (single backtest) takes the converged/inference branch even if the seeded model
|
|
|
|
|
//--- wasn't flagged complete, so a backtest never drops into Train()'s era loop - see m_inferenceOnly.
|
fix: the DB backfill could never run, and HEAD did not compile
Four defects in 64c5dd5/1a05e63, found by review + a baseline compile.
Goals 1-8 of that session are unchanged; this makes 6 and 8 actually reachable.
1. HEAD DID NOT COMPILE - 6 errors. CControlPanel::Minimize/Maximize were
declared `virtual bool ... override`, but CAppDialog declares both as
`virtual void` (Controls\Dialog.mqh). errors 265 + 404 on each, plus 151
on `bool ok = CAppDialog::Minimize()`. Return type is void now; there was
never a success flag to forward. Verified: 0 errors, 0 warnings.
2. THE BACKFILL COULD NEVER ADVANCE, and neither could the OOS continual
simulation (that one has been dead since it was written). Both are armed
at the instant convergence is declared, and both advance only from inside
Train(), one chunk per call. But ScheduleTrainingIfNeeded's only per-tick
ArmStudyEvent site sits in the `else` of a branch taken whenever
m_trainingComplete is set and m_trainRunActive is clear - which is exactly
the state FinalizeTrainRun() leaves behind one line before they are armed.
Train() was never called again, so the walks sat at their start index
forever: no "simulation complete" line, and not one row written to the DB
this feature exists to fill. Only a manual Resume/Retrain unstuck them.
Both flags now keep the model schedulable.
3. IN AI_HYBRID - the mode this ships in - the backfill was never even armed.
Ensemble members deploy at Train() ENTRY and return immediately (so no era
is wasted), which skips the era-end block the backfill was started from.
All four members were a no-op for a second, independent reason. Armed on
the ensemble deploy path too, from m_resumeBars/m_resumeOosCutoff.
4. RE-RUNS DUPLICATED ROWS. RegisterSignal inserts unconditionally - no key,
no duplicate check - and m_dbBackfillDone is in-memory, so every later
attach that retrained to convergence wrote a second full set of rows for
the same bars. The ranking would count one bar once per model that ever
deployed, weighting superseded opinions as heavily as the live one. A
.dbfill marker stamps the deployed era; written only on completion (an
interrupted walk redoes itself rather than ranking a partial window) and
deleted with the other sidecars on reset-weights.
Also: WarmBlocking's timeout was silent, which restored the exact silent
pin failure it was added to prevent - it now says so in the journal, and
returns true for "no reference pairs to wait for" so the warning stays rare
enough to be read.
Not addressed, needs a decision: the backfill scores the OOS window with the
checkpoint that was SELECTED as best on that same window, then writes those
win rates into the table filter weights rank on - the selection set consumed
twice, undiscounted, while the deploy gate right next to it applies a
family-wise correction for exactly that effect. The rows are also simulated
triple-barrier outcomes at today's spread sharing a table with realised
fills. The completion log line now states both plainly.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-16 21:25:51 -04:00
|
|
|
if((m_trainingComplete || m_inferenceOnly) && !m_trainingStopRequested && !m_trainRunActive &&
|
|
|
|
|
!postTrainWalkPending)
|
2026-08-01 11:27:28 -04:00
|
|
|
{
|
|
|
|
|
if(newBarPending)
|
|
|
|
|
RefreshConvergedSignal();
|
|
|
|
|
}
|
|
|
|
|
else
|
fix(ensemble): responsive panel + synchronized eras + combined-vote accuracy
Four user-reported/requested items, one root cause chain:
1) DEAD CONTROL PANEL in AI_HYBRID mode. All members posted custom event
id 1 and handled id 1001, and CExpertCustom broadcasts every chart
event to every filter - so each posted event ran a train chunk in ALL
N members (N*N chunks per round) and the chart thread never idled
long enough to deliver clicks/drags. profiling.csv: 99.45% of time in
OnChartEventHandler. Fix: per-instance study-event ids
(STUDY_EVENT_ID_BASE + construction order, offset above the Controls
library's ON_* codes - id 1 was also ON_DBL_CLICK, so panel
double-clicks fired training chunks). ArmStudyEvent() is the single
post site; lost-event watchdog replaces the accidental
sibling-clears-my-flag rescue.
2) WARM-UP DUPLICATION. The auto-tune sweep is deterministic over
identical features/labels, and it ends in the full MI diagnostic
suite, which the MI-share gate never intercepted on the sweep path -
four members ran four identical ~36s sweep+report blocks. First
member publishes outcome (g_ensembleChartTuneDone/Installed/Settings);
the rest apply it and skip both.
3) DEINIT STRANDED PANEL+ARROWS (user repro 18:52). Root cause from the
log: the 4,500ms budget runs from MetaTrader's stop REQUEST - a heavy
autosave in flight ate it, OnDeinit got ~430ms and died in the first
member's arrow persist ("Abnormal termination" 432ms in). Fix: early
visible-UI sweep (native prefix deletes for status/panel/dialog)
right after ClearStatusLabel, and a fast path for still-training
models - their arrows are re-rendered every era, so they get one bulk
purge instead of scan+atomic-write in the death window.
4) ENSEMBLE FEATURES (user requests): era BARRIER - members advance era
by era together; a member ahead of the slowest still-training member
declines Train() calls and its chunk budget is donated
(TRAIN_TIME_BUDGET_MS = 120/activeTrainers, UI headroom constant).
COMBINED-VOTE OOS SCORE - each member's pass-3 scan contributes its
adjusted per-bar decision (0.0 on abstain) to a shared row buffer;
the last member to finish the era scores the averaged vote vs the
mirrored Min_Vote_Open against the same target-before-stop outcomes
members grade themselves on, publishing an "Ensemble vote" line on
the aggregated panel. Member headlines now carry their lifetime win
rate with break-even.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-16 19:06:04 -04:00
|
|
|
{
|
|
|
|
|
//--- Lost-event watchdog: an armed event that never arrived (chart-event queue overflow) would
|
|
|
|
|
//--- otherwise leave bEventStudy true forever and silently stall training - the accidental rescue
|
|
|
|
|
//--- (any sibling's event clearing this flag) went away with the per-instance ids. See
|
|
|
|
|
//--- STUDY_EVENT_LOST_MS for why a false trip is not a realistic concern.
|
|
|
|
|
if(bEventStudy && GetTickCount() - m_studyArmedTick > STUDY_EVENT_LOST_MS)
|
|
|
|
|
{
|
|
|
|
|
Print(ID + ": study event armed " + IntegerToString(STUDY_EVENT_LOST_MS / 1000) +
|
|
|
|
|
"s ago never arrived (chart-event queue overflow?) - re-arming.");
|
|
|
|
|
bEventStudy = false;
|
|
|
|
|
}
|
fix: training could only advance one 120ms chunk per bar
ScheduleTrainingIfNeeded() armed the next Train() call only when
dtStudied < lastBarDate. That watermark test is right for a CONVERGED
model - one inference refresh per new bar - and wrong for a training
run, because Train() is chunked: it does ~120ms of work and yields,
needing thousands of calls to finish one era, and every one of those
calls has to be armed from there.
dtStudied is two incompatible things. Train() sets it to the training
WINDOW START (~2008); FinalizeTrainRun() sets it to the last bar
SCANNED (~now). So the moment any run finalized, the scheduler went
silent until the next candle closed. On H1 that is one chunk per hour.
The symptom was indistinguishable from a hang: no era lines, no
heartbeats, not one of the six instrumented stall branches - because
Train() was not being CALLED. The TRAIN STALL line that caught it
reported runActive=Y only because m_trainRunActive had been set
microseconds earlier in that same call, and eraResume=N proved no era
was in flight. Two log bursts, 28 minutes apart, exactly one H1 bar.
Before 0c85c54 this was survivable rather than correct: the saved
watermark left almost no bars eligible per era, so eras were nearly
free and one call per bar still looked like progress.
An unconverged model is now always pending. Pause/stop are handled by
m_trainingPaused/m_trainingStopRequested, which Train() checks itself.
Also: the one Train() exit that tears down the whole run on a buffer
failure was completely silent - it now says so. And the build tag moves
to train-dispatch-v2; it had not moved since ce52654, which is why the
running binary could not be identified from its own log.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-10 10:06:49 -04:00
|
|
|
if(!m_trainingStopRequested && !bEventStudy && (newBarPending || trainingPending))
|
fix(ensemble): responsive panel + synchronized eras + combined-vote accuracy
Four user-reported/requested items, one root cause chain:
1) DEAD CONTROL PANEL in AI_HYBRID mode. All members posted custom event
id 1 and handled id 1001, and CExpertCustom broadcasts every chart
event to every filter - so each posted event ran a train chunk in ALL
N members (N*N chunks per round) and the chart thread never idled
long enough to deliver clicks/drags. profiling.csv: 99.45% of time in
OnChartEventHandler. Fix: per-instance study-event ids
(STUDY_EVENT_ID_BASE + construction order, offset above the Controls
library's ON_* codes - id 1 was also ON_DBL_CLICK, so panel
double-clicks fired training chunks). ArmStudyEvent() is the single
post site; lost-event watchdog replaces the accidental
sibling-clears-my-flag rescue.
2) WARM-UP DUPLICATION. The auto-tune sweep is deterministic over
identical features/labels, and it ends in the full MI diagnostic
suite, which the MI-share gate never intercepted on the sweep path -
four members ran four identical ~36s sweep+report blocks. First
member publishes outcome (g_ensembleChartTuneDone/Installed/Settings);
the rest apply it and skip both.
3) DEINIT STRANDED PANEL+ARROWS (user repro 18:52). Root cause from the
log: the 4,500ms budget runs from MetaTrader's stop REQUEST - a heavy
autosave in flight ate it, OnDeinit got ~430ms and died in the first
member's arrow persist ("Abnormal termination" 432ms in). Fix: early
visible-UI sweep (native prefix deletes for status/panel/dialog)
right after ClearStatusLabel, and a fast path for still-training
models - their arrows are re-rendered every era, so they get one bulk
purge instead of scan+atomic-write in the death window.
4) ENSEMBLE FEATURES (user requests): era BARRIER - members advance era
by era together; a member ahead of the slowest still-training member
declines Train() calls and its chunk budget is donated
(TRAIN_TIME_BUDGET_MS = 120/activeTrainers, UI headroom constant).
COMBINED-VOTE OOS SCORE - each member's pass-3 scan contributes its
adjusted per-bar decision (0.0 on abstain) to a shared row buffer;
the last member to finish the era scores the averaged vote vs the
mirrored Min_Vote_Open against the same target-before-stop outcomes
members grade themselves on, publishing an "Ensemble vote" line on
the aggregated panel. Member headlines now carry their lifetime win
rate with break-even.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-16 19:06:04 -04:00
|
|
|
ArmStudyEvent((long)MathMax(0, MathMin(iTime(m_symbol.Name(), PERIOD_CURRENT, (int)(100 * Net.recentAverageSmoothingFactor * (m_trainingComplete ? 1 : 10))), dtStudied)), "New Bar");
|
|
|
|
|
}
|
2026-08-01 11:27:28 -04:00
|
|
|
//--- Train() (see its declaration comment) now yields every ~TRAIN_TIME_BUDGET_MS instead of
|
|
|
|
|
//--- blocking for a whole era, so while a run is active this per-tick line would otherwise
|
|
|
|
|
//--- overwrite Train()'s own full-detail status label on every single tick between chunks -
|
|
|
|
|
//--- flickering between the two instead of showing one steady picture. Only write this terse
|
|
|
|
|
//--- summary when nothing else is actively updating the status label (idle/stopped/paused/cooldown).
|
fix(ensemble+depth): the barrier had no liveness escape, and the depth gate could not report the one state the evidence pointed at
Two charts (USDJPY 50,179 bars / XAUUSD 33,982) sat at era 0 for 38 minutes with
four of their eight members completely silent. Nothing in this commit guesses at
why the sweep fails - the last five guesses were all wrong. It makes the failure
say what it is, and stops one broken member taking its whole chart down with it.
WHAT THE LOG ACTUALLY SAYS, before any of this.
- The running build IS d9f834d (pulled 14:18, compiled 14:19:01, 0 errors), so
every depth instrument from 1dda479/7e63a8b/45c9e21 was live.
- It printed NOTHING. Zero "PRIMING", zero "CAPPED", zero "Per-indicator depth"
in 27 MB of journal. The instrument built to find the depth shortfall returned
"not this".
- On USDJPY at 14:24, CONV-cad8 completed eras 0 AND 1 across all 50,179 bars -
same chart, same 832-value window, same indicators, byte-identical fingerprint -
while LSTM-cad8 and HYB-cad8 reported ok=0 failed=50163. So it is not the
symbol, the history, the bar count or the indicator depth. It is per-member.
- ok=0 means the NEWEST anchors failed too, and a short indicator cannot do that.
The depth reading in project_silent_block_failures is therefore retired by its own
instrumentation. THE ROOT CAUSE IS STILL UNKNOWN and this commit does not claim one.
1. THE DEPTH GATE'S SILENT PATH WAS THE STATE IT WAS HUNTING.
ServableBars() read `if(servable <= 0 || servable >= want) return want;` - one
branch over three unrelated states, silent in all of them:
enabled == 0 -> nothing tunable is on. No cap. Healthy.
enabled > 0, servable == -1 -> a handle answered INVALID.
enabled > 0, servable == 0 -> created, never calculated.
BarsCalculated() returns -1 for a dead handle, so a dead MA is indistinguishable
from "no tunable indicators enabled" - and both returned `want` without printing a
character. That is exactly the state a per-member, every-index, depth-independent
failure produces, and it is the single reason a build carrying full depth
instrumentation logged nothing through the whole outage.
TunableBarsCalculated() now also reports HOW MANY indicators it consulted, and the
dead-handle case is reported (latched, with per-handle depths). The RETURN is
deliberately unchanged - what to do about a dead handle is not yet known, and
changing control flow on an unproven cause is how the last four fixes here went
wrong. SettledBars() routes its three pass-through states via ServableBars() so the
report is reachable from the training sweep, which is the only caller that hits it.
2. THE STALL REPORT NAMED A SLOT, NEVER A BLOCK.
"lookback slot 0 REJECTED (window had 24 of 832 values)" plus a guess ("an
indicator warm-up or a history-edge read"). Which guard fired was INFERRED by
counting 4+5+4+4+6+1 = 24 and concluding feature 25 must be the MA. The arithmetic
was right; every conclusion drawn from it was wrong, because a value count names a
POSITION and a position cannot tell cold from capped from invalid from off-the-end.
Every guard that can reject a bar now records itself - m_featureFailBlock - and the
report carries it, the series index, IndicatorDepthReport()'s per-handle depths,
and for each indicator whether the NEWEST bar reads. That last field is the whole
diagnosis in one word: newest-also-EMPTY means the buffer is unreadable everywhere
(cold or dead handle), newest-reads means a genuine history edge. Instrumented:
open, ATR, MA, RSI, MACD, Ichimoku, and all five AD blocks via ADIndicatorCold().
3. THE TOTAL-FAILURE BACKOFF WAS GATED ON THE WRONG QUESTION.
It armed only when m_featureFailTransient was set. Keeping that flag correct across
every guard is a list that has to stay right forever - the same shape of fix the
feature cache abandoned for the same reason - and the gate is pointless anyway: a
sweep where ZERO of 50,163 bars produced a window will produce zero again if it
restarts a millisecond later, transient or not. Doing that at full speed is what
starved six indicator threads on a six-core box. The backoff is now unconditional
on a total failure. The flag keeps its real job, deciding whether a MISS may be
cached, which is a per-bar question and not a scheduling one.
4. THE ERA BARRIER DEADLOCKED, AND SILENCED THE MEMBERS IT FROZE.
EnsembleMinTrainingEra() exempted deployed, stopped and paused members and its
comment concluded "so nothing deadlocks". Those three are all VOLUNTARY. A member
that simply CANNOT finish an era is none of them, so it pinned the minimum at its
own era with no time limit - and the hold branch's only action was
`m_lastEraCompleteTick = GetTickCount()`, which silences the stall watchdog. So on
USDJPY the two members that could not train reported, and the two healthy members
frozen behind them wrote nothing anywhere. The outage was visible only through the
members that were not suffering it.
- BarrierEraHeartbeat() stamps a clock on real era CHANGE, kept separate from
m_lastEraCompleteTick precisely because the barrier resets that one. Only a
member AT the minimum can be a blocker; a member ahead is idle by design and is
never counted as stuck.
- After ENSEMBLE_BARRIER_STUCK_MS (12 min) a non-advancing member is dropped from
the barrier minimum. It keeps training and rejoins the instant it completes an
era - at which point, being behind, it legitimately becomes the minimum again,
which is the documented resumed-laggard behaviour.
- Both transitions say so loudly, and the release states plainly that the
combined-vote score cannot be computed while the ensemble is desynchronised.
- A held member now writes a rate-limited journal line naming WHICH members it is
waiting on, so the blocker is read off one line.
5. THE PANEL FLICKER.
OnTickHandler gates its terse writer on !m_trainRunActive, and a barrier-held member
returns from Train() before ever setting it - so both writers thought they were the
only one updating the label and fought every tick. That is the reported "Getting
ready..." <-> "Waiting at era N for slower ensemble members" oscillation, and it hit
Perceptron but not Convolutional purely because Convolutional had a run active from
a completed era and Perceptron, resumed from disk, never did. Train()'s message is
the specific one, so it wins.
NEXT STEP once this is running: the stall line now ends in "REJECTED BY: ..." and
the per-handle depths. Read it. Do not reason around it.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-17 15:21:15 -04:00
|
|
|
//--- ...and the same reasoning covers a member held at the ENSEMBLE ERA BARRIER, which the
|
|
|
|
|
//--- !m_trainRunActive test above does not: a held member returns from Train() before it ever sets
|
|
|
|
|
//--- m_trainRunActive, so BOTH writers considered themselves the only one updating the label and
|
|
|
|
|
//--- fought over it every tick. That is the "Getting ready..." <-> "Waiting at era N for slower
|
|
|
|
|
//--- ensemble members" flicker reported on 2026-08-17 - and it appeared on Perceptron but not
|
|
|
|
|
//--- Convolutional purely because Convolutional had a run active from a completed era and Perceptron,
|
|
|
|
|
//--- resumed from disk, never did. Train()'s message is the specific one, so it wins.
|
|
|
|
|
if(m_ensembleMember && EnsembleEraBarrierHolds())
|
|
|
|
|
return;
|
2026-08-01 11:27:28 -04:00
|
|
|
if(!m_trainRunActive)
|
|
|
|
|
{
|
|
|
|
|
//--- Compact, accurate end-state text. The completed state distinguishes a model that is genuinely
|
|
|
|
|
//--- adapting live (online learning active - a live chart with EnableOnlineLearning, not the tester)
|
|
|
|
|
//--- from one running pure inference (the Strategy Tester, or online learning off), so the label is
|
|
|
|
|
//--- literally true either way and never over-promises "keeps learning" when it doesn't - see
|
|
|
|
|
//--- OnlineLearnStep()'s gate for exactly when adaptation runs.
|
|
|
|
|
bool onlineActive = m_enableOnlineLearning && !m_inferenceOnly
|
|
|
|
|
&& !MQLInfoInteger(MQL_TESTER) && !MQLInfoInteger(MQL_OPTIMIZATION) && !MQLInfoInteger(MQL_FORWARD)
|
|
|
|
|
&& CheckPointer(Net) != POINTER_INVALID && !Net.CpuInference();
|
|
|
|
|
//--- Simple end-state panel (default, VerboseMode off): plain-language status + the model's
|
|
|
|
|
//--- compounded/persistent Buy/Sell win-rate (directional accuracy, Neutral excluded - persisted in
|
|
|
|
|
//--- .stats WST5, so it survives a fresh chart reload and is meaningful the moment a drop-and-go user
|
|
|
|
|
//--- attaches the EA) + the current call. The verbose era/forecast dump below stays for power users.
|
|
|
|
|
if(!VerboseMode)
|
|
|
|
|
{
|
|
|
|
|
string statusPlain;
|
|
|
|
|
if(m_trainingComplete)
|
|
|
|
|
statusPlain = onlineActive ? "Live - learning from new bars" : "Ready for live trading";
|
|
|
|
|
else if(m_trainingStopRequested)
|
|
|
|
|
statusPlain = "Paused - progress saved";
|
|
|
|
|
else if(m_trainingPaused)
|
|
|
|
|
statusPlain = "Paused";
|
|
|
|
|
else
|
|
|
|
|
statusPlain = "Getting ready...";
|
|
|
|
|
ENUM_SIGNAL liveSig = DoubleToSignal(dPrevSignal);
|
|
|
|
|
string liveSigPlain = (liveSig == Buy) ? "Buy" : (liveSig == Sell) ? "Sell" : "Neutral (no trade)";
|
|
|
|
|
string simpleLive = DisplayName() + " - " + statusPlain + "\n";
|
|
|
|
|
//--- Only show the accuracy line once at least one signal has been validated (compounded counts
|
|
|
|
|
//--- persist across restarts, so a deployed model shows real numbers immediately, not "measuring").
|
|
|
|
|
if(m_cumIsTotal > 0 || m_cumOosTotal > 0)
|
|
|
|
|
simpleLive += ComputeCompoundedAccuracyLine() + "\n";
|
2026-08-15 16:54:43 -04:00
|
|
|
//--- The ensemble headline is the FIRST line, so lead with the signal - on the combined
|
|
|
|
|
//--- panel each member's one line must answer "what is this model saying right now".
|
|
|
|
|
if(m_ensembleMember)
|
fix(ensemble): responsive panel + synchronized eras + combined-vote accuracy
Four user-reported/requested items, one root cause chain:
1) DEAD CONTROL PANEL in AI_HYBRID mode. All members posted custom event
id 1 and handled id 1001, and CExpertCustom broadcasts every chart
event to every filter - so each posted event ran a train chunk in ALL
N members (N*N chunks per round) and the chart thread never idled
long enough to deliver clicks/drags. profiling.csv: 99.45% of time in
OnChartEventHandler. Fix: per-instance study-event ids
(STUDY_EVENT_ID_BASE + construction order, offset above the Controls
library's ON_* codes - id 1 was also ON_DBL_CLICK, so panel
double-clicks fired training chunks). ArmStudyEvent() is the single
post site; lost-event watchdog replaces the accidental
sibling-clears-my-flag rescue.
2) WARM-UP DUPLICATION. The auto-tune sweep is deterministic over
identical features/labels, and it ends in the full MI diagnostic
suite, which the MI-share gate never intercepted on the sweep path -
four members ran four identical ~36s sweep+report blocks. First
member publishes outcome (g_ensembleChartTuneDone/Installed/Settings);
the rest apply it and skip both.
3) DEINIT STRANDED PANEL+ARROWS (user repro 18:52). Root cause from the
log: the 4,500ms budget runs from MetaTrader's stop REQUEST - a heavy
autosave in flight ate it, OnDeinit got ~430ms and died in the first
member's arrow persist ("Abnormal termination" 432ms in). Fix: early
visible-UI sweep (native prefix deletes for status/panel/dialog)
right after ClearStatusLabel, and a fast path for still-training
models - their arrows are re-rendered every era, so they get one bulk
purge instead of scan+atomic-write in the death window.
4) ENSEMBLE FEATURES (user requests): era BARRIER - members advance era
by era together; a member ahead of the slowest still-training member
declines Train() calls and its chunk budget is donated
(TRAIN_TIME_BUDGET_MS = 120/activeTrainers, UI headroom constant).
COMBINED-VOTE OOS SCORE - each member's pass-3 scan contributes its
adjusted per-bar decision (0.0 on abstain) to a shared row buffer;
the last member to finish the era scores the averaged vote vs the
mirrored Min_Vote_Open against the same target-before-stop outcomes
members grade themselves on, publishing an "Ensemble vote" line on
the aggregated panel. Member headlines now carry their lifetime win
rate with break-even.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-16 19:06:04 -04:00
|
|
|
{
|
2026-08-15 16:54:43 -04:00
|
|
|
simpleLive = statusPlain + " -> " + liveSigPlain;
|
fix(ensemble): responsive panel + synchronized eras + combined-vote accuracy
Four user-reported/requested items, one root cause chain:
1) DEAD CONTROL PANEL in AI_HYBRID mode. All members posted custom event
id 1 and handled id 1001, and CExpertCustom broadcasts every chart
event to every filter - so each posted event ran a train chunk in ALL
N members (N*N chunks per round) and the chart thread never idled
long enough to deliver clicks/drags. profiling.csv: 99.45% of time in
OnChartEventHandler. Fix: per-instance study-event ids
(STUDY_EVENT_ID_BASE + construction order, offset above the Controls
library's ON_* codes - id 1 was also ON_DBL_CLICK, so panel
double-clicks fired training chunks). ArmStudyEvent() is the single
post site; lost-event watchdog replaces the accidental
sibling-clears-my-flag rescue.
2) WARM-UP DUPLICATION. The auto-tune sweep is deterministic over
identical features/labels, and it ends in the full MI diagnostic
suite, which the MI-share gate never intercepted on the sweep path -
four members ran four identical ~36s sweep+report blocks. First
member publishes outcome (g_ensembleChartTuneDone/Installed/Settings);
the rest apply it and skip both.
3) DEINIT STRANDED PANEL+ARROWS (user repro 18:52). Root cause from the
log: the 4,500ms budget runs from MetaTrader's stop REQUEST - a heavy
autosave in flight ate it, OnDeinit got ~430ms and died in the first
member's arrow persist ("Abnormal termination" 432ms in). Fix: early
visible-UI sweep (native prefix deletes for status/panel/dialog)
right after ClearStatusLabel, and a fast path for still-training
models - their arrows are re-rendered every era, so they get one bulk
purge instead of scan+atomic-write in the death window.
4) ENSEMBLE FEATURES (user requests): era BARRIER - members advance era
by era together; a member ahead of the slowest still-training member
declines Train() calls and its chunk budget is donated
(TRAIN_TIME_BUDGET_MS = 120/activeTrainers, UI headroom constant).
COMBINED-VOTE OOS SCORE - each member's pass-3 scan contributes its
adjusted per-bar decision (0.0 on abstain) to a shared row buffer;
the last member to finish the era scores the averaged vote vs the
mirrored Min_Vote_Open against the same target-before-stop outcomes
members grade themselves on, publishing an "Ensemble vote" line on
the aggregated panel. Member headlines now carry their lifetime win
rate with break-even.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-16 19:06:04 -04:00
|
|
|
//--- one-line accuracy on the member headline (user request 2026-08-16: the solo panel
|
|
|
|
|
//--- shows accuracy, the ensemble panel did not). Same counters and same
|
|
|
|
|
//--- always-show-the-break-even doctrine as ComputeCompoundedAccuracyLine - a win rate
|
|
|
|
|
//--- without its geometry's base rate reads as skill when it is chance.
|
|
|
|
|
if(m_cumOosTotal > 0)
|
|
|
|
|
{
|
|
|
|
|
double slBeH = 0.0, tpBeH = 0.0;
|
|
|
|
|
BarrierMultiples(slBeH, tpBeH);
|
|
|
|
|
simpleLive += StringFormat(" | win %d%%", (int)MathRound(m_cumOosCorrect * 100.0 / m_cumOosTotal));
|
|
|
|
|
if(slBeH > 0.0 && tpBeH > 0.0)
|
|
|
|
|
simpleLive += StringFormat(" (need %d%%)", (int)MathRound(100.0 * slBeH / (slBeH + tpBeH)));
|
|
|
|
|
}
|
|
|
|
|
}
|
2026-08-15 16:54:43 -04:00
|
|
|
PublishStatus(simpleLive);
|
2026-08-01 11:27:28 -04:00
|
|
|
return;
|
|
|
|
|
}
|
|
|
|
|
string completeText = onlineActive ? "Complete - live (adapting to new bars)" : "Complete - ready for live (inference)";
|
|
|
|
|
string trainingState = m_trainingStopRequested
|
|
|
|
|
? (m_trainingComplete ? completeText : "Stopped - resumable (weights kept)")
|
|
|
|
|
: (m_trainingPaused ? "Paused" : (m_trainingComplete ? completeText : "In progress"));
|
|
|
|
|
//--- same "Forecast: <signal> -> <value>" line the active training loop's status label ends on
|
|
|
|
|
//--- (see the classLine-terminated StringFormat below), instead of a raw bEventStudy/dPrevSignal/
|
|
|
|
|
//--- dtStudied debug dump - this is what stays on screen once training stops/pauses/completes.
|
2026-08-15 16:54:43 -04:00
|
|
|
PublishStatus(StringFormat(
|
|
|
|
|
ID + " : Era %d -> Training %s\n" +
|
|
|
|
|
"Forecast: %s -> %.2f",
|
|
|
|
|
m_eraCount, trainingState,
|
|
|
|
|
EnumToString(DoubleToSignal(dPrevSignal)), dPrevSignal));
|
2026-08-01 11:27:28 -04:00
|
|
|
}
|
|
|
|
|
}
|
|
|
|
|
//+------------------------------------------------------------------+
|
|
|
|
|
//| Timer-driven equivalent of OnTickHandler()'s scheduling, called |
|
|
|
|
|
//| from Warrior_EA.mq5's always-on OnTimer() so training keeps |
|
|
|
|
|
//| progressing purely on wall-clock time - no dependency on ticks, |
|
|
|
|
|
//| which simply don't arrive while the market is closed. |
|
|
|
|
|
//+------------------------------------------------------------------+
|
|
|
|
|
void CExpertSignalAIBase::PollTraining(void)
|
|
|
|
|
{
|
|
|
|
|
//--- Drain a slice of the queued chart-arrow restore FIRST, and skip training work on any tick where
|
|
|
|
|
//--- restoring is still in flight. Both compete for the one MQL5 thread; letting the arrows finish
|
|
|
|
|
//--- quickly (a few hundred ms of slices) means the user sees a complete chart almost immediately,
|
|
|
|
|
//--- whereas interleaving them with 80ms training chunks would stretch the restore over minutes.
|
fix(shutdown): make ExitPolicy public, and stop every long loop the moment MT5 asks
Two things, one of which was a compile error.
1. ExitPolicy() was declared in the protected block but is pushed in from
Warrior_EA.mq5:770. Moved to public beside the other EA-facing setters.
2. Chart objects surviving OnDeinit. The 4,500 ms teardown budget is measured
from the STOP REQUEST, not from OnDeinit's first line, and OnDeinit cannot
begin until whatever is in flight returns - so a scan still running after
_StopFlag is raised does not delay the cleanup, it SPENDS it, and the purge
never gets its turn.
New CExpertSignalAIBase::ShutdownRequested() = IsStopped() || m_shutdownInProgress.
Deliberately NOT m_trainingStopRequested: that latches, and a latched flag
would permanently disable scans that must run again on the next Start.
Guarded, longest first:
- TuneIndicatorsByFilter - per candidate, restoring the OPERATOR's settings
on the way out (best[] is mutated in place; the tuner otherwise keeps the
last trial's parameters, which nothing chose).
- ReportBarrierGeometryScan - per pairing, breaking to ONE restore point so
m_barrierScanLiveLabels can never be left true (that makes ComputeLabelForBar
read the last candidate's multiples as the configured geometry).
- ReportFeatureLabelInformation / ReportExcursionInformation / lag profile -
nulls ABANDON rather than truncate: fewer draws is not a smaller null, it
is a wrong one, and p shifts toward significance. m_dirEvidence staying
false is the safe direction.
- SimulateExitPolicyOutcomes - zeroes its accumulators so the divergence line
is dropped instead of latching a partial expectancy as the run's only report.
- ReportGeometryExpectancyScan - per ladder rung.
- HttpGet - one choke point for up to a dozen blocking WebRequests per
first-pass Update(). An in-flight request cannot be cancelled; refusing to
start another is the whole remedy.
- PollTraining, OnChartEventHandler's study event, TuneIndicatorsAndTrain -
entry points, so a queued event cannot open an era during teardown.
TuneIndicatorsAndTrain's guard is the first statement, ahead of the
m_tuneFilterDone / g_ensembleChartTuneDone latches.
- OnTick / OnTimer / OnChartEvent.
Training's own bar loops already honoured this (pass 1 per bar, passes 2/2.5/3
yield on a 120 ms budget); the warm-up scans did not, and they are the longest
uninterruptible stretches the EA has.
StopTraining() is unchanged: the operator's Stop still finalises synchronously.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-17 17:03:11 -04:00
|
|
|
//--- STOP BEFORE ANY OF IT. Everything below either draws objects on the chart (the arrow restore and
|
|
|
|
|
//--- the rescan) or starts a training chunk, and OnDeinit is about to delete every one of those objects
|
|
|
|
|
//--- again - while the budget it has to do it in is already running. Drawing arrows for a chart that is
|
|
|
|
|
//--- unloading is the worst case of the three: it lengthens the very sweep that has to finish.
|
|
|
|
|
if(ShutdownRequested())
|
|
|
|
|
return;
|
2026-08-01 11:27:28 -04:00
|
|
|
if(m_arrowRestorePending)
|
|
|
|
|
{
|
|
|
|
|
AdvanceChartSignalRestore();
|
|
|
|
|
return;
|
|
|
|
|
}
|
|
|
|
|
//--- Same one-thread reasoning as the arrow restore above: a manual rescan (Show Signals) also competes
|
|
|
|
|
//--- for the single MQL5 thread, and its per-bar feedForward is real compute rather than a cheap object
|
|
|
|
|
//--- write, so it must finish its own slices before training resumes rather than interleaving with it.
|
|
|
|
|
if(m_rescanPending)
|
|
|
|
|
{
|
|
|
|
|
AdvanceChartSignalRescan();
|
|
|
|
|
return;
|
|
|
|
|
}
|
|
|
|
|
if(m_isInitialized)
|
|
|
|
|
ScheduleTrainingIfNeeded();
|
|
|
|
|
}
|
|
|
|
|
//+------------------------------------------------------------------+
|
|
|
|
|
//| |
|
|
|
|
|
//+------------------------------------------------------------------+
|
|
|
|
|
void CExpertSignalAIBase::OnChartEventHandler(const int id,
|
|
|
|
|
const long &lparam,
|
|
|
|
|
const double &dparam,
|
|
|
|
|
const string &sparam)
|
|
|
|
|
{
|
fix(ensemble): responsive panel + synchronized eras + combined-vote accuracy
Four user-reported/requested items, one root cause chain:
1) DEAD CONTROL PANEL in AI_HYBRID mode. All members posted custom event
id 1 and handled id 1001, and CExpertCustom broadcasts every chart
event to every filter - so each posted event ran a train chunk in ALL
N members (N*N chunks per round) and the chart thread never idled
long enough to deliver clicks/drags. profiling.csv: 99.45% of time in
OnChartEventHandler. Fix: per-instance study-event ids
(STUDY_EVENT_ID_BASE + construction order, offset above the Controls
library's ON_* codes - id 1 was also ON_DBL_CLICK, so panel
double-clicks fired training chunks). ArmStudyEvent() is the single
post site; lost-event watchdog replaces the accidental
sibling-clears-my-flag rescue.
2) WARM-UP DUPLICATION. The auto-tune sweep is deterministic over
identical features/labels, and it ends in the full MI diagnostic
suite, which the MI-share gate never intercepted on the sweep path -
four members ran four identical ~36s sweep+report blocks. First
member publishes outcome (g_ensembleChartTuneDone/Installed/Settings);
the rest apply it and skip both.
3) DEINIT STRANDED PANEL+ARROWS (user repro 18:52). Root cause from the
log: the 4,500ms budget runs from MetaTrader's stop REQUEST - a heavy
autosave in flight ate it, OnDeinit got ~430ms and died in the first
member's arrow persist ("Abnormal termination" 432ms in). Fix: early
visible-UI sweep (native prefix deletes for status/panel/dialog)
right after ClearStatusLabel, and a fast path for still-training
models - their arrows are re-rendered every era, so they get one bulk
purge instead of scan+atomic-write in the death window.
4) ENSEMBLE FEATURES (user requests): era BARRIER - members advance era
by era together; a member ahead of the slowest still-training member
declines Train() calls and its chunk budget is donated
(TRAIN_TIME_BUDGET_MS = 120/activeTrainers, UI headroom constant).
COMBINED-VOTE OOS SCORE - each member's pass-3 scan contributes its
adjusted per-bar decision (0.0 on abstain) to a shared row buffer;
the last member to finish the era scores the averaged vote vs the
mirrored Min_Vote_Open against the same target-before-stop outcomes
members grade themselves on, publishing an "Ensemble vote" line on
the aggregated panel. Member headlines now carry their lifetime win
rate with break-even.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-16 19:06:04 -04:00
|
|
|
//--- Match THIS instance's id only. CExpertCustom broadcasts every chart event to every filter, so
|
|
|
|
|
//--- matching a shared id here (the old `id == 1001`) made each posted event run a train chunk in
|
|
|
|
|
//--- ALL ensemble members - N members posting per round times N members handling each = N*N chunks,
|
|
|
|
|
//--- and the chart thread never idled long enough to deliver the control panel's clicks and drags.
|
|
|
|
|
//--- See STUDY_EVENT_ID_BASE (ExpertSignalAIBase.mqh) for the full failure shape.
|
|
|
|
|
if(id == CHARTEVENT_CUSTOM + m_studyEventId)
|
2026-08-01 11:27:28 -04:00
|
|
|
{
|
fix(shutdown): make ExitPolicy public, and stop every long loop the moment MT5 asks
Two things, one of which was a compile error.
1. ExitPolicy() was declared in the protected block but is pushed in from
Warrior_EA.mq5:770. Moved to public beside the other EA-facing setters.
2. Chart objects surviving OnDeinit. The 4,500 ms teardown budget is measured
from the STOP REQUEST, not from OnDeinit's first line, and OnDeinit cannot
begin until whatever is in flight returns - so a scan still running after
_StopFlag is raised does not delay the cleanup, it SPENDS it, and the purge
never gets its turn.
New CExpertSignalAIBase::ShutdownRequested() = IsStopped() || m_shutdownInProgress.
Deliberately NOT m_trainingStopRequested: that latches, and a latched flag
would permanently disable scans that must run again on the next Start.
Guarded, longest first:
- TuneIndicatorsByFilter - per candidate, restoring the OPERATOR's settings
on the way out (best[] is mutated in place; the tuner otherwise keeps the
last trial's parameters, which nothing chose).
- ReportBarrierGeometryScan - per pairing, breaking to ONE restore point so
m_barrierScanLiveLabels can never be left true (that makes ComputeLabelForBar
read the last candidate's multiples as the configured geometry).
- ReportFeatureLabelInformation / ReportExcursionInformation / lag profile -
nulls ABANDON rather than truncate: fewer draws is not a smaller null, it
is a wrong one, and p shifts toward significance. m_dirEvidence staying
false is the safe direction.
- SimulateExitPolicyOutcomes - zeroes its accumulators so the divergence line
is dropped instead of latching a partial expectancy as the run's only report.
- ReportGeometryExpectancyScan - per ladder rung.
- HttpGet - one choke point for up to a dozen blocking WebRequests per
first-pass Update(). An in-flight request cannot be cancelled; refusing to
start another is the whole remedy.
- PollTraining, OnChartEventHandler's study event, TuneIndicatorsAndTrain -
entry points, so a queued event cannot open an era during teardown.
TuneIndicatorsAndTrain's guard is the first statement, ahead of the
m_tuneFilterDone / g_ensembleChartTuneDone latches.
- OnTick / OnTimer / OnChartEvent.
Training's own bar loops already honoured this (pass 1 per bar, passes 2/2.5/3
yield on a 120 ms budget); the warm-up scans did not, and they are the longest
uninterruptible stretches the EA has.
StopTraining() is unchanged: the operator's Stop still finalises synchronously.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-17 17:03:11 -04:00
|
|
|
//--- The study event IS the training driver, so a queued one landing after the stop request would
|
|
|
|
|
//--- open a whole era inside the teardown window. Checked here as well as in Warrior_EA.mq5's
|
|
|
|
|
//--- OnChartEvent because CExpertCustom re-posts these between members.
|
|
|
|
|
if(ShutdownRequested())
|
|
|
|
|
return;
|
2026-08-01 11:27:28 -04:00
|
|
|
TuneIndicatorsAndTrain(lparam);
|
|
|
|
|
bEventStudy = false;
|
|
|
|
|
OnTickHandler();
|
|
|
|
|
}
|
|
|
|
|
}
|
|
|
|
|
//+------------------------------------------------------------------+
|
|
|
|
|
//| Claim m_activeFileName for this chart, terminal-wide. |
|
|
|
|
|
//| |
|
feat(ensemble): per-NN inputs replace the preset selector - the meta head becomes the vote's gate
User design (2026-08-19): 'remove the enum menu that selects neural networks... individual
inputs for every NN just like classic signals... the META NN should be integrated into the
voting decision pipeline when enabled... as a bonus meta labelling is applied to enabled NNs.'
- AI_CHOICE is GONE (tombstoned per the stale-.set doctrine). Use_MLP/Use_CONV/Use_LSTM/
Use_CONVLSTM are ordinary bools like the classic votes; the ensemble arithmetic adapts to
any subset because the consensus divisor is the enabled capable weight. Two or more
enabled = ensemble (|ENS1 token + joint gate, exactly the old AI_HYBRID fingerprints, so
existing weight files keep loading); one = the old solo preset; none = classic-only.
- Use_MetaLabeling un-couples META from the direction NNs (the old selector made them
mutually exclusive). S3 ships: CSignalMETA::LiveMetaGate scores each vote-cleared entry
(shared window at bar 1 + proposal descriptor: side, net vote, live geometry, spread/ATR;
pattern one-hot ZEROED - ranking, not calibrated probability, documented in the body) and
vetoes below the cost-adjusted break-even. Entries only; fail-open everywhere, loudly.
- COEXISTENCE HAZARDS closed: VoteCapableWeight()=0 and ProspectiveVote()=false for the
meta target - solo-only until today, a trained META would otherwise sit in the consensus
divisor as a permanent abstainer and shrink every vote by its module weight.
- CERTIFIED == TRADED: the ensemble era verdict replays the identical veto through the same
g_warriorMetaGate pointer over its OOS fired bars (bar re-resolved from the row's own
time; fail-open counted as fires and reported: 'metaGate: N approved, M vetoed, K
unscored'). The overlay deliberately does NOT replay it (veto-filter-in-replay class,
calendar-cliff precedent) - documented at the sweep site. Solo charts' own gate does not
model the veto - the standing solo-gate caveat, documented at the input.
- DB continuity: the pattern/journal DB fingerprint's first slot was (int)AIType;
DbLegacyAiSlot() maps every legacy-expressible config to its OLD value (new 2-3 member
subsets get 100+bitmask, outside the legacy range) so no existing database re-keys.
filterID becomes the enabled roster via one EnabledNNSummary().
- HUD: the meta line shows the gate (armed/(trn), last P vs BE, ok/veto tally); the
armed/disarmed announcement fires on state change via one latch (MetaGateArmedNow), not
only when an entry happens to be proposed.
NOT COMPILED - user compiles in MetaEditor.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-19 13:01:02 -04:00
|
|
|
//| Two charts running the same NN set with the same retrain-affecting|
|
2026-08-01 11:27:28 -04:00
|
|
|
//| inputs resolve to the SAME .nnw/.cfg/.stats/checkpoint set. Both |
|
|
|
|
|
//| then train independently and save over each other, so whichever |
|
|
|
|
|
//| writes last wins and the other's eras are discarded - silently, |
|
|
|
|
|
//| because every individual file operation succeeds. A five-chart |
|
|
|
|
|
//| comparison run on 2026-07-29 lost both its HYBRID models this way |
|
feat(ensemble): per-NN inputs replace the preset selector - the meta head becomes the vote's gate
User design (2026-08-19): 'remove the enum menu that selects neural networks... individual
inputs for every NN just like classic signals... the META NN should be integrated into the
voting decision pipeline when enabled... as a bonus meta labelling is applied to enabled NNs.'
- AI_CHOICE is GONE (tombstoned per the stale-.set doctrine). Use_MLP/Use_CONV/Use_LSTM/
Use_CONVLSTM are ordinary bools like the classic votes; the ensemble arithmetic adapts to
any subset because the consensus divisor is the enabled capable weight. Two or more
enabled = ensemble (|ENS1 token + joint gate, exactly the old AI_HYBRID fingerprints, so
existing weight files keep loading); one = the old solo preset; none = classic-only.
- Use_MetaLabeling un-couples META from the direction NNs (the old selector made them
mutually exclusive). S3 ships: CSignalMETA::LiveMetaGate scores each vote-cleared entry
(shared window at bar 1 + proposal descriptor: side, net vote, live geometry, spread/ATR;
pattern one-hot ZEROED - ranking, not calibrated probability, documented in the body) and
vetoes below the cost-adjusted break-even. Entries only; fail-open everywhere, loudly.
- COEXISTENCE HAZARDS closed: VoteCapableWeight()=0 and ProspectiveVote()=false for the
meta target - solo-only until today, a trained META would otherwise sit in the consensus
divisor as a permanent abstainer and shrink every vote by its module weight.
- CERTIFIED == TRADED: the ensemble era verdict replays the identical veto through the same
g_warriorMetaGate pointer over its OOS fired bars (bar re-resolved from the row's own
time; fail-open counted as fires and reported: 'metaGate: N approved, M vetoed, K
unscored'). The overlay deliberately does NOT replay it (veto-filter-in-replay class,
calendar-cliff precedent) - documented at the sweep site. Solo charts' own gate does not
model the veto - the standing solo-gate caveat, documented at the input.
- DB continuity: the pattern/journal DB fingerprint's first slot was (int)AIType;
DbLegacyAiSlot() maps every legacy-expressible config to its OLD value (new 2-3 member
subsets get 100+bitmask, outside the legacy range) so no existing database re-keys.
filterID becomes the enabled roster via one EnabledNNSummary().
- HUD: the meta line shows the gate (armed/(trn), last P vs BE, ok/veto tally); the
armed/disarmed announcement fires on state change via one latch (MetaGateArmedNow), not
only when an entry happens to be proposed.
NOT COMPILED - user compiles in MetaEditor.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-19 13:01:02 -04:00
|
|
|
//| (one chart left at the default NN set), and the only evidence |
|
2026-08-01 11:27:28 -04:00
|
|
|
//| anywhere was that model path appearing twice as often in the log. |
|
|
|
|
|
//| |
|
|
|
|
|
//| A terminal-wide global variable is the right lock rather than a |
|
|
|
|
|
//| lock FILE: GlobalVariableTemp() is an atomic create-if-absent, |
|
|
|
|
|
//| and a TEMPORARY variable dies with the terminal, so a crash can |
|
|
|
|
|
//| never leave a stale lock that blocks the next start. Within one |
|
|
|
|
|
//| session a stale entry is still possible (an EA removed without a |
|
|
|
|
|
//| clean deinit), so the owner's chart id is stored and revalidated. |
|
|
|
|
|
//+------------------------------------------------------------------+
|
|
|
|
|
bool CExpertSignalAIBase::AcquireConfigLock(void)
|
|
|
|
|
{
|
|
|
|
|
//--- FNV-1a over the resolved filename: every retrain-affecting input is already folded into that
|
|
|
|
|
//--- name, so equal names mean genuinely equal configs and nothing else has to be compared. Hashed
|
|
|
|
|
//--- because MQL5 caps global-variable names at 63 characters and the path alone exceeds that.
|
|
|
|
|
uint h = 2166136261;
|
|
|
|
|
int len = StringLen(m_activeFileName);
|
|
|
|
|
for(int i = 0; i < len; i++)
|
|
|
|
|
{
|
|
|
|
|
h ^= (uint)StringGetCharacter(m_activeFileName, i);
|
|
|
|
|
h *= 16777619;
|
|
|
|
|
}
|
|
|
|
|
string name = "WarriorAI_" + m_id + "_" + StringFormat("%08x", h);
|
|
|
|
|
long self = ChartID();
|
|
|
|
|
//--- Atomic: true means it did not exist and is now ours.
|
|
|
|
|
if(GlobalVariableTemp(name))
|
|
|
|
|
{
|
|
|
|
|
GlobalVariableSet(name, (double)self);
|
|
|
|
|
m_configLockName = name;
|
|
|
|
|
return true;
|
|
|
|
|
}
|
|
|
|
|
long owner = (long)GlobalVariableGet(name);
|
|
|
|
|
//--- Our own entry: this chart is re-initializing after a parameter change or a recompile whose
|
|
|
|
|
//--- OnDeinit never reached ReleaseConfigLock(). Reclaim it instead of refusing to start.
|
|
|
|
|
if(owner == self)
|
|
|
|
|
{
|
|
|
|
|
m_configLockName = name;
|
|
|
|
|
return true;
|
|
|
|
|
}
|
|
|
|
|
//--- Owner recorded but its chart no longer runs an expert - take the claim over. owner == 0 is
|
|
|
|
|
//--- deliberately NOT treated as stale: it means another instance created the variable microseconds
|
|
|
|
|
//--- ago and has not stamped its id yet, which is a live claim, not a dead one.
|
|
|
|
|
bool ownerAlive = false;
|
|
|
|
|
if(owner != 0)
|
|
|
|
|
{
|
|
|
|
|
long id = ChartFirst();
|
|
|
|
|
while(id >= 0)
|
|
|
|
|
{
|
|
|
|
|
if(id == owner)
|
|
|
|
|
{
|
|
|
|
|
ownerAlive = (StringLen(ChartGetString(id, CHART_EXPERT_NAME)) > 0);
|
|
|
|
|
break;
|
|
|
|
|
}
|
|
|
|
|
id = ChartNext(id);
|
|
|
|
|
}
|
|
|
|
|
}
|
|
|
|
|
if(owner != 0 && !ownerAlive)
|
|
|
|
|
{
|
|
|
|
|
GlobalVariableSet(name, (double)self);
|
|
|
|
|
m_configLockName = name;
|
|
|
|
|
return true;
|
|
|
|
|
}
|
|
|
|
|
Print(ID + ": REFUSED to start - another chart is already training this exact configuration. Both" +
|
|
|
|
|
" would save into the same files (" + m_activeFileName + ".nnw plus its .cfg/.stats/checkpoints)" +
|
|
|
|
|
" and overwrite each other's progress with no error reported anywhere. Owner: " +
|
|
|
|
|
(owner != 0 ? "chart " + IntegerToString(owner) + " (" + ChartSymbol(owner) + " " +
|
|
|
|
|
EnumToString((ENUM_TIMEFRAMES)ChartPeriod(owner)) + ")" : "another chart, still initializing") +
|
feat(ensemble): per-NN inputs replace the preset selector - the meta head becomes the vote's gate
User design (2026-08-19): 'remove the enum menu that selects neural networks... individual
inputs for every NN just like classic signals... the META NN should be integrated into the
voting decision pipeline when enabled... as a bonus meta labelling is applied to enabled NNs.'
- AI_CHOICE is GONE (tombstoned per the stale-.set doctrine). Use_MLP/Use_CONV/Use_LSTM/
Use_CONVLSTM are ordinary bools like the classic votes; the ensemble arithmetic adapts to
any subset because the consensus divisor is the enabled capable weight. Two or more
enabled = ensemble (|ENS1 token + joint gate, exactly the old AI_HYBRID fingerprints, so
existing weight files keep loading); one = the old solo preset; none = classic-only.
- Use_MetaLabeling un-couples META from the direction NNs (the old selector made them
mutually exclusive). S3 ships: CSignalMETA::LiveMetaGate scores each vote-cleared entry
(shared window at bar 1 + proposal descriptor: side, net vote, live geometry, spread/ATR;
pattern one-hot ZEROED - ranking, not calibrated probability, documented in the body) and
vetoes below the cost-adjusted break-even. Entries only; fail-open everywhere, loudly.
- COEXISTENCE HAZARDS closed: VoteCapableWeight()=0 and ProspectiveVote()=false for the
meta target - solo-only until today, a trained META would otherwise sit in the consensus
divisor as a permanent abstainer and shrink every vote by its module weight.
- CERTIFIED == TRADED: the ensemble era verdict replays the identical veto through the same
g_warriorMetaGate pointer over its OOS fired bars (bar re-resolved from the row's own
time; fail-open counted as fires and reported: 'metaGate: N approved, M vetoed, K
unscored'). The overlay deliberately does NOT replay it (veto-filter-in-replay class,
calendar-cliff precedent) - documented at the sweep site. Solo charts' own gate does not
model the veto - the standing solo-gate caveat, documented at the input.
- DB continuity: the pattern/journal DB fingerprint's first slot was (int)AIType;
DbLegacyAiSlot() maps every legacy-expressible config to its OLD value (new 2-3 member
subsets get 100+bitmask, outside the legacy range) so no existing database re-keys.
filterID becomes the enabled roster via one EnabledNNSummary().
- HUD: the meta line shows the gate (armed/(trn), last P vs BE, ok/veto tally); the
armed/disarmed announcement fires on state change via one latch (MetaGateArmedNow), not
only when an entry happens to be proposed.
NOT COMPILED - user compiles in MetaEditor.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-19 13:01:02 -04:00
|
|
|
". Change the enabled NN set (Use_MLP/Use_CONV/Use_LSTM/Use_CONVLSTM) or any retrain-affecting" +
|
|
|
|
|
" input on THIS chart so it trains its own model, or remove one of the two charts. Note the" +
|
|
|
|
|
" private build defaults ALL FOUR direction NNs on - two same-symbol charts left at their" +
|
|
|
|
|
" defaults land here.");
|
2026-08-01 11:27:28 -04:00
|
|
|
return false;
|
|
|
|
|
}
|
|
|
|
|
//+------------------------------------------------------------------+
|
|
|
|
|
//| Drop this instance's claim (see AcquireConfigLock). |
|
|
|
|
|
//+------------------------------------------------------------------+
|
|
|
|
|
void CExpertSignalAIBase::ReleaseConfigLock(void)
|
|
|
|
|
{
|
|
|
|
|
if(StringLen(m_configLockName) == 0)
|
|
|
|
|
return;
|
|
|
|
|
//--- Only delete a claim we still hold: if a later instance took this entry over via the stale-owner
|
|
|
|
|
//--- path above, deleting it here would silently hand the config to a third chart.
|
|
|
|
|
if((long)GlobalVariableGet(m_configLockName) == ChartID())
|
|
|
|
|
GlobalVariableDel(m_configLockName);
|
|
|
|
|
m_configLockName = "";
|
|
|
|
|
}
|
|
|
|
|
#endif
|