Warrior_EA/Expert/AIBase/Lifecycle.mqh

1144 lines
57 KiB
MQL5
Raw Permalink Normal View History

feat(target): delete the barrier/geometry stack - the label is the verdict Step 3 of the swing-pivot plan, whole-hog. The swing label is now the ONE target and the era verdict is precision + recall per class against the label's own base rate - no win rate, no break-even, no expectancy, no geometry anywhere in training. DELETED - Expert/Excursion/ (4), Expert/BarrierHorizon/ (4), GeometrySweep, FirstPassageLadder, Labeling/TripleBarrier.mqh (CLabelOverlap survives in Labeling/LabelOverlap.mqh), 3 test EAs. - TripleBarrierLabel + walk, fractal label, geometry derivation/scan/ adoption, exit-policy replay, excursion MI targets, the drift verdict (DIRECTION_INTELLIGENT), the recall floor, balanced-accuracy telemetry, the barrier defines, the .cfg geometry adopt (slots kept as zeros for the positional layout), the derived-geometry live-order override. - TRAINING_TARGET input/enum: direction models are always swing; META2 re-keys the meta head onto label agreement (descriptor loses its two geometry slots). REWORKED - Labels.mqh (1795 -> ~370 lines): AdvanceSwingLabelState with FINALITY-GATED CACHING - an unresolved bar (pivot pair uncommitted) is never cached, so it can never freeze as a false Neutral; training, calibration, OOS scoring and online learning all skip unresolved bars. - SDeployVerdict: significance-only; SOosTally chance = larger directional class share; pooled gate poolability = timeframe (record v2). - Purge/embargo/declustering gaps: the measured mean label resolution lag (LabelResolutionBars), not a barrier horizon. - Pool purge key + backfill DB rows: marked at the bar the label resolved on (m_labelResolveAge), not a fabricated barrier touch. - Online learning frontier: finality, not a horizon delay. - m_bestBalancedOos -> m_bestSelectionScore, m_erasSinceBestBalanced -> m_erasSinceBest, ensemble vote outcome arrays -> label arrays. STEP 4 folded in: Entry_Multiplier / SL_Mode / TP_Mode / tradingdirection are inputs again - trade management is the tester GA's search space. Fingerprints: every direction model re-keys (TGT:SWG1 now unconditional, CUT token gone); META1 -> META2. Full retrain, as planned. Compile-verified in _claude_stage: Warrior_EA + both surviving test EAs, 0 errors, 0 warnings each. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-24 20:42:31 -04:00
//+------------------------------------------------------------------+
//+------------------------------------------------------------------+
//| Lifecycle.mqh |
//| |
//| Construction/destruction (the composition root - every |
//| collaborator's Bind() call lives here), the CExpertSignal vote |
//| API (LongCondition/ShortCondition/ConfidenceTier/pattern |
//| weights), and tick + chart-event dispatch. The per-config chart |
//| lock now lives on CConfigLock - see Expert\ConfigLock\ConfigLock.mqh. |
//+------------------------------------------------------------------+
#ifndef WARRIOR_AIBASE_LIFECYCLE_MQH
#define WARRIOR_AIBASE_LIFECYCLE_MQH
//+------------------------------------------------------------------+
//| Constructor |
//+------------------------------------------------------------------+
//--- These are only fallback defaults for a fresh object before the EA's OnInit() applies the
//--- active input values via the public setters in Warrior_EA.mq5. The input-driven values are the
//--- source of truth for the actual run configuration.
CExpertSignalAIBase::CExpertSignalAIBase(void) :
ID("NULL"),
m_neuronsCount(0),
m_minTrainYear(1970),
m_optimizationAlgo(TrainingOptimizer), // see the member declaration comment
//--- Placeholder only; InitNeuralNetwork() replaces it with ComputeFirstLayerWidth() before anything
//--- reads it. Deliberately the floor rather than 0, so a hypothetical path that built a topology
//--- without going through init would produce a small usable net instead of a zero-width layer.
m_initialNeuronsCount(FIRST_LAYER_MIN_WIDTH),
m_outputNeuronsCount(OUTPUT_CLASSIFICATION),
//--- Frozen. Nothing reads these to build a topology any more - the taper derives its own
//--- endpoints (BuildFreshTopology) - but they still occupy positional slots in the .cfg sidecar
//--- and the weights fingerprint.
m_minNeuronsCount(MIN_NEURONS_20),
m_neuronsReduction(RF_70),
m_hiddenLayersCount(3),
m_lstmHiddenSize(32),
m_convFilterCount(16),
m_historyBars(14),
m_fractalPeriods(5),
m_pattern_0(25),
m_pattern_1(50),
m_pattern_2(75),
m_pattern_3(100),
m_ensembleMember(false),
m_ensemblePanelSlot(-1),
m_useVolumes(true),
m_useTime(true),
m_useATR(true),
m_useMA(false),
m_useSwingContext(false),
m_useNews(false),
fix(signals): revive a dead MA model, and demote Sanyaku from state to event Two defects surfaced by research/test_classic.py, both verified fixed by re-running the transcription against 178k bars of EURUSD H1. CSignalMA model 1 could never fire. For any recursive average - and MA_TYPE_EMA is the shipped default - MA(i) = a*Close(i) + (1-a)*MA(i+1), so DiffMA(i) = a * (Close(i) - MA(i+1)) DiffCloseMA(i) = (1-a) * (Close(i) - MA(i+1)) are positive multiples of one quantity and always share a sign. Model 1 asks for a close BELOW a RISING average, which is precisely the combination that identity forbids: 0.000% of bars, either direction, any symbol. The MQL5 standard library this was ported from defaults to MODE_SMA, where the two are merely correlated - the bug arrived with the EMA default, not with the port. Reading the slope one bar back (DiffMAPrev) breaks the tie for every MA type while keeping the model's stated meaning. Now fires on 7.92% of bars. CSignalIchimoku model 11 fired on 27% of bars at weight 100. Sanyaku is three standing STATES conjoined with no transition term, so it held across long stretches - and being last in the if-chain at the top weight, the module's highest-conviction reading was also its most common one, overwriting all eight event models below it on a quarter of all bars. The old comment rejected an event form because "demanding all three flip on the same bar would fire almost never" - true, but that is not the alternative. Kouten is the TURN: the ALIGNMENT transitions, and only one role need change for it to. Testing !Sanyaku(idx+1) fires once per aligned stretch. Now 2.17%, in line with Kumo breakout (2.4%) and the strong TK cross (1.1%). DataReady() extended one bar deeper to cover the lookback. Neither pattern showed edge before or after; this is about the models meaning what they say and the vote not being dominated by a constant. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-01 17:14:34 -04:00
m_useCrossAsset(false),
feat(ai): spread as a volatility-regime feature, and fix a stale-index cache in both new blocks Adds spread/ATR and the spread change ratio as network inputs (EnableSpreadFeature, default on). Spread is the one microstructure channel that is both FX-available and genuinely historical in the Strategy Tester - "during testing, the spread is not modeled but is taken from historical data" - so unlike swap, signed tick flow or depth of market it is something a backtest can honestly validate. What it encodes, stated precisely because the raw measurement overstates it. research/test_spread.py found spr/atr the strongest single feature in this codebase, on 5 of 8 instrument/geometry cells at 2-4x any volume feature. But the barrier LABEL charges the spread inside its own barriers, so a wide-spread bar is mechanically likelier to resolve as a loss and the feature would partly be predicting its own cost model. Relabelling at zero cost and re-measuring the identical feature showed 20-40% of it WAS that tautology and the majority was not (XAUUSD retained 97%). What survives is a volatility-regime reading: spread is near-fixed while ATR is not, so the ratio runs high exactly when realised volatility is below its own ATR estimate, which genuinely predicts whether ATR-scaled barriers get reached. It is UNSIGNED - Neutral-vs-directional only, never a side. Also fixes a stale-index bug I introduced with the cross-asset panel and had just repeated in the spread series. Both cached on length alone: if(m_crossAsset.Bars() >= bars) return true; MQL5 series indices are relative to NOW, so one new closed candle shifts every index by one. Keyed only on length, the panel keeps serving its index 0 as a bar that is no longer the newest, and every cross-asset value is read one bar out of step with the price features sitting beside it in the same vector - silently, with no error and no shape change. This is the same class of defect as the dtStudied watermark behind the zero-direction backtests. Both now carry a datetime anchor on m_Time.GetData(0), the same invalidation key the label/feature bar caches already use. And a performance fix that fell out of it: with correct invalidation the panel rebuilds on every new bar, and RefreshConvergedSignal runs per bar - which in the tester would mean one full multi-symbol resample per simulated bar at training depth. Inference only reads bars 0..m_historyBars-1 plus the panel's own slow window, so it now requests exactly that. The cache check is >=, so a deeper panel left from training still satisfies it. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-01 17:42:40 -04:00
m_useSpreadFeature(false),
m_crossAssetPairsPinned(""),
m_crossAssetCfgSaved(false),
m_useAltData(false),
m_altDataEnabled(true),
m_altDataLateWarned(false),
m_altDataNamesPinned(""),
m_newsFeatureWindowMinutes(60),
m_autoTuneIndicators(false),
m_indicatorsPtr(NULL),
Net(NULL),
TempData(NULL),
dError(-1),
dUndefine(0),
dForecast(0),
dPrevSignal(0),
m_refreshOk(0),
m_refreshFailFeatures(0),
m_refreshFailShort(0),
m_refreshBuy(0),
m_refreshSell(0),
m_refreshNeutral(0),
m_voteGateBlocked(0),
m_voteGatePassed(0),
m_voteGateCompleteAtFirst(-1),
m_voteGateLoadedAtFirst(-1),
m_signalClusterWindow(6),
m_nmsLiveBuyTime(0),
m_nmsLiveSellTime(0),
m_nmsLiveBuyAccept(false),
m_nmsLiveSellAccept(false),
m_nmsLiveKeptTime(0),
m_nmsLiveKeptDir(Neutral),
m_nmsLiveKeptConf(0),
dtStudied(0),
perf(deinit): I/O-free chart cleanup, dead-panel purge, skip clean weight saves The 18:23 terminal close (20260825.log) killed two of six charts inside OnDeinit: they printed "shutting down" then nothing for 5.9 s until "Abnormal termination", stranding ~700 objects each - including the one family no prefix sweep can reach, the control panel (CAppDialog names its 15 objects <numeric instance id><control>, and a re-attach mints a new id, so a killed panel is a permanent ghost; XTIUSD carried one across sessions). The stall sat in the two file writes that preceded all visible cleanup while the four sibling charts flooded the same 2013-era disk - the ~4x18MB-per-chart shutdown weight saves. Three changes: 1. OnDeinit touches no file until the chart is clean. CVoteArrowStore splits Save() into Snapshot() (the chart scan, in memory) and WriteSnapshot() (the disk half, consuming). New order: status label, vote-arrow snapshot, prefix sweep, panel destroy - all object ops - then member sidecars, final sweep, timings, and only then the visibility file, the vote-arrow write and the weight saves. 2. PurgeOrphanedPanelObjects() at OnInit: deletes numeric-prefix CAppDialog ghosts by name (6 chrome + 9 buttons), qualifying a prefix only when >=4 of OUR button names carry it, so a foreign dialog sharing stock chrome names is never touched. 3. m_netDirty: set by every net mutation (both backProp sites, both RestoreWeights sites, online learning conservatively, panel reset), cleared only on a successful Net.Save. Shutdown AND the per-bar autosave now skip the ~18MB write when the net is provably unchanged - for converged ensembles that is every save - which removes the very flood that starved the sibling charts. .stats still writes every time (small; carries the vote record and calibration). A skipped save leaves the .nnw header dtStudied stale, which is the already-handled attach-after-offline-gap case. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-25 18:46:51 -04:00
m_netDirty(true),
m_eraCount(0),
m_trainingComplete(false),
m_inferenceOnly(false),
m_modelLoadedFromDisk(false),
m_topologySuperseded(false),
m_mqlInferenceValidated(false),
m_freezePriorCalibration(false),
bEventStudy(false),
m_oosSplitPct(30),
dOosError(-1),
dOosForecast(0),
m_oosSamples(0),
m_cumIsCorrect(0),
m_cumIsTotal(0),
m_cumOosCorrect(0),
m_cumOosTotal(0),
m_oosOutSpreadSum(0),
m_oosOutCount(0),
feat(diagnostics): split a reported "Neutral" into CHOSE vs TIED - they need opposite fixes ApplyClassificationSoftmax() requires a STRICT majority over both rivals and sends every tie, 2-way or 3-way, to Neutral. So "OOS recall Neutral:100%" is two completely different events sharing one label: CHOSE - the net genuinely ranks Neutral highest. A class-prior/label problem. TIED - the top two are EXACTLY equal, so the net expressed no preference and the tie-break reported Neutral. A SATURATION problem: the head is SIGMOID, and a saturated sigmoid returns exactly 0.0f or 1.0f in the DLL's float32, so two classes pinned to the same rail compare equal and the bar is silently discarded. Nothing in the logs could tell them apart, and the fixes point opposite ways. Eras 1-25 of the 2026-08-17 solo PAI run read "Neutral 100%" at spread avg 0.99 - fully saturated - and broke out at era 27 as the spread fell to 0.75. That is consistent with EITHER story. The user reports the Neutral phase on most runs, so it is worth four longs to stop guessing. Four per-era counters on the pass 3 OOS walk, reported as: | Neutral CHOSE 12.4% / TIED 38.1% (of which B=S 1204) | rail 61.2% m_oosNeutralStrict - Neutral strictly highest m_oosNeutralTie - no strict winner; the tie-break produced Neutral m_oosTieBuySell - the costly subset: Buy and Sell tied AT the top, i.e. a DIRECTIONAL reading thrown away by float equality m_oosRailBars - any raw output sitting on a sigmoid asymptote, the saturation that makes exact ties possible at all Read on the RAW logits, before ApplyClassificationSoftmax() overwrites TempData in place. Legitimate because softmax is strictly monotone: it cannot change the ordering and cannot break a tie either, so the raw reading and the decision always agree. Placed alongside the existing min/max/spread capture so all the output diagnostics describe the same values. Measurement only - no decision path reads these. NOT COMPILED - user compiles. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-17 00:06:54 -04:00
m_oosNeutralStrict(0),
m_oosNeutralTie(0),
m_oosTieBuySell(0),
m_oosRailBars(0),
m_countBuySignals(0),
m_countSellSignals(0),
m_countNeutralSignals(0),
m_trueBuyCount(0),
m_trueSellCount(0),
m_trueNeutralCount(0),
m_logitAdjustLogged(false),
m_logitAdjustSkipWarned(false),
m_prevEraTrueBuyCount(0),
m_prevEraTrueSellCount(0),
m_prevEraTrueNeutralCount(0),
m_confidenceCalScale(1.0),
m_lastProgressLogTick(0),
m_priorBuy(0.0),
m_priorSell(0.0),
m_priorNeutral(0.0),
fix: NMS gates the TRADE, not just the arrow - one arrow is now one trade NmsLiveAccept() appeared in exactly one place: wrapped around DrawObject(). It never touched dPrevSignal, and dPrevSignal is what LongCondition() / ShortCondition() / SignedAIConfidence() read. So a declustered bar lost its arrow and still opened a position. Measured on SP500 H1 2026-08-09: CONV called a direction on 64% of bars, so the ~500 bars visible on screen held ~320 decisions - and ~40 arrows were drawn. Roughly one arrow per eight positions the EA would take. And the survivors are not a random eighth. Rule 2 of the declustering keeps the HIGHER-CONFIDENCE side of a cluster, so the visible set is systematically the best member of each run. A chart showing the best of every eight decisions and hiding the rest reads far better than the model is - the same best-of-N selection error already corrected in the geometry scan, the indicator tuner, the lag profile and the deploy gate, this time on the display layer, where it is most likely to mislead the person deciding whether to trade. Fixed by neutralising dPrevSignal when NMS rejects, rather than adding a "may trade" flag consulted at each read site: that leaves exactly ONE definition of what the model decided this bar, so the arrow, the panel's "Current signal", the confidence feeding sizing/SL/TP/trailing, the refresh tally and the order itself cannot drift apart again. Also reports the consequence instead of hiding it. Every OOS counter on the era line still scores every directional call - a population ~8x larger than what now trades - so the line carries a second figure: | TRADED (declustered) NN% on N calls (edge +Npp) replaying the identical rule over pass 3 (which walks OOS bars oldest to newest, the same order the live sweep sees). Its cursors are separate members from the live ones so a training pass can never disturb the live chart's declustering. Deliberately NOT switched into selectionScore yet. Declustering cuts coverage from ~64% of bars to ~8%, well under MIN_COVERAGE_FRACTION_OF_BASE_RATE, which would make every checkpoint undeployable overnight - the minRR collision and the recall-floor catch-22 twice over. The floor gets re-derived from these measurements first. Compiles clean: 0 errors, 0 warnings. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-09 10:22:31 -04:00
m_oosNmsFired(0),
m_oosNmsHits(0),
m_oosNmsLastBuyIdx(-1),
m_oosNmsLastSellIdx(-1),
m_oosNmsKeptIdx(-1),
m_oosNmsKeptConf(0.0),
m_oosNmsKeptDir(Neutral),
m_lastBuyFiredPrecPct(-1),
m_lastSellFiredPrecPct(-1),
m_lastBuyFired(0),
m_lastSellFired(0),
m_swingConfirmationBars(100),
m_maxErasPerRun(300),
refactor(chart): ChartUI is a real collaborator, not a raw-include partial (S2) Expert/AIBase/ChartUI.mqh was 869 lines of method bodies of CExpertSignalAIBase, #included after the class declaration - free to touch any of its ~500 members. First of the eleven AIBase/*.mqh partials to come out (fewest inbound edges - see the SOLID campaign session order), using the same view+adapter shape already proven for CTrainingDataView. CChartView (Expert/Chart/IChartView.mqh) is the abstract read/behaviour surface a chart-rendering collaborator needs - identity, bar/model access, the prediction cache, and the training/vote/meta scalars the panel and HUD line summarise. CAIBaseChartView is the adapter the signal owns and binds to itself (MQL5 gives a class exactly one base, so CExpertSignalAIBase cannot implement the view directly). CChartUI is the real collaborator: it owns the arrow-restore queue, the rescan queue/tally, the last-arrows-saved count and the purge-mismatch latch as its own fields (verified via grep to be touched nowhere else in Expert/), and reaches everything else - including StartChartSignalRescan, moved in from its old inline home in the header since it drives the exact same rescan state machine AdvanceChartSignalRescan drains - through the view. m_arrowSignalCache and m_signalClusterWindow stay on the signal: Training.mqh writes the cache directly every era and the training-data view already reads it, so moving it would mean rewriting Training.mqh's write sites too - out of scope here. CChartUI reaches it through four bounds-checked accessors instead of a raw member poke. All 10 public methods keep their exact signatures and become one-line forwards on the signal, so no other file's call sites change except Training.mqh's one era-end status refresh, which now reads RefreshStatusLabel() rather than reaching into CChartUI's now-private last-displayed-neuron cache directly. Verified structurally, not compiled (never compile - the operator does, in MetaEditor): brace balance checked on every touched/new file against HEAD, and the view/adapter/impl method lists cross-diffed to confirm all 59 accessors match 1:1 across the interface, the adapter declaration and the adapter body. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-08-23 20:03:52 -04:00
//--- The arrow-restore queue and the rescan queue/tally now default-construct on CChartUI itself
//--- (m_chartUI, declared below) - see its own constructor.
m_trainRunActive(false),
m_eraResumePending(false),
m_resumeBars(0),
m_resumeTotalIter(0),
m_resumeOosCutoff(0),
m_resumeBarIndex(0),
m_resumeAddLoop(false),
m_isTrainQueueCount(0),
m_isTrainCursor(0),
m_isPass2Active(false),
m_isPass2Done(false),
m_isPass3Active(false),
m_oosScoreIndex(0),
m_oosScoreStartIndex(0),
fix: the operating point was fitted on bars the net had memorized FitDirConfThreshold harvested its margin histogram from pass 2's own backprop samples. Pairing every fit against the same era's OOS result shows what that measured: PAI era 1 IS 25% cov @ 66.1% (-0.8pp) -> OOS 64% (-3pp) gap +2.1pp PAI era 76 IS 90% cov @ 79.6% (+12.7pp) -> OOS 65% (-2pp) gap +14.6pp LSTM era 9 IS 77% cov @ 81.6% (+14.6pp) -> OOS 63% (-4pp) gap +18.6pp The gap grows monotonically while OOS stays flat, so within a handful of eras the curve stops describing behaviour on unseen bars. That is fatal here specifically, because the objective branches on the SIGN of (p - break-even): the memorized curve reads +12pp at 95% coverage, so coverage x (p - p0) correctly maximises coverage and returns ~0.02 - fire on every bar. The "p < p0 -> get more selective" branch, which is the actual regime and the entire point of 983a6a3, could never fire because IS never showed p < p0. Carve a calibration slice out of the IS span - DIR_CONF_CALIB_PCT_OF_IS, purged from backprop by one label horizon on BOTH sides (the far-side purge is not optional: without it the newest training bars carry labels partly decided by price action inside the slice, putting the memorization straight back into the curve). Score it in a new chunked pass 2.5, after pass 2 has trained and before pass 3 grades - the only position where the histogram is simultaneously not-trained-on, not-graded, and current with the weights it will be applied to. Costs 15% of the training data. Worth it beyond honesty: the deploy gate needs dirPrecPct > chance + EDGE_MIN_SIGMAS*SE, and a threshold pinned near zero dilutes any edge concentrated in the confident bars across every bar the model calls, driving dirPrecPct toward chance by construction. A threshold that can be selective is the only mechanism by which a small, concentrated edge could ever clear that gate. Also: a sparse histogram now KEEPS the previous threshold instead of resetting to 0.0. A failed measurement must not decay to the most exposed setting in the range. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-10 15:58:18 -04:00
m_isCalibActive(false),
m_isCalibDone(false),
m_calibIndex(0),
m_calibStartIndex(0),
2026-08-23 22:28:31 -04:00
//--- The excursion head's own state (m_excNet and its accumulators) now default-constructs on
//--- CExcursionHead (S4 - see its own constructor), same doctrine as CChartUI below.
refactor(chart): ChartUI is a real collaborator, not a raw-include partial (S2) Expert/AIBase/ChartUI.mqh was 869 lines of method bodies of CExpertSignalAIBase, #included after the class declaration - free to touch any of its ~500 members. First of the eleven AIBase/*.mqh partials to come out (fewest inbound edges - see the SOLID campaign session order), using the same view+adapter shape already proven for CTrainingDataView. CChartView (Expert/Chart/IChartView.mqh) is the abstract read/behaviour surface a chart-rendering collaborator needs - identity, bar/model access, the prediction cache, and the training/vote/meta scalars the panel and HUD line summarise. CAIBaseChartView is the adapter the signal owns and binds to itself (MQL5 gives a class exactly one base, so CExpertSignalAIBase cannot implement the view directly). CChartUI is the real collaborator: it owns the arrow-restore queue, the rescan queue/tally, the last-arrows-saved count and the purge-mismatch latch as its own fields (verified via grep to be touched nowhere else in Expert/), and reaches everything else - including StartChartSignalRescan, moved in from its old inline home in the header since it drives the exact same rescan state machine AdvanceChartSignalRescan drains - through the view. m_arrowSignalCache and m_signalClusterWindow stay on the signal: Training.mqh writes the cache directly every era and the training-data view already reads it, so moving it would mean rewriting Training.mqh's write sites too - out of scope here. CChartUI reaches it through four bounds-checked accessors instead of a raw member poke. All 10 public methods keep their exact signatures and become one-line forwards on the signal, so no other file's call sites change except Training.mqh's one era-end status refresh, which now reads RefreshStatusLabel() rather than reaching into CChartUI's now-private last-displayed-neuron cache directly. Verified structurally, not compiled (never compile - the operator does, in MetaEditor): brace balance checked on every touched/new file against HEAD, and the view/adapter/impl method lists cross-diffed to confirm all 59 accessors match 1:1 across the interface, the adapter declaration and the adapter body. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-08-23 20:03:52 -04:00
//--- m_lastStatusLabelUpdateTick and the four m_lastDisplay* fields now default-construct on
//--- CChartUI (see its own constructor); m_lastBuyRecallPct/m_lastSellRecallPct stay here - they
//--- are read by other subsystems too, not exclusive to the panel.
m_lastBuyRecallPct(-1),
m_lastSellRecallPct(-1),
m_lastBarTime(0),
m_modelEta(InitialEtaForOptimizer()),
m_etaCeiling(InitialEtaForOptimizer()),
m_erasSinceCooldown(0),
m_bestOosForecast(-1),
feat(target): delete the barrier/geometry stack - the label is the verdict Step 3 of the swing-pivot plan, whole-hog. The swing label is now the ONE target and the era verdict is precision + recall per class against the label's own base rate - no win rate, no break-even, no expectancy, no geometry anywhere in training. DELETED - Expert/Excursion/ (4), Expert/BarrierHorizon/ (4), GeometrySweep, FirstPassageLadder, Labeling/TripleBarrier.mqh (CLabelOverlap survives in Labeling/LabelOverlap.mqh), 3 test EAs. - TripleBarrierLabel + walk, fractal label, geometry derivation/scan/ adoption, exit-policy replay, excursion MI targets, the drift verdict (DIRECTION_INTELLIGENT), the recall floor, balanced-accuracy telemetry, the barrier defines, the .cfg geometry adopt (slots kept as zeros for the positional layout), the derived-geometry live-order override. - TRAINING_TARGET input/enum: direction models are always swing; META2 re-keys the meta head onto label agreement (descriptor loses its two geometry slots). REWORKED - Labels.mqh (1795 -> ~370 lines): AdvanceSwingLabelState with FINALITY-GATED CACHING - an unresolved bar (pivot pair uncommitted) is never cached, so it can never freeze as a false Neutral; training, calibration, OOS scoring and online learning all skip unresolved bars. - SDeployVerdict: significance-only; SOosTally chance = larger directional class share; pooled gate poolability = timeframe (record v2). - Purge/embargo/declustering gaps: the measured mean label resolution lag (LabelResolutionBars), not a barrier horizon. - Pool purge key + backfill DB rows: marked at the bar the label resolved on (m_labelResolveAge), not a fabricated barrier touch. - Online learning frontier: finality, not a horizon delay. - m_bestBalancedOos -> m_bestSelectionScore, m_erasSinceBestBalanced -> m_erasSinceBest, ensemble vote outcome arrays -> label arrays. STEP 4 folded in: Entry_Multiplier / SL_Mode / TP_Mode / tradingdirection are inputs again - trade management is the tester GA's search space. Fingerprints: every direction model re-keys (TGT:SWG1 now unconditional, CUT token gone); META1 -> META2. Full retrain, as planned. Compile-verified in _claude_stage: Warrior_EA + both surviving test EAs, 0 errors, 0 warnings each. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-24 20:42:31 -04:00
m_bestSelectionScore(-1),
m_bestPassedRecall(false),
m_bestBothSidesLive(false),
diag: slow eras must explain themselves - heartbeat + era time split + pass-1 paint The 23:42 restart left all four charts grinding ~25x slower than the 18:01 baseline (era lines in 86 seconds there; 20+ minutes of nothing here), and NOTHING could say why from outside: pass 1 logs nothing, its status paint sat inside the !wouldQueue branch so the IS sweep - 80% of the pass, processed FIRST - painted nothing either, the VPS has no debugger for a thread stack, and the hourly new-bar cache invalidation cancels and restarts an unfinished era, so a slow era can stay invisible FOREVER. Externals gave: four chart threads at ~95% pure user-mode compute, DLL pool idle, no file writes. That narrows it to "MQL5-side per-item work in the era passes" and no further. So training now explains itself: - TrainHeartbeat: one line per 4096 processed items, only after an era has already run 60s, at most 6 lines per era - a healthy era stays exactly as quiet as before. Reports position and the cumulative split: feature-window builds vs net forward/backprop vs everything else. Hooked into all three passes. - The era summary line gains "| ERA TOOK Ns (feature windows X, net fwd/back Y, other Z)" whenever an era exceeded 120s. - Pass 1 paints its progress for QUEUED bars too, not just the OOS slice, so the panel shows "learning (era N)" instead of sitting on the idle writer's "Getting ready..." for the entire IS sweep. The label is throttled internally; painting per bar costs nothing. Both build variants compile 0 errors, 0 warnings. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-10 00:10:45 -04:00
m_eraStartTick(0),
m_passFeatUs(0),
m_passNetUs(0),
m_passHeartbeatPrints(0),
m_passWindowOk(0),
m_passWindowFail(0),
fix: the recall gate was unsatisfiable and the LR decay was a spiral Both made the run structurally unable to succeed, independently of any signal in the data. Found by reading the 13:01 log. RECALL GATE. m_objectiveMet required Buy, Sell AND Neutral OOS recall each >= 40%. First-touch resolution (ce52654) collapsed Neutral from the ~94% majority it was under exact-pivot labels to a same-bar-tie residue - 250 of 38,261 bars, 0.65% - so the floor was asking the model to identify 40% of coin-flip ties before it could converge. Measured: CONV, LSTM and HYBRID all logged "Neutral:0% (need >=40% each)" on every era. No model could ever satisfy it; every run was destined for the plateau ladder or the era cap. Only the DIRECTIONAL floors are load-bearing for the anti-collapse job the gate exists to do: an all-Neutral model shows Buy and Sell recall at 0% and is blocked by them. Neutral's own floor guarded the mirror bias (over-calling Buy/Sell at Neutral's expense), which was real at 94% prevalence and is not at 0.65% - there, almost never calling Neutral is correct rather than biased. Prevalence-guarded rather than hardcoded off, so it returns by itself if a future label rule makes Neutral substantial again. Deliberately NOT extended to Buy/Sell: exempting a thin directional class reopens the era-44-46 hole, which directionalRecallMeasured only half-covers - it checks those classes were MEASURED, not that they passed. ETA DECAY. A regressing era restored the checkpoint, reset the optimizer and cut eta - all on the FIRST regression. The next era then started from an identical state with a smaller step, regressed again, and got the same treatment. The loop is self-sustaining and cannot discover anything, because rolling the weights back is exactly what removes the exploration that would end it. Measured on PAI: eras 2-11 every one a regression against era 1, eta 0.000594 -> 0.000024, dW/W 0.000%/0.000% from era 2 onward. Ten eras, ~45s each, reproducing era 1 exactly and unable to do anything else. Now requires ETA_DECAY_PATIENCE_ERAS consecutive regressions - the standard ReduceLROnPlateau formulation. A single bad era is noise, and an improving era clears the counter so alternating runs never accumulate into a decay. Build tag -> gate-patience-v3. It had not moved in six commits, which is why the running binary could not be identified from its own log. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-10 13:28:58 -04:00
m_consecutiveRegressions(0),
fix(ensemble+depth): the barrier had no liveness escape, and the depth gate could not report the one state the evidence pointed at Two charts (USDJPY 50,179 bars / XAUUSD 33,982) sat at era 0 for 38 minutes with four of their eight members completely silent. Nothing in this commit guesses at why the sweep fails - the last five guesses were all wrong. It makes the failure say what it is, and stops one broken member taking its whole chart down with it. WHAT THE LOG ACTUALLY SAYS, before any of this. - The running build IS d9f834d (pulled 14:18, compiled 14:19:01, 0 errors), so every depth instrument from 1dda479/7e63a8b/45c9e21 was live. - It printed NOTHING. Zero "PRIMING", zero "CAPPED", zero "Per-indicator depth" in 27 MB of journal. The instrument built to find the depth shortfall returned "not this". - On USDJPY at 14:24, CONV-cad8 completed eras 0 AND 1 across all 50,179 bars - same chart, same 832-value window, same indicators, byte-identical fingerprint - while LSTM-cad8 and HYB-cad8 reported ok=0 failed=50163. So it is not the symbol, the history, the bar count or the indicator depth. It is per-member. - ok=0 means the NEWEST anchors failed too, and a short indicator cannot do that. The depth reading in project_silent_block_failures is therefore retired by its own instrumentation. THE ROOT CAUSE IS STILL UNKNOWN and this commit does not claim one. 1. THE DEPTH GATE'S SILENT PATH WAS THE STATE IT WAS HUNTING. ServableBars() read `if(servable <= 0 || servable >= want) return want;` - one branch over three unrelated states, silent in all of them: enabled == 0 -> nothing tunable is on. No cap. Healthy. enabled > 0, servable == -1 -> a handle answered INVALID. enabled > 0, servable == 0 -> created, never calculated. BarsCalculated() returns -1 for a dead handle, so a dead MA is indistinguishable from "no tunable indicators enabled" - and both returned `want` without printing a character. That is exactly the state a per-member, every-index, depth-independent failure produces, and it is the single reason a build carrying full depth instrumentation logged nothing through the whole outage. TunableBarsCalculated() now also reports HOW MANY indicators it consulted, and the dead-handle case is reported (latched, with per-handle depths). The RETURN is deliberately unchanged - what to do about a dead handle is not yet known, and changing control flow on an unproven cause is how the last four fixes here went wrong. SettledBars() routes its three pass-through states via ServableBars() so the report is reachable from the training sweep, which is the only caller that hits it. 2. THE STALL REPORT NAMED A SLOT, NEVER A BLOCK. "lookback slot 0 REJECTED (window had 24 of 832 values)" plus a guess ("an indicator warm-up or a history-edge read"). Which guard fired was INFERRED by counting 4+5+4+4+6+1 = 24 and concluding feature 25 must be the MA. The arithmetic was right; every conclusion drawn from it was wrong, because a value count names a POSITION and a position cannot tell cold from capped from invalid from off-the-end. Every guard that can reject a bar now records itself - m_featureFailBlock - and the report carries it, the series index, IndicatorDepthReport()'s per-handle depths, and for each indicator whether the NEWEST bar reads. That last field is the whole diagnosis in one word: newest-also-EMPTY means the buffer is unreadable everywhere (cold or dead handle), newest-reads means a genuine history edge. Instrumented: open, ATR, MA, RSI, MACD, Ichimoku, and all five AD blocks via ADIndicatorCold(). 3. THE TOTAL-FAILURE BACKOFF WAS GATED ON THE WRONG QUESTION. It armed only when m_featureFailTransient was set. Keeping that flag correct across every guard is a list that has to stay right forever - the same shape of fix the feature cache abandoned for the same reason - and the gate is pointless anyway: a sweep where ZERO of 50,163 bars produced a window will produce zero again if it restarts a millisecond later, transient or not. Doing that at full speed is what starved six indicator threads on a six-core box. The backoff is now unconditional on a total failure. The flag keeps its real job, deciding whether a MISS may be cached, which is a per-bar question and not a scheduling one. 4. THE ERA BARRIER DEADLOCKED, AND SILENCED THE MEMBERS IT FROZE. EnsembleMinTrainingEra() exempted deployed, stopped and paused members and its comment concluded "so nothing deadlocks". Those three are all VOLUNTARY. A member that simply CANNOT finish an era is none of them, so it pinned the minimum at its own era with no time limit - and the hold branch's only action was `m_lastEraCompleteTick = GetTickCount()`, which silences the stall watchdog. So on USDJPY the two members that could not train reported, and the two healthy members frozen behind them wrote nothing anywhere. The outage was visible only through the members that were not suffering it. - BarrierEraHeartbeat() stamps a clock on real era CHANGE, kept separate from m_lastEraCompleteTick precisely because the barrier resets that one. Only a member AT the minimum can be a blocker; a member ahead is idle by design and is never counted as stuck. - After ENSEMBLE_BARRIER_STUCK_MS (12 min) a non-advancing member is dropped from the barrier minimum. It keeps training and rejoins the instant it completes an era - at which point, being behind, it legitimately becomes the minimum again, which is the documented resumed-laggard behaviour. - Both transitions say so loudly, and the release states plainly that the combined-vote score cannot be computed while the ensemble is desynchronised. - A held member now writes a rate-limited journal line naming WHICH members it is waiting on, so the blocker is read off one line. 5. THE PANEL FLICKER. OnTickHandler gates its terse writer on !m_trainRunActive, and a barrier-held member returns from Train() before ever setting it - so both writers thought they were the only one updating the label and fought every tick. That is the reported "Getting ready..." <-> "Waiting at era N for slower ensemble members" oscillation, and it hit Perceptron but not Convolutional purely because Convolutional had a run active from a completed era and Perceptron, resumed from disk, never did. Train()'s message is the specific one, so it wins. NEXT STEP once this is running: the stall line now ends in "REJECTED BY: ..." and the per-handle depths. Read it. Do not reason around it. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-17 15:21:15 -04:00
m_featureFailBlock(""),
m_featureFailIdx(-1),
diag: name the cause when every feature window fails, and enforce the width contract Era 0 stalls with "NOT ONE of 54681 scanned bars produced a usable feature window, windows ok=0 failed=54681" and nothing else. That line reads identically for a cold ATR, a conditionally-missing optional feature block and an out-of-range index, so it cannot be diagnosed without one restart per hypothesis. Two changes: 1. WIDTH CONTRACT in BufferTempData. Every enabled block must emit exactly m_neuronsCount values on EVERY bar. A block that emits its values on some bars and skips them on others (indicator, panel or series unavailable for that bar) does not merely shorten the window - it SHIFTS every feature after it into the wrong slot, and the net then trains on silently misaligned inputs that still look like a valid window to everything downstream. Now rejected, rolled back and reported once, naming the optional blocks (XA / SPR / swing context) as the ones carrying an availability test. Worth having independently of the current stall. 2. BuildFeatureWindow records WHICH lookback slot rejected and how much of the window was assembled, and the pass-1 stall report renders it: "slot 0 of 20 REJECTED (window had 0 of 760)" is an indicator warm-up or history-edge read; "every lookback bar ACCEPTED and the window was still short: 640 of 760" is a missing 6-value block. No behaviour change on a healthy run: the width check is an equality that already holds, and the diagnostics render only inside the total-failure branch. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-11 10:31:56 -04:00
m_windowFailSlot(-2),
m_windowFailTotal(0),
m_lastHeartbeatTick(0),
m_passProgressPct(0),
m_passLabel("starting"),
fix: prebuild and era sized different windows; diag: Train() names its branch TWO things, one incident. 1) THE BUG I SHIPPED IN 0c85c54. m_tuneStartTrainBar is declared, initialised to 0, and NEVER ASSIGNED - the assignment existed before the God-class split and the split dropped it, leaving a dead member. Harmless while nothing read it; a real defect the moment 0c85c54 made StartLabelCachePrebuild() reset dtStudied from it. Train() then computed the window as max(StartTrainBar, floor) while the prebuild computed max(0, floor), where StartTrainBar is the non-zero datetime OnChartEventHandler passes through from the "New Bar" event. The two therefore disagreed about `bars`, so EnsureBarCachesCapacity() saw a changed size at era start, wiped the caches, and re-armed a full 38k-bar prebuild - instead of training. Restored the assignment so both sides evaluate the identical expression. 2) THE REASON IT TOOK ALL NIGHT TO FIND. Train() is a state machine with six early-return branches above the era loop and every one of them is silent. Four charts burned a core each for 15 minutes with an empty journal: the pass heartbeats (694b756) proved the era loop was never reached, no prebuild completion line appeared either, and nothing external can see inside a single MQL5 thread - per-thread CPU says "busy", file writes say nothing, and the VPS has no debugger. That is an undiagnosable state, and it is the thing to fix, not just the bug of the day. ReportTrainStall() now names the branch Train() is taking whenever no era has completed for 3 minutes, at most once a minute per signal, with the state that decides the branch: run/prebuild/simOos/resume flags, era, dtStudied, and - for the cache-invalidation branch specifically - BOTH bar counts, since two sizings disagreeing is exactly what re-arms the prebuild forever. Silent on a healthy run: an era completing resets the clock. Both build variants compile 0 errors, 0 warnings. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-10 07:32:08 -04:00
m_lastEraCompleteTick(0),
m_lastStallReportTick(0),
m_lastEraWindowBars(-1),
m_haveOosCheckpoint(false),
feat: gate deployment on the null of the MAXIMUM, not the per-era null EDGE_MIN_SIGMAS is a PER-ERA test and the deployed model is the MAXIMUM over every era a run ranks. A 2-sigma one-sided test passes on noise with probability 0.0228 per era, so over N eras the chance at least one clears it is 1-(1-0.0228)^N: 34% by era 18, 80% by era 70, 93% by era 112. The gate was near-certain to open on a long run whatever the data held. It did. HYBRID deployed 2026-08-08 at dir-precision 35.5% vs 34% chance - +1.5pp, best of 112 eras whose per-era values wandered 30%..35.5%. At the call counts these runs produce that is p_family 0.92..0.9999. Every OTHER best-of-N decision here already carries this correction, and every one REJECTS on this data: the barrier-geometry winner (null of the maximum over 6, p=0.3902), the indicator tuner (Sidak, p=1.0000), the MI lag profile (null of the maximum over 21 lags). The one decision that ships a model to a live account had none. BestCheckpointSurvivesSelection() re-tests the checkpoint that is about to deploy: z = (precision - chance)/SE, SE = sqrt(p0(1-p0)/n) p_single = P(Z >= z) p_family = 1 - (1-p_single)^N against DEPLOY_FAMILY_WISE_ALPHA. It uses the checkpoint's OWN snapshotted precision/chance/call-count, not the latest era's, because the model that ships is the one that has to clear the bar. N counts CANDIDATE eras (coverage measurable, at least one directional call) - an era that called nothing directional could never have become the best, so counting it would make the gate stricter than the search that actually happened. Conservative on purpose: consecutive eras share OOS bars and differ by one gradient step, so they are nowhere near N independent draws and the true family-wise error is below this bound. This gate decides what trades real money and the house posture is reject-unless-demonstrated. Effect at 2900 directional calls / N=112: required edge goes 1.76pp -> 2.92pp. A real edge clears it; +1.5pp does not. Applied to BOTH automatic paths - the plateau ladder's stage-3 deploy and the m_trainingComplete assignment - which must stay identical or the flag persisted into the .nnw disagrees with the decision to stop, and a reload runs inference on a model the ladder refused. NOT applied to the two operator paths (era-cap deploy, panel Deploy button). Those stay the operator's call; ReportSelectionGateVerdict() logs the verdict beside them so an authorised deploy can never later be misread as a validated one. NormalUpperTail() is A&S 26.2.17 (|err| < 7.5e-8), self-contained rather than pulling in Math\Stat. Verified against reference values to 6dp: Q(1.645)=0.049985, Q(1.96)=0.024998, Q(3.0)=0.001350. Its locals are ntB1..ntB5 because AI\Network.mqh line 79 does "#define b1 AdamBeta1". Compiles clean: 0 errors, 0 warnings. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-08 18:01:04 -04:00
m_bestDirPrecPct(-1.0),
m_bestChancePrecPct(-1.0),
m_bestDirCalls(0),
m_deployCandidateEras(0),
m_oosStable(false),
m_objectiveMet(false),
feat(target): delete the barrier/geometry stack - the label is the verdict Step 3 of the swing-pivot plan, whole-hog. The swing label is now the ONE target and the era verdict is precision + recall per class against the label's own base rate - no win rate, no break-even, no expectancy, no geometry anywhere in training. DELETED - Expert/Excursion/ (4), Expert/BarrierHorizon/ (4), GeometrySweep, FirstPassageLadder, Labeling/TripleBarrier.mqh (CLabelOverlap survives in Labeling/LabelOverlap.mqh), 3 test EAs. - TripleBarrierLabel + walk, fractal label, geometry derivation/scan/ adoption, exit-policy replay, excursion MI targets, the drift verdict (DIRECTION_INTELLIGENT), the recall floor, balanced-accuracy telemetry, the barrier defines, the .cfg geometry adopt (slots kept as zeros for the positional layout), the derived-geometry live-order override. - TRAINING_TARGET input/enum: direction models are always swing; META2 re-keys the meta head onto label agreement (descriptor loses its two geometry slots). REWORKED - Labels.mqh (1795 -> ~370 lines): AdvanceSwingLabelState with FINALITY-GATED CACHING - an unresolved bar (pivot pair uncommitted) is never cached, so it can never freeze as a false Neutral; training, calibration, OOS scoring and online learning all skip unresolved bars. - SDeployVerdict: significance-only; SOosTally chance = larger directional class share; pooled gate poolability = timeframe (record v2). - Purge/embargo/declustering gaps: the measured mean label resolution lag (LabelResolutionBars), not a barrier horizon. - Pool purge key + backfill DB rows: marked at the bar the label resolved on (m_labelResolveAge), not a fabricated barrier touch. - Online learning frontier: finality, not a horizon delay. - m_bestBalancedOos -> m_bestSelectionScore, m_erasSinceBestBalanced -> m_erasSinceBest, ensemble vote outcome arrays -> label arrays. STEP 4 folded in: Entry_Multiplier / SL_Mode / TP_Mode / tradingdirection are inputs again - trade management is the tester GA's search space. Fingerprints: every direction model re-keys (TGT:SWG1 now unconditional, CUT token gone); META1 -> META2. Full retrain, as planned. Compile-verified in _claude_stage: Warrior_EA + both surviving test EAs, 0 errors, 0 warnings each. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-24 20:42:31 -04:00
m_erasSinceBest(0),
m_plateauStage(0),
feat(search): stop on the IN-SAMPLE plateau, and shrink every best-of-K effect before quoting it Points 3 and 4 of the four-point plan. 1. IN-SAMPLE EARLY STOP - and the reason it is worth having is not compute. The plateau ladder stops on the OOS SELECTION score. That is a peek: by the time it fires, every one of those eras has been evaluated out of sample, so all of them sit in the family the deploy gate corrects over (g_ensCandidateEras, Sidak). Training longer therefore does not merely cost time - it RAISES the bar the eventual winner has to clear. The new stop reads the TRAINING error, which the gate never looks at. When the optimiser has stopped improving on data it can see, more eras will not find a better model; they will only enlarge the OOS family. Ending there shrinks the correction, and the shrinkage is legitimate precisely BECAUSE the stopping rule never consulted an out-of-sample number. That distinction is the whole point and it is the one this project has got wrong four times: stop on IS and the family really is smaller; stop on OOS and those eras were searched and still count. Both stops now exist; only this one buys a lower bar. Deliberately more patient than the OOS ladder (IS_ERROR_PATIENCE_MULT = 3x): training error is noisy per era - mini-batch order alone moves it - and ending a run that is still learning costs far more than a few wasted eras. Improvement is RELATIVE (IS_ERROR_IMPROVE_FRAC = 1%), so it does not depend on the loss's absolute scale, and it only acts when a checkpoint exists, since otherwise it would end a run with nothing to deploy. Reset per RUN alongside the ladder, so a resumed run cannot early-stop on its first era against a previous run's best. 2. WINNER'S-CURSE SHRINKAGE ON THE BARRIER-GEOMETRY WINNER. The family-wise permutation gate already establishes that the RANKING is not noise. It says nothing about the SIZE of the winner's effect - and a best-of-K maximum is biased upward by construction, being the largest of K noisy draws. The adoption message quotes that raw maximum and compares it against the incumbent, so the number a reader plans on is the inflated one. The penalty is now measured, not assumed: the same permutation draws that produce the p-value also produce, per draw, the MAXIMUM excess across all candidates under pure noise. The mean of those maxima is exactly what a best-of-K selection is expected to report when there is nothing there. This is the empirical form of the sqrt(2 ln K) x SE penalty the SQX EdgeFinder plugin applies to every maximum it reports (Stats.java:79-88), and it needs no normality assumption because the draws ARE the null distribution. Applied in James-Stein form - effect x max(0, 1 - penalty^2/effect^2) - so a large effect is nearly untouched and a marginal one collapses toward zero. Reported, not gated. The adoption decision still turns on the permutation p-value, which is the right test for "is the ranking real"; the shrunk number is there so the magnitude quoted beside it is one worth planning on. Closes the first of the two EdgeFinder ports identified on 2026-08-12. NOTE on the second EdgeFinder port, deliberately not done here: "let the measurement steer the target" is already true where it matters most - ReportGeometryExpectancyScan ADOPTS the winning barrier geometry under the family-wise gate rather than advising it, and the MI excursion suite publishes a verdict per instrument per config. What is still missing is steering the TRAINING TARGET itself (direction vs excursion) off those verdicts, and that is a design change rather than a surgical one - direction is a closed verdict while excursion SIZE keeps clearing, so the honest version of that change is a target-selection policy, not a flag. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-17 16:29:00 -04:00
m_bestIsError(-1.0),
m_erasSinceBestIsError(0),
fix(plateau): the IS-error early stop was inert for every ensemble member 15 hours of training, and the stop that exists to END a run announced itself 1,299 consecutive times without ending anything: SP500 ConvLSTM IN-SAMPLE ERROR PLATEAU - not improved in 1297 / 1298 / 1299 eras (best 0.2689, now 0.3269) ... era 1396, 1397, 1398 SP500 LSTM 536 eras SP500 CONV 442 eras SP500 PAI 150 eras XAUUSD HYB 478 eras XAUUSD LSTM 296 eras XAUUSD CONV 366 eras CAUSE: it wrote its decision into m_plateauStage, and EnsembleEraVerdict mirrors the shared ladder onto every member - `mm.m_plateauStage = g_ensPlateauStage` - on EVERY era, purely so each member's status line shows the collective stage. A display mirror was silently overwriting a decision, so the stop re-armed and re-fired the next era, forever. This is the worst possible direction for this particular bug. Every one of those 1,299 eras was scored out of sample and joined the family the deploy gate corrects over (Sidak, g_ensCandidateEras). The stop's entire purpose is to make that family SMALLER; instead the run spent fifteen hours raising its own bar. - m_isErrorPlateaued: a one-way per-member latch, cleared only by a fresh run. Nothing in the ladder may reset it. The stop condition and the two solo deploy conditions read the latch, not the mirrored stage. - The orchestrator combines: EnsembleEraVerdict requires UNANIMITY across participating members (same participation test the era barrier uses, so an excluded or finished member cannot veto). One member still learning can still move the combined vote, and the vote is what the gate certifies. - Fed in as `dueStage = PLATEAU_STAGE_DEPLOY`, NOT written to g_ensPlateauStage. The block that actually ends the run sits under `dueStage > g_ensPlateauStage`, so assigning the stage directly makes that test false and the deploy never happens - the same inert-write shape as the bug being fixed. Caught before committing; raising dueStage carries it through the ladder's own path (warm restarts skipped, family-wise vote test, measurement screen, joint checkpoint) unchanged. - g_ensIsPlateauAnnounced: announce once per run, not once per era. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-18 13:09:26 -04:00
m_isErrorPlateaued(false),
fix: training-stability audit fixes F1/F2/F3/F5 - unbiased shuffle, real plateau escapes, fresh optimizer state on restore, pure OOS metric Four of the six findings from research/training_pipeline_audit_2026-08-09.md (F4 mini-batching and F6 feature re-encode deliberately deferred - see the report's implementation-status section for why): - F1: pass-2 Fisher-Yates (and AutoTune's MI block shuffle) used MathRand()%, which is 15-bit - provably non-uniform on every full-history era over 32,768 queued samples. New 30-bit ShuffleRandomIndex(). - F2: plateau warm restarts were a no-op whenever eta already sat at its ceiling (the normal state of a non-regressing plateau) - the ladder was just a 24-era countdown. Restarts now overshoot to 5x the ceiling (PLATEAU_RESTART_BOOST) and anneal geometrically back over the patience window, SGDR-style; ETA_MIN widened 1e-4 -> 1e-5 so the decay schedule has real range. - F3: checkpoint restores put weights back but kept the rejected trajectory's Adam moments, so the optimizer immediately pushed back toward the rolled-back state (the restore->regress->restore oscillation). CNet::ResetOptimizerState() zeroes moments/momentum/step counters (weights, BN statistics, gamma/beta untouched) on every mid-run restore, every boosted restart, and the deploy-time restore that online learning continues from. - F5: batch-norm running statistics now freeze for the pass-3 OOS scoring walk, so the selection metric the checkpoint ranking and deploy gate read is a pure function of the checkpoint instead of partly measuring BN drift. Defensive unfreeze in FinalizeTrainRun covers stop-mid-pass; live/online adaptation and the OOS continual-learning simulation stay adaptive by design. Compiled clean (0 errors, 0 warnings) via the staged-tree recipe. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-09 10:54:09 -04:00
m_restartBoostErasLeft(0),
m_syncWaitStartTick(0),
m_warmupPassesRemaining(0),
2026-08-13 10:23:11 -04:00
m_coldSweepTick(0),
fix(ensemble+depth): the barrier had no liveness escape, and the depth gate could not report the one state the evidence pointed at Two charts (USDJPY 50,179 bars / XAUUSD 33,982) sat at era 0 for 38 minutes with four of their eight members completely silent. Nothing in this commit guesses at why the sweep fails - the last five guesses were all wrong. It makes the failure say what it is, and stops one broken member taking its whole chart down with it. WHAT THE LOG ACTUALLY SAYS, before any of this. - The running build IS d9f834d (pulled 14:18, compiled 14:19:01, 0 errors), so every depth instrument from 1dda479/7e63a8b/45c9e21 was live. - It printed NOTHING. Zero "PRIMING", zero "CAPPED", zero "Per-indicator depth" in 27 MB of journal. The instrument built to find the depth shortfall returned "not this". - On USDJPY at 14:24, CONV-cad8 completed eras 0 AND 1 across all 50,179 bars - same chart, same 832-value window, same indicators, byte-identical fingerprint - while LSTM-cad8 and HYB-cad8 reported ok=0 failed=50163. So it is not the symbol, the history, the bar count or the indicator depth. It is per-member. - ok=0 means the NEWEST anchors failed too, and a short indicator cannot do that. The depth reading in project_silent_block_failures is therefore retired by its own instrumentation. THE ROOT CAUSE IS STILL UNKNOWN and this commit does not claim one. 1. THE DEPTH GATE'S SILENT PATH WAS THE STATE IT WAS HUNTING. ServableBars() read `if(servable <= 0 || servable >= want) return want;` - one branch over three unrelated states, silent in all of them: enabled == 0 -> nothing tunable is on. No cap. Healthy. enabled > 0, servable == -1 -> a handle answered INVALID. enabled > 0, servable == 0 -> created, never calculated. BarsCalculated() returns -1 for a dead handle, so a dead MA is indistinguishable from "no tunable indicators enabled" - and both returned `want` without printing a character. That is exactly the state a per-member, every-index, depth-independent failure produces, and it is the single reason a build carrying full depth instrumentation logged nothing through the whole outage. TunableBarsCalculated() now also reports HOW MANY indicators it consulted, and the dead-handle case is reported (latched, with per-handle depths). The RETURN is deliberately unchanged - what to do about a dead handle is not yet known, and changing control flow on an unproven cause is how the last four fixes here went wrong. SettledBars() routes its three pass-through states via ServableBars() so the report is reachable from the training sweep, which is the only caller that hits it. 2. THE STALL REPORT NAMED A SLOT, NEVER A BLOCK. "lookback slot 0 REJECTED (window had 24 of 832 values)" plus a guess ("an indicator warm-up or a history-edge read"). Which guard fired was INFERRED by counting 4+5+4+4+6+1 = 24 and concluding feature 25 must be the MA. The arithmetic was right; every conclusion drawn from it was wrong, because a value count names a POSITION and a position cannot tell cold from capped from invalid from off-the-end. Every guard that can reject a bar now records itself - m_featureFailBlock - and the report carries it, the series index, IndicatorDepthReport()'s per-handle depths, and for each indicator whether the NEWEST bar reads. That last field is the whole diagnosis in one word: newest-also-EMPTY means the buffer is unreadable everywhere (cold or dead handle), newest-reads means a genuine history edge. Instrumented: open, ATR, MA, RSI, MACD, Ichimoku, and all five AD blocks via ADIndicatorCold(). 3. THE TOTAL-FAILURE BACKOFF WAS GATED ON THE WRONG QUESTION. It armed only when m_featureFailTransient was set. Keeping that flag correct across every guard is a list that has to stay right forever - the same shape of fix the feature cache abandoned for the same reason - and the gate is pointless anyway: a sweep where ZERO of 50,163 bars produced a window will produce zero again if it restarts a millisecond later, transient or not. Doing that at full speed is what starved six indicator threads on a six-core box. The backoff is now unconditional on a total failure. The flag keeps its real job, deciding whether a MISS may be cached, which is a per-bar question and not a scheduling one. 4. THE ERA BARRIER DEADLOCKED, AND SILENCED THE MEMBERS IT FROZE. EnsembleMinTrainingEra() exempted deployed, stopped and paused members and its comment concluded "so nothing deadlocks". Those three are all VOLUNTARY. A member that simply CANNOT finish an era is none of them, so it pinned the minimum at its own era with no time limit - and the hold branch's only action was `m_lastEraCompleteTick = GetTickCount()`, which silences the stall watchdog. So on USDJPY the two members that could not train reported, and the two healthy members frozen behind them wrote nothing anywhere. The outage was visible only through the members that were not suffering it. - BarrierEraHeartbeat() stamps a clock on real era CHANGE, kept separate from m_lastEraCompleteTick precisely because the barrier resets that one. Only a member AT the minimum can be a blocker; a member ahead is idle by design and is never counted as stuck. - After ENSEMBLE_BARRIER_STUCK_MS (12 min) a non-advancing member is dropped from the barrier minimum. It keeps training and rejoins the instant it completes an era - at which point, being behind, it legitimately becomes the minimum again, which is the documented resumed-laggard behaviour. - Both transitions say so loudly, and the release states plainly that the combined-vote score cannot be computed while the ensemble is desynchronised. - A held member now writes a rate-limited journal line naming WHICH members it is waiting on, so the blocker is read off one line. 5. THE PANEL FLICKER. OnTickHandler gates its terse writer on !m_trainRunActive, and a barrier-held member returns from Train() before ever setting it - so both writers thought they were the only one updating the label and fought every tick. That is the reported "Getting ready..." <-> "Waiting at era N for slower ensemble members" oscillation, and it hit Perceptron but not Convolutional purely because Convolutional had a run active from a completed era and Perceptron, resumed from disk, never did. Train()'s message is the specific one, so it wins. NEXT STEP once this is running: the stall line now ends in "REJECTED BY: ..." and the per-handle depths. Read it. Do not reason around it. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-17 15:21:15 -04:00
m_barrierEraSeen(-1),
m_barrierEraTick(0),
m_barrierExcluded(false),
fix(ensemble): the era barrier read healthy startup work as a dead member Reported symptom: one member at era 17 while the rest sat at era 2, with the combined vote never scoring. Two faults compound to produce exactly that, and neither needs a broken model to trigger. FIRST - BUSY WAS READ AS STUCK. BarrierEraHeartbeat() decides liveness from one signal: has m_eraCount changed in the last 12 minutes. But Train() returns early, before the era loop, for three ONE-TIME phases that never touch m_eraCount - the label-cache prebuild, the pattern-DB backfill and the OOS simulation walk - and those are precisely what a slow topology spends its first many minutes doing. A member grinding steadily through a prebuild therefore looked identical to a dead one and was dropped from the barrier at startup, before it had trained a single era. The constant's own comment states the flawed premise: "comfortably past the slowest healthy ERA on the deepest chart" - true, and not the question being asked. Those three branches now call NoteBarrierProgress() and a chunk of phase work re-arms the watchdog exactly as an era does. SECOND - EXCLUSION HAD NO BOUND. Once dropped, a member is skipped by EnsembleMinTrainingEra(). Drop every OTHER member and that loop finds nothing to take a minimum over, falls through to its `return m_eraCount` fallback - the CALLER'S own era - and EnsembleEraBarrierHolds() evaluates `era > era`, false, for everybody. The barrier silently becomes a no-op and the fastest member runs away unbounded. EnsembleMinEraAnyMember() now measures against every still-training member, excluded or not, and a member may lead it by at most ENSEMBLE_MAX_ERA_LEAD eras. The cap is deliberately a real stop rather than a warning. A desynchronised ensemble is not a degraded one: the combined-vote score and the joint checkpoint both require every member on the same era, so weights trained past the cap can never be certified by any gate. The hold reports which of the two it is, because the operator's next move differs - an ordinary barrier hold resolves itself, a lead-cap hold names a member that needs diagnosing and will not resolve on its own. Not yet explained: "only one NN listened to the stop command". The panel now dispatches down the filter tree and reports the count it reached ("training stopped (N model(s))"), so the next run answers that definitively instead of leaving it to inference. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-20 09:35:25 -04:00
m_barrierPhaseProgress(false),
fix(ensemble+depth): the barrier had no liveness escape, and the depth gate could not report the one state the evidence pointed at Two charts (USDJPY 50,179 bars / XAUUSD 33,982) sat at era 0 for 38 minutes with four of their eight members completely silent. Nothing in this commit guesses at why the sweep fails - the last five guesses were all wrong. It makes the failure say what it is, and stops one broken member taking its whole chart down with it. WHAT THE LOG ACTUALLY SAYS, before any of this. - The running build IS d9f834d (pulled 14:18, compiled 14:19:01, 0 errors), so every depth instrument from 1dda479/7e63a8b/45c9e21 was live. - It printed NOTHING. Zero "PRIMING", zero "CAPPED", zero "Per-indicator depth" in 27 MB of journal. The instrument built to find the depth shortfall returned "not this". - On USDJPY at 14:24, CONV-cad8 completed eras 0 AND 1 across all 50,179 bars - same chart, same 832-value window, same indicators, byte-identical fingerprint - while LSTM-cad8 and HYB-cad8 reported ok=0 failed=50163. So it is not the symbol, the history, the bar count or the indicator depth. It is per-member. - ok=0 means the NEWEST anchors failed too, and a short indicator cannot do that. The depth reading in project_silent_block_failures is therefore retired by its own instrumentation. THE ROOT CAUSE IS STILL UNKNOWN and this commit does not claim one. 1. THE DEPTH GATE'S SILENT PATH WAS THE STATE IT WAS HUNTING. ServableBars() read `if(servable <= 0 || servable >= want) return want;` - one branch over three unrelated states, silent in all of them: enabled == 0 -> nothing tunable is on. No cap. Healthy. enabled > 0, servable == -1 -> a handle answered INVALID. enabled > 0, servable == 0 -> created, never calculated. BarsCalculated() returns -1 for a dead handle, so a dead MA is indistinguishable from "no tunable indicators enabled" - and both returned `want` without printing a character. That is exactly the state a per-member, every-index, depth-independent failure produces, and it is the single reason a build carrying full depth instrumentation logged nothing through the whole outage. TunableBarsCalculated() now also reports HOW MANY indicators it consulted, and the dead-handle case is reported (latched, with per-handle depths). The RETURN is deliberately unchanged - what to do about a dead handle is not yet known, and changing control flow on an unproven cause is how the last four fixes here went wrong. SettledBars() routes its three pass-through states via ServableBars() so the report is reachable from the training sweep, which is the only caller that hits it. 2. THE STALL REPORT NAMED A SLOT, NEVER A BLOCK. "lookback slot 0 REJECTED (window had 24 of 832 values)" plus a guess ("an indicator warm-up or a history-edge read"). Which guard fired was INFERRED by counting 4+5+4+4+6+1 = 24 and concluding feature 25 must be the MA. The arithmetic was right; every conclusion drawn from it was wrong, because a value count names a POSITION and a position cannot tell cold from capped from invalid from off-the-end. Every guard that can reject a bar now records itself - m_featureFailBlock - and the report carries it, the series index, IndicatorDepthReport()'s per-handle depths, and for each indicator whether the NEWEST bar reads. That last field is the whole diagnosis in one word: newest-also-EMPTY means the buffer is unreadable everywhere (cold or dead handle), newest-reads means a genuine history edge. Instrumented: open, ATR, MA, RSI, MACD, Ichimoku, and all five AD blocks via ADIndicatorCold(). 3. THE TOTAL-FAILURE BACKOFF WAS GATED ON THE WRONG QUESTION. It armed only when m_featureFailTransient was set. Keeping that flag correct across every guard is a list that has to stay right forever - the same shape of fix the feature cache abandoned for the same reason - and the gate is pointless anyway: a sweep where ZERO of 50,163 bars produced a window will produce zero again if it restarts a millisecond later, transient or not. Doing that at full speed is what starved six indicator threads on a six-core box. The backoff is now unconditional on a total failure. The flag keeps its real job, deciding whether a MISS may be cached, which is a per-bar question and not a scheduling one. 4. THE ERA BARRIER DEADLOCKED, AND SILENCED THE MEMBERS IT FROZE. EnsembleMinTrainingEra() exempted deployed, stopped and paused members and its comment concluded "so nothing deadlocks". Those three are all VOLUNTARY. A member that simply CANNOT finish an era is none of them, so it pinned the minimum at its own era with no time limit - and the hold branch's only action was `m_lastEraCompleteTick = GetTickCount()`, which silences the stall watchdog. So on USDJPY the two members that could not train reported, and the two healthy members frozen behind them wrote nothing anywhere. The outage was visible only through the members that were not suffering it. - BarrierEraHeartbeat() stamps a clock on real era CHANGE, kept separate from m_lastEraCompleteTick precisely because the barrier resets that one. Only a member AT the minimum can be a blocker; a member ahead is idle by design and is never counted as stuck. - After ENSEMBLE_BARRIER_STUCK_MS (12 min) a non-advancing member is dropped from the barrier minimum. It keeps training and rejoins the instant it completes an era - at which point, being behind, it legitimately becomes the minimum again, which is the documented resumed-laggard behaviour. - Both transitions say so loudly, and the release states plainly that the combined-vote score cannot be computed while the ensemble is desynchronised. - A held member now writes a rate-limited journal line naming WHICH members it is waiting on, so the blocker is read off one line. 5. THE PANEL FLICKER. OnTickHandler gates its terse writer on !m_trainRunActive, and a barrier-held member returns from Train() before ever setting it - so both writers thought they were the only one updating the label and fought every tick. That is the reported "Getting ready..." <-> "Waiting at era N for slower ensemble members" oscillation, and it hit Perceptron but not Convolutional purely because Convolutional had a run active from a completed era and Perceptron, resumed from disk, never did. Train()'s message is the specific one, so it wins. NEXT STEP once this is running: the stall line now ends in "REJECTED BY: ..." and the per-handle depths. Read it. Do not reason around it. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-17 15:21:15 -04:00
m_barrierHoldReportTick(0),
fix(depth): route EVERY ResizeBuffers call site through one indicator-depth gate 1dda479 clamped the training sweep. It left five other paths asking the indicators for a depth they cannot serve, and on a live account the quiet ones are worse than the stall was - a stalled chart is visible, a chart trading on a degraded feature window is not. ServableBars(want, context) is now the single gate, and all six go through it: training sweep clamp, floored at TRAIN_MIN_CLAMPED_BARS (below that a small positive BarsCalculated is warm-up, which m_coldSweepTick owns) label prebuild clamp - labels come from price/ADZigZag and would survive a capped MA, but ResizeBuffers sizes EVERY buffer and a failed CopyBuffer leaves m_MA EMPTY for the next reader, so this path could silently re-break the block Train()'s clamp just fixed live inference HOLD. Below `need` the swing block takes its degraded path and inference runs on a different feature distribution than the model was fitted on. This EA sizes real positions off that output, so no signal beats a mismatched one online learning HOLD, same reason and worse - this path WRITES to a live trading model, so a mismatched (features, label) pair is not a wrong arrow, it is a wrong weight update that compounds every bar chart rescan clamp - SIGNAL_RESCAN_LOOKBACK_BARS is 5000 and MT5's smallest "Max bars in chart" is also 5000, so this one is genuinely reachable; uncapped it repaints the window all-Neutral research export clamp before the emptiness test, so a capped symbol exports the depth it has rather than writing a CSV with a dead feature block - an artefact that looks complete and is silently wrong Both HOLDs are insurance, not expected states: `need` tops out near 1,152 bars (16 + 750 + 384 + 2) against a 5,000 floor on the terminal setting. They exist so the failure mode is unreachable rather than merely unlikely. Not changed: a genuinely SHORT price history still takes the old degraded path at every site. That is pre-existing behaviour and narrowing it would mute charts that trade today, so it stays a separate decision rather than a side effect of this fix. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-17 13:33:16 -04:00
m_inferenceDepthRefusalWarned(false),
fix(buffers): revert the MA +1 - it asked for a bar that does not exist and stopped every chart REGRESSION I INTRODUCED IN 1dda479, live for ~15 minutes. CSeries::BufferResize -> CheckLoadHistory -> CheckTerminalHistory succeeds only when Bars() >= size. Train() calls ResizeBuffers with barIndex == Bars(), so sizing the MA buffer to barIndex + 1 asks for one bar more than the symbol has and fails the WHOLE ResizeBuffers call. The log named it exactly: failed to get 50180 bars for USDJPY,PERIOD_H4 (Bars() = 50,179) failed to get 33983 bars for XAUUSD,PERIOD_H4 (Bars() = 33,982) StartLabelCachePrebuild() then bailed on the false return and stayed silent, so the only symptom was Train() reporting "arming the first label-cache prebuild" forever with labelCacheBars=0 - the panel's "getting ready". The premise was wrong, not just the arithmetic. The MA block reads GetData(idx) AND GetData(idx + 1), and at the OLDEST bar that second read is SUPPOSED to fail - there is no older bar to difference against. Rejecting that one bar is correct behaviour; buying it cost the entire history. Two more things, since the same defect had a second instance and no alarm: - The Ichimoku pair (closeBars and m_Ichimoku, both barIndex + ichiKijun) is the same bug with a far larger constant, latent only because the feature is off. Both are now clamped to Bars(). The oldest ichiKijun bars then have no cloud, which that block's EMPTY_VALUE guard already handles per-bar - the right outcome. - The prebuild's bare `return` on a false ResizeBuffers now says so once, naming the depth and Bars(). MQL5's own "failed to get N bars" was in the log the entire time, from a stack frame nothing connected to the prebuild. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-17 14:12:47 -04:00
m_prebuildBlockWarned(false),
m_labelCacheBars(0),
m_labelCacheAnchorTime(0),
m_labelCachePrebuilt(false),
fix(labels): overlapping-label sample correction + horizon cap on the scale ladder Three defects, all surfaced by the 2026-08-17 SP500 H4 run that shipped stop 4.86 / target 9.71 (width 14.57*ATR, horizon 384). 1. EVERY STANDARD ERROR ASSUMED INDEPENDENT SAMPLES. Triple-barrier labels started one per bar overlap by the label's lifespan, so n calls are worth ~n/L independent observations (Lopez de Prado, AFML ch. 4 - sample uniqueness). All three sqrt(p(1-p)/n) sites divided by the RAW count. The tell: the operating point's null-of-the-maximum gate is family-wise and should fire on ~5% of eras under the null. Measured fire rates - PAI 47/73 (64%), ConvLSTM 9/24, LSTM 8/21 (38%), CONV 4/62 (6%). CONV, the only model whose margin distribution admits few bins, sat on the null; the rest cleared a bar that was too low by ~sqrt(L). PAI's deployed threshold consequently alternated between the ENDS of its own range era to era (0.10 -> 0.88 -> 0.86 -> 0.66; coverage 16% <-> 73%). TripleBarrierLabel now records when each label became KNOWABLE - the first winning touch, or both stops, or the timeout - and the prebuild accumulates the mean. EffectiveSampleSize() feeds the operating point, the member deploy gate and the ensemble vote gate. Conservative by construction (n/L is an upper bound on the damage); gates get harder, never easier. 2. THE SCALE LADDER RAN AWAY, again. Horizon scales as swingMedian*sl*tp, and since 4d8cb08 reachability is measured OVER that horizon - so a wider rung buys itself the time that makes it look reachable. Same target -> horizon -> reach -> target loop the excursion window is kept short to avoid; fixing the window confusion reopened it through the other door. It walked 128 -> 256 -> 384 bars and stopped at q90, the widest rung there is, with every rung reading 39-48% against a 20% floor. A floor nothing fails selects nothing. Rungs whose required horizon exceeds BARRIER_HORIZON_MAX are now rejected - the same rule ReportGeometryExpectancyScan already applied. It was printing the shipped pair as CLAMPED and disqualified ('h384!') two lines under the deriver that chose it: two subsystems, one geometry, opposite verdicts. 3. THE RUNG SNAP DESTROYED THE RATIO IT WAS COMPARING. Both legs snapped independently to the coarse first-passage grid, re-rating each candidate: q90 4.86/9.71 -> 5.00/10.00 (2.00), q85 4.07/8.14 -> 5.00/10.00 (IDENTICAL measurement), q75 3.07/6.13 -> 4.00/6.50 (1.63 - a nearer target). So the ladder compared win shares taken at ratios from 1.63 to 2.17 and read the differences as scale. It is why the reach column came out non-monotone in width (q75 48.5% above q90 42.9%). The stop now snaps to its nearest rung in log space and the target follows the ratio off it; the pair actually measured is returned and logged, so a collision reads as a collision. Also: LadderWinShare guarded against the conditional (fractal) geometry path, which fills n from m_fracLegCount while leaving idxList empty - a latent out-of-bounds on a currently-dead path. New log lines: mean label lifespan and effective n on the label-cache line, the required-vs-available horizon per rung, and the grid pair the reconciliation actually measured (its tolerance now scales with the grid skew instead of a flat 5pp). Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-17 01:12:05 -04:00
//--- 0 samples => MeanLabelLifespan() returns 1.0 => EffectiveSampleSize() is the identity, so an
//--- un-prebuilt model behaves exactly as it did before the overlap correction existed rather than
refactor(labeling): CTripleBarrier - one copy of the fill/barrier arithmetic Session B of the feature-selection/labeling refactor track. Extracts the two pieces of triple-barrier arithmetic that were genuinely duplicated or scattered, taking price/ATR/geometry as plain arguments - no chart, no indicator handle - so it is testable with synthetic numbers. CTripleBarrier::ComputeLevels() replaces the fill/barrier level arithmetic that TripleBarrierLabel() and SimulateTradeOutcome() each spelled out by hand; their own comments already called it "IDENTICAL... deliberately and by copy." One caller resolves both sides at once (the both-won tie-break needs both); the other selects the side its isLong argument names. Same for ApplyMinStopWidening(), the broker-minimum-stop floor both walks applied. Fuzzed 200k random (entry, spread, risk, reward, minStop, isLong) tuples against both original hand-written forms: 0 mismatches. CLabelOverlap replaces m_labelLifespanSum/m_labelLifespanCount - two members reset from three separate call sites (constructor, label-cache rebuild), the exact "N loose members cleared in more than one place" shape a candidate- geometry incident (7452bd1) turned into a live bug. One object, one Reset(), default-constructed like every other object member. MeanLabelLifespan() and EffectiveSampleSize() on the signal become thin forwarders with an unchanged signature - every one of their ~15 existing callers, direct and through the CAIBaseTrainingData adapter, is unaffected. SnapHorizonToLadder() forwards to CTripleBarrier::SnapToLadder(), the ladder array's one remaining copy; EffectiveHorizonMax() (the close-all cache) stays on the signal since that state has no clean argument form. NOT extracted: TripleBarrierLabel()'s ~200-line walk itself. It resolves both sides simultaneously, tracks the first-passage ladder, and feeds the label every live order is sized from; a rewrite of it cannot be checked without a compiler, so only the two pieces provably identical to their originals moved. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-08-23 18:57:17 -04:00
//--- shrinking its own samples on a guess. See m_lastLabelLifespan. m_labelOverlap is a class
//--- member and default-constructs itself (CLabelOverlap::CLabelOverlap() calls Reset()) - it has
//--- no init-list form here, same as m_ladder and every other object member below.
fix(labels): overlapping-label sample correction + horizon cap on the scale ladder Three defects, all surfaced by the 2026-08-17 SP500 H4 run that shipped stop 4.86 / target 9.71 (width 14.57*ATR, horizon 384). 1. EVERY STANDARD ERROR ASSUMED INDEPENDENT SAMPLES. Triple-barrier labels started one per bar overlap by the label's lifespan, so n calls are worth ~n/L independent observations (Lopez de Prado, AFML ch. 4 - sample uniqueness). All three sqrt(p(1-p)/n) sites divided by the RAW count. The tell: the operating point's null-of-the-maximum gate is family-wise and should fire on ~5% of eras under the null. Measured fire rates - PAI 47/73 (64%), ConvLSTM 9/24, LSTM 8/21 (38%), CONV 4/62 (6%). CONV, the only model whose margin distribution admits few bins, sat on the null; the rest cleared a bar that was too low by ~sqrt(L). PAI's deployed threshold consequently alternated between the ENDS of its own range era to era (0.10 -> 0.88 -> 0.86 -> 0.66; coverage 16% <-> 73%). TripleBarrierLabel now records when each label became KNOWABLE - the first winning touch, or both stops, or the timeout - and the prebuild accumulates the mean. EffectiveSampleSize() feeds the operating point, the member deploy gate and the ensemble vote gate. Conservative by construction (n/L is an upper bound on the damage); gates get harder, never easier. 2. THE SCALE LADDER RAN AWAY, again. Horizon scales as swingMedian*sl*tp, and since 4d8cb08 reachability is measured OVER that horizon - so a wider rung buys itself the time that makes it look reachable. Same target -> horizon -> reach -> target loop the excursion window is kept short to avoid; fixing the window confusion reopened it through the other door. It walked 128 -> 256 -> 384 bars and stopped at q90, the widest rung there is, with every rung reading 39-48% against a 20% floor. A floor nothing fails selects nothing. Rungs whose required horizon exceeds BARRIER_HORIZON_MAX are now rejected - the same rule ReportGeometryExpectancyScan already applied. It was printing the shipped pair as CLAMPED and disqualified ('h384!') two lines under the deriver that chose it: two subsystems, one geometry, opposite verdicts. 3. THE RUNG SNAP DESTROYED THE RATIO IT WAS COMPARING. Both legs snapped independently to the coarse first-passage grid, re-rating each candidate: q90 4.86/9.71 -> 5.00/10.00 (2.00), q85 4.07/8.14 -> 5.00/10.00 (IDENTICAL measurement), q75 3.07/6.13 -> 4.00/6.50 (1.63 - a nearer target). So the ladder compared win shares taken at ratios from 1.63 to 2.17 and read the differences as scale. It is why the reach column came out non-monotone in width (q75 48.5% above q90 42.9%). The stop now snaps to its nearest rung in log space and the target follows the ratio off it; the pair actually measured is returned and logged, so a collision reads as a collision. Also: LadderWinShare guarded against the conditional (fractal) geometry path, which fills n from m_fracLegCount while leaving idxList empty - a latent out-of-bounds on a currently-dead path. New log lines: mean label lifespan and effective n on the label-cache line, the required-vs-available horizon per rung, and the grid pair the reconciliation actually measured (its tolerance now scales with the grid skew instead of a flat 5pp). Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-17 01:12:05 -04:00
m_lastLabelLifespan(0),
feat(measurement): fix zero-skill denominator, publish the deploy bar, measure lifespan per rung, add a MEASURE scale objective The last run could not have demonstrated an edge either way, and nothing in the log said so. Four changes so it does. 1. THE ZERO-SKILL LINE DIVIDED BY THE WRONG DENOMINATOR. m_oosWinLongTotal resets every era; m_oosSamples only resets on a full model reset. So 'always-long %' decayed as ~1/era: a run whose true rate is 37% printed 1.2% at era 33 and 0.0% at era 2219. This is the SAME bug already found and fixed for logBuyPredPct thirty lines above ('era-15 Buy:2% that was really ~30%'), left in the one line whose whole job is to be the reference every other number is read against. Correct at era 1, wrong everywhere after - including the '62% zero-skill' figure in the 2026-08-16 notes. Now per-era, and always-short is finally readable. 2. THE DEPLOY GATE STATES ITS OWN BAR. 'edge -1pp' era after era cannot separate 'short by a hair' from 'short by an amount no strategy could cover'. The era line now prints the required win rate, the SE, the effective n and the lifespan it was deflated by; above 100% it says UNREACHABLE. At 4,738 OOS bars and L=75.6 there are ~63 independent observations, putting the bar near 66% at typical coverage. 3. LIFESPAN MEASURED PER RUNG. The first-passage cache already stores touch ages at every ladder level, so each candidate geometry's resolution time is readable without training on it - L-vs-width becomes a measurement across the whole ladder in ONE run rather than a second chart. Each rung reports L, n_eff, min provable edge and min provable EV. 4. SCALE OBJECTIVE IS PHASE-AWARE, defaulting to MEASURE. Width and detectability are opposed: labels overlap by L, L grows like m*k = width^2 at fixed ratio, so min provable EV ~ width^2 while the cost saving from width is only linear. Doubling width quadruples the smallest EV you can prove. DEPLOY (widest that clears reachability) is right once an edge is known; MEASURE (narrowest that keeps round-trip spread under BARRIER_MAX_COST_FRACTION_PCT) is right while it still has to be shown. The direction does not depend on the exponent, and item 3 makes the exponent checkable. Fixed in review: m_lastRungLifespan is cleared on every LadderWinShare entry or a rejected rung reports the previous rung's lifespan as its own; per-rung detectability is labelled IS-sample based (the deriver may not see the holdout), so absolute figures are optimistic while the ranking is unaffected. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-17 10:37:05 -04:00
//--- -1 = the deploy gate has not run its arithmetic yet this era; the era line then omits the bar
//--- rather than printing a stale one from a previous era.
m_lastEdgeFloorPct(-1.0),
m_lastPrecSE(-1.0),
m_lastEffN(-1.0),
feat(gate): cross-instrument pooled certification The deploy bottleneck is CERTIFICATION, not training. A 4,738-bar OOS window at L=75.6 holds ~63 independent observations; certifying a 3pp edge at 2 sigma needs ~1,036. More bars of the same symbol barely help - they overlap. Other symbols do not. WHAT POOLS. Not win rates: symbols have different derived geometries, different break-evens and different drifts, so averaging raw rates across them is meaningless. What pools is each symbol's EXCESS OVER ITS OWN CHANCE RATE, combined by inverse-variance weighting (fixed-effects meta-analysis). Each symbol keeps its own model, geometry and chance rate; only the evidence is combined. THE CORRELATION PROBLEM, bracketed rather than assumed away. SP500 and NAS100 are ~0.9 correlated and pooling them as independent inflates the evidence. Nothing here can measure that without sharing return series, so instead of guessing a correction the gate reports both ends: SE_INDEP = sqrt(1/SUM(1/var_i)) all members independent SE_CORR = SUM(w_i * sqrt(var_i)) all members perfectly correlated The truth is always between. THE GATE USES SE_CORR, so a pass cannot be an artifact of correlated instruments - that bound already assumes the worst. The ratio is logged as the diversification credit the gate declines to claim, so the cost of that conservatism is visible instead of hidden. SCOPE, deliberately limited: the pooled result is REPORTED, never folded into tradeableOK. The local gate certifies the model that actually trades this symbol; the pool answers the different question of whether the strategy has an edge at all. Letting a cross-symbol result license a local deploy would ship a model that never cleared its own bar - so it cannot. Mechanics: one file per instrument (no concurrent-write path to get wrong), every FileOpen carrying FILE_SHARE_READ|FILE_SHARE_WRITE, records skipped rather than reinterpreted on a version mismatch, 12h staleness cutoff so a stopped chart cannot vote, and pooling refused below 3 instruments. Poolability requires matching timeframe and ratio; differing SYMBOL is the entire point. Publishing is unconditional - a pool that only hears from winners is a selection effect, not a meta-analysis. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-17 10:45:37 -04:00
m_lastPoolPasses(false),
m_lastPoolReport(""),
feat: fitted directional confidence threshold - selectivity gets a mechanism The training loss and the selection metric wanted different things and only the second one knew it. Logit-adjusted cross-entropy has no term for "how often should I trade", so the head calls a direction on 87-91% of bars. The selection metric is precision x coverage credit, saturating at the coverage floor - above the floor extra calls earn NOTHING and only precision counts. So selection wanted few good calls, the loss produced many mediocre ones, and all selection could do was pick the least-bad era out of what it was handed. Nothing pushed the model toward selectivity. This gives the decision RULE the policy instead of distorting the loss (which is estimating class probabilities correctly, and a probability estimate should not be bent to encode a trading policy - Elkan 2001: estimate, then choose the operating point separately). AdjustedSignalFromSoftmax now abstains unless the winning direction's softmax margin over its best rival clears a fitted threshold. Margin, not the winning probability: the latter moves with overall calibration rather than with how close the decision actually was. Fitted on IS, applied to OOS and live. Pass 2 already forward-passes every IS sample, so the margin histogram is harvested there for free (primary occurrences only, so the oversampled replay queue cannot skew the operating point); the fit runs at the end of pass 2, BEFORE pass 3, so the deploy gate grades the thresholded model on bars the threshold never saw. Fitting on pass 3's own predictions would be choosing the operating point on the data being graded - the best-of-N error corrected in five other places here. Objective: maximise IS directional precision subject to still clearing the SAME coverage floor the deploy gate uses (base rate x 0.25, re-derived locally so the two cannot drift apart). Swept top-down in one pass; ties go to the LOWER threshold, since equal precision for less coverage is strictly worse. Under DIR_CONF_MIN_FIT_CALLS (200) it runs unthresholded rather than on a guess. The threshold is part of the MODEL, not the run: captured with Net.CaptureWeights(), restored with the weights at both restore sites, and appended to the .cfg under the same length-guard convention so a deployed model reloads at the operating point its gate actually cleared. A pre-2026-08-09 .cfg reads 0.0, which is exactly the behaviour it was trained under. Per-era line now prints "@margin>=X.XX" next to coverage, so a coverage drop can be attributed to the operating point rather than guessed at. Both build variants compile 0 errors / 0 warnings. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-09 15:04:37 -04:00
//--- 0.0 = unthresholded until the first pass 2 fits it (see DIR_CONF_THRESHOLD_BINS). Deliberately
//--- the permissive value: a model that has not measured its own operating point must not silently
//--- abstain on everything.
m_dirConfThreshold(0.0),
m_bestDirConfThreshold(0.0),
m_dirConfPrimaryBars(0),
m_dirConfSparseWarned(false),
m_labelPrebuildActive(false),
m_prebuildSeedPending(false),
m_labelPrebuildBars(0),
m_labelPrebuildOosCutoff(0),
m_labelPrebuildIndex(-1),
m_labelPrebuildBuyCount(0),
m_labelPrebuildSellCount(0),
m_labelPrebuildNeutralCount(0),
refactor(online-learning): OnlineLearning is a real collaborator, not a raw-include partial (S5) Expert\AIBase\OnlineLearning.mqh (595 lines) -> Expert\OnlineLearning\: IOnlineLearningView.mqh (abstract, ~50 accessors) + AIBaseOnlineLearningView.mqh/ AIBaseOnlineLearningViewImpl.mqh (the adapter) + OnlineLearning.mqh (COnlineLearning). STATEFUL, unlike CModelPersistence: grep-verified the shadow net, the OOS continual-learning simulation state and the pattern-database backfill state are genuinely exclusive to this file's own methods - Training.mqh/Topology.mqh/ Lifecycle.mqh/the signal's own header only ever CHECKED or RESET this state at era/lifecycle boundaries, never owned it, so it moved onto the collaborator as real members (same doctrine as Excursion). Those external touch points became consolidated view/forward calls instead of raw field pokes - AbortSimIfActive() replaces THREE separate copies of the same delete/null/false triple (Training.mqh's stop path, FlushTrainRun, ResetWeights), matching the geometry-scan duplicate-reset precedent in project memory. ResetForFreshTopology() replaces Topology.mqh's five- field reset block, DeployNet() replaces the shadow-preferred net selection duplicated in Inference.mqh and ChartScoreBarForRescan, and BlendTowardNet() replaces the era-end blend Training.mqh used to poke m_shadowNet for directly. Reused the signal's existing Data*()/Chart*()/Persist*() getters wherever one already answered the question (labels/outcome/history/horizon/priors/servable-bars/etc.); added ~30 new Online*() wrappers only for what nothing else exposed yet. The three PersistOnline*() get/set pairs (WST3 .stats fields) now forward through the owning member instead of touching the field directly - CModelPersistence is unaffected. Every method body is a pure relocation of the original's statements in original order; verified against `git show HEAD~1:Expert/AIBase/Excursion.mqh`-style diff against the pre-extraction file kept in the working tree until this commit. Compiled 0 errors, 0 warnings (stage mirror + MetaEditor64 /compile, ~91s). Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-08-23 22:59:02 -04:00
//--- The shadow net, the OOS continual-learning simulation state, the pattern-database backfill
//--- state, and the online-learning watermark/guardrail/counters now default-construct on
//--- COnlineLearning (m_onlineLearning, declared below) - see its own constructor.
m_tuneTrialIndex(-1),
m_tuneBestOosForecast(-1),
m_tuneLastTrialWasWin(true),
m_tuneHaveBestCheckpoint(false),
m_tuneStartTrainBar(0),
m_tuneFilterDone(false),
m_trainingPaused(false),
m_trainingStopRequested(false),
m_activeFileCommon(true),
//--- m_configLockName now default-constructs on CConfigLock (m_configLock, declared below) - see
//--- its own constructor.
m_isInitialized(false),
fix(deinit): a full model write was running ahead of the cheap cleanup "Abnormal termination" is back, and this time it is not the arrows. The timing names the culprit exactly: 16:02:31.547 OnDeinit: shutting down 16:02:36.003 Abnormal termination <- 4.46 s, MetaTrader gave up 16:02:36.226 chart signals - persisted <- cleanup finished 0.2 s LATE OnDeinit called StopTraining() BEFORE the chart cleanup. StopTraining() finalises an in-flight run, and FinalizeTrainRun() restores the best checkpoint and then persists it - a full ~1MB model write per signal. So the expensive step ran ahead of the cheap bounded one, which is precisely the inversion the shutdown ordering exists to prevent. The previous fix put PersistWeightsOnShutdown last and missed that StopTraining smuggles a second save in at the front. Two changes: Cleanup now runs FIRST, then StopTraining, then the weight save. The visible teardown is cheap and bounded, so it always completes even when everything after it is killed. And the deploy-persist inside FinalizeTrainRun is suppressed during shutdown. RestoreWeights() is an in-MEMORY swap, so the best checkpoint is already the live net by that line, and PersistWeightsOnShutdown writes exactly those weights moments later. The old path wrote the same model twice per signal - eight full writes across four charts - for no benefit. A user-pressed Stop still persists immediately, because nothing else would. Compiles 0 errors / 0 warnings. Build tag deinit-order-v2. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-01 16:06:40 -04:00
m_shutdownInProgress(false),
refactor(chart): ChartUI is a real collaborator, not a raw-include partial (S2) Expert/AIBase/ChartUI.mqh was 869 lines of method bodies of CExpertSignalAIBase, #included after the class declaration - free to touch any of its ~500 members. First of the eleven AIBase/*.mqh partials to come out (fewest inbound edges - see the SOLID campaign session order), using the same view+adapter shape already proven for CTrainingDataView. CChartView (Expert/Chart/IChartView.mqh) is the abstract read/behaviour surface a chart-rendering collaborator needs - identity, bar/model access, the prediction cache, and the training/vote/meta scalars the panel and HUD line summarise. CAIBaseChartView is the adapter the signal owns and binds to itself (MQL5 gives a class exactly one base, so CExpertSignalAIBase cannot implement the view directly). CChartUI is the real collaborator: it owns the arrow-restore queue, the rescan queue/tally, the last-arrows-saved count and the purge-mismatch latch as its own fields (verified via grep to be touched nowhere else in Expert/), and reaches everything else - including StartChartSignalRescan, moved in from its old inline home in the header since it drives the exact same rescan state machine AdvanceChartSignalRescan drains - through the view. m_arrowSignalCache and m_signalClusterWindow stay on the signal: Training.mqh writes the cache directly every era and the training-data view already reads it, so moving it would mean rewriting Training.mqh's write sites too - out of scope here. CChartUI reaches it through four bounds-checked accessors instead of a raw member poke. All 10 public methods keep their exact signatures and become one-line forwards on the signal, so no other file's call sites change except Training.mqh's one era-end status refresh, which now reads RefreshStatusLabel() rather than reaching into CChartUI's now-private last-displayed-neuron cache directly. Verified structurally, not compiled (never compile - the operator does, in MetaEditor): brace balance checked on every touched/new file against HEAD, and the view/adapter/impl method lists cross-diffed to confirm all 59 accessors match 1:1 across the interface, the adapter declaration and the adapter body. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-08-23 20:03:52 -04:00
//--- m_lastArrowsSaved and the purge-mismatch latch now default-construct on CChartUI.
m_miBestColumn(0.0),
m_miLabelEntropy(0.0),
diag(autotune): a positive control, and a scan that separates "no signal" from "signal knocked out of step" Four architecturally different networks landed on the same precision - Buy 23-25% against a 25.4% base rate, Sell 19-22% against 22.0% - while making completely different calls (HYBRID votes Sell on 69% of bars, PAI on 41%). Precision equal to the base rate is what INDEPENDENCE looks like, and precision under independence is fixed by the label distribution, not by the architecture, so all four converging on it is arithmetic rather than coincidence. Accuracy meanwhile tracks coverage exactly as independence predicts (31.1/30.3/25.0 predicted vs 31.8/28.9/24.6 observed for PAI/CONV/HYB). But "no information in the data" and "information destroyed upstream of every topology" produce that identical picture, and the MI test alone cannot tell them apart either. Two additions: POSITIVE CONTROL. Three "measurements" in this codebase have turned out to be silent no-ops that produced plausible numbers - the MI scorer reading an array nobody filled, the eval-mode guard that switched off the imbalance correction, the alternation gate whose premise was never true. So the estimator now has to prove it responds to a signal known to be present before any floor reading is believed: the label of a neighbouring sample row, ~19 bars away and far inside the 128-bar barrier horizon, so the two outcome windows overlap heavily and MUST be associated. Same binning, same estimator. Near the floor => every MI figure is void. ALIGNMENT SCAN. Re-scores against the label taken from bar i+k for k in -5..+5. A peak at k != 0 is a feature/label misalignment - an off-by-one in the label index, a horizon applied to the wrong bar, a feature window that lags what it claims - which would destroy the information before any topology saw it and would look identical in every accuracy number this EA prints. A flat profile says the features simply do not carry this target. The sampled range is trimmed by |k| at both ends so a shift is measured rather than an edge effect, and both bars must carry a real label. Also: BuildMiSample publishes its stride instead of the report recomputing that arithmetic (it would drift), and the control sizes its buffers from its own sample count rather than the caller's. Compiles 0 errors / 0 warnings, standard and Market. Build tag mi-control-align-v1. Redeploy only - no retrain, no model deletion; the diagnostic runs on resumed models. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-01 14:12:10 -04:00
m_miStrideBars(0),
fix(diag): the symbol sweep was measuring its own sampling, not the market Twelve cells came back with higher-timeframe "signal" 5-9x anything on H1, at p=0.005. It was an artifact, and the sweep's own columns gave it away: excess tracked the sampling STRIDE almost monotonically, and the three D1 cells - stride collapsed to 1-5 bars against a 128-bar horizon, i.e. ~99% window overlap - were the three highest. Three flaws, all the same family: comparing numbers without the spread that belongs to them. 1. THE NULL ASSUMED INDEPENDENCE THE LABELS DO NOT HAVE. Triple-barrier labels overlap; two rows less than one horizon apart share most of their outcome window. A free Fisher-Yates shuffle destroys that dependence along with the association, making the null far narrower than the truth and handing out significance that isn't there - Lopez de Prado ch. 4 arriving through the back door of the significance test. Now permutes contiguous BLOCKS of at least one horizon, so the null keeps the autocorrelation and the p-value means what it says. It degrades honestly: severe overlap leaves few blocks, the null widens, nothing reaches significance. The block count is now printed, because THAT - not the row count - is the sample size a p-value rests on, and a warning fires under 30 blocks so "not significant" is not misread as "no signal" when it means "not enough independent history to tell". 2. THE POSITIVE CONTROL'S STRENGTH DEPENDED ON THE DATASET. It paired each row's label with the NEXT SAMPLE ROW's, whose distance is the stride - so on M5, where stride ran 160-717 bars against a 128-bar horizon, it was pairing two windows that never overlap. All three M5 cells duly reported a FAILED estimator and voided their own results with nothing wrong. A control whose strength varies with the cell cannot certify the cell. Now pinned to a quarter of the horizon, where ~75% overlap is guaranteed by construction. 3. THE LOOKAHEAD VERDICT HAD NO MARGIN. It flagged 7 of 12 cells on gaps of 0.00008-0.00040 nats against a measured null sd of ~0.00030 - noise, every one. Now requires 3 sd, the same discipline the deploy floor applies to precision. Compiles 0 errors / 0 warnings, standard and Market. Build tag blockperm-v1. Supersedes every number from the sweep. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-01 15:11:40 -04:00
m_miNullBlocks(0),
feat(labels): measure which barrier is predictable at entry, don't guess The alignment scan settled the shape of the problem: 4.7x more is knowable 5 bars into a 128-bar window than at the entry the model actually trades. A 6xATR target reached over 128 bars is decided overwhelmingly by what happens DURING the window, so whatever the entry state knows is buried under 128 bars of later noise. That is a property of the TARGET, and it is why four different architectures all landed on precision exactly equal to the base rate - no topology can undo it. So measure the target. For each SL/TP pairing a user can actually select, relabel the same sampled bars and score how much the SAME features say about THAT outcome at entry. Seconds, no training, no topology, and it runs on the diagnostic path that already exists. Ranked on excess over its OWN null as a share of its OWN H(Y), never on raw nats: each geometry has a different class balance, hence a different finite-sample bias and a different amount of information there to find, so raw MI would rank the most BALANCED label rather than the most PREDICTABLE one. The break-even win rate m/(m+k) is printed beside each so the ranking is read next to the bar the model must clear. Stated in the output because it is the easy thing to get wrong: chance precision EQUALS break-even at every geometry, so a tighter target does not hand you expectancy. It buys predictability - less noise piled on top of what the entry state knows - which is the one thing changing topology cannot do. Read-only by construction: it relabels a sampled copy via TripleBarrierLabel(), never writes the label cache (which belongs to the configured geometry), and restores the horizon and overrides it borrowed. The overrides apply only when BOTH are positive, so a half-set pair can never silently relabel a live run. Compiles 0 errors / 0 warnings, standard and Market. Build tag geometry-scan-v1. Redeploy only - no retrain to READ the ranking. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-01 14:32:04 -04:00
m_miReportDone(false),
feat(rank): AI models rank their own confidence tiers from held-out outcomes Closes the caveat 4858507 shipped with: the vote is a confidence percentage, but only to the extent the pattern weights are measured. AI tier weights sat at their designed defaults (25/50/75/100) because AI rows only ever arrive from LIVE journaling, of which a training run produces almost none. AND A STALE-TIER BUG THAT MADE THE EVIDENCE MEANINGLESS. The OOS scan bucketed every scanned bar by ConfidenceTier(), which reads dPrevSignal - and dPrevSignal is assigned in PASS 1 only, never anywhere in the OOS scan. So an entire era's fires were bucketed by one stale, unrelated bar's confidence and landed in a SINGLE tier. That is the "tier prec T0:72%(828) T1:n/a(0) T2:n/a(0) T3:n/a(0)" symptom recorded on 2026-08-16 and attributed to the calibration clamp. The clamp was real and was fixed then; this is a second, independent cause of the identical output that survived that fix untouched - which is why the log kept reading the same afterwards. Two causes, one symptom. Now ConfidenceTierFor(adjSig): the bar this iteration actually scored. WHY THIS DOES NOT WRITE ROWS TO THE SIGNAL DB, which was the obvious reading of "fill the database during training". The user's own observation is the reason: a classic Pattern_2 is a fixed geometric condition, so its win rate is legitimately accumulated over years, but an AI Pattern_2 means "confidence landed in tier 2" and tier 2 under era 100's weights is a different statement from tier 2 under era 500's. The DB's value is ACCUMULATION, and accumulation is exactly what is wrong here - it would average together models that no longer exist, while colliding with the per-table row cap and mixing measured-on-holdout outcomes into the live ledger's own tables. What the DB actually supplies is a measured win rate per pattern, and pass 3 already computes that on held-out bars, thousands at a time. So the model ranks itself once per era, REPLACING rather than accumulating, which makes the weights describe the current weights by construction. ESTIMATOR. Not WinRateFromCounts(): it returns NO_DATA below 100 raw trades BEFORE shrinking, which here would fire on every tier every era and hand all four the pooled rate - the tiers could never separate and the mechanism would be inert. Shrinkage is the answer to a small sample; a floor in front of it means the shrinkage never runs. Instead: a Beta prior of TIER_PRIOR_EFF_N pseudo-observations centred on the model's pooled holdout rate, counted in EFFECTIVE observations, because overlapping triple-barrier labels mean 800 raw fires can be worth ~12 independent ones. Rounded to the integer, not to the decade NormalizeWinRate() uses, which would collapse the shrunk tiers back into one number. NO SAME-ERA CIRCULARITY, and it falls out of the ordering rather than a guard: weights are computed at the END of era N, so the vote scored during era N was cast with era N-1's weights. The deploy gate never grades a vote whose weights were fitted on the bars it is scoring. Residual leakage remains - the same OOS bars each era under a different model - and is stated in the code rather than papered over. Both DB clobber paths are closed: ApplyPatternWeight() declines once self-ranked, and UpdateSignalsWeights()' filter.Weight() call is guarded by SelfRanked() - guarding only the tiers would have let the hourly ranking pass undo half the self-ranking. NOT COMPILED - user compiles in MetaEditor. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-18 16:00:32 -04:00
m_tiersSelfRanked(false),
fix(chart): display now reads era-end SNAPSHOTS - the live cache is wiped mid-era Full-pipeline analysis after "threshold 30, attained often, nothing drawn, still glued to buy". The log falsified the premise before any code did: 21:40:43 swept 4999, 794 voters, drew 491. Strongest 43.0% vs 30.0% 21:42:07 swept 4999, 0 voters, drew 0 21:51:30 swept 4999, 0 voters, drew 0 21:56:30 swept 4999, 922 voters, drew 382. Strongest 44.0% vs 30.0% The arrows WERE drawn - 491 of them, then 382 - and then erased. ONE root cause, three symptoms: every display path read m_arrowSignalCache, which is wiped to sentinel at each era start and only complete again when pass 3 finishes. With eras at ~30s and a sweep at ~17s: * ARROW FLICKER: a sweep landing mid-era found no voters anywhere, and its else-branch deleted the arrow on every voteless bar - erasing the previous sweep's entire output. The chart cycled populated -> blank -> populated; the user kept catching the blank phase. * READOUT GLUE: the newest-cache walk found only sentinel for ~90% of every era and fell through to dPrevSignal - the frozen purge-band edge bar that reads Buy. 659638e fixed which bar was frozen, not the freezing. * VOTER FLAP: 1299 -> 257 -> 1113 across back-to-back sweeps - each saw a different fraction of half-rebuilt caches. THE FIX, structural rather than another patch: 1. Era-end snapshots. RankTiersFromOos() runs at pass-3 completion - the one moment the cache is complete - and now copies it (raw signals, newest LOOKBACK+16 bars) into member-owned snapshot state, unconditionally, BEFORE its early return: an all-Neutral era is a snapshot worth showing, not an absence of one. Raw signals rather than votes, so a tier re-rank between eras reprices them at read time via LiveVoteContribution for free. 2. The sweep (SnapshotVoteAt) and the prospective readout both read snapshots; the readout's fallback chain is live-cache -> snapshot -> dPrevSignal, and the snapshot leg is the one that fires most of the time. 3. NO DATA IS NOT A VERDICT: a den==0 bar no longer deletes - only an actual sub-threshold vote takes an arrow down. This alone ends the wipe half of the flicker even where snapshots are missing (before the first era). 4. Arming moved from an era-counter diff (which fires at era BOUNDARIES, i.e. precisely when caches are about to be wiped) to g_warriorOverlayArmRequest, set by each RankTiersFromOos - "a member's snapshot just got fresher", the only event a redraw can act on. 60s rate limit collapses the four members' burst into one sweep. Classic-only charts arm once at start. 5. Census now reports the direction split - "922 had a voter (610 buy / 312 sell)" - so "the vote leans buy" is checkable from the log instead of inferred from arrow colours. Also visible in the log and worth knowing: the threshold flip-flopped 30 -> 40 -> 30 across the evening's re-inits (census lines at 21:42-21:51 ran at 40), so part of the observed blankness was configuration, not code. NOT COMPILED - user compiles in MetaEditor. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-18 22:03:20 -04:00
m_overlaySnapBars(0),
m_prospectiveSigSnap(-2.0),
feat(vote): replay pass rebuilds a deployed model's ladder without retraining The previous commit persisted the tier ladder, which fixes this going forward but did nothing for models whose .stats predates WST7 - they still had to retrain to mint one. They never did. Every number a converged model needs in order to vote is a pure function of weights already on disk plus labels derivable from the chart, so replay them: stage 1 build the label cache (existing chunked prebuild) stage 2 rescan history (existing chunked rescan, deployed net) stage 3 score + rank + persist (one walk over two arrays) ScoreReplayFromCache() walks m_arrowSignalCache against m_labelCacheBuy/Sell, fills the same m_oosTierFired/Hits and per-class totals pass 3 fills, and hands them to RankTiersFromOos() - deliberately feeding the existing ranker rather than reimplementing it. The shrinkage, the chance reference and the module trust weight are subtle enough that a second copy would drift, and a ladder measured by a slightly different rule would be silently incomparable with every ladder training produced. AdvanceDeployedRebuild() sequences the three stages off the timer. It has to be a sequence: stages 1 and 2 are each minutes of work draining in time-boxed slices, and stage 2's output is meaningless until stage 1 has labels to score against. The previous version ran the rescan with no labels at all, which is why it could only ever rebuild arrows and never the ladder - the thing actually blocking the vote. The result is written to .stats immediately. The failure being repaired is state that lived in memory and was never written down; recomputing it and not saving it would repeat that exactly. Also routes every rescan completion through one hook, so there is a single place that knows what a finished rescan means - republish for a manual one, score and rank for a rebuild. Compile-verified in _claude_stage: 0 errors, 0 warnings. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-25 15:01:24 -04:00
m_deployedRebuildStage(0),
feat(hud): per-member neuron lines + a vote label that moves as the nets learn Both 2026-08-19 reports were the same staleness: every source behind the label was an ERA artifact (live cache refills at pass-3 completion, the snapshot copies once per era, dPrevSignal is the frozen purge-band edge bar) - so the readout stepped at era cadence at best, stayed glued to one direction, and lagged the era counter. DisplayInference(): throttled (4s, 1s across an era boundary), SIDE-EFFECT-FREE forward of the current decision bar (window ending on bar 1, same question the live path asks) through the LEARNER net. Batch-norm running stats are bracketed frozen/RESTORED via the new CNet::GetBatchNormFrozen() + CNeuronBatchNormOCL::StatsFrozen() - restore, not unfreeze, because a display tick can land between pass-3 chunks whose whole scan holds them frozen. Writes nothing a trading or training path reads (dPrevSignal, NMS state, tallies, watermarks all untouched; RefreshLatestSignal is not reusable here precisely because it writes all of them). LSTM safe by construction: h/c zeroed per forward. ProspectiveVote() reads the fresh forward as its FIRST source; the era-artifact chain becomes the fallback (meta head, warm-up, window holes). DisplayHudLine(): the reference library's training label, per ensemble member - name, output activations (softmax probs or raw scalar), the decision, its weighted vote (the exact consensus numerator term), era, recent average error, "(trn)" while not vote-capable. Rendered under the vote line in RefreshVoteReadout BEFORE the live-vote defer (member lines are telemetry, not tradable readings), coloured by the member's own direction in muted tones - the vote line's strict green-only-when-it-would-trade rule is untouched. NOT COMPILED - user compiles in MetaEditor. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-19 08:48:34 -04:00
m_dispSignal(0.0),
m_dispValid(false),
m_dispStamp(0),
m_dispEra(-1),
feat(logs): throttle the settled per-era diagnostics - measured 22MB/9.5h of confirmed-working systems Measured from the journal (2026-08-19): the era deep-dive line (~2KB) plus the excursion verdict, tier re-rank, calibration move, barrier hold and selection-regressed note each printed EVERY era for EVERY member - ~940 eras/member/day - long after the systems they watch were confirmed working. Yesterday's file was 1.3GB (70% of it the news-filter calendar spam the sweep fix already removed). VerboseMode returns as an INPUT (demoted 2026-08-01 for the marketplace; that track is dead since the 2026-08-16 pivot) and gains a second job: false throttles each settled per-era print to eras 0-3 plus every TRAIN_LOG_EVERY_ERAS-th (25 ~= one deep-dive per ~15min per member); true restores the per-era firehose, flippable live. Never throttled: anything that marks a CHANGE - new bests, restores + eta decays, plateau stage transitions, deploy approvals, warnings, errors, the label-cache/adoption one-shots, and the combined-vote gate line (the active system's primary telemetry, still every era). Semantic fixes over blanket gating: - barrier hold now ARMS silently and prints only when the hold outlasts the 2-min report interval - a brief hold every era is the design, the long hold is the watchdog case the line exists for; - the ensemble deploy REFUSAL prints immediately when its reason changes (that is a finding), on cadence when unchanged; - the filtered-view census prints when its RESULT moves (drawn count, or strongest vote by >=2pp) and at least every 10th sweep. NOT COMPILED - user compiles in MetaEditor. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-19 09:36:15 -04:00
m_lastEnsRefusalKey(0),
feat(pool,mi): one feature layout fleet-wide, and the keep-screen stops self-disabling on a cold start TWO CHANGES, BOTH RETRAIN-FORCING BY INTENT. 1. SP500 was training alone, and one alt-data column was the reason. The alt block's width joins the model fingerprint, and the pool reader only adopts peer rows whose fingerprint and width match. The exporter gives each instrument the series that apply to it - FX 15 columns, metals/oil 14, SP500 13 - so the fleet ran as three incompatible pools: EURUSD/USDJPY/USDCAD adopt ~57-60k peer rows each XAUUSD/XTIUSD adopt 6.4k / 20.3k SP500 "EVERY peer file was REJECTED, so this chart is training alone" - 0 rows SP500 therefore trained on 2279 independent observations against a 600-wide input with its first layer floored at 16, printing its own "expect overfitting" warning. It is the one chart with no pool and the worst capacity ratio in the fleet by a factor of three. Fresh models now pin ALTDATA_FLEET_COLUMNS - the 12-column intersection - instead of their own file header. An existing model still adopts its .cfg pin, so this re-keys nothing that is already trained. Intersection rather than union: filling an absent series with its median makes that column constant per instrument, which lets a pooled model identify the source instrument and stop learning the shared mechanism. It is also 6 columns narrower. Cost is six columns whose retained information is UNMEASURED - the keep-screen reports a bitmask nothing has mapped back to names. 2. The MI keep-screen disabled itself for the whole run on any cold start. ReportFeatureLabelInformation set m_miReportDone on ENTRY. On a cold start the label cache is allocated before it is filled, so BuildMiSample finds no row carrying a resolved label and returns 0 - a sixth exit, and the only one the 8c1266d instrumentation did not cover, which is why it printed nothing. observed then stayed -1, the permutation loop never iterated, and the report emitted "-1.00000 nats over 0 permutations" beside a plausible "strongest single feature 0.05979" that was a STALE m_miBestColumn from an earlier scoring call. The first ensemble member propagated the latch to g_ensembleChartMiReportDone and silenced every member on the chart. The flag now latches only once a measurement exists. A short sample is reported as a deferral naming the two numbers that identify it (cached bars vs bars carrying a resolved label) and retried, up to MI_REPORT_MAX_ATTEMPTS. Build tag -> fleet-pool-v1. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-26 17:41:14 -04:00
m_miReportDeferrals(0),
m_miReportAttempts(0)
{
fix(ensemble): responsive panel + synchronized eras + combined-vote accuracy Four user-reported/requested items, one root cause chain: 1) DEAD CONTROL PANEL in AI_HYBRID mode. All members posted custom event id 1 and handled id 1001, and CExpertCustom broadcasts every chart event to every filter - so each posted event ran a train chunk in ALL N members (N*N chunks per round) and the chart thread never idled long enough to deliver clicks/drags. profiling.csv: 99.45% of time in OnChartEventHandler. Fix: per-instance study-event ids (STUDY_EVENT_ID_BASE + construction order, offset above the Controls library's ON_* codes - id 1 was also ON_DBL_CLICK, so panel double-clicks fired training chunks). ArmStudyEvent() is the single post site; lost-event watchdog replaces the accidental sibling-clears-my-flag rescue. 2) WARM-UP DUPLICATION. The auto-tune sweep is deterministic over identical features/labels, and it ends in the full MI diagnostic suite, which the MI-share gate never intercepted on the sweep path - four members ran four identical ~36s sweep+report blocks. First member publishes outcome (g_ensembleChartTuneDone/Installed/Settings); the rest apply it and skip both. 3) DEINIT STRANDED PANEL+ARROWS (user repro 18:52). Root cause from the log: the 4,500ms budget runs from MetaTrader's stop REQUEST - a heavy autosave in flight ate it, OnDeinit got ~430ms and died in the first member's arrow persist ("Abnormal termination" 432ms in). Fix: early visible-UI sweep (native prefix deletes for status/panel/dialog) right after ClearStatusLabel, and a fast path for still-training models - their arrows are re-rendered every era, so they get one bulk purge instead of scan+atomic-write in the death window. 4) ENSEMBLE FEATURES (user requests): era BARRIER - members advance era by era together; a member ahead of the slowest still-training member declines Train() calls and its chunk budget is donated (TRAIN_TIME_BUDGET_MS = 120/activeTrainers, UI headroom constant). COMBINED-VOTE OOS SCORE - each member's pass-3 scan contributes its adjusted per-bar decision (0.0 on abstain) to a shared row buffer; the last member to finish the era scores the averaged vote vs the mirrored Min_Vote_Open against the same target-before-stop outcomes members grade themselves on, publishing an "Ensemble vote" line on the aggregated panel. Member headlines now carry their lifetime win rate with break-even. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-16 19:06:04 -04:00
//--- Claim this instance's study-event id - see STUDY_EVENT_ID_BASE (ExpertSignalAIBase.mqh) for why
//--- these are per instance and offset above the Controls library's event codes.
refactor(arch): a read-only training-data view, so modules stop being #included code The AIBase\*.mqh files are not modules. They are method bodies of one 3,400-line class, textually #included after its declaration. Every one of them can touch every member of every other, which is why "move this out" has so far meant "move the whole class". Introduce the seam that ends that: CTrainingDataView abstract - the ONLY thing a training-side collaborator may see: a feature row, a label, an outcome, an excursion, the shape they share, and the identity to log under. CAIBaseTrainingData the adapter. MQL5 gives a class exactly one base and CExpertSignalAIBase is already a CExpertSignalCustom, so it cannot implement the view itself. It owns one of these instead. Data*() on the the published read API the adapter forwards to. signal MQL5 has no `friend`, so reaching in from outside was never an option - and making it explicit is the point rather than a workaround. Every row accessor OWNS ITS BOUNDS TEST and answers false for a bar it has nothing for. Thirty-odd call sites currently carry their own ArraySize() guard; one that forgets reads past a cache that is shorter than the bar count for the whole warm-up. The -2.0 "never scored" sentinel on the arrow cache is folded in the same way, so it can no longer be mistaken for a small confidence. Nothing uses it yet - this is the seam only, kept as its own commit so the pattern compiles before 951 lines of Baselines move onto it. The pattern is the stdlib's own: abstract base with =0 (Canvas\DX\DXObject), concrete override, forward-declared owner pointer. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-23 11:53:00 -04:00
//--- BIND THE VIEW FIRST. Collaborators are handed TrainingData() and nothing else, so an unbound
//--- adapter would answer every question with a safe default and a diagnostic would quietly report
//--- nothing at all - which is worse than one that fails loudly.
m_trainingData.Bind(GetPointer(this));
refactor(baselines): the first real module - a class, not an #included partial Baselines was 951 lines of CExpertSignalAIBase method bodies in a file that only looked like a module. It is now CBaselineComparator: a class the signal OWNS, which reads a CTrainingDataView and prints. It does not name the signal anywhere in its code. What the seam forced out into the open: - Thirty-odd ArraySize() bounds tests, each carried by its caller, are now one test per accessor next to the data. The two `hasValueN` and one `arrowN` locals are gone with them. - The -2.0 "never scored" sentinel on the arrow cache was tested at the call site. It is now inside DataDirectionalCall, where it cannot be read as a small confidence. - DoubleToSignal needs m_outputNeuronsCount, so a raw double could not be turned into a side by any reader. The view answers DirectionalCall(bar, isBuy, magnitude) instead - the conversion happens where the head width lives, and the module no longer needs ENUM_SIGNAL at all. - m_baselineDone was a latch on the signal for a decision only this module makes. It is m_done, private, where it belongs. Correction to my own earlier claim: I said Baselines had nine exclusive members "polluting the signal class". It had none. m_x, m_f, m_ngrad, m_AvgCE and the rest are FIELDS OF ALGLIB REPORT OBJECTS (state.m_x, mrep.m_AvgCE) that my `\bm_\w+` scan matched after the dot. The module needs no private state but its view pointer and that latch - which is why it came out this cleanly. The include sits below the g_ens* vote globals and the Alglib headers it reads, because unlike the AIBase\*.mqh partials this is a real class declaration compiled where it stands. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-23 12:00:18 -04:00
//--- ...and every collaborator gets the VIEW, never `this`. That is what stops a module from
//--- quietly growing a second dependency on the signal the way the AIBase\*.mqh files all did.
m_baselines.Bind(GetPointer(m_trainingData));
refactor(chart): ChartUI is a real collaborator, not a raw-include partial (S2) Expert/AIBase/ChartUI.mqh was 869 lines of method bodies of CExpertSignalAIBase, #included after the class declaration - free to touch any of its ~500 members. First of the eleven AIBase/*.mqh partials to come out (fewest inbound edges - see the SOLID campaign session order), using the same view+adapter shape already proven for CTrainingDataView. CChartView (Expert/Chart/IChartView.mqh) is the abstract read/behaviour surface a chart-rendering collaborator needs - identity, bar/model access, the prediction cache, and the training/vote/meta scalars the panel and HUD line summarise. CAIBaseChartView is the adapter the signal owns and binds to itself (MQL5 gives a class exactly one base, so CExpertSignalAIBase cannot implement the view directly). CChartUI is the real collaborator: it owns the arrow-restore queue, the rescan queue/tally, the last-arrows-saved count and the purge-mismatch latch as its own fields (verified via grep to be touched nowhere else in Expert/), and reaches everything else - including StartChartSignalRescan, moved in from its old inline home in the header since it drives the exact same rescan state machine AdvanceChartSignalRescan drains - through the view. m_arrowSignalCache and m_signalClusterWindow stay on the signal: Training.mqh writes the cache directly every era and the training-data view already reads it, so moving it would mean rewriting Training.mqh's write sites too - out of scope here. CChartUI reaches it through four bounds-checked accessors instead of a raw member poke. All 10 public methods keep their exact signatures and become one-line forwards on the signal, so no other file's call sites change except Training.mqh's one era-end status refresh, which now reads RefreshStatusLabel() rather than reaching into CChartUI's now-private last-displayed-neuron cache directly. Verified structurally, not compiled (never compile - the operator does, in MetaEditor): brace balance checked on every touched/new file against HEAD, and the view/adapter/impl method lists cross-diffed to confirm all 59 accessors match 1:1 across the interface, the adapter declaration and the adapter body. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-08-23 20:03:52 -04:00
m_chartView.Bind(GetPointer(this));
m_chartUI.Bind(GetPointer(m_chartView));
refactor(persistence): ModelPersistence is a real collaborator, not a raw-include partial (S3) Expert/AIBase/Persistence.mqh (598 lines, 8 methods) -> Expert/Persistence/: IPersistenceView.mqh (abstract, 68 read+write accessors) + AIBasePersistenceView.mqh/ AIBasePersistenceViewImpl.mqh (the adapter) + ModelPersistence.mqh (CModelPersistence, the real collaborator - signal owns m_modelPersistence and binds it to m_persistenceView, same shape as ChartUI's S2). Grep-verified before starting: every field these 8 methods touch is ALSO touched elsewhere in the class (Training/Lifecycle/OnlineLearning/Topology/FeatureScreen/ Labels.mqh) or already exposed via ChartView. Zero exclusive state, unlike ChartUI's arrow-restore/rescan queues - CModelPersistence is stateless, holding only the borrowed view pointer, operating entirely through 68 Persist*/PersistSet*() accessors on the signal. ValidateCpuInference's Net-pointer/throwaway-clone core is ONE consolidated view call (PersistRunCpuInferenceSelfCheck) rather than field-by-field - irreducible pointer/ object work, not signal state, same doctrine as ChartScoreBarForRescan. LoadNetWithRetry keeps its original CheckPointer(Net)-free Net.Load() call unchanged (no guard added - would change failure behaviour on what must be a pure relocation). This code writes the actual on-disk .cfg/.stats binary layouts every deployed model depends on (explicit "DO NOT REORDER" comment in the original), so beyond compiling clean (0 errors, 0 warnings) this was verified with a positional field-order diff: every FileWrite*/FileRead* call's target field, extracted and normalized from both the original and the new file, matches 1:1 in the same order (43/43 on the write side covering SaveModelStats+SaveTopologyConfiguration, 17/17 on LoadModelStats' read side; LoadAndCompareTopologyConfiguration's local-variable read block was copied verbatim, untouched, so nothing to diff there). The magic-version conditionals (WST2-6, haveDerivedStages/haveBarrierGeometry/etc.) moved unchanged. All 8 methods keep their exact original signatures as one-line forwards - zero external call sites changed.
2026-08-23 21:45:09 -04:00
m_persistenceView.Bind(GetPointer(this));
m_modelPersistence.Bind(GetPointer(m_persistenceView));
refactor(online-learning): OnlineLearning is a real collaborator, not a raw-include partial (S5) Expert\AIBase\OnlineLearning.mqh (595 lines) -> Expert\OnlineLearning\: IOnlineLearningView.mqh (abstract, ~50 accessors) + AIBaseOnlineLearningView.mqh/ AIBaseOnlineLearningViewImpl.mqh (the adapter) + OnlineLearning.mqh (COnlineLearning). STATEFUL, unlike CModelPersistence: grep-verified the shadow net, the OOS continual-learning simulation state and the pattern-database backfill state are genuinely exclusive to this file's own methods - Training.mqh/Topology.mqh/ Lifecycle.mqh/the signal's own header only ever CHECKED or RESET this state at era/lifecycle boundaries, never owned it, so it moved onto the collaborator as real members (same doctrine as Excursion). Those external touch points became consolidated view/forward calls instead of raw field pokes - AbortSimIfActive() replaces THREE separate copies of the same delete/null/false triple (Training.mqh's stop path, FlushTrainRun, ResetWeights), matching the geometry-scan duplicate-reset precedent in project memory. ResetForFreshTopology() replaces Topology.mqh's five- field reset block, DeployNet() replaces the shadow-preferred net selection duplicated in Inference.mqh and ChartScoreBarForRescan, and BlendTowardNet() replaces the era-end blend Training.mqh used to poke m_shadowNet for directly. Reused the signal's existing Data*()/Chart*()/Persist*() getters wherever one already answered the question (labels/outcome/history/horizon/priors/servable-bars/etc.); added ~30 new Online*() wrappers only for what nothing else exposed yet. The three PersistOnline*() get/set pairs (WST3 .stats fields) now forward through the owning member instead of touching the field directly - CModelPersistence is unaffected. Every method body is a pure relocation of the original's statements in original order; verified against `git show HEAD~1:Expert/AIBase/Excursion.mqh`-style diff against the pre-extraction file kept in the working tree until this commit. Compiled 0 errors, 0 warnings (stage mirror + MetaEditor64 /compile, ~91s). Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-08-23 22:59:02 -04:00
m_onlineLearningView.Bind(GetPointer(this));
m_onlineLearning.Bind(GetPointer(m_onlineLearningView));
refactor(topology): split shape derivation into CTopology, leave the boot sequence in place Expert/AIBase/Topology.mqh (1191 lines) held two genuinely different jobs: the fingerprint/derived-shape/BuildFreshTopology math, and InitNeuralNetwork/ InitFeatureIndicators - the network boot sequence (config-lock, tester-cache seeding, load/save the .cfg, net-load backend fallback, chart/persistence/ online-learning orchestration). Extracted the first job to Expert/Topology/ as CTopology + CTopologyView/ CAIBaseTopologyView (20 methods: BuildModelFingerprint, the Estimated*/Compute* budget math, the Conv*/Lstm* shape helpers, Add*Stage, BuildFreshTopology). STATELESS, like ModelPersistence - grep-verified zero exclusive fields, every member these methods touch is shared elsewhere in the signal. Reused ~15 existing Data*/Chart*/Persist*/Exc* getters per the established convention; added ~20 new getter overloads next to their existing setters (UseVolumes(), MinDirectionalRecall(), etc. - same pattern as SignalClusterWindow) and ~16 new Topology*() wrappers for fields with no prior accessor. The Net-pointer swap in BuildFreshTopology is one consolidated view call (TopologyReplaceNetFromTopology), same doctrine as Persistence's RunCpuInferenceSelfCheck - irreducible pointer work, not signal state. Deliberately did NOT extract InitNeuralNetwork/InitFeatureIndicators: they orchestrate nearly every other collaborator (chart, persistence, online- learning, cross-asset, config-lock) rather than deriving a shape, so moving them would just relocate a hub, not reduce coupling - same judgment call as Inference.mqh (assessed, not extracted). They stay in the AIBase/Topology.mqh partial, byte-identical to before (diffed against git HEAD to confirm), and now call the extracted math through the same public forwards every other caller already used. Verified: string- and numeric-literal diff of the old file's 20 method bodies against the new CTopology methods (0 differences), InitNeuralNetwork/ InitFeatureIndicators byte-identical, self-compiled 0 errors/0 warnings. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-08-23 23:23:00 -04:00
m_topologyView.Bind(GetPointer(this));
m_topology.Bind(GetPointer(m_topologyView));
refactor(features): FeatureBuilder is a real collaborator, not a raw-include partial Expert/AIBase/Features.mqh (2017 lines, 38 methods) split by exclusivity grep (whole-repo, not just Expert/): 30 methods -> Expert/Features/FeatureBuilder.mqh (CFeatureBuilder + CFeaturesView/CAIBaseFeaturesView), 8 stay behind as a much smaller raw partial. CFeatureBuilder is STATEFUL, same shape as Excursion/OnlineLearning: owns the 10 feature-only indicator handles (m_Volumes/m_MA/m_RSI/m_MACDFeature/ m_Ichimoku/5 AD* CiCustom indicators - grep-verified touched nowhere else in the repo, only their bare declarations) plus the depth-probe/handle-repair/ spread-series/detectability-latch scalars (exclusive, Lifecycle.mqh ctor-init only elsewhere). m_Open/m_Close/m_High/m_Low/m_Time/m_ATR/m_ADZigZag stay signal-owned - Labels.mqh/AutoTune.mqh/Training.mqh read them directly - and are reached read-only through the view (FeatureOpenAt/FeatureHighAt/ FeatureLowAt/ChartBarClose/ChartBarTime/OnlineAtrMain, all reused where a forward already existed). Deliberately did NOT move InitOpen/InitClose/InitHigh/InitLow/InitTime/ InitADZigZag/ResizeBuffers/RefreshData: they manage the 7 shared indicators' Create/BufferResize/Refresh lifecycle, which would need a pure-relay wrapper per operation per indicator for zero coupling benefit - same judgment as Topology's boot sequence. They stay in Expert/AIBase/Features.mqh and reach CFeatureBuilder's 10 owned indicators through 20 new Feature*BufferResize()/ Feature*Refresh() forwards (signal calling into its own owned collaborator directly, no view needed in that direction). Whole-repo grep (not just Expert/) caught a real external miss the campaign's own doctrine warns about: Signals/SignalMETA.mqh read m_spreadSeries/ m_spreadSeriesBars directly as an inherited protected field (a subclass, not an AIBase/*.mqh partial) - fixed with two new FeatureSpreadSeriesBars()/ FeatureSpreadSeriesAt() forwards. Verified: if(/for(/while( counts identical between the original file and the new split (269/20/1); return-count delta (+12) fully accounted for by the 12 new trivial one-line forwards added (10 indicator BufferResize + 2 spread- series getters); quoted-string-literal diff empty except two doc-comment paraphrases. Self-compiled 0 errors, 0 warnings.
2026-08-24 00:00:31 -04:00
m_featuresView.Bind(GetPointer(this));
m_featureBuilder.Bind(GetPointer(m_featuresView));
m_configLockView.Bind(GetPointer(this));
m_configLock.Bind(GetPointer(m_configLockView));
fix(ensemble): responsive panel + synchronized eras + combined-vote accuracy Four user-reported/requested items, one root cause chain: 1) DEAD CONTROL PANEL in AI_HYBRID mode. All members posted custom event id 1 and handled id 1001, and CExpertCustom broadcasts every chart event to every filter - so each posted event ran a train chunk in ALL N members (N*N chunks per round) and the chart thread never idled long enough to deliver clicks/drags. profiling.csv: 99.45% of time in OnChartEventHandler. Fix: per-instance study-event ids (STUDY_EVENT_ID_BASE + construction order, offset above the Controls library's ON_* codes - id 1 was also ON_DBL_CLICK, so panel double-clicks fired training chunks). ArmStudyEvent() is the single post site; lost-event watchdog replaces the accidental sibling-clears-my-flag rescue. 2) WARM-UP DUPLICATION. The auto-tune sweep is deterministic over identical features/labels, and it ends in the full MI diagnostic suite, which the MI-share gate never intercepted on the sweep path - four members ran four identical ~36s sweep+report blocks. First member publishes outcome (g_ensembleChartTuneDone/Installed/Settings); the rest apply it and skip both. 3) DEINIT STRANDED PANEL+ARROWS (user repro 18:52). Root cause from the log: the 4,500ms budget runs from MetaTrader's stop REQUEST - a heavy autosave in flight ate it, OnDeinit got ~430ms and died in the first member's arrow persist ("Abnormal termination" 432ms in). Fix: early visible-UI sweep (native prefix deletes for status/panel/dialog) right after ClearStatusLabel, and a fast path for still-training models - their arrows are re-rendered every era, so they get one bulk purge instead of scan+atomic-write in the death window. 4) ENSEMBLE FEATURES (user requests): era BARRIER - members advance era by era together; a member ahead of the slowest still-training member declines Train() calls and its chunk budget is donated (TRAIN_TIME_BUDGET_MS = 120/activeTrainers, UI headroom constant). COMBINED-VOTE OOS SCORE - each member's pass-3 scan contributes its adjusted per-bar decision (0.0 on abstain) to a shared row buffer; the last member to finish the era scores the averaged vote vs the mirrored Min_Vote_Open against the same target-before-stop outcomes members grade themselves on, publishing an "Ensemble vote" line on the aggregated panel. Member headlines now carry their lifetime win rate with break-even. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-16 19:06:04 -04:00
m_studyEventId = (ushort)(STUDY_EVENT_ID_BASE + g_warriorStudyEventSeq++);
m_studyArmedTick = 0;
m_ensembleIndex = -1; // not an ensemble member until EnsembleMember(true) registers one (the flag itself is in the init list)
feat(ensemble): deploy gate on the COMBINED VOTE, with a joint checkpoint The unit of evaluation in ensemble mode becomes the vote, because the vote is what trades (user: "at the end of the day they will vote together during live trading so that would make sense"). Four decisions move from the member to the ensemble: * which era is "best" -> the era whose COMBINED VOTE scored best * what is checkpointed -> a JOINT snapshot: every member's weights at that one era * when the run gives up -> one shared plateau ladder * whether it may deploy -> family-wise gate on the vote WHY THE JOINT CHECKPOINT IS THE POINT: per-member selection picks each net's own best era, and those eras differ. The resulting quartet was never measured together at any instant, so the vote it casts live is a configuration no OOS number ever described. Capturing all four at the era whose vote won makes the deployed ensemble exactly the measured one. Correct because of the era barrier (b77e7b4): Train() runs at most one era per call and a member that finished era N is held until every member reaches N, so when the last member scores the vote no member's weights have advanced past end-of-era-N. That makes the deferred simultaneous capture a guarantee rather than a race. Each snapshot is era-STAMPED and deploy requires every stamp to equal the winning era - otherwise a member whose capture failed would still hold an older snapshot and the deployed quartet would again be one nothing measured. Partial capture rolls the era back out of "best" so the search continues instead of freezing behind a checkpoint that does not exist. Statistics mirror the per-member gate one for one - same coverage floor (MIN_COVERAGE_FRACTION_OF_BASE_RATE), same always-call-one-direction chance reference, same EDGE_MIN_SIGMAS margin, same Sidak correction over the eras ranked (DEPLOY_FAMILY_WISE_ALPHA). Only the population differs: the bars the VOTE fired on, at Min_Vote_Open, rather than the bars one member called. Two-sidedness is required of the vote itself - a vote that never goes short IS the always-long model the chance reference prices in. Members keep their own per-era statistics and their own learning-rate dynamics (regression restore, eta decay); those are per-net training mechanics, not deployment decisions. The shared ladder is mirrored onto each member so per-era log lines report the state that actually governs them. Solo charts are untouched on every path. Verified: full MetaEditor compile, 0 errors 0 warnings. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-16 19:25:25 -04:00
//--- per-era stash the ensemble verdict reads (see EnsembleStashEraStats); -1/false = "no era yet"
m_eraStatPrecPct = -1.0;
m_eraStatChancePct = -1.0;
fix(vote): persist the member's skill verdict - a converged model was ruled no-skill on every restart SP500 resumed converged at era 136 with its tier ladder correctly restored and still swept 4999 bars reporting "0 had a snapshot, drew 0 arrow(s)" while the other five charts drew 221-312. HasDemonstratedEdge() - added with the no-skill exclusion - compares m_eraStatPrecPct against m_eraStatChancePct. Both are written once per era by EnsembleStashEraStats. A converged model runs no eras, so after a restart both sat at their -1 ctor defaults, every member was ruled no-skill, ReconstructionWeight() returned 0 for all four, and the overlay divisor was zero on every bar. Exactly the failure the WST7 ladder persistence fixed one level down: the ladder says how much a member votes, this says whether it may. RankTiersFromOos already computes the pair (pooled holdout precision and the zero-skill reference rate) and now records it as the CERTIFIED edge. That path is reached by the era end AND by the deployed replay, which is the only measurement a converged model will ever make. Persisted as WST8; HasDemonstratedEdge() prefers the era pair and falls back to it. The census line also had to be fixed: it reported "NOT ONE of those bars had a single member snapshot ... no enrolled member has published m_overlaySigSnap" for a condition that was purely a skill verdict. The snapshots were there. It now counts the two causes separately and names the one that fired. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-26 16:53:16 -04:00
m_certifiedPrecPct = -1.0;
m_certifiedChancePct = -1.0;
feat(ensemble): deploy gate on the COMBINED VOTE, with a joint checkpoint The unit of evaluation in ensemble mode becomes the vote, because the vote is what trades (user: "at the end of the day they will vote together during live trading so that would make sense"). Four decisions move from the member to the ensemble: * which era is "best" -> the era whose COMBINED VOTE scored best * what is checkpointed -> a JOINT snapshot: every member's weights at that one era * when the run gives up -> one shared plateau ladder * whether it may deploy -> family-wise gate on the vote WHY THE JOINT CHECKPOINT IS THE POINT: per-member selection picks each net's own best era, and those eras differ. The resulting quartet was never measured together at any instant, so the vote it casts live is a configuration no OOS number ever described. Capturing all four at the era whose vote won makes the deployed ensemble exactly the measured one. Correct because of the era barrier (b77e7b4): Train() runs at most one era per call and a member that finished era N is held until every member reaches N, so when the last member scores the vote no member's weights have advanced past end-of-era-N. That makes the deferred simultaneous capture a guarantee rather than a race. Each snapshot is era-STAMPED and deploy requires every stamp to equal the winning era - otherwise a member whose capture failed would still hold an older snapshot and the deployed quartet would again be one nothing measured. Partial capture rolls the era back out of "best" so the search continues instead of freezing behind a checkpoint that does not exist. Statistics mirror the per-member gate one for one - same coverage floor (MIN_COVERAGE_FRACTION_OF_BASE_RATE), same always-call-one-direction chance reference, same EDGE_MIN_SIGMAS margin, same Sidak correction over the eras ranked (DEPLOY_FAMILY_WISE_ALPHA). Only the population differs: the bars the VOTE fired on, at Min_Vote_Open, rather than the bars one member called. Two-sidedness is required of the vote itself - a vote that never goes short IS the always-long model the chance reference prices in. Members keep their own per-era statistics and their own learning-rate dynamics (regression restore, eta decay); those are per-net training mechanics, not deployment decisions. The shared ladder is mirrored onto each member so per-era log lines report the state that actually governs them. Solo charts are untouched on every path. Verified: full MetaEditor compile, 0 errors 0 warnings. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-16 19:25:25 -04:00
m_eraStatCalls = 0;
m_eraStatTradeable = false;
m_eraStatTwoSided = false;
m_eraStatScore = 0.0;
m_eraStatBlended = 0.0;
m_eraStatThreshold = 0.0;
m_checkpointEra = -1;
//--- indicator tuning defaults live in CADIndicatorTuner's own constructor (Expert\ADIndicatorTuner.mqh),
//--- which runs automatically for the m_indicatorTuner member above.
}
//+------------------------------------------------------------------+
//| Destructor |
//+------------------------------------------------------------------+
CExpertSignalAIBase::~CExpertSignalAIBase(void)
{
//--- deliberately NOT calling PersistOnShutdown() here: OnDeinit() (Warrior_EA.mq5) already
//--- calls it explicitly for every signal, one call stack frame shallower, BEFORE
//--- Expert.Deinit() tears these objects down.
if(CheckPointer(Net) != POINTER_INVALID)
delete Net;
if(CheckPointer(TempData) != POINTER_INVALID)
delete TempData;
2026-08-23 22:28:31 -04:00
//--- Excursion head teardown (m_excNet/m_excTgt/m_excOut) now happens in CExcursionHead's own
refactor(online-learning): OnlineLearning is a real collaborator, not a raw-include partial (S5) Expert\AIBase\OnlineLearning.mqh (595 lines) -> Expert\OnlineLearning\: IOnlineLearningView.mqh (abstract, ~50 accessors) + AIBaseOnlineLearningView.mqh/ AIBaseOnlineLearningViewImpl.mqh (the adapter) + OnlineLearning.mqh (COnlineLearning). STATEFUL, unlike CModelPersistence: grep-verified the shadow net, the OOS continual-learning simulation state and the pattern-database backfill state are genuinely exclusive to this file's own methods - Training.mqh/Topology.mqh/ Lifecycle.mqh/the signal's own header only ever CHECKED or RESET this state at era/lifecycle boundaries, never owned it, so it moved onto the collaborator as real members (same doctrine as Excursion). Those external touch points became consolidated view/forward calls instead of raw field pokes - AbortSimIfActive() replaces THREE separate copies of the same delete/null/false triple (Training.mqh's stop path, FlushTrainRun, ResetWeights), matching the geometry-scan duplicate-reset precedent in project memory. ResetForFreshTopology() replaces Topology.mqh's five- field reset block, DeployNet() replaces the shadow-preferred net selection duplicated in Inference.mqh and ChartScoreBarForRescan, and BlendTowardNet() replaces the era-end blend Training.mqh used to poke m_shadowNet for directly. Reused the signal's existing Data*()/Chart*()/Persist*() getters wherever one already answered the question (labels/outcome/history/horizon/priors/servable-bars/etc.); added ~30 new Online*() wrappers only for what nothing else exposed yet. The three PersistOnline*() get/set pairs (WST3 .stats fields) now forward through the owning member instead of touching the field directly - CModelPersistence is unaffected. Every method body is a pure relocation of the original's statements in original order; verified against `git show HEAD~1:Expert/AIBase/Excursion.mqh`-style diff against the pre-extraction file kept in the working tree until this commit. Compiled 0 errors, 0 warnings (stage mirror + MetaEditor64 /compile, ~91s). Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-08-23 22:59:02 -04:00
//--- destructor (S4), same as CChartUI/CModelPersistence needing none here. The shadow net and the
//--- OOS-simulation net teardown now happens in COnlineLearning's own destructor, same doctrine.
fix(chart): arrows survived the EA that drew them - persist, then clear Reported: on deinit the panel and status label go, the signal arrows stay. Two independent causes, both fixed here. 1. It was partly deliberate. ShutdownChartCleanup carried a second behaviour selected by a `preserveChartArrows` flag derived from the deinit reason: on RECOMPILE / PARAMETERS / CHARTCHANGE / TEMPLATE the arrows were left on the chart on purpose, to avoid a reload flicker. That branch IS the reported symptom, an operator cannot tell it apart from a cleanup that failed, and it was outright wrong whenever the reload changed the config - REASON_PARAMETERS means exactly that, and the preserved arrows then belonged to a model the chart no longer runs, with nothing marking them stale. It is gone, along with the flag and m_purgeChartOnDestruct. One path now: persist, clear, restore on the next attach. 2. Whatever remains was unfalsifiable. PurgeChart was a single ObjectsDeleteAll(prefix) whose return value was discarded, with no caller ever looking at the chart again - so "the arrows are still there" and "the arrows were never there" produced identical evidence, which is why the report survived three sessions. It now verifies: after the bulk delete it walks the OBJ_ARROW-typed list (a handful of objects, not the whole chart), deletes any surviving WarSig_ by name, and says so. Costs one typed scan when the bulk delete works, which is the normal case; names the root cause when it does not. Every failure mode of SaveChartSignals was also silent - it returned void and had three bare early returns. It returns bool now, logs the open error with the filename, and the shutdown purge is CONDITIONAL on it: for a converged model the chart objects are the only copy of its signal history (nothing redraws them - the renderer runs per training era and a deployed model has none left), so a chart left littered because the disk write failed beats a clean chart bought by destroying the history. Either way the log now says which happened. Also states the user's rule once, where arrows come back rather than across InitNeuralNetwork's several exits: no weights loaded for this config => clear the sidecar and start visually clean. A fresh run must not inherit calls it never made, and the first save would otherwise adopt them (the sidecar is rebuilt by scanning the chart). Compiles 0 errors / 0 warnings, standard and Market. Needs redeploy. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-01 13:28:34 -04:00
//--- Unconditional now that the warm-reload "leave the arrows up" branch is gone (see
//--- ShutdownChartCleanup). Cheap and idempotent: OnDeinit already purged, so this normally deletes
//--- nothing - it exists for the teardown paths that never reach OnDeinit (a failed OnInit).
PurgeChart();
//--- Last, and cheap by design (one global-variable delete): the claim must outlive every save above
//--- it, or a chart re-attaching during this teardown could start writing the same files mid-save.
ReleaseConfigLock();
}
//+------------------------------------------------------------------+
//| Sets the file/id identity a subclass constructor would otherwise |
//| repeat verbatim (ID, m_id, m_folderPath, m_fileName, pattern count)|
//+------------------------------------------------------------------+
void CExpertSignalAIBase::SetIdentity(string id, string shortId, int patternCount = 4)
{
ID = id;
m_id = shortId;
m_folderPath = eaName + "\\" + "Neural Networks" + "\\" + "State" + "\\" + m_id + "\\";
m_fileName = m_folderPath + _Symbol + "_" + IntegerToString(_Period);
m_pattern_count = patternCount;
}
//+------------------------------------------------------------------+
feat(panel): commands reach signals down the filter tree, not through a registry The control panel drove training by looping g_aiSignals[] - a hand-maintained, MAX_AI_SIGNALS-capped, AI-only registry that had already dropped an ensemble member on the floor once (609be10). A model missing from it still trains and still votes, it just cannot be paused, stopped, deployed or reset, and every button label is computed from the same short list, so the panel described one set of models while acting on another. Classic signals could not respond to a panel action at all. Commands now walk the signal tree CExpert already owns: Expert.DispatchSignalCommand(cmd) -> root signal -> every filter, recursively, returning how many actually acted. CExpertSignalCustom carries the seam (OnSignalCommand / HasSignalTrait, both no-ops by default), so a classic signal opts in by overriding two methods and needs no registration and no cap. CExpertSignalAIBase implements the training commands over its existing Pause/Stop/Deploy/ Reset methods - the behaviour is unchanged, only its reach reported. Button labels ask the same tree via CountSignalTrait, with SIGTRAIT_TRAINABLE as an explicit denominator: "all paused" is meaningless without knowing how many could be paused. Pause/Stop resolve their toggle direction ONCE in the EA and hand every model the same plain command, instead of each re-deriving the direction from its own local state - which is how a mixed set ends up half paused. The alerts now report the count acted on rather than assuming it. Two dispatch bugs found on the way, both from a database guard copied onto event delivery: CExpertSignalCustom::OnTickHandler and ::OnChartEventHandler each skipped any filter whose GetFilterID() is "NULL". That id is a DB folder name, and CSignalNewsFilter, CSignalSessionFilter and CSignalRiskGuard never set one - so all three were silently receiving neither ticks nor chart events. The guard stays where it belongs, on the paths that write pattern tables. ENUM_CP_ACTION moves to Enumerations\GlobalEnums.mqh (now include- guarded) because the Expert bases have to name it and the panel is included long after them. The AI-only lifecycle loops - PollTraining, the weight autosave, AltDataReload, OnDeinit's shutdown cascade - still use g_aiSignals[] and are untouched here. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-20 09:05:50 -04:00
//| CONTROL-PANEL COMMAND. One switch, so a button can never reach |
//| some models and miss others: the panel resolves the direction |
//| once and every model in the tree is told the same thing. |
//+------------------------------------------------------------------+
bool CExpertSignalAIBase::OnSignalCommand(const ENUM_SIGNAL_COMMAND cmd)
{
switch(cmd)
{
case SIGCMD_PAUSE_TRAINING:
PauseTraining();
return true;
case SIGCMD_RESUME_TRAINING:
ResumeTraining();
return true;
case SIGCMD_STOP_TRAINING:
StopTraining();
return true;
case SIGCMD_START_TRAINING:
StartTraining();
return true;
//--- These three report SUCCESS rather than "I understood the command", because the panel
//--- counts them back to the operator and a failed deploy or a failed rebuild is exactly what
//--- they need told about.
case SIGCMD_DEPLOY:
return DeployNow();
case SIGCMD_RESET_WEIGHTS:
return ResetWeights();
case SIGCMD_SAVE_WEIGHTS:
return SaveWeightsNow();
case SIGCMD_LOAD_WEIGHTS:
return LoadWeightsNow();
case SIGCMD_RETRAIN_DEPLOYED:
RetrainDeployed();
return true;
//--- Queues a rescan; returns whether one was actually queued, which is what tells the EA to
//--- defer the "arrows shown" alert until every queued scan has drained.
case SIGCMD_RESCAN_SIGNALS:
return StartChartSignalRescan();
case SIGCMD_REPORT_IDENTITY:
Print(" " + RegistryLine());
return true;
}
return false;
}
//+------------------------------------------------------------------+
//| Panel button labels ask these; see ENUM_SIGNAL_TRAIT. |
//+------------------------------------------------------------------+
bool CExpertSignalAIBase::HasSignalTrait(const ENUM_SIGNAL_TRAIT trait)
{
switch(trait)
{
case SIGTRAIT_TRAINABLE:
return true;
case SIGTRAIT_TRAINING_PAUSED:
return m_trainingPaused;
case SIGTRAIT_TRAINING_STOPPED:
return m_trainingStopRequested;
case SIGTRAIT_TRAINING_COMPLETE:
return m_trainingComplete;
//--- "still trainable AND has never checkpointed an era that cleared the per-class recall
//--- floor" - the same bar the plateau ladder refuses to cross on its own.
case SIGTRAIT_DEPLOY_SKIPS_RECALL:
return (!m_trainingComplete && !m_bestPassedRecall);
case SIGTRAIT_RESCAN_PENDING:
refactor(chart): ChartUI is a real collaborator, not a raw-include partial (S2) Expert/AIBase/ChartUI.mqh was 869 lines of method bodies of CExpertSignalAIBase, #included after the class declaration - free to touch any of its ~500 members. First of the eleven AIBase/*.mqh partials to come out (fewest inbound edges - see the SOLID campaign session order), using the same view+adapter shape already proven for CTrainingDataView. CChartView (Expert/Chart/IChartView.mqh) is the abstract read/behaviour surface a chart-rendering collaborator needs - identity, bar/model access, the prediction cache, and the training/vote/meta scalars the panel and HUD line summarise. CAIBaseChartView is the adapter the signal owns and binds to itself (MQL5 gives a class exactly one base, so CExpertSignalAIBase cannot implement the view directly). CChartUI is the real collaborator: it owns the arrow-restore queue, the rescan queue/tally, the last-arrows-saved count and the purge-mismatch latch as its own fields (verified via grep to be touched nowhere else in Expert/), and reaches everything else - including StartChartSignalRescan, moved in from its old inline home in the header since it drives the exact same rescan state machine AdvanceChartSignalRescan drains - through the view. m_arrowSignalCache and m_signalClusterWindow stay on the signal: Training.mqh writes the cache directly every era and the training-data view already reads it, so moving it would mean rewriting Training.mqh's write sites too - out of scope here. CChartUI reaches it through four bounds-checked accessors instead of a raw member poke. All 10 public methods keep their exact signatures and become one-line forwards on the signal, so no other file's call sites change except Training.mqh's one era-end status refresh, which now reads RefreshStatusLabel() rather than reaching into CChartUI's now-private last-displayed-neuron cache directly. Verified structurally, not compiled (never compile - the operator does, in MetaEditor): brace balance checked on every touched/new file against HEAD, and the view/adapter/impl method lists cross-diffed to confirm all 59 accessors match 1:1 across the interface, the adapter declaration and the adapter body. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-08-23 20:03:52 -04:00
return RescanPending();
feat(panel): commands reach signals down the filter tree, not through a registry The control panel drove training by looping g_aiSignals[] - a hand-maintained, MAX_AI_SIGNALS-capped, AI-only registry that had already dropped an ensemble member on the floor once (609be10). A model missing from it still trains and still votes, it just cannot be paused, stopped, deployed or reset, and every button label is computed from the same short list, so the panel described one set of models while acting on another. Classic signals could not respond to a panel action at all. Commands now walk the signal tree CExpert already owns: Expert.DispatchSignalCommand(cmd) -> root signal -> every filter, recursively, returning how many actually acted. CExpertSignalCustom carries the seam (OnSignalCommand / HasSignalTrait, both no-ops by default), so a classic signal opts in by overriding two methods and needs no registration and no cap. CExpertSignalAIBase implements the training commands over its existing Pause/Stop/Deploy/ Reset methods - the behaviour is unchanged, only its reach reported. Button labels ask the same tree via CountSignalTrait, with SIGTRAIT_TRAINABLE as an explicit denominator: "all paused" is meaningless without knowing how many could be paused. Pause/Stop resolve their toggle direction ONCE in the EA and hand every model the same plain command, instead of each re-deriving the direction from its own local state - which is how a mixed set ends up half paused. The alerts now report the count acted on rather than assuming it. Two dispatch bugs found on the way, both from a database guard copied onto event delivery: CExpertSignalCustom::OnTickHandler and ::OnChartEventHandler each skipped any filter whose GetFilterID() is "NULL". That id is a DB folder name, and CSignalNewsFilter, CSignalSessionFilter and CSignalRiskGuard never set one - so all three were silently receiving neither ticks nor chart events. The guard stays where it belongs, on the paths that write pattern tables. ENUM_CP_ACTION moves to Enumerations\GlobalEnums.mqh (now include- guarded) because the Expert bases have to name it and the panel is included long after them. The AI-only lifecycle loops - PollTraining, the weight autosave, AltDataReload, OnDeinit's shutdown cascade - still use g_aiSignals[] and are untouched here. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-20 09:05:50 -04:00
}
return false;
}
//+------------------------------------------------------------------+
//| "Voting" that price will grow. |
//+------------------------------------------------------------------+
int CExpertSignalAIBase::LongCondition(void)
{
int result = 0;
//--- Readiness gate: live trading still requires a converged model, but an inference-only tester
//--- run may replay a model that was ACTUALLY loaded from disk even if its persisted
//--- trainingComplete flag is false.
NoteVoteGate(DoubleToSignal(dPrevSignal) == Buy);
if(!m_trainingComplete && !(m_inferenceOnly && m_modelLoadedFromDisk))
return 0;
//--- No alternation gate any more - see the removal note at m_voteGateBlocked's declaration.
//--- Under triple-barrier labels consecutive same-direction setups are ordinary and correct.
if(dPrevSignal == -2)
return 0;
//--- NO confidence floor here, by design - see m_minSignalConfidence's former declaration site. A
//--- weak call is not blocked at the AI's own boundary; it votes at its tier weight (as low as
//--- m_pattern_0) and is then filtered by Min vote to open, exactly like a weak classic vote.
if(DoubleToSignal(dPrevSignal) == Buy)
{
int tier = ConfidenceTier();
result = PatternWeightForTier(tier);
m_active_pattern = "Pattern_" + IntegerToString(tier);
m_active_direction = "Buy";
}
return(result);
}
//+------------------------------------------------------------------+
//| "Voting" that price will fall. |
//+------------------------------------------------------------------+
int CExpertSignalAIBase::ShortCondition(void)
{
int result = 0;
//--- Readiness gate - see LongCondition's matching comment.
NoteVoteGate(DoubleToSignal(dPrevSignal) == Sell);
if(!m_trainingComplete && !(m_inferenceOnly && m_modelLoadedFromDisk))
return 0;
//--- "not yet studied" sentinel, and no confidence floor - see LongCondition's matching comments.
if(dPrevSignal == -2)
return 0;
if(DoubleToSignal(dPrevSignal) == Sell)
{
int tier = ConfidenceTier();
result = PatternWeightForTier(tier);
m_active_pattern = "Pattern_" + IntegerToString(tier);
m_active_direction = "Sell";
}
return result;
}
//+------------------------------------------------------------------+
fix(gate): the ensemble gate certified a vote the EA never casts g_ensembleVoteThreshold's comment claims the combined-vote scorer "fires on the same criterion the live trade does". It did not. Two independent mismatches, both silent: CURRENCY. Each member contributed its raw signed confidence x100 - a 33..100 number straight off the softmax head. Live contributes m_weight x the tier's pattern weight, and BOTH of those are rewritten from the signal DB by UpdateSignalsWeights(). A head output and a DB-ranked win-rate weight share an axis and nothing relates them, so the same bar was one number to the gate and a different one to the order path. Same shape as the 2026-08-09 geometry incident: certified on one game, paid on another. DENOMINATOR. The gate divided by the member count, so an abstaining member pulled the average toward zero. CExpertSignalCustom::Direction() skips a zero contribution in BOTH the sum and the count (`if(direction == 0) continue;` before `number++`) - live is a mean over VOTERS. The gate was therefore scoring a strictly more agreement-heavy set of bars than the EA trades. The contribution hook's own comment asserted the opposite ("abstentions dilute the average exactly as they do in the live vote"), while the AI_CHOICE enum 20 lines away correctly documented union semantics. LiveVoteContribution() is now the single definition of "what this member votes", called from the gate; the live path reaches the same arithmetic through LongCondition/ShortCondition. g_ensVoteVoterMask records who actually voted, separately from who evaluated the bar, because those are the divisor and the shared-population test respectively. ConfidenceTier() is split into ConfidenceTierFor(signal) plus a thin live-bar wrapper - the OOS scan holds the scanned bar's decision in a local, and dPrevSignal is a different bar. NOT changed, deliberately: the gate still does not model live NMS declustering, and the per-member solo gate still scores every directional call rather than threshold-clearing ones. Both are selection-metric changes and this codebase has twice been bitten by switching one blind. Also corrects a stale paragraph in m_pattern_0's declaration block quoting 80/87/93/100 as the tier defaults. The constructor is 25/50/75/100 and has been since the confidence floor and alternation gate were removed; the block carried both tables at once, and the dead one was quoted back as fact. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-18 15:23:33 -04:00
//| Buckets a confidence magnitude into one of 4 equal bands between |
//| the head's own structural floor and 1.0 - see m_pattern_0's |
//| declaration comment for the resulting tier/weight table. Neither |
//| head has a configurable floor: the boundary is 1/3 for the 3-class|
//| softmax and 0.5 for regression (DoubleToSignal's own threshold), |
//| both arithmetic properties of the head rather than settings. |
//+------------------------------------------------------------------+
fix(gate): the ensemble gate certified a vote the EA never casts g_ensembleVoteThreshold's comment claims the combined-vote scorer "fires on the same criterion the live trade does". It did not. Two independent mismatches, both silent: CURRENCY. Each member contributed its raw signed confidence x100 - a 33..100 number straight off the softmax head. Live contributes m_weight x the tier's pattern weight, and BOTH of those are rewritten from the signal DB by UpdateSignalsWeights(). A head output and a DB-ranked win-rate weight share an axis and nothing relates them, so the same bar was one number to the gate and a different one to the order path. Same shape as the 2026-08-09 geometry incident: certified on one game, paid on another. DENOMINATOR. The gate divided by the member count, so an abstaining member pulled the average toward zero. CExpertSignalCustom::Direction() skips a zero contribution in BOTH the sum and the count (`if(direction == 0) continue;` before `number++`) - live is a mean over VOTERS. The gate was therefore scoring a strictly more agreement-heavy set of bars than the EA trades. The contribution hook's own comment asserted the opposite ("abstentions dilute the average exactly as they do in the live vote"), while the AI_CHOICE enum 20 lines away correctly documented union semantics. LiveVoteContribution() is now the single definition of "what this member votes", called from the gate; the live path reaches the same arithmetic through LongCondition/ShortCondition. g_ensVoteVoterMask records who actually voted, separately from who evaluated the bar, because those are the divisor and the shared-population test respectively. ConfidenceTier() is split into ConfidenceTierFor(signal) plus a thin live-bar wrapper - the OOS scan holds the scanned bar's decision in a local, and dPrevSignal is a different bar. NOT changed, deliberately: the gate still does not model live NMS declustering, and the per-member solo gate still scores every directional call rather than threshold-clearing ones. Both are selection-metric changes and this codebase has twice been bitten by switching one blind. Also corrects a stale paragraph in m_pattern_0's declaration block quoting 80/87/93/100 as the tier defaults. The constructor is 25/50/75/100 and has been since the confidence floor and alternation gate were removed; the block carried both tables at once, and the dead one was quoted back as fact. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-18 15:23:33 -04:00
int CExpertSignalAIBase::ConfidenceTierFor(const double signal)
{
//--- The head's own STRUCTURAL decision boundary - the lowest confidence magnitude that head can
//--- possibly report for a directional call - not a user setting: - 3-class classification: the
//--- winning class of a 3-way softmax is arithmetically >= 1/3, since three probabilities
//--- summing to 1 cannot all be below it.
double floorConf = (m_outputNeuronsCount == 3) ? (1.0 / 3.0) : 0.5;
double span = MathMax(1.0 - floorConf, 0.0001);
//--- RAW magnitude, not the calibrated one (fixed 2026-08-16).
fix(gate): the ensemble gate certified a vote the EA never casts g_ensembleVoteThreshold's comment claims the combined-vote scorer "fires on the same criterion the live trade does". It did not. Two independent mismatches, both silent: CURRENCY. Each member contributed its raw signed confidence x100 - a 33..100 number straight off the softmax head. Live contributes m_weight x the tier's pattern weight, and BOTH of those are rewritten from the signal DB by UpdateSignalsWeights(). A head output and a DB-ranked win-rate weight share an axis and nothing relates them, so the same bar was one number to the gate and a different one to the order path. Same shape as the 2026-08-09 geometry incident: certified on one game, paid on another. DENOMINATOR. The gate divided by the member count, so an abstaining member pulled the average toward zero. CExpertSignalCustom::Direction() skips a zero contribution in BOTH the sum and the count (`if(direction == 0) continue;` before `number++`) - live is a mean over VOTERS. The gate was therefore scoring a strictly more agreement-heavy set of bars than the EA trades. The contribution hook's own comment asserted the opposite ("abstentions dilute the average exactly as they do in the live vote"), while the AI_CHOICE enum 20 lines away correctly documented union semantics. LiveVoteContribution() is now the single definition of "what this member votes", called from the gate; the live path reaches the same arithmetic through LongCondition/ShortCondition. g_ensVoteVoterMask records who actually voted, separately from who evaluated the bar, because those are the divisor and the shared-population test respectively. ConfidenceTier() is split into ConfidenceTierFor(signal) plus a thin live-bar wrapper - the OOS scan holds the scanned bar's decision in a local, and dPrevSignal is a different bar. NOT changed, deliberately: the gate still does not model live NMS declustering, and the per-member solo gate still scores every directional call rather than threshold-clearing ones. Both are selection-metric changes and this codebase has twice been bitten by switching one blind. Also corrects a stale paragraph in m_pattern_0's declaration block quoting 80/87/93/100 as the tier defaults. The constructor is 25/50/75/100 and has been since the confidence floor and alternation gate were removed; the block carried both tables at once, and the dead one was quoted back as fact. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-18 15:23:33 -04:00
double rawMag = MathAbs(signal);
fix: drop the ranking slice for the calibration band; un-collapse the tiers NOT COMPILED - user compiles. (1) THE RANKING SLICE IS GONE. It reserved 20% of the OOS window so the pattern-DB backfill would read bars the deployed checkpoint was not SELECTED on. That objection stands; carving a new region to answer it did not. The calibration band already has every property the slice was buying: never trained on | never graded by pass 3 (which walks [0, oosCutoff) and so never reaches it) | never seen by the deploy gate | purged by a full label horizon on BOTH sides | and larger besides - 1,684 bars vs the ~970 carved So the backfill now walks [calibLo, calibHi) and pass 3 goes back to grading the entire OOS window, exactly as before any of this. The gate gets its full sample back (~10% of a sigma), the split loses a region, and the failure mode found an hour ago - a reserved region silently blanking ~10 months of chart arrows, because arrows are only drawn on bars pass 3 grades - becomes impossible. One impurity, stated in the completion log rather than hidden: m_dirConfThreshold is FITTED on that band and the walk applies it to decide which bars fired, so coverage there is mildly optimistic. One scalar under a coverage floor, against checkpoint selection over hundreds of eras. This backfill IS the deploy-time warm-up: it runs right after FinalizeTrainRun() restores the deployed weights, so it scores with exactly what is about to trade. (2) EVERY CALL WAS TIER 0, AND IT WAS ARITHMETIC. ConfidenceTier() quartiles [floorConf, 1] where floorConf = 1/3 - the lowest magnitude a 3-way softmax winner can hold. But it was fed CalibratedConfidenceMagnitude(), which multiplies by m_confidenceCalScale, clamped to [0.3, 1.5]. That lower clamp is BELOW 1/3. Whenever calibration bottoms out, t goes negative and MathMax(0, ...) pins every call to tier 0. Which is what the live run does. m_confidenceCalScale is EMA'd toward empiricalAccuracy / avgClaimedConfidence; with the model over-calling Neutral, 3-class agreement sits near 10% against a claimed confidence near 0.9, so the ratio is ~0.11 and clamps to 0.3 every era. Logged: tier prec T0:72%(828) T1:n/a(0) T2:n/a(0) T3:n/a(0) 828 calls, one bucket - the four tier weights and the entire per-tier pattern-DB ranking reduced to a single number. The backfill was feeding a mechanism that structurally could not rank. Tiering now reads the RAW head magnitude, which genuinely lives on the [1/3, 1] range these bounds were written for. Calibration keeps its real jobs - AIConfidence() for MM sizing and SignedAIConfidence() for the vote are unchanged. STILL OPEN, deliberately not touched here: the calibration TARGET itself. empiricalAccuracy is 3-class agreement, which is the wrong quantity to scale a DIRECTIONAL confidence against - it counts a Neutral class that is 0.19% of labels. The honest target is the win rate on the calls the confidence describes (directional precision), with the claimed-confidence average taken over those same called bars. That needs a new accumulator and it interacts with the Neutral over-calling being fixed elsewhere, so it wants one clean run first. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-16 23:06:11 -04:00
if(!MathIsValidNumber(rawMag))
return 0;
double t = (MathMin(1.0, rawMag) - floorConf) / span;
int tier = (int)MathFloor(t * 4.0);
return MathMax(0, MathMin(tier, 3));
}
//+------------------------------------------------------------------+
fix(gate): the ensemble gate certified a vote the EA never casts g_ensembleVoteThreshold's comment claims the combined-vote scorer "fires on the same criterion the live trade does". It did not. Two independent mismatches, both silent: CURRENCY. Each member contributed its raw signed confidence x100 - a 33..100 number straight off the softmax head. Live contributes m_weight x the tier's pattern weight, and BOTH of those are rewritten from the signal DB by UpdateSignalsWeights(). A head output and a DB-ranked win-rate weight share an axis and nothing relates them, so the same bar was one number to the gate and a different one to the order path. Same shape as the 2026-08-09 geometry incident: certified on one game, paid on another. DENOMINATOR. The gate divided by the member count, so an abstaining member pulled the average toward zero. CExpertSignalCustom::Direction() skips a zero contribution in BOTH the sum and the count (`if(direction == 0) continue;` before `number++`) - live is a mean over VOTERS. The gate was therefore scoring a strictly more agreement-heavy set of bars than the EA trades. The contribution hook's own comment asserted the opposite ("abstentions dilute the average exactly as they do in the live vote"), while the AI_CHOICE enum 20 lines away correctly documented union semantics. LiveVoteContribution() is now the single definition of "what this member votes", called from the gate; the live path reaches the same arithmetic through LongCondition/ShortCondition. g_ensVoteVoterMask records who actually voted, separately from who evaluated the bar, because those are the divisor and the shared-population test respectively. ConfidenceTier() is split into ConfidenceTierFor(signal) plus a thin live-bar wrapper - the OOS scan holds the scanned bar's decision in a local, and dPrevSignal is a different bar. NOT changed, deliberately: the gate still does not model live NMS declustering, and the per-member solo gate still scores every directional call rather than threshold-clearing ones. Both are selection-metric changes and this codebase has twice been bitten by switching one blind. Also corrects a stale paragraph in m_pattern_0's declaration block quoting 80/87/93/100 as the tier defaults. The constructor is 25/50/75/100 and has been since the confidence floor and alternation gate were removed; the block carried both tables at once, and the dead one was quoted back as fact. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-18 15:23:33 -04:00
//| The live bar's tier - the only caller shape that existed before |
//| ConfidenceTierFor() was split out, kept so LongCondition()/ |
//| ShortCondition() read exactly as they did. |
//+------------------------------------------------------------------+
int CExpertSignalAIBase::ConfidenceTier(void)
{
return ConfidenceTierFor(dPrevSignal);
}
//+------------------------------------------------------------------+
//| The signed vote this member would cast for a given decision, in |
//| the units CExpertSignalCustom::Direction() sums - see the |
//| declaration comment for why this is a shared function and not |
//| two parallel expressions. |
fix(gate): the ensemble gate certified a vote the EA never casts g_ensembleVoteThreshold's comment claims the combined-vote scorer "fires on the same criterion the live trade does". It did not. Two independent mismatches, both silent: CURRENCY. Each member contributed its raw signed confidence x100 - a 33..100 number straight off the softmax head. Live contributes m_weight x the tier's pattern weight, and BOTH of those are rewritten from the signal DB by UpdateSignalsWeights(). A head output and a DB-ranked win-rate weight share an axis and nothing relates them, so the same bar was one number to the gate and a different one to the order path. Same shape as the 2026-08-09 geometry incident: certified on one game, paid on another. DENOMINATOR. The gate divided by the member count, so an abstaining member pulled the average toward zero. CExpertSignalCustom::Direction() skips a zero contribution in BOTH the sum and the count (`if(direction == 0) continue;` before `number++`) - live is a mean over VOTERS. The gate was therefore scoring a strictly more agreement-heavy set of bars than the EA trades. The contribution hook's own comment asserted the opposite ("abstentions dilute the average exactly as they do in the live vote"), while the AI_CHOICE enum 20 lines away correctly documented union semantics. LiveVoteContribution() is now the single definition of "what this member votes", called from the gate; the live path reaches the same arithmetic through LongCondition/ShortCondition. g_ensVoteVoterMask records who actually voted, separately from who evaluated the bar, because those are the divisor and the shared-population test respectively. ConfidenceTier() is split into ConfidenceTierFor(signal) plus a thin live-bar wrapper - the OOS scan holds the scanned bar's decision in a local, and dPrevSignal is a different bar. NOT changed, deliberately: the gate still does not model live NMS declustering, and the per-member solo gate still scores every directional call rather than threshold-clearing ones. Both are selection-metric changes and this codebase has twice been bitten by switching one blind. Also corrects a stale paragraph in m_pattern_0's declaration block quoting 80/87/93/100 as the tier defaults. The constructor is 25/50/75/100 and has been since the confidence floor and alternation gate were removed; the block carried both tables at once, and the dead one was quoted back as fact. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-18 15:23:33 -04:00
//+------------------------------------------------------------------+
double CExpertSignalAIBase::LiveVoteContribution(const double signal)
{
fix(vote): unranked members voted with the stock 25/50/75/100 ladder Two defects behind "arrows drawn while members are still mid-era". 1. THE DRAW. The filtered overlay armed on the FIRST member to finish pass 3 and leaned on a 60 s rate limit to "collapse the burst", assuming members finish seconds apart. They do not - on USDJPY one member was at sample 10496 of pass 2 while another was at 2304, minutes apart. A member with no era-end snapshot returns false from SnapshotVoteAt, and the sweep's `if(!hasData) continue;` skips it BEFORE `den += ModuleWeight()`, so the one finished model's tier weight became the entire vote and was drawn as a consensus arrow. An abstention is a member that looked at the bar and said nothing; a missing snapshot is a member that has not looked. The first must dilute the vote, the second must suppress the draw. The arm is now a readiness MASK - one bit per m_ensembleIndex, set at that member's pass-3 completion, cleared when a sweep arms - and a sweep waits for every enrolled member. Bounded at 10 minutes so a member that stops cannot freeze the chart, and the partial draw PRINTS which members were missing: the be39674 lesson is that a hold must never silence the thing that reports it. 2. THE VOTE ITSELF, which is the worse half and is not display-only. Tier weights are not persisted in the .nnw - they exist only as the output of a completed pass 3 - so before a member's first RankTiersFromOos() it holds the constructor's stock 25/50/75/100. Since 4858507 the vote currency is a WIN RATE, so an unranked tier-3 call enters the capability-weighted mean claiming a 100% win rate beside ranked members contributing ~25. Not a strong opinion: the wrong unit. One unranked member drags the ensemble over any threshold, on every fresh deploy and every resume. USDJPY has a measured ceiling of ~19 and was firing anyway. LiveVoteContribution() now abstains until self-ranked, which drops the member from the sum AND the divisor. One function, so live and the gate move together (2c443ba). Era 0 will therefore report 0 coverage until each member completes one era. The ensemble line says so explicitly rather than leaving it to look like the USDJPY unreachable-threshold case - the two are identical in the coverage number and completely different problems. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-23 08:55:04 -04:00
//--- A MEMBER THAT HAS NEVER RANKED ITSELF DOES NOT VOTE. Until RankTiersFromOos() runs once,
//--- m_pattern_0..3 hold the constructor's stock 25/50/75/100 ladder - and since 4858507 the vote
//--- currency is a WIN RATE, so an unranked tier-3 call enters the mean claiming a 100% win rate
//--- against ranked members contributing ~25. That is not a strong opinion, it is the wrong unit:
//--- one unranked member drags the whole ensemble over any threshold. It bites on every fresh
//--- deploy AND every resume, because the tier weights are not persisted in the .nnw - they exist
//--- only as the output of a completed pass 3. Found 2026-08-23 on USDJPY, whose measured ceiling
//--- is ~19 and which was firing anyway.
if(!m_tiersSelfRanked)
return 0.0;
fix(gate): the ensemble gate certified a vote the EA never casts g_ensembleVoteThreshold's comment claims the combined-vote scorer "fires on the same criterion the live trade does". It did not. Two independent mismatches, both silent: CURRENCY. Each member contributed its raw signed confidence x100 - a 33..100 number straight off the softmax head. Live contributes m_weight x the tier's pattern weight, and BOTH of those are rewritten from the signal DB by UpdateSignalsWeights(). A head output and a DB-ranked win-rate weight share an axis and nothing relates them, so the same bar was one number to the gate and a different one to the order path. Same shape as the 2026-08-09 geometry incident: certified on one game, paid on another. DENOMINATOR. The gate divided by the member count, so an abstaining member pulled the average toward zero. CExpertSignalCustom::Direction() skips a zero contribution in BOTH the sum and the count (`if(direction == 0) continue;` before `number++`) - live is a mean over VOTERS. The gate was therefore scoring a strictly more agreement-heavy set of bars than the EA trades. The contribution hook's own comment asserted the opposite ("abstentions dilute the average exactly as they do in the live vote"), while the AI_CHOICE enum 20 lines away correctly documented union semantics. LiveVoteContribution() is now the single definition of "what this member votes", called from the gate; the live path reaches the same arithmetic through LongCondition/ShortCondition. g_ensVoteVoterMask records who actually voted, separately from who evaluated the bar, because those are the divisor and the shared-population test respectively. ConfidenceTier() is split into ConfidenceTierFor(signal) plus a thin live-bar wrapper - the OOS scan holds the scanned bar's decision in a local, and dPrevSignal is a different bar. NOT changed, deliberately: the gate still does not model live NMS declustering, and the per-member solo gate still scores every directional call rather than threshold-clearing ones. Both are selection-metric changes and this codebase has twice been bitten by switching one blind. Also corrects a stale paragraph in m_pattern_0's declaration block quoting 80/87/93/100 as the tier defaults. The constructor is 25/50/75/100 and has been since the confidence floor and alternation gate were removed; the block carried both tables at once, and the dead one was quoted back as fact. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-18 15:23:33 -04:00
ENUM_SIGNAL s = DoubleToSignal(signal);
if(s != Buy && s != Sell)
return 0.0; // abstention - live drops it from the sum AND the divisor
double w = (double)PatternWeightForTier(ConfidenceTierFor(signal));
if(!MathIsValidNumber(w) || w <= 0.0)
return 0.0; // a pattern ranked to weight 0 contributes nothing and is not a voter
feat(vote): edge-over-chance currency, no-skill exclusion, checkpoint burn-in RETRAIN-FORCING and deliberately so. Two independent fixes for the same symptom - charts that go quiet while others overtrade. 1. THE VOTE CURRENCY IS NOW EDGE OVER CHANCE, not an absolute win rate. A tier weight is a raw win rate and a raw win rate means nothing without the chance rate behind it: 30% is strong under a 14% base rate and catastrophic under 50%, yet both entered the mean as "30". That is why the threshold needed re-tuning every time the label changed - 25 was permissive at ~70% win rates under the old direction label and a near-unanimity rule at ~30% under the pivot-event one - and why one chart's 25% was never the same statement as another's. Subtracting the member's own chance rate makes the units percentage points of demonstrated edge, comparable across charts, labels and regimes. Clamped at zero: a below-chance tier is anti-informative, and contributing negatively would act on a broken model as an inverted oracle rather than discarding it. 2. A NO-SKILL MEMBER IS NOW ABSENT, NOT ABSTAINING. Measured on XTIUSD: a Perceptron collapsed to B97/S6/N3, pooled win rate 11.5% against a 14% chance rate - worse than guessing - and still voting. Three healthy members voting Sell scored -21.06/0.77 = -27.4 and cleared; with the dead one voting Buy it became (-21.06+1.44)/0.89 = -22.0 and was BLOCKED. It vetoed its own ensemble on ~95% of bars, and that WAS the chart's 3.3% coverage. Neither existing guard caught it: it IS self-ranked and its tier weights were 11-14. The fix has to remove it from the DIVISOR, not just the sum - an abstainer contributes weight by design, so zeroing only the contribution makes the dilution worse. VoteCapableWeight() already means exactly "may this member's weight sit in the denominator", so the skill test belongs there. ReconstructionWeight() and the OOS scorer's divisor move with it or the scorer certifies a vote live does not cast. The skill test reads the PREVIOUS era's measurement - gating this era's vote on this era's own outcome would be circular. 3. CHECKPOINT BURN-IN (ENSEMBLE_CHECKPOINT_MIN_ERA 20). XAUUSD deployed the checkpoint from ERA 2, XTIUSD from ERA 4, each after 69 and 65 further eras failed to beat it. Ensemble coverage measures AGREEMENT, and four models that have barely moved off their initialisation agree almost by construction - so coverage is inflated exactly when the models know least and decays as they differentiate (XAUUSD 6.6% at era 8 -> 0.4% at era 75). Since selectionScore is precision discounted by coverage, an early era outscores every mature one and the ladder freezes on it. INTENDED CONSEQUENCE: a chart whose MATURE coverage cannot clear the floor now refuses to deploy rather than shipping era-2 weights. Fewer deploys, honest ones. Burn-in eras are also kept out of g_ensCandidateEras (they could not have won, so counting them inflates the family-wise N and raises the bar for nothing) and out of g_ensErasSinceBest (or the run reaches "no better vote for N eras" with no best to beat, exhausting the escalation ladder before the first era may compete). Every pinned threshold and .stats record is in the OLD currency and is now meaningless - this forces a fresh start on its own. Nothing needs re-tuning because the threshold is DERIVED: the sweep re-picks the rung by itself. Compiled clean; NOT yet run. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-26 15:38:26 -04:00
//--- THE CURRENCY IS EDGE OVER CHANCE, NOT AN ABSOLUTE WIN RATE (2026-08-26).
//---
//--- A tier weight is a raw win rate, and a raw win rate means nothing without the chance rate it
//--- is measured against. 30% is a strong call under a 14% base rate and a catastrophic one under
//--- 50% - yet both entered the mean as "30". That is why the threshold had to be re-tuned every
//--- time the LABEL changed (25 was permissive at ~70% win rates under the old direction label and
//--- a near-unanimity rule at ~30% under the pivot-event one), and why one chart's 25% was never
//--- the same statement as another's.
//---
//--- Subtracting the member's own chance rate fixes both: a chance-level call contributes 0 on its
//--- own, the units become percentage points of demonstrated edge, and the number is comparable
//--- across charts, labels and regimes. The threshold no longer needs re-tuning when any of those
//--- move - and since it is DERIVED rather than configured, the sweep re-picks the rung by itself.
//---
//--- CLAMPED AT ZERO, deliberately. A below-chance tier is anti-informative, and treating it as an
//--- inverted oracle (contributing negatively, i.e. voting the other way) would be acting on a
//--- broken model's output rather than discarding it. Zero means "this call carries nothing"; the
//--- member stays in the divisor because it DID look - only a member with no demonstrated skill at
//--- all leaves the denominator, via VoteCapableWeight().
double chancePct = m_eraStatChancePct;
if(chancePct < 0.0)
return 0.0; // no reference rate yet: an unmeasurable edge is not a strong one
double edge = w - chancePct;
if(edge <= 0.0)
return 0.0;
double contribution = m_weight * edge;
fix(gate): the ensemble gate certified a vote the EA never casts g_ensembleVoteThreshold's comment claims the combined-vote scorer "fires on the same criterion the live trade does". It did not. Two independent mismatches, both silent: CURRENCY. Each member contributed its raw signed confidence x100 - a 33..100 number straight off the softmax head. Live contributes m_weight x the tier's pattern weight, and BOTH of those are rewritten from the signal DB by UpdateSignalsWeights(). A head output and a DB-ranked win-rate weight share an axis and nothing relates them, so the same bar was one number to the gate and a different one to the order path. Same shape as the 2026-08-09 geometry incident: certified on one game, paid on another. DENOMINATOR. The gate divided by the member count, so an abstaining member pulled the average toward zero. CExpertSignalCustom::Direction() skips a zero contribution in BOTH the sum and the count (`if(direction == 0) continue;` before `number++`) - live is a mean over VOTERS. The gate was therefore scoring a strictly more agreement-heavy set of bars than the EA trades. The contribution hook's own comment asserted the opposite ("abstentions dilute the average exactly as they do in the live vote"), while the AI_CHOICE enum 20 lines away correctly documented union semantics. LiveVoteContribution() is now the single definition of "what this member votes", called from the gate; the live path reaches the same arithmetic through LongCondition/ShortCondition. g_ensVoteVoterMask records who actually voted, separately from who evaluated the bar, because those are the divisor and the shared-population test respectively. ConfidenceTier() is split into ConfidenceTierFor(signal) plus a thin live-bar wrapper - the OOS scan holds the scanned bar's decision in a local, and dPrevSignal is a different bar. NOT changed, deliberately: the gate still does not model live NMS declustering, and the per-member solo gate still scores every directional call rather than threshold-clearing ones. Both are selection-metric changes and this codebase has twice been bitten by switching one blind. Also corrects a stale paragraph in m_pattern_0's declaration block quoting 80/87/93/100 as the tier defaults. The constructor is 25/50/75/100 and has been since the confidence floor and alternation gate were removed; the block carried both tables at once, and the dead one was quoted back as fact. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-18 15:23:33 -04:00
if(!MathIsValidNumber(contribution))
return 0.0;
return (s == Buy) ? contribution : -contribution;
}
//+------------------------------------------------------------------+
//| Returns the given tier's current pattern weight (0-100) |
//+------------------------------------------------------------------+
int CExpertSignalAIBase::PatternWeightForTier(int tier)
{
switch(tier)
{
case 0:
return m_pattern_0;
case 1:
return m_pattern_1;
case 2:
return m_pattern_2;
default:
return m_pattern_3;
}
}
//+------------------------------------------------------------------+
fix(chart): a deployed model rescans history to rebuild its vote arrows The sidecar added in 484a9d8 restores the vote arrows from the previous session - but there was no previous session to restore from, and a deployed ensemble could never produce one. The overlay that draws the vote layer replays each member's m_overlaySigSnap, published in exactly one place: RankTiersFromOos, at pass-3 completion. A converged model runs no further eras. So after a restart every member's snapshot was empty, would never fill, the sweep had nothing to replay and the chart stayed blank permanently - no route back by any path. The chart rescan is the route: it runs the DEPLOYED net forward over history and rebuilds the per-bar cache, which is the same quantity pass 3 produces, obtained without training. It already existed for the panel's Show-Signals button; it just never handed its result to the overlay, so on the default filtered view a rescan rebuilt only the RAW per-member layer - the one that is hidden - and appeared to do nothing. - PublishOverlaySnapshotFromCache() extracted from RankTiersFromOos, so the era end and a completed rescan publish through one implementation. - A completed rescan now calls it, which also arms the sweep. - PollTraining auto-arms one rescan for a model that is converged, has no snapshot, and is on the filtered view. One-shot: a model that legitimately calls Neutral everywhere must not rescan forever chasing a snapshot that is correctly empty. On the timer, not in OnInit - it is a full feedForward per bar over up to 5000 bars and drains in the same time-boxed slices as a manual rescan. Together with the sidecar this closes both halves: the rescan covers the first session and any chart whose file was lost or invalidated by a threshold change; the sidecar covers every session after one is saved. Compile-verified in _claude_stage: 0 errors, 0 warnings. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-25 13:11:36 -04:00
//| COPY THE PER-BAR CACHE INTO THE OVERLAY'S SNAPSHOT, and tell the |
//| EA this member is ready to be swept. |
//| |
//| Display-side code reads the SNAPSHOT rather than m_arrowSignalCache|
//| because the cache is wiped at every era start - a sweep reading it |
//| directly would draw from a half-rebuilt array. See |
//| project_chart_filtered_view: the display may forward the live net, |
//| but must never read a live TRAINING cache. |
//| |
//| TWO CALLERS, AND THE SECOND ONE IS THE POINT. RankTiersFromOos |
//| calls it at pass-3 completion, which covers a model that is still |
//| training. A DEPLOYED model runs no further eras, so on a restart |
//| its snapshot was empty and stayed empty forever - the sweep had |
//| nothing to replay, drew nothing, and the vote arrows could never |
//| come back (user report 2026-08-25, "still no signals drawn on |
//| chart"). A completed chart rescan now publishes here too: the |
//| rescan runs the DEPLOYED net forward over history, which is |
//| exactly the same quantity pass 3 would have produced, obtained |
//| without training. |
//+------------------------------------------------------------------+
void CExpertSignalAIBase::PublishOverlaySnapshotFromCache(void)
feat(rank): AI models rank their own confidence tiers from held-out outcomes Closes the caveat 4858507 shipped with: the vote is a confidence percentage, but only to the extent the pattern weights are measured. AI tier weights sat at their designed defaults (25/50/75/100) because AI rows only ever arrive from LIVE journaling, of which a training run produces almost none. AND A STALE-TIER BUG THAT MADE THE EVIDENCE MEANINGLESS. The OOS scan bucketed every scanned bar by ConfidenceTier(), which reads dPrevSignal - and dPrevSignal is assigned in PASS 1 only, never anywhere in the OOS scan. So an entire era's fires were bucketed by one stale, unrelated bar's confidence and landed in a SINGLE tier. That is the "tier prec T0:72%(828) T1:n/a(0) T2:n/a(0) T3:n/a(0)" symptom recorded on 2026-08-16 and attributed to the calibration clamp. The clamp was real and was fixed then; this is a second, independent cause of the identical output that survived that fix untouched - which is why the log kept reading the same afterwards. Two causes, one symptom. Now ConfidenceTierFor(adjSig): the bar this iteration actually scored. WHY THIS DOES NOT WRITE ROWS TO THE SIGNAL DB, which was the obvious reading of "fill the database during training". The user's own observation is the reason: a classic Pattern_2 is a fixed geometric condition, so its win rate is legitimately accumulated over years, but an AI Pattern_2 means "confidence landed in tier 2" and tier 2 under era 100's weights is a different statement from tier 2 under era 500's. The DB's value is ACCUMULATION, and accumulation is exactly what is wrong here - it would average together models that no longer exist, while colliding with the per-table row cap and mixing measured-on-holdout outcomes into the live ledger's own tables. What the DB actually supplies is a measured win rate per pattern, and pass 3 already computes that on held-out bars, thousands at a time. So the model ranks itself once per era, REPLACING rather than accumulating, which makes the weights describe the current weights by construction. ESTIMATOR. Not WinRateFromCounts(): it returns NO_DATA below 100 raw trades BEFORE shrinking, which here would fire on every tier every era and hand all four the pooled rate - the tiers could never separate and the mechanism would be inert. Shrinkage is the answer to a small sample; a floor in front of it means the shrinkage never runs. Instead: a Beta prior of TIER_PRIOR_EFF_N pseudo-observations centred on the model's pooled holdout rate, counted in EFFECTIVE observations, because overlapping triple-barrier labels mean 800 raw fires can be worth ~12 independent ones. Rounded to the integer, not to the decade NormalizeWinRate() uses, which would collapse the shrunk tiers back into one number. NO SAME-ERA CIRCULARITY, and it falls out of the ordering rather than a guard: weights are computed at the END of era N, so the vote scored during era N was cast with era N-1's weights. The deploy gate never grades a vote whose weights were fitted on the bars it is scoring. Residual leakage remains - the same OOS bars each era under a different model - and is stated in the code rather than papered over. Both DB clobber paths are closed: ApplyPatternWeight() declines once self-ranked, and UpdateSignalsWeights()' filter.Weight() call is guarded by SelfRanked() - guarding only the tiers would have let the hourly ranking pass undo half the self-ranking. NOT COMPILED - user compiles in MetaEditor. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-18 16:00:32 -04:00
{
fix(chart): display now reads era-end SNAPSHOTS - the live cache is wiped mid-era Full-pipeline analysis after "threshold 30, attained often, nothing drawn, still glued to buy". The log falsified the premise before any code did: 21:40:43 swept 4999, 794 voters, drew 491. Strongest 43.0% vs 30.0% 21:42:07 swept 4999, 0 voters, drew 0 21:51:30 swept 4999, 0 voters, drew 0 21:56:30 swept 4999, 922 voters, drew 382. Strongest 44.0% vs 30.0% The arrows WERE drawn - 491 of them, then 382 - and then erased. ONE root cause, three symptoms: every display path read m_arrowSignalCache, which is wiped to sentinel at each era start and only complete again when pass 3 finishes. With eras at ~30s and a sweep at ~17s: * ARROW FLICKER: a sweep landing mid-era found no voters anywhere, and its else-branch deleted the arrow on every voteless bar - erasing the previous sweep's entire output. The chart cycled populated -> blank -> populated; the user kept catching the blank phase. * READOUT GLUE: the newest-cache walk found only sentinel for ~90% of every era and fell through to dPrevSignal - the frozen purge-band edge bar that reads Buy. 659638e fixed which bar was frozen, not the freezing. * VOTER FLAP: 1299 -> 257 -> 1113 across back-to-back sweeps - each saw a different fraction of half-rebuilt caches. THE FIX, structural rather than another patch: 1. Era-end snapshots. RankTiersFromOos() runs at pass-3 completion - the one moment the cache is complete - and now copies it (raw signals, newest LOOKBACK+16 bars) into member-owned snapshot state, unconditionally, BEFORE its early return: an all-Neutral era is a snapshot worth showing, not an absence of one. Raw signals rather than votes, so a tier re-rank between eras reprices them at read time via LiveVoteContribution for free. 2. The sweep (SnapshotVoteAt) and the prospective readout both read snapshots; the readout's fallback chain is live-cache -> snapshot -> dPrevSignal, and the snapshot leg is the one that fires most of the time. 3. NO DATA IS NOT A VERDICT: a den==0 bar no longer deletes - only an actual sub-threshold vote takes an arrow down. This alone ends the wipe half of the flicker even where snapshots are missing (before the first era). 4. Arming moved from an era-counter diff (which fires at era BOUNDARIES, i.e. precisely when caches are about to be wiped) to g_warriorOverlayArmRequest, set by each RankTiersFromOos - "a member's snapshot just got fresher", the only event a redraw can act on. 60s rate limit collapses the four members' burst into one sweep. Classic-only charts arm once at start. 5. Census now reports the direction split - "922 had a voter (610 buy / 312 sell)" - so "the vote leans buy" is checkable from the log instead of inferred from arrow colours. Also visible in the log and worth knowing: the threshold flip-flopped 30 -> 40 -> 30 across the evening's re-inits (census lines at 21:42-21:51 ran at 40), so part of the observed blankness was configuration, not code. NOT COMPILED - user compiles in MetaEditor. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-18 22:03:20 -04:00
int snapN = MathMin(ArraySize(m_arrowSignalCache), SIGNAL_RESCAN_LOOKBACK_BARS + 16);
if(snapN > 0)
{
ArrayResize(m_overlaySigSnap, snapN);
ArrayCopy(m_overlaySigSnap, m_arrowSignalCache, 0, 0, snapN);
}
m_overlaySnapBars = snapN;
m_prospectiveSigSnap = -2.0;
for(int pi = 1; pi <= 16 && pi < snapN; pi++)
if(m_overlaySigSnap[pi] != -2.0 && MathIsValidNumber(m_overlaySigSnap[pi]))
{
m_prospectiveSigSnap = m_overlaySigSnap[pi];
break;
}
fix(vote): unranked members voted with the stock 25/50/75/100 ladder Two defects behind "arrows drawn while members are still mid-era". 1. THE DRAW. The filtered overlay armed on the FIRST member to finish pass 3 and leaned on a 60 s rate limit to "collapse the burst", assuming members finish seconds apart. They do not - on USDJPY one member was at sample 10496 of pass 2 while another was at 2304, minutes apart. A member with no era-end snapshot returns false from SnapshotVoteAt, and the sweep's `if(!hasData) continue;` skips it BEFORE `den += ModuleWeight()`, so the one finished model's tier weight became the entire vote and was drawn as a consensus arrow. An abstention is a member that looked at the bar and said nothing; a missing snapshot is a member that has not looked. The first must dilute the vote, the second must suppress the draw. The arm is now a readiness MASK - one bit per m_ensembleIndex, set at that member's pass-3 completion, cleared when a sweep arms - and a sweep waits for every enrolled member. Bounded at 10 minutes so a member that stops cannot freeze the chart, and the partial draw PRINTS which members were missing: the be39674 lesson is that a hold must never silence the thing that reports it. 2. THE VOTE ITSELF, which is the worse half and is not display-only. Tier weights are not persisted in the .nnw - they exist only as the output of a completed pass 3 - so before a member's first RankTiersFromOos() it holds the constructor's stock 25/50/75/100. Since 4858507 the vote currency is a WIN RATE, so an unranked tier-3 call enters the capability-weighted mean claiming a 100% win rate beside ranked members contributing ~25. Not a strong opinion: the wrong unit. One unranked member drags the ensemble over any threshold, on every fresh deploy and every resume. USDJPY has a measured ceiling of ~19 and was firing anyway. LiveVoteContribution() now abstains until self-ranked, which drops the member from the sum AND the divisor. One function, so live and the gate move together (2c443ba). Era 0 will therefore report 0 coverage until each member completes one era. The ensemble line says so explicitly rather than leaving it to look like the USDJPY unreachable-threshold case - the two are identical in the coverage number and completely different problems. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-23 08:55:04 -04:00
//--- ...and tell the EA THIS member's snapshot is fresh. The sweep waits for every enrolled
//--- member's bit - see g_warriorOverlayReadyMask for why a rate limit was not enough.
if(m_ensembleIndex >= 0 && m_ensembleIndex < ENS_MAX_MEMBERS)
g_warriorOverlayReadyMask |= (((uint)1) << m_ensembleIndex);
fix(chart): display now reads era-end SNAPSHOTS - the live cache is wiped mid-era Full-pipeline analysis after "threshold 30, attained often, nothing drawn, still glued to buy". The log falsified the premise before any code did: 21:40:43 swept 4999, 794 voters, drew 491. Strongest 43.0% vs 30.0% 21:42:07 swept 4999, 0 voters, drew 0 21:51:30 swept 4999, 0 voters, drew 0 21:56:30 swept 4999, 922 voters, drew 382. Strongest 44.0% vs 30.0% The arrows WERE drawn - 491 of them, then 382 - and then erased. ONE root cause, three symptoms: every display path read m_arrowSignalCache, which is wiped to sentinel at each era start and only complete again when pass 3 finishes. With eras at ~30s and a sweep at ~17s: * ARROW FLICKER: a sweep landing mid-era found no voters anywhere, and its else-branch deleted the arrow on every voteless bar - erasing the previous sweep's entire output. The chart cycled populated -> blank -> populated; the user kept catching the blank phase. * READOUT GLUE: the newest-cache walk found only sentinel for ~90% of every era and fell through to dPrevSignal - the frozen purge-band edge bar that reads Buy. 659638e fixed which bar was frozen, not the freezing. * VOTER FLAP: 1299 -> 257 -> 1113 across back-to-back sweeps - each saw a different fraction of half-rebuilt caches. THE FIX, structural rather than another patch: 1. Era-end snapshots. RankTiersFromOos() runs at pass-3 completion - the one moment the cache is complete - and now copies it (raw signals, newest LOOKBACK+16 bars) into member-owned snapshot state, unconditionally, BEFORE its early return: an all-Neutral era is a snapshot worth showing, not an absence of one. Raw signals rather than votes, so a tier re-rank between eras reprices them at read time via LiveVoteContribution for free. 2. The sweep (SnapshotVoteAt) and the prospective readout both read snapshots; the readout's fallback chain is live-cache -> snapshot -> dPrevSignal, and the snapshot leg is the one that fires most of the time. 3. NO DATA IS NOT A VERDICT: a den==0 bar no longer deletes - only an actual sub-threshold vote takes an arrow down. This alone ends the wipe half of the flicker even where snapshots are missing (before the first era). 4. Arming moved from an era-counter diff (which fires at era BOUNDARIES, i.e. precisely when caches are about to be wiped) to g_warriorOverlayArmRequest, set by each RankTiersFromOos - "a member's snapshot just got fresher", the only event a redraw can act on. 60s rate limit collapses the four members' burst into one sweep. Classic-only charts arm once at start. 5. Census now reports the direction split - "922 had a voter (610 buy / 312 sell)" - so "the vote leans buy" is checkable from the log instead of inferred from arrow colours. Also visible in the log and worth knowing: the threshold flip-flopped 30 -> 40 -> 30 across the evening's re-inits (census lines at 21:42-21:51 ran at 40), so part of the observed blankness was configuration, not code. NOT COMPILED - user compiles in MetaEditor. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-18 22:03:20 -04:00
g_warriorOverlayArmRequest = true;
fix(chart): a deployed model rescans history to rebuild its vote arrows The sidecar added in 484a9d8 restores the vote arrows from the previous session - but there was no previous session to restore from, and a deployed ensemble could never produce one. The overlay that draws the vote layer replays each member's m_overlaySigSnap, published in exactly one place: RankTiersFromOos, at pass-3 completion. A converged model runs no further eras. So after a restart every member's snapshot was empty, would never fill, the sweep had nothing to replay and the chart stayed blank permanently - no route back by any path. The chart rescan is the route: it runs the DEPLOYED net forward over history and rebuilds the per-bar cache, which is the same quantity pass 3 produces, obtained without training. It already existed for the panel's Show-Signals button; it just never handed its result to the overlay, so on the default filtered view a rescan rebuilt only the RAW per-member layer - the one that is hidden - and appeared to do nothing. - PublishOverlaySnapshotFromCache() extracted from RankTiersFromOos, so the era end and a completed rescan publish through one implementation. - A completed rescan now calls it, which also arms the sweep. - PollTraining auto-arms one rescan for a model that is converged, has no snapshot, and is on the filtered view. One-shot: a model that legitimately calls Neutral everywhere must not rescan forever chasing a snapshot that is correctly empty. On the timer, not in OnInit - it is a full feedForward per bar over up to 5000 bars and drains in the same time-boxed slices as a manual rescan. Together with the sidecar this closes both halves: the rescan covers the first session and any chart whose file was lost or invalidated by a threshold change; the sidecar covers every session after one is saved. Compile-verified in _claude_stage: 0 errors, 0 warnings. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-25 13:11:36 -04:00
}
//+------------------------------------------------------------------+
feat(vote): replay pass rebuilds a deployed model's ladder without retraining The previous commit persisted the tier ladder, which fixes this going forward but did nothing for models whose .stats predates WST7 - they still had to retrain to mint one. They never did. Every number a converged model needs in order to vote is a pure function of weights already on disk plus labels derivable from the chart, so replay them: stage 1 build the label cache (existing chunked prebuild) stage 2 rescan history (existing chunked rescan, deployed net) stage 3 score + rank + persist (one walk over two arrays) ScoreReplayFromCache() walks m_arrowSignalCache against m_labelCacheBuy/Sell, fills the same m_oosTierFired/Hits and per-class totals pass 3 fills, and hands them to RankTiersFromOos() - deliberately feeding the existing ranker rather than reimplementing it. The shrinkage, the chance reference and the module trust weight are subtle enough that a second copy would drift, and a ladder measured by a slightly different rule would be silently incomparable with every ladder training produced. AdvanceDeployedRebuild() sequences the three stages off the timer. It has to be a sequence: stages 1 and 2 are each minutes of work draining in time-boxed slices, and stage 2's output is meaningless until stage 1 has labels to score against. The previous version ran the rescan with no labels at all, which is why it could only ever rebuild arrows and never the ladder - the thing actually blocking the vote. The result is written to .stats immediately. The failure being repaired is state that lived in memory and was never written down; recomputing it and not saving it would repeat that exactly. Also routes every rescan completion through one hook, so there is a single place that knows what a finished rescan means - republish for a manual one, score and rank for a rebuild. Compile-verified in _claude_stage: 0 errors, 0 warnings. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-25 15:01:24 -04:00
//| THE REPLAY PASS: score the deployed net's own history, then rank. |
//| |
//| A converged model's tier ladder is produced by pass 3 and by |
//| nothing else, so before this existed a deployed model that had |
//| lost its ladder (a .stats predating WST7, or a wipe) could only |
//| get one back by RETRAINING - hours of work to recompute numbers |
//| that are a pure function of weights already on disk. |
//| |
//| This is the same measurement without the training. The rescan has |
//| already run the DEPLOYED net over every bar in the window and left |
//| its prior-corrected decision in m_arrowSignalCache; the label |
//| prebuild has filled m_labelCacheBuy/Sell for the same bars. So the |
//| ladder is one walk over two arrays that already agree on indexing. |
//| |
//| It feeds RankTiersFromOos() rather than reimplementing it. The |
//| shrinkage, the chance reference and the module trust weight are |
//| subtle enough that a second copy would drift from the first, and |
//| a ladder measured by a slightly different rule is worse than no |
//| ladder - it would be silently incomparable with every ladder any |
//| training run ever produced. |
//| |
//| INDEXING: both arrays are series-indexed (0 = newest). Bar 0 is |
//| skipped because it is still forming. |
//+------------------------------------------------------------------+
feat(vote): backfill the ensemble win-rate record from the overlay sweep "Vote win rate: measuring..." never resolved on a deployed chart whose .stats predate the WST7 ensemble record: g_ensCumOosTotal is fed only by the era-end combined-vote scorer (Training.mqh), and a deployed ensemble runs no further eras. The replay pass rebuilt every MEMBER's ladder (64-71% each, per the 16:12 log) but nothing ever scored the COMBINED vote, so the aggregate line sat on "measuring" while 300+ arrows drew. The overlay sweep already reconstructs the vote per bar with the live threshold and direction policy - so it now also tallies, BEFORE declustering (NMS thins arrows, not calls), each threshold-clearing bar against the inline swing-pivot label (same resolution ScoreReplayFromCache uses, same window-mismatch reason). On sweep completion Warrior_EA.mq5 harvests the tally through a consuming one-shot read and adopts it ONLY when the record is empty and the models are deployed - a training-time sweep can never pre-empt the era scorer, and a restored record always wins. The result is persisted immediately into every member's .stats. Also verified against the same log: the sweep does NOT ignore DrawUnfilteredSignals - 4986 voter bars -> ~300 arrows, all gated on the 25% open threshold. The arrow increase vs the restored set (41-312 saved) is the replay-minted ladder reading stronger (partly in-sample), plus the reconstruction deliberately not replaying order validation/session hours (tooltip says so); the backfilled record carries the same caveat and is labelled so in the log. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-25 18:22:12 -04:00
//+------------------------------------------------------------------+
//| The resolved swing-pivot label at one bar, for the head's |
//| combined-vote scoring during the overlay sweep. Inline label |
//| resolution, NOT the prebuilt cache - identical reasoning to the |
//| comment inside ScoreReplayFromCache below: the cache's window is |
//| anchored at dtStudied and need not overlap the sweep's. |
//+------------------------------------------------------------------+
bool CExpertSignalAIBase::ReplayTruthAt(const int idx, ENUM_SIGNAL &truth)
{
truth = SwingPivotDirectionLabel(idx);
if(m_lastLabelLifespan <= 0)
{
truth = Neutral;
return false; // pivot pair not committed: no label exists for this bar
}
return true;
}
feat(vote): replay pass rebuilds a deployed model's ladder without retraining The previous commit persisted the tier ladder, which fixes this going forward but did nothing for models whose .stats predates WST7 - they still had to retrain to mint one. They never did. Every number a converged model needs in order to vote is a pure function of weights already on disk plus labels derivable from the chart, so replay them: stage 1 build the label cache (existing chunked prebuild) stage 2 rescan history (existing chunked rescan, deployed net) stage 3 score + rank + persist (one walk over two arrays) ScoreReplayFromCache() walks m_arrowSignalCache against m_labelCacheBuy/Sell, fills the same m_oosTierFired/Hits and per-class totals pass 3 fills, and hands them to RankTiersFromOos() - deliberately feeding the existing ranker rather than reimplementing it. The shrinkage, the chance reference and the module trust weight are subtle enough that a second copy would drift, and a ladder measured by a slightly different rule would be silently incomparable with every ladder training produced. AdvanceDeployedRebuild() sequences the three stages off the timer. It has to be a sequence: stages 1 and 2 are each minutes of work draining in time-boxed slices, and stage 2's output is meaningless until stage 1 has labels to score against. The previous version ran the rescan with no labels at all, which is why it could only ever rebuild arrows and never the ladder - the thing actually blocking the vote. The result is written to .stats immediately. The failure being repaired is state that lived in memory and was never written down; recomputing it and not saving it would repeat that exactly. Also routes every rescan completion through one hook, so there is a single place that knows what a finished rescan means - republish for a manual one, score and rank for a rebuild. Compile-verified in _claude_stage: 0 errors, 0 warnings. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-25 15:01:24 -04:00
void CExpertSignalAIBase::ScoreReplayFromCache(void)
{
fix(replay): resolve labels inline - the prebuilt cache's window never overlapped the rescan The 15:13 session proved the replay pass ran end-to-end on all 24 models and scored ZERO labelled bars on every one of them, while each rescan sat on ~5000 scored predictions (~2755 Buy / ~2232 Sell). The two windows never overlapped: StartLabelCachePrebuild deliberately keeps a CONVERGED model's dtStudied watermark (it gates inference recency and must not move), so the prebuild's window was the handful of bars since the last studied bar - all with uncommitted pivots, hence "label cache pre-built - Buy: 0 | Sell: 0 | Neutral: 0" on every member. The label never needed a cache. SwingPivotDirectionLabel(idx) is a pure function of the ZigZag/Close/ATR buffers the rescan itself refreshes over exactly the scoring window, and m_lastLabelLifespan == 0 is its own unresolved flag - the same finality gate the cache applies, applied directly. ScoreReplayFromCache now resolves each bar's label inline and the label-prebuild stage is deleted from the rebuild state machine outright; going through a cache built for a different window was indirection that changed the answer. Also splits the empty-result diagnostics: "no resolved labels" (a windowing/data fault) is now distinguished from "labels present, every call Neutral" (a calibration verdict). The first version reported the second message for both, which mislabelled this very bug as a calibration outcome in the same breath as reporting scored=0. Honest limitation, stated in the code too: the replay window includes bars the model trained on, so a replay-minted ladder is measured partly in-sample and will read stronger than a holdout-measured one. It is replaced by the genuine article at the next completed scoring pass; until then it is what makes a restarted deployed model able to vote at all. Compile-verified in _claude_stage: 0 errors, 0 warnings. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-25 15:22:01 -04:00
int n = ArraySize(m_arrowSignalCache);
feat(vote): replay pass rebuilds a deployed model's ladder without retraining The previous commit persisted the tier ladder, which fixes this going forward but did nothing for models whose .stats predates WST7 - they still had to retrain to mint one. They never did. Every number a converged model needs in order to vote is a pure function of weights already on disk plus labels derivable from the chart, so replay them: stage 1 build the label cache (existing chunked prebuild) stage 2 rescan history (existing chunked rescan, deployed net) stage 3 score + rank + persist (one walk over two arrays) ScoreReplayFromCache() walks m_arrowSignalCache against m_labelCacheBuy/Sell, fills the same m_oosTierFired/Hits and per-class totals pass 3 fills, and hands them to RankTiersFromOos() - deliberately feeding the existing ranker rather than reimplementing it. The shrinkage, the chance reference and the module trust weight are subtle enough that a second copy would drift, and a ladder measured by a slightly different rule would be silently incomparable with every ladder training produced. AdvanceDeployedRebuild() sequences the three stages off the timer. It has to be a sequence: stages 1 and 2 are each minutes of work draining in time-boxed slices, and stage 2's output is meaningless until stage 1 has labels to score against. The previous version ran the rescan with no labels at all, which is why it could only ever rebuild arrows and never the ladder - the thing actually blocking the vote. The result is written to .stats immediately. The failure being repaired is state that lived in memory and was never written down; recomputing it and not saving it would repeat that exactly. Also routes every rescan completion through one hook, so there is a single place that knows what a finished rescan means - republish for a manual one, score and rank for a rebuild. Compile-verified in _claude_stage: 0 errors, 0 warnings. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-25 15:01:24 -04:00
if(n <= 1)
{
fix(replay): resolve labels inline - the prebuilt cache's window never overlapped the rescan The 15:13 session proved the replay pass ran end-to-end on all 24 models and scored ZERO labelled bars on every one of them, while each rescan sat on ~5000 scored predictions (~2755 Buy / ~2232 Sell). The two windows never overlapped: StartLabelCachePrebuild deliberately keeps a CONVERGED model's dtStudied watermark (it gates inference recency and must not move), so the prebuild's window was the handful of bars since the last studied bar - all with uncommitted pivots, hence "label cache pre-built - Buy: 0 | Sell: 0 | Neutral: 0" on every member. The label never needed a cache. SwingPivotDirectionLabel(idx) is a pure function of the ZigZag/Close/ATR buffers the rescan itself refreshes over exactly the scoring window, and m_lastLabelLifespan == 0 is its own unresolved flag - the same finality gate the cache applies, applied directly. ScoreReplayFromCache now resolves each bar's label inline and the label-prebuild stage is deleted from the rebuild state machine outright; going through a cache built for a different window was indirection that changed the answer. Also splits the empty-result diagnostics: "no resolved labels" (a windowing/data fault) is now distinguished from "labels present, every call Neutral" (a calibration verdict). The first version reported the second message for both, which mislabelled this very bug as a calibration outcome in the same breath as reporting scored=0. Honest limitation, stated in the code too: the replay window includes bars the model trained on, so a replay-minted ladder is measured partly in-sample and will read stronger than a holdout-measured one. It is replaced by the genuine article at the next completed scoring pass; until then it is what makes a restarted deployed model able to vote at all. Compile-verified in _claude_stage: 0 errors, 0 warnings. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-25 15:22:01 -04:00
Print(ID + ": replay scoring skipped - the rescan left no per-bar signals to score.");
feat(vote): replay pass rebuilds a deployed model's ladder without retraining The previous commit persisted the tier ladder, which fixes this going forward but did nothing for models whose .stats predates WST7 - they still had to retrain to mint one. They never did. Every number a converged model needs in order to vote is a pure function of weights already on disk plus labels derivable from the chart, so replay them: stage 1 build the label cache (existing chunked prebuild) stage 2 rescan history (existing chunked rescan, deployed net) stage 3 score + rank + persist (one walk over two arrays) ScoreReplayFromCache() walks m_arrowSignalCache against m_labelCacheBuy/Sell, fills the same m_oosTierFired/Hits and per-class totals pass 3 fills, and hands them to RankTiersFromOos() - deliberately feeding the existing ranker rather than reimplementing it. The shrinkage, the chance reference and the module trust weight are subtle enough that a second copy would drift, and a ladder measured by a slightly different rule would be silently incomparable with every ladder training produced. AdvanceDeployedRebuild() sequences the three stages off the timer. It has to be a sequence: stages 1 and 2 are each minutes of work draining in time-boxed slices, and stage 2's output is meaningless until stage 1 has labels to score against. The previous version ran the rescan with no labels at all, which is why it could only ever rebuild arrows and never the ladder - the thing actually blocking the vote. The result is written to .stats immediately. The failure being repaired is state that lived in memory and was never written down; recomputing it and not saving it would repeat that exactly. Also routes every rescan completion through one hook, so there is a single place that knows what a finished rescan means - republish for a manual one, score and rank for a rebuild. Compile-verified in _claude_stage: 0 errors, 0 warnings. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-25 15:01:24 -04:00
return;
}
//--- Same counters pass 3 fills, cleared the same way. RankTiersFromOos reads ONLY these plus the
//--- per-class totals below, which is what makes this substitution exact.
ArrayInitialize(m_oosTierFired, 0);
ArrayInitialize(m_oosTierHits, 0);
m_oos.Reset();
fix(replay): resolve labels inline - the prebuilt cache's window never overlapped the rescan The 15:13 session proved the replay pass ran end-to-end on all 24 models and scored ZERO labelled bars on every one of them, while each rescan sat on ~5000 scored predictions (~2755 Buy / ~2232 Sell). The two windows never overlapped: StartLabelCachePrebuild deliberately keeps a CONVERGED model's dtStudied watermark (it gates inference recency and must not move), so the prebuild's window was the handful of bars since the last studied bar - all with uncommitted pivots, hence "label cache pre-built - Buy: 0 | Sell: 0 | Neutral: 0" on every member. The label never needed a cache. SwingPivotDirectionLabel(idx) is a pure function of the ZigZag/Close/ATR buffers the rescan itself refreshes over exactly the scoring window, and m_lastLabelLifespan == 0 is its own unresolved flag - the same finality gate the cache applies, applied directly. ScoreReplayFromCache now resolves each bar's label inline and the label-prebuild stage is deleted from the rebuild state machine outright; going through a cache built for a different window was indirection that changed the answer. Also splits the empty-result diagnostics: "no resolved labels" (a windowing/data fault) is now distinguished from "labels present, every call Neutral" (a calibration verdict). The first version reported the second message for both, which mislabelled this very bug as a calibration outcome in the same breath as reporting scored=0. Honest limitation, stated in the code too: the replay window includes bars the model trained on, so a replay-minted ladder is measured partly in-sample and will read stronger than a holdout-measured one. It is replaced by the genuine article at the next completed scoring pass; until then it is what makes a restarted deployed model able to vote at all. Compile-verified in _claude_stage: 0 errors, 0 warnings. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-25 15:22:01 -04:00
int scored = 0, unresolved = 0;
feat(vote): replay pass rebuilds a deployed model's ladder without retraining The previous commit persisted the tier ladder, which fixes this going forward but did nothing for models whose .stats predates WST7 - they still had to retrain to mint one. They never did. Every number a converged model needs in order to vote is a pure function of weights already on disk plus labels derivable from the chart, so replay them: stage 1 build the label cache (existing chunked prebuild) stage 2 rescan history (existing chunked rescan, deployed net) stage 3 score + rank + persist (one walk over two arrays) ScoreReplayFromCache() walks m_arrowSignalCache against m_labelCacheBuy/Sell, fills the same m_oosTierFired/Hits and per-class totals pass 3 fills, and hands them to RankTiersFromOos() - deliberately feeding the existing ranker rather than reimplementing it. The shrinkage, the chance reference and the module trust weight are subtle enough that a second copy would drift, and a ladder measured by a slightly different rule would be silently incomparable with every ladder training produced. AdvanceDeployedRebuild() sequences the three stages off the timer. It has to be a sequence: stages 1 and 2 are each minutes of work draining in time-boxed slices, and stage 2's output is meaningless until stage 1 has labels to score against. The previous version ran the rescan with no labels at all, which is why it could only ever rebuild arrows and never the ladder - the thing actually blocking the vote. The result is written to .stats immediately. The failure being repaired is state that lived in memory and was never written down; recomputing it and not saving it would repeat that exactly. Also routes every rescan completion through one hook, so there is a single place that knows what a finished rescan means - republish for a manual one, score and rank for a rebuild. Compile-verified in _claude_stage: 0 errors, 0 warnings. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-25 15:01:24 -04:00
for(int i = 1; i < n; i++)
{
double sig = m_arrowSignalCache[i];
if(sig == -2.0 || !MathIsValidNumber(sig))
continue; // the net never scored this bar (window could not be built)
fix(replay): resolve labels inline - the prebuilt cache's window never overlapped the rescan The 15:13 session proved the replay pass ran end-to-end on all 24 models and scored ZERO labelled bars on every one of them, while each rescan sat on ~5000 scored predictions (~2755 Buy / ~2232 Sell). The two windows never overlapped: StartLabelCachePrebuild deliberately keeps a CONVERGED model's dtStudied watermark (it gates inference recency and must not move), so the prebuild's window was the handful of bars since the last studied bar - all with uncommitted pivots, hence "label cache pre-built - Buy: 0 | Sell: 0 | Neutral: 0" on every member. The label never needed a cache. SwingPivotDirectionLabel(idx) is a pure function of the ZigZag/Close/ATR buffers the rescan itself refreshes over exactly the scoring window, and m_lastLabelLifespan == 0 is its own unresolved flag - the same finality gate the cache applies, applied directly. ScoreReplayFromCache now resolves each bar's label inline and the label-prebuild stage is deleted from the rebuild state machine outright; going through a cache built for a different window was indirection that changed the answer. Also splits the empty-result diagnostics: "no resolved labels" (a windowing/data fault) is now distinguished from "labels present, every call Neutral" (a calibration verdict). The first version reported the second message for both, which mislabelled this very bug as a calibration outcome in the same breath as reporting scored=0. Honest limitation, stated in the code too: the replay window includes bars the model trained on, so a replay-minted ladder is measured partly in-sample and will read stronger than a holdout-measured one. It is replaced by the genuine article at the next completed scoring pass; until then it is what makes a restarted deployed model able to vote at all. Compile-verified in _claude_stage: 0 errors, 0 warnings. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-25 15:22:01 -04:00
//--- THE LABEL IS RESOLVED INLINE, NOT READ FROM THE PREBUILT CACHE. The cache's window is
//--- anchored at dtStudied, which for a CONVERGED model is deliberately left at its watermark
//--- (inference recency) - a few bars ago. Scoring against it produced "0 labelled bars" on
//--- 24/24 models (2026-08-25 15:13 session) while the rescan sat on ~5000 scored predictions:
//--- the two windows simply never overlapped. SwingPivotDirectionLabel is a pure function of
//--- the ZigZag/Close/ATR buffers the rescan just refreshed over EXACTLY this window, and
//--- m_lastLabelLifespan == 0 is its own unresolved flag - the same finality gate the cache
//--- uses, applied directly. The cache itself is deliberately not touched: it belongs to the
//--- incremental training path, whose window this is not.
ENUM_SIGNAL truth = SwingPivotDirectionLabel(i);
if(m_lastLabelLifespan <= 0)
{
unresolved++;
continue; // pivot pair not committed: no label exists for this bar
}
feat(vote): replay pass rebuilds a deployed model's ladder without retraining The previous commit persisted the tier ladder, which fixes this going forward but did nothing for models whose .stats predates WST7 - they still had to retrain to mint one. They never did. Every number a converged model needs in order to vote is a pure function of weights already on disk plus labels derivable from the chart, so replay them: stage 1 build the label cache (existing chunked prebuild) stage 2 rescan history (existing chunked rescan, deployed net) stage 3 score + rank + persist (one walk over two arrays) ScoreReplayFromCache() walks m_arrowSignalCache against m_labelCacheBuy/Sell, fills the same m_oosTierFired/Hits and per-class totals pass 3 fills, and hands them to RankTiersFromOos() - deliberately feeding the existing ranker rather than reimplementing it. The shrinkage, the chance reference and the module trust weight are subtle enough that a second copy would drift, and a ladder measured by a slightly different rule would be silently incomparable with every ladder training produced. AdvanceDeployedRebuild() sequences the three stages off the timer. It has to be a sequence: stages 1 and 2 are each minutes of work draining in time-boxed slices, and stage 2's output is meaningless until stage 1 has labels to score against. The previous version ran the rescan with no labels at all, which is why it could only ever rebuild arrows and never the ladder - the thing actually blocking the vote. The result is written to .stats immediately. The failure being repaired is state that lived in memory and was never written down; recomputing it and not saving it would repeat that exactly. Also routes every rescan completion through one hook, so there is a single place that knows what a finished rescan means - republish for a manual one, score and rank for a rebuild. Compile-verified in _claude_stage: 0 errors, 0 warnings. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-25 15:01:24 -04:00
ENUM_SIGNAL pred = DoubleToSignal(sig);
scored++;
//--- Per-class totals: RankTiersFromOos derives its zero-skill reference from these
//--- (50% x the directional base rate), so they must be counted over the SAME population the
//--- tiers were counted over, not over a wider one.
switch(truth)
{
case Buy:
m_oos.buyTotal++;
break;
case Sell:
m_oos.sellTotal++;
break;
default:
m_oos.neutralTotal++;
break;
}
//--- FIRED = a directional call under the live decision rule, which is exactly the population
//--- the tier weights are meant to describe. A Neutral is an abstention, neither right nor wrong.
if(pred != Buy && pred != Sell)
continue;
int tier = ConfidenceTierFor(sig);
if(tier < 0 || tier > 3)
continue;
m_oosTierFired[tier]++;
if(pred == truth)
m_oosTierHits[tier]++;
}
int fired = 0, hits = 0;
for(int t = 0; t < 4; t++)
{
fired += m_oosTierFired[t];
hits += m_oosTierHits[t];
}
fix(replay): resolve labels inline - the prebuilt cache's window never overlapped the rescan The 15:13 session proved the replay pass ran end-to-end on all 24 models and scored ZERO labelled bars on every one of them, while each rescan sat on ~5000 scored predictions (~2755 Buy / ~2232 Sell). The two windows never overlapped: StartLabelCachePrebuild deliberately keeps a CONVERGED model's dtStudied watermark (it gates inference recency and must not move), so the prebuild's window was the handful of bars since the last studied bar - all with uncommitted pivots, hence "label cache pre-built - Buy: 0 | Sell: 0 | Neutral: 0" on every member. The label never needed a cache. SwingPivotDirectionLabel(idx) is a pure function of the ZigZag/Close/ATR buffers the rescan itself refreshes over exactly the scoring window, and m_lastLabelLifespan == 0 is its own unresolved flag - the same finality gate the cache applies, applied directly. ScoreReplayFromCache now resolves each bar's label inline and the label-prebuild stage is deleted from the rebuild state machine outright; going through a cache built for a different window was indirection that changed the answer. Also splits the empty-result diagnostics: "no resolved labels" (a windowing/data fault) is now distinguished from "labels present, every call Neutral" (a calibration verdict). The first version reported the second message for both, which mislabelled this very bug as a calibration outcome in the same breath as reporting scored=0. Honest limitation, stated in the code too: the replay window includes bars the model trained on, so a replay-minted ladder is measured partly in-sample and will read stronger than a holdout-measured one. It is replaced by the genuine article at the next completed scoring pass; until then it is what makes a restarted deployed model able to vote at all. Compile-verified in _claude_stage: 0 errors, 0 warnings. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-25 15:22:01 -04:00
Print(ID + StringFormat(": replay scored %d labelled bar(s) against the deployed weights"
" (%d unresolved bars excluded) - %d directional call(s), %d correct"
" (%.1f%%). No training was run.",
scored, unresolved, fired, hits, (fired > 0 ? 100.0 * hits / fired : 0.0)));
//--- TWO DISTINCT EMPTY OUTCOMES, and naming the wrong one cost a session: "no labels overlapped"
//--- is a windowing/data fault to be fixed, while "labels present, every call Neutral" is a
//--- calibration verdict to be respected. The first version of this reported the second for both.
if(scored <= 0)
{
Print(ID + ": WARNING - the replay found NO RESOLVED LABELS in the rescan window, so nothing"
" could be scored. That is a data/windowing fault (ZigZag pivots missing over the whole"
" window?), not a verdict on the model. No ladder was ranked.");
return;
}
feat(vote): replay pass rebuilds a deployed model's ladder without retraining The previous commit persisted the tier ladder, which fixes this going forward but did nothing for models whose .stats predates WST7 - they still had to retrain to mint one. They never did. Every number a converged model needs in order to vote is a pure function of weights already on disk plus labels derivable from the chart, so replay them: stage 1 build the label cache (existing chunked prebuild) stage 2 rescan history (existing chunked rescan, deployed net) stage 3 score + rank + persist (one walk over two arrays) ScoreReplayFromCache() walks m_arrowSignalCache against m_labelCacheBuy/Sell, fills the same m_oosTierFired/Hits and per-class totals pass 3 fills, and hands them to RankTiersFromOos() - deliberately feeding the existing ranker rather than reimplementing it. The shrinkage, the chance reference and the module trust weight are subtle enough that a second copy would drift, and a ladder measured by a slightly different rule would be silently incomparable with every ladder training produced. AdvanceDeployedRebuild() sequences the three stages off the timer. It has to be a sequence: stages 1 and 2 are each minutes of work draining in time-boxed slices, and stage 2's output is meaningless until stage 1 has labels to score against. The previous version ran the rescan with no labels at all, which is why it could only ever rebuild arrows and never the ladder - the thing actually blocking the vote. The result is written to .stats immediately. The failure being repaired is state that lived in memory and was never written down; recomputing it and not saving it would repeat that exactly. Also routes every rescan completion through one hook, so there is a single place that knows what a finished rescan means - republish for a manual one, score and rank for a rebuild. Compile-verified in _claude_stage: 0 errors, 0 warnings. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-25 15:01:24 -04:00
if(fired <= 0)
{
fix(replay): resolve labels inline - the prebuilt cache's window never overlapped the rescan The 15:13 session proved the replay pass ran end-to-end on all 24 models and scored ZERO labelled bars on every one of them, while each rescan sat on ~5000 scored predictions (~2755 Buy / ~2232 Sell). The two windows never overlapped: StartLabelCachePrebuild deliberately keeps a CONVERGED model's dtStudied watermark (it gates inference recency and must not move), so the prebuild's window was the handful of bars since the last studied bar - all with uncommitted pivots, hence "label cache pre-built - Buy: 0 | Sell: 0 | Neutral: 0" on every member. The label never needed a cache. SwingPivotDirectionLabel(idx) is a pure function of the ZigZag/Close/ATR buffers the rescan itself refreshes over exactly the scoring window, and m_lastLabelLifespan == 0 is its own unresolved flag - the same finality gate the cache applies, applied directly. ScoreReplayFromCache now resolves each bar's label inline and the label-prebuild stage is deleted from the rebuild state machine outright; going through a cache built for a different window was indirection that changed the answer. Also splits the empty-result diagnostics: "no resolved labels" (a windowing/data fault) is now distinguished from "labels present, every call Neutral" (a calibration verdict). The first version reported the second message for both, which mislabelled this very bug as a calibration outcome in the same breath as reporting scored=0. Honest limitation, stated in the code too: the replay window includes bars the model trained on, so a replay-minted ladder is measured partly in-sample and will read stronger than a holdout-measured one. It is replaced by the genuine article at the next completed scoring pass; until then it is what makes a restarted deployed model able to vote at all. Compile-verified in _claude_stage: 0 errors, 0 warnings. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-25 15:22:01 -04:00
Print(ID + ": WARNING - the replay produced NO directional calls on " + IntegerToString(scored) +
" labelled bar(s), so no ladder can be ranked. This model calls Neutral everywhere; that"
" is a calibration outcome, not a drawing or persistence fault, and it will stay silent"
" until retrained.");
feat(vote): replay pass rebuilds a deployed model's ladder without retraining The previous commit persisted the tier ladder, which fixes this going forward but did nothing for models whose .stats predates WST7 - they still had to retrain to mint one. They never did. Every number a converged model needs in order to vote is a pure function of weights already on disk plus labels derivable from the chart, so replay them: stage 1 build the label cache (existing chunked prebuild) stage 2 rescan history (existing chunked rescan, deployed net) stage 3 score + rank + persist (one walk over two arrays) ScoreReplayFromCache() walks m_arrowSignalCache against m_labelCacheBuy/Sell, fills the same m_oosTierFired/Hits and per-class totals pass 3 fills, and hands them to RankTiersFromOos() - deliberately feeding the existing ranker rather than reimplementing it. The shrinkage, the chance reference and the module trust weight are subtle enough that a second copy would drift, and a ladder measured by a slightly different rule would be silently incomparable with every ladder training produced. AdvanceDeployedRebuild() sequences the three stages off the timer. It has to be a sequence: stages 1 and 2 are each minutes of work draining in time-boxed slices, and stage 2's output is meaningless until stage 1 has labels to score against. The previous version ran the rescan with no labels at all, which is why it could only ever rebuild arrows and never the ladder - the thing actually blocking the vote. The result is written to .stats immediately. The failure being repaired is state that lived in memory and was never written down; recomputing it and not saving it would repeat that exactly. Also routes every rescan completion through one hook, so there is a single place that knows what a finished rescan means - republish for a manual one, score and rank for a rebuild. Compile-verified in _claude_stage: 0 errors, 0 warnings. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-25 15:01:24 -04:00
return;
}
//--- ...and rank exactly as an era end would, including publishing the overlay snapshot and
//--- arming the sweep, which is the whole reason this returns nothing.
RankTiersFromOos();
}
//+------------------------------------------------------------------+
fix(chart): a deployed model rescans history to rebuild its vote arrows The sidecar added in 484a9d8 restores the vote arrows from the previous session - but there was no previous session to restore from, and a deployed ensemble could never produce one. The overlay that draws the vote layer replays each member's m_overlaySigSnap, published in exactly one place: RankTiersFromOos, at pass-3 completion. A converged model runs no further eras. So after a restart every member's snapshot was empty, would never fill, the sweep had nothing to replay and the chart stayed blank permanently - no route back by any path. The chart rescan is the route: it runs the DEPLOYED net forward over history and rebuilds the per-bar cache, which is the same quantity pass 3 produces, obtained without training. It already existed for the panel's Show-Signals button; it just never handed its result to the overlay, so on the default filtered view a rescan rebuilt only the RAW per-member layer - the one that is hidden - and appeared to do nothing. - PublishOverlaySnapshotFromCache() extracted from RankTiersFromOos, so the era end and a completed rescan publish through one implementation. - A completed rescan now calls it, which also arms the sweep. - PollTraining auto-arms one rescan for a model that is converged, has no snapshot, and is on the filtered view. One-shot: a model that legitimately calls Neutral everywhere must not rescan forever chasing a snapshot that is correctly empty. On the timer, not in OnInit - it is a full feedForward per bar over up to 5000 bars and drains in the same time-boxed slices as a manual rescan. Together with the sidecar this closes both halves: the rescan covers the first session and any chart whose file was lost or invalidated by a threshold change; the sidecar covers every session after one is saved. Compile-verified in _claude_stage: 0 errors, 0 warnings. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-25 13:11:36 -04:00
//| TURN THIS ERA'S HELD-OUT OUTCOMES INTO THE VOTE WEIGHTS. |
//+------------------------------------------------------------------+
void CExpertSignalAIBase::RankTiersFromOos(void)
{
//--- SNAPSHOT FIRST, unconditionally - this runs at pass-3 completion, the single moment the
//--- arrow cache is complete for the era.
PublishOverlaySnapshotFromCache();
feat(rank): AI models rank their own confidence tiers from held-out outcomes Closes the caveat 4858507 shipped with: the vote is a confidence percentage, but only to the extent the pattern weights are measured. AI tier weights sat at their designed defaults (25/50/75/100) because AI rows only ever arrive from LIVE journaling, of which a training run produces almost none. AND A STALE-TIER BUG THAT MADE THE EVIDENCE MEANINGLESS. The OOS scan bucketed every scanned bar by ConfidenceTier(), which reads dPrevSignal - and dPrevSignal is assigned in PASS 1 only, never anywhere in the OOS scan. So an entire era's fires were bucketed by one stale, unrelated bar's confidence and landed in a SINGLE tier. That is the "tier prec T0:72%(828) T1:n/a(0) T2:n/a(0) T3:n/a(0)" symptom recorded on 2026-08-16 and attributed to the calibration clamp. The clamp was real and was fixed then; this is a second, independent cause of the identical output that survived that fix untouched - which is why the log kept reading the same afterwards. Two causes, one symptom. Now ConfidenceTierFor(adjSig): the bar this iteration actually scored. WHY THIS DOES NOT WRITE ROWS TO THE SIGNAL DB, which was the obvious reading of "fill the database during training". The user's own observation is the reason: a classic Pattern_2 is a fixed geometric condition, so its win rate is legitimately accumulated over years, but an AI Pattern_2 means "confidence landed in tier 2" and tier 2 under era 100's weights is a different statement from tier 2 under era 500's. The DB's value is ACCUMULATION, and accumulation is exactly what is wrong here - it would average together models that no longer exist, while colliding with the per-table row cap and mixing measured-on-holdout outcomes into the live ledger's own tables. What the DB actually supplies is a measured win rate per pattern, and pass 3 already computes that on held-out bars, thousands at a time. So the model ranks itself once per era, REPLACING rather than accumulating, which makes the weights describe the current weights by construction. ESTIMATOR. Not WinRateFromCounts(): it returns NO_DATA below 100 raw trades BEFORE shrinking, which here would fire on every tier every era and hand all four the pooled rate - the tiers could never separate and the mechanism would be inert. Shrinkage is the answer to a small sample; a floor in front of it means the shrinkage never runs. Instead: a Beta prior of TIER_PRIOR_EFF_N pseudo-observations centred on the model's pooled holdout rate, counted in EFFECTIVE observations, because overlapping triple-barrier labels mean 800 raw fires can be worth ~12 independent ones. Rounded to the integer, not to the decade NormalizeWinRate() uses, which would collapse the shrunk tiers back into one number. NO SAME-ERA CIRCULARITY, and it falls out of the ordering rather than a guard: weights are computed at the END of era N, so the vote scored during era N was cast with era N-1's weights. The deploy gate never grades a vote whose weights were fitted on the bars it is scoring. Residual leakage remains - the same OOS bars each era under a different model - and is stated in the code rather than papered over. Both DB clobber paths are closed: ApplyPatternWeight() declines once self-ranked, and UpdateSignalsWeights()' filter.Weight() call is guarded by SelfRanked() - guarding only the tiers would have let the hourly ranking pass undo half the self-ranking. NOT COMPILED - user compiles in MetaEditor. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-18 16:00:32 -04:00
int pooledFired = 0, pooledHits = 0;
for(int t = 0; t < 4; t++)
{
pooledFired += m_oosTierFired[t];
pooledHits += m_oosTierHits[t];
}
//--- Nothing fired this era (all-Neutral, or a stopped/cap-hit era): leave the previous era's
//--- weights standing rather than collapsing every tier to a prior built on no evidence at all.
if(pooledFired <= 0)
return;
double pooledPct = 100.0 * pooledHits / pooledFired;
fix(geometry): a free zero made "never resolve" the winning geometry The CANDIDATE GEOMETRY line shipped in 05f1a53 said per-candidate geometry beats the global pair on every SP500 member at 2-3 sigma. It does not. It said so because a bar that reached neither barrier scored 0 R, and the incumbent's mean is NEGATIVE (-0.07 to -0.21 R). Against a losing baseline a free zero is a win, so the widest candidate always came out ahead - and the reported gain ordered itself by timeout share, not by skill: PAI 95.1% timed out -> +0.189 R (head measured -2.42 sigma, HARMFUL) HYB 73.8% -> +0.182 R (head at chance, +0.68 sigma) CONV 61.8% -> +0.163 R (head measured -2.47 sigma, HARMFUL) LSTM 27.1% -> +0.158 R (head +1.67 sigma) Monotone in the timeout share and inverted against the sigma gate. The acceptance test written when this was built - "the sigma gate predicts LSTM helps and CONV hurts; if the R difference does not reproduce that ordering, something is wrong" - is what caught it. A trade that reaches neither barrier is not worth zero. It is closed at the horizon, which is what the scheduled close-all does live and what SimulateTradeOutcome's timeout path already charges. So mark it there: TripleBarrierLabel now publishes the signed close-to-close travel at the last bar it actually visited (m_termTravelCache, same validity flag as the excursion and ladder caches), and LadderOutcomeR prices a timeout off it instead of returning false. A bar that cannot be evaluated under BOTH pairs is now dropped whole - scoring one leg and defaulting the other is the same bug in a smaller costume. Second defect, same function: CandidateGeometryFor applied neither of the floors the global derivation applies, so on USDJPY it chose stop 2.00 / target 1.00 - a 67% break-even, forbidden by the 1:2 policy floor. c3daded in miniature: a selector optimising its own criterion with no reference to the decision criterion. Both floors now apply, and the ratio is re-checked AFTER the per-leg rung snap, which can lose it. Also: the module weight was an unshrunk pooled win rate. USDJPY ConvLSTM fired 19 times (2.0 effective), won 36.8%, and took module weight 0.37 - 41% of the ensemble's capable weight and the loudest voice on the chart, off two effective observations. It also lifted the computed vote ceiling to 26.3 against a 25 threshold, which is why THRESHOLD UNREACHABLE never printed on a chart whose peak vote is 14 and whose practical ceiling without that member is 18.8. The pooled rate is now shrunk toward the coin-flip rate on the era's own OOS bars over 30 prior-equivalent calls, and the tiers shrink toward the shrunk value rather than the raw one. A member with ~300 effective calls moves by ~0.4pp; the 19-fire member goes 0.37 -> ~0.15. MEASUREMENT ONLY still - no order reads any of this. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-22 11:30:54 -04:00
//--- WHAT A MEMBER WITH NO EVIDENCE IS WORTH: the coin-flip rate on this era's OOS bars, which is
//--- what the gate calls zero skill. Shrinking toward it means "few fires -> speaks at chance",
//--- where shrinking toward the member's own pooled rate would mean "few fires -> speaks at
//--- whatever those few fires said", which is no shrinkage at all.
refactor(oos): twenty-one counters with one lifetime become one object SOosTally holds this era's OOS confusion counts and the rates they imply. The signal keeps one member where it kept twenty-one, and the era-reset block loses twenty of its twenty-one clearing lines. THE SHAPE THIS ENDS is the one that produced 7452bd1: a group of tallies read together but cleared one-per-line, so a second reset path could clear a subset and leave stale numerators over restarted denominators. Reset() is now the only way to clear them and it clears all of them. The pair had already started to drift. m_oosBuyFired/m_oosBuyFiredHits sat at line 1085 and their Sell twins at line 1140 - 55 lines and an unrelated member apart, with the Buy comment still claiming to describe both. DERIVED RATES MOVE WITH THE DATA. `(bars > 0) ? (int)MathRound(100.0 * x / bars) : -1` was written out twelve times, and the "-1 means not measurable, never 0" convention re-spelled at each - a convention the deploy gate depends on, since every caller tests `< 0` to mean "this does not block". One rounding rule and one sentinel now. GROUPED BY LIFETIME, NOT BY NAME. m_oosSamples looks like it belongs here and does not: it is RUN-level, reset only with the weights, and the status panel prints it beside dOosError which is also a run-level EMA. That pairing is correct and stays. But the confidence-calibration block divided per-era numerators by it, naming the results `empiricalAccuracy` and `avgClaimedConfidence` when neither is that - the run-level denominator cancels in their ratio, so eraScale was right and the two named intermediates were not. Now written as the ratio it actually is, with the cancellation stated, so nobody logs or gates on a half that decays with era count. BEHAVIOUR UNCHANGED: every moved expression preserves its formula, its denominator and its sentinel. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-23 14:15:22 -04:00
int zsBars = m_oos.Bars();
fix(geometry): a free zero made "never resolve" the winning geometry The CANDIDATE GEOMETRY line shipped in 05f1a53 said per-candidate geometry beats the global pair on every SP500 member at 2-3 sigma. It does not. It said so because a bar that reached neither barrier scored 0 R, and the incumbent's mean is NEGATIVE (-0.07 to -0.21 R). Against a losing baseline a free zero is a win, so the widest candidate always came out ahead - and the reported gain ordered itself by timeout share, not by skill: PAI 95.1% timed out -> +0.189 R (head measured -2.42 sigma, HARMFUL) HYB 73.8% -> +0.182 R (head at chance, +0.68 sigma) CONV 61.8% -> +0.163 R (head measured -2.47 sigma, HARMFUL) LSTM 27.1% -> +0.158 R (head +1.67 sigma) Monotone in the timeout share and inverted against the sigma gate. The acceptance test written when this was built - "the sigma gate predicts LSTM helps and CONV hurts; if the R difference does not reproduce that ordering, something is wrong" - is what caught it. A trade that reaches neither barrier is not worth zero. It is closed at the horizon, which is what the scheduled close-all does live and what SimulateTradeOutcome's timeout path already charges. So mark it there: TripleBarrierLabel now publishes the signed close-to-close travel at the last bar it actually visited (m_termTravelCache, same validity flag as the excursion and ladder caches), and LadderOutcomeR prices a timeout off it instead of returning false. A bar that cannot be evaluated under BOTH pairs is now dropped whole - scoring one leg and defaulting the other is the same bug in a smaller costume. Second defect, same function: CandidateGeometryFor applied neither of the floors the global derivation applies, so on USDJPY it chose stop 2.00 / target 1.00 - a 67% break-even, forbidden by the 1:2 policy floor. c3daded in miniature: a selector optimising its own criterion with no reference to the decision criterion. Both floors now apply, and the ratio is re-checked AFTER the per-leg rung snap, which can lose it. Also: the module weight was an unshrunk pooled win rate. USDJPY ConvLSTM fired 19 times (2.0 effective), won 36.8%, and took module weight 0.37 - 41% of the ensemble's capable weight and the loudest voice on the chart, off two effective observations. It also lifted the computed vote ceiling to 26.3 against a 25 threshold, which is why THRESHOLD UNREACHABLE never printed on a chart whose peak vote is 14 and whose practical ceiling without that member is 18.8. The pooled rate is now shrunk toward the coin-flip rate on the era's own OOS bars over 30 prior-equivalent calls, and the tiers shrink toward the shrunk value rather than the raw one. A member with ~300 effective calls moves by ~0.4pp; the 19-fire member goes 0.37 -> ~0.15. MEASUREMENT ONLY still - no order reads any of this. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-22 11:30:54 -04:00
double chancePct = (zsBars > 0)
feat(target): delete the barrier/geometry stack - the label is the verdict Step 3 of the swing-pivot plan, whole-hog. The swing label is now the ONE target and the era verdict is precision + recall per class against the label's own base rate - no win rate, no break-even, no expectancy, no geometry anywhere in training. DELETED - Expert/Excursion/ (4), Expert/BarrierHorizon/ (4), GeometrySweep, FirstPassageLadder, Labeling/TripleBarrier.mqh (CLabelOverlap survives in Labeling/LabelOverlap.mqh), 3 test EAs. - TripleBarrierLabel + walk, fractal label, geometry derivation/scan/ adoption, exit-policy replay, excursion MI targets, the drift verdict (DIRECTION_INTELLIGENT), the recall floor, balanced-accuracy telemetry, the barrier defines, the .cfg geometry adopt (slots kept as zeros for the positional layout), the derived-geometry live-order override. - TRAINING_TARGET input/enum: direction models are always swing; META2 re-keys the meta head onto label agreement (descriptor loses its two geometry slots). REWORKED - Labels.mqh (1795 -> ~370 lines): AdvanceSwingLabelState with FINALITY-GATED CACHING - an unresolved bar (pivot pair uncommitted) is never cached, so it can never freeze as a false Neutral; training, calibration, OOS scoring and online learning all skip unresolved bars. - SDeployVerdict: significance-only; SOosTally chance = larger directional class share; pooled gate poolability = timeframe (record v2). - Purge/embargo/declustering gaps: the measured mean label resolution lag (LabelResolutionBars), not a barrier horizon. - Pool purge key + backfill DB rows: marked at the bar the label resolved on (m_labelResolveAge), not a fabricated barrier touch. - Online learning frontier: finality, not a horizon delay. - m_bestBalancedOos -> m_bestSelectionScore, m_erasSinceBestBalanced -> m_erasSinceBest, ensemble vote outcome arrays -> label arrays. STEP 4 folded in: Entry_Multiplier / SL_Mode / TP_Mode / tradingdirection are inputs again - trade management is the tester GA's search space. Fingerprints: every direction model re-keys (TGT:SWG1 now unconditional, CUT token gone); META1 -> META2. Full retrain, as planned. Compile-verified in _claude_stage: Warrior_EA + both surviving test EAs, 0 errors, 0 warnings each. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-24 20:42:31 -04:00
? 50.0 * ((double)m_oos.buyTotal + (double)m_oos.sellTotal) / zsBars
fix(geometry): a free zero made "never resolve" the winning geometry The CANDIDATE GEOMETRY line shipped in 05f1a53 said per-candidate geometry beats the global pair on every SP500 member at 2-3 sigma. It does not. It said so because a bar that reached neither barrier scored 0 R, and the incumbent's mean is NEGATIVE (-0.07 to -0.21 R). Against a losing baseline a free zero is a win, so the widest candidate always came out ahead - and the reported gain ordered itself by timeout share, not by skill: PAI 95.1% timed out -> +0.189 R (head measured -2.42 sigma, HARMFUL) HYB 73.8% -> +0.182 R (head at chance, +0.68 sigma) CONV 61.8% -> +0.163 R (head measured -2.47 sigma, HARMFUL) LSTM 27.1% -> +0.158 R (head +1.67 sigma) Monotone in the timeout share and inverted against the sigma gate. The acceptance test written when this was built - "the sigma gate predicts LSTM helps and CONV hurts; if the R difference does not reproduce that ordering, something is wrong" - is what caught it. A trade that reaches neither barrier is not worth zero. It is closed at the horizon, which is what the scheduled close-all does live and what SimulateTradeOutcome's timeout path already charges. So mark it there: TripleBarrierLabel now publishes the signed close-to-close travel at the last bar it actually visited (m_termTravelCache, same validity flag as the excursion and ladder caches), and LadderOutcomeR prices a timeout off it instead of returning false. A bar that cannot be evaluated under BOTH pairs is now dropped whole - scoring one leg and defaulting the other is the same bug in a smaller costume. Second defect, same function: CandidateGeometryFor applied neither of the floors the global derivation applies, so on USDJPY it chose stop 2.00 / target 1.00 - a 67% break-even, forbidden by the 1:2 policy floor. c3daded in miniature: a selector optimising its own criterion with no reference to the decision criterion. Both floors now apply, and the ratio is re-checked AFTER the per-leg rung snap, which can lose it. Also: the module weight was an unshrunk pooled win rate. USDJPY ConvLSTM fired 19 times (2.0 effective), won 36.8%, and took module weight 0.37 - 41% of the ensemble's capable weight and the loudest voice on the chart, off two effective observations. It also lifted the computed vote ceiling to 26.3 against a 25 threshold, which is why THRESHOLD UNREACHABLE never printed on a chart whose peak vote is 14 and whose practical ceiling without that member is 18.8. The pooled rate is now shrunk toward the coin-flip rate on the era's own OOS bars over 30 prior-equivalent calls, and the tiers shrink toward the shrunk value rather than the raw one. A member with ~300 effective calls moves by ~0.4pp; the 19-fire member goes 0.37 -> ~0.15. MEASUREMENT ONLY still - no order reads any of this. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-22 11:30:54 -04:00
: pooledPct;
double pooledEffN = MathMax(0.0, EffectiveSampleSize((double)pooledFired));
double trustPct = ShrunkRatePct(pooledEffN * ((double)pooledHits / pooledFired), pooledEffN,
chancePct, MODULE_PRIOR_EFF_N);
feat(rank): AI models rank their own confidence tiers from held-out outcomes Closes the caveat 4858507 shipped with: the vote is a confidence percentage, but only to the extent the pattern weights are measured. AI tier weights sat at their designed defaults (25/50/75/100) because AI rows only ever arrive from LIVE journaling, of which a training run produces almost none. AND A STALE-TIER BUG THAT MADE THE EVIDENCE MEANINGLESS. The OOS scan bucketed every scanned bar by ConfidenceTier(), which reads dPrevSignal - and dPrevSignal is assigned in PASS 1 only, never anywhere in the OOS scan. So an entire era's fires were bucketed by one stale, unrelated bar's confidence and landed in a SINGLE tier. That is the "tier prec T0:72%(828) T1:n/a(0) T2:n/a(0) T3:n/a(0)" symptom recorded on 2026-08-16 and attributed to the calibration clamp. The clamp was real and was fixed then; this is a second, independent cause of the identical output that survived that fix untouched - which is why the log kept reading the same afterwards. Two causes, one symptom. Now ConfidenceTierFor(adjSig): the bar this iteration actually scored. WHY THIS DOES NOT WRITE ROWS TO THE SIGNAL DB, which was the obvious reading of "fill the database during training". The user's own observation is the reason: a classic Pattern_2 is a fixed geometric condition, so its win rate is legitimately accumulated over years, but an AI Pattern_2 means "confidence landed in tier 2" and tier 2 under era 100's weights is a different statement from tier 2 under era 500's. The DB's value is ACCUMULATION, and accumulation is exactly what is wrong here - it would average together models that no longer exist, while colliding with the per-table row cap and mixing measured-on-holdout outcomes into the live ledger's own tables. What the DB actually supplies is a measured win rate per pattern, and pass 3 already computes that on held-out bars, thousands at a time. So the model ranks itself once per era, REPLACING rather than accumulating, which makes the weights describe the current weights by construction. ESTIMATOR. Not WinRateFromCounts(): it returns NO_DATA below 100 raw trades BEFORE shrinking, which here would fire on every tier every era and hand all four the pooled rate - the tiers could never separate and the mechanism would be inert. Shrinkage is the answer to a small sample; a floor in front of it means the shrinkage never runs. Instead: a Beta prior of TIER_PRIOR_EFF_N pseudo-observations centred on the model's pooled holdout rate, counted in EFFECTIVE observations, because overlapping triple-barrier labels mean 800 raw fires can be worth ~12 independent ones. Rounded to the integer, not to the decade NormalizeWinRate() uses, which would collapse the shrunk tiers back into one number. NO SAME-ERA CIRCULARITY, and it falls out of the ordering rather than a guard: weights are computed at the END of era N, so the vote scored during era N was cast with era N-1's weights. The deploy gate never grades a vote whose weights were fitted on the bars it is scoring. Residual leakage remains - the same OOS bars each era under a different model - and is stated in the code rather than papered over. Both DB clobber paths are closed: ApplyPatternWeight() declines once self-ranked, and UpdateSignalsWeights()' filter.Weight() call is guarded by SelfRanked() - guarding only the tiers would have let the hourly ranking pass undo half the self-ranking. NOT COMPILED - user compiles in MetaEditor. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-18 16:00:32 -04:00
//--- WHY NOT WinRateFromCounts(), which is the estimator the classic ladders use: it returns
//--- NO_DATA_WIN_RATE for anything under MIN_TRADES_FOR_WIN_RATE raw trades, BEFORE it shrinks.
feat(rank): AI models rank their own confidence tiers from held-out outcomes Closes the caveat 4858507 shipped with: the vote is a confidence percentage, but only to the extent the pattern weights are measured. AI tier weights sat at their designed defaults (25/50/75/100) because AI rows only ever arrive from LIVE journaling, of which a training run produces almost none. AND A STALE-TIER BUG THAT MADE THE EVIDENCE MEANINGLESS. The OOS scan bucketed every scanned bar by ConfidenceTier(), which reads dPrevSignal - and dPrevSignal is assigned in PASS 1 only, never anywhere in the OOS scan. So an entire era's fires were bucketed by one stale, unrelated bar's confidence and landed in a SINGLE tier. That is the "tier prec T0:72%(828) T1:n/a(0) T2:n/a(0) T3:n/a(0)" symptom recorded on 2026-08-16 and attributed to the calibration clamp. The clamp was real and was fixed then; this is a second, independent cause of the identical output that survived that fix untouched - which is why the log kept reading the same afterwards. Two causes, one symptom. Now ConfidenceTierFor(adjSig): the bar this iteration actually scored. WHY THIS DOES NOT WRITE ROWS TO THE SIGNAL DB, which was the obvious reading of "fill the database during training". The user's own observation is the reason: a classic Pattern_2 is a fixed geometric condition, so its win rate is legitimately accumulated over years, but an AI Pattern_2 means "confidence landed in tier 2" and tier 2 under era 100's weights is a different statement from tier 2 under era 500's. The DB's value is ACCUMULATION, and accumulation is exactly what is wrong here - it would average together models that no longer exist, while colliding with the per-table row cap and mixing measured-on-holdout outcomes into the live ledger's own tables. What the DB actually supplies is a measured win rate per pattern, and pass 3 already computes that on held-out bars, thousands at a time. So the model ranks itself once per era, REPLACING rather than accumulating, which makes the weights describe the current weights by construction. ESTIMATOR. Not WinRateFromCounts(): it returns NO_DATA below 100 raw trades BEFORE shrinking, which here would fire on every tier every era and hand all four the pooled rate - the tiers could never separate and the mechanism would be inert. Shrinkage is the answer to a small sample; a floor in front of it means the shrinkage never runs. Instead: a Beta prior of TIER_PRIOR_EFF_N pseudo-observations centred on the model's pooled holdout rate, counted in EFFECTIVE observations, because overlapping triple-barrier labels mean 800 raw fires can be worth ~12 independent ones. Rounded to the integer, not to the decade NormalizeWinRate() uses, which would collapse the shrunk tiers back into one number. NO SAME-ERA CIRCULARITY, and it falls out of the ordering rather than a guard: weights are computed at the END of era N, so the vote scored during era N was cast with era N-1's weights. The deploy gate never grades a vote whose weights were fitted on the bars it is scoring. Residual leakage remains - the same OOS bars each era under a different model - and is stated in the code rather than papered over. Both DB clobber paths are closed: ApplyPatternWeight() declines once self-ranked, and UpdateSignalsWeights()' filter.Weight() call is guarded by SelfRanked() - guarding only the tiers would have let the hourly ranking pass undo half the self-ranking. NOT COMPILED - user compiles in MetaEditor. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-18 16:00:32 -04:00
string line = "";
for(int t = 0; t < 4; t++)
{
int fired = m_oosTierFired[t];
int hits = m_oosTierHits[t];
fix(geometry): a free zero made "never resolve" the winning geometry The CANDIDATE GEOMETRY line shipped in 05f1a53 said per-candidate geometry beats the global pair on every SP500 member at 2-3 sigma. It does not. It said so because a bar that reached neither barrier scored 0 R, and the incumbent's mean is NEGATIVE (-0.07 to -0.21 R). Against a losing baseline a free zero is a win, so the widest candidate always came out ahead - and the reported gain ordered itself by timeout share, not by skill: PAI 95.1% timed out -> +0.189 R (head measured -2.42 sigma, HARMFUL) HYB 73.8% -> +0.182 R (head at chance, +0.68 sigma) CONV 61.8% -> +0.163 R (head measured -2.47 sigma, HARMFUL) LSTM 27.1% -> +0.158 R (head +1.67 sigma) Monotone in the timeout share and inverted against the sigma gate. The acceptance test written when this was built - "the sigma gate predicts LSTM helps and CONV hurts; if the R difference does not reproduce that ordering, something is wrong" - is what caught it. A trade that reaches neither barrier is not worth zero. It is closed at the horizon, which is what the scheduled close-all does live and what SimulateTradeOutcome's timeout path already charges. So mark it there: TripleBarrierLabel now publishes the signed close-to-close travel at the last bar it actually visited (m_termTravelCache, same validity flag as the excursion and ladder caches), and LadderOutcomeR prices a timeout off it instead of returning false. A bar that cannot be evaluated under BOTH pairs is now dropped whole - scoring one leg and defaulting the other is the same bug in a smaller costume. Second defect, same function: CandidateGeometryFor applied neither of the floors the global derivation applies, so on USDJPY it chose stop 2.00 / target 1.00 - a 67% break-even, forbidden by the 1:2 policy floor. c3daded in miniature: a selector optimising its own criterion with no reference to the decision criterion. Both floors now apply, and the ratio is re-checked AFTER the per-leg rung snap, which can lose it. Also: the module weight was an unshrunk pooled win rate. USDJPY ConvLSTM fired 19 times (2.0 effective), won 36.8%, and took module weight 0.37 - 41% of the ensemble's capable weight and the loudest voice on the chart, off two effective observations. It also lifted the computed vote ceiling to 26.3 against a 25 threshold, which is why THRESHOLD UNREACHABLE never printed on a chart whose peak vote is 14 and whose practical ceiling without that member is 18.8. The pooled rate is now shrunk toward the coin-flip rate on the era's own OOS bars over 30 prior-equivalent calls, and the tiers shrink toward the shrunk value rather than the raw one. A member with ~300 effective calls moves by ~0.4pp; the 19-fire member goes 0.37 -> ~0.15. MEASUREMENT ONLY still - no order reads any of this. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-22 11:30:54 -04:00
double w = trustPct;
feat(rank): AI models rank their own confidence tiers from held-out outcomes Closes the caveat 4858507 shipped with: the vote is a confidence percentage, but only to the extent the pattern weights are measured. AI tier weights sat at their designed defaults (25/50/75/100) because AI rows only ever arrive from LIVE journaling, of which a training run produces almost none. AND A STALE-TIER BUG THAT MADE THE EVIDENCE MEANINGLESS. The OOS scan bucketed every scanned bar by ConfidenceTier(), which reads dPrevSignal - and dPrevSignal is assigned in PASS 1 only, never anywhere in the OOS scan. So an entire era's fires were bucketed by one stale, unrelated bar's confidence and landed in a SINGLE tier. That is the "tier prec T0:72%(828) T1:n/a(0) T2:n/a(0) T3:n/a(0)" symptom recorded on 2026-08-16 and attributed to the calibration clamp. The clamp was real and was fixed then; this is a second, independent cause of the identical output that survived that fix untouched - which is why the log kept reading the same afterwards. Two causes, one symptom. Now ConfidenceTierFor(adjSig): the bar this iteration actually scored. WHY THIS DOES NOT WRITE ROWS TO THE SIGNAL DB, which was the obvious reading of "fill the database during training". The user's own observation is the reason: a classic Pattern_2 is a fixed geometric condition, so its win rate is legitimately accumulated over years, but an AI Pattern_2 means "confidence landed in tier 2" and tier 2 under era 100's weights is a different statement from tier 2 under era 500's. The DB's value is ACCUMULATION, and accumulation is exactly what is wrong here - it would average together models that no longer exist, while colliding with the per-table row cap and mixing measured-on-holdout outcomes into the live ledger's own tables. What the DB actually supplies is a measured win rate per pattern, and pass 3 already computes that on held-out bars, thousands at a time. So the model ranks itself once per era, REPLACING rather than accumulating, which makes the weights describe the current weights by construction. ESTIMATOR. Not WinRateFromCounts(): it returns NO_DATA below 100 raw trades BEFORE shrinking, which here would fire on every tier every era and hand all four the pooled rate - the tiers could never separate and the mechanism would be inert. Shrinkage is the answer to a small sample; a floor in front of it means the shrinkage never runs. Instead: a Beta prior of TIER_PRIOR_EFF_N pseudo-observations centred on the model's pooled holdout rate, counted in EFFECTIVE observations, because overlapping triple-barrier labels mean 800 raw fires can be worth ~12 independent ones. Rounded to the integer, not to the decade NormalizeWinRate() uses, which would collapse the shrunk tiers back into one number. NO SAME-ERA CIRCULARITY, and it falls out of the ordering rather than a guard: weights are computed at the END of era N, so the vote scored during era N was cast with era N-1's weights. The deploy gate never grades a vote whose weights were fitted on the bars it is scoring. Residual leakage remains - the same OOS bars each era under a different model - and is stated in the code rather than papered over. Both DB clobber paths are closed: ApplyPatternWeight() declines once self-ranked, and UpdateSignalsWeights()' filter.Weight() call is guarded by SelfRanked() - guarding only the tiers would have let the hourly ranking pass undo half the self-ranking. NOT COMPILED - user compiles in MetaEditor. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-18 16:00:32 -04:00
if(fired > 0)
{
double effN = MathMax(0.0, EffectiveSampleSize((double)fired));
double effHits = effN * ((double)hits / fired);
//--- Same estimator the classic ladders use (ShrunkRatePct), with the prior deliberately
//--- far smaller: it is counted in the same EFFECTIVE units as the evidence, and a tier
//--- holds ~8-15 of those, so the classic path's 100 would drown every tier in the pool.
fix(geometry): a free zero made "never resolve" the winning geometry The CANDIDATE GEOMETRY line shipped in 05f1a53 said per-candidate geometry beats the global pair on every SP500 member at 2-3 sigma. It does not. It said so because a bar that reached neither barrier scored 0 R, and the incumbent's mean is NEGATIVE (-0.07 to -0.21 R). Against a losing baseline a free zero is a win, so the widest candidate always came out ahead - and the reported gain ordered itself by timeout share, not by skill: PAI 95.1% timed out -> +0.189 R (head measured -2.42 sigma, HARMFUL) HYB 73.8% -> +0.182 R (head at chance, +0.68 sigma) CONV 61.8% -> +0.163 R (head measured -2.47 sigma, HARMFUL) LSTM 27.1% -> +0.158 R (head +1.67 sigma) Monotone in the timeout share and inverted against the sigma gate. The acceptance test written when this was built - "the sigma gate predicts LSTM helps and CONV hurts; if the R difference does not reproduce that ordering, something is wrong" - is what caught it. A trade that reaches neither barrier is not worth zero. It is closed at the horizon, which is what the scheduled close-all does live and what SimulateTradeOutcome's timeout path already charges. So mark it there: TripleBarrierLabel now publishes the signed close-to-close travel at the last bar it actually visited (m_termTravelCache, same validity flag as the excursion and ladder caches), and LadderOutcomeR prices a timeout off it instead of returning false. A bar that cannot be evaluated under BOTH pairs is now dropped whole - scoring one leg and defaulting the other is the same bug in a smaller costume. Second defect, same function: CandidateGeometryFor applied neither of the floors the global derivation applies, so on USDJPY it chose stop 2.00 / target 1.00 - a 67% break-even, forbidden by the 1:2 policy floor. c3daded in miniature: a selector optimising its own criterion with no reference to the decision criterion. Both floors now apply, and the ratio is re-checked AFTER the per-leg rung snap, which can lose it. Also: the module weight was an unshrunk pooled win rate. USDJPY ConvLSTM fired 19 times (2.0 effective), won 36.8%, and took module weight 0.37 - 41% of the ensemble's capable weight and the loudest voice on the chart, off two effective observations. It also lifted the computed vote ceiling to 26.3 against a 25 threshold, which is why THRESHOLD UNREACHABLE never printed on a chart whose peak vote is 14 and whose practical ceiling without that member is 18.8. The pooled rate is now shrunk toward the coin-flip rate on the era's own OOS bars over 30 prior-equivalent calls, and the tiers shrink toward the shrunk value rather than the raw one. A member with ~300 effective calls moves by ~0.4pp; the 19-fire member goes 0.37 -> ~0.15. MEASUREMENT ONLY still - no order reads any of this. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-22 11:30:54 -04:00
//--- Toward the SHRUNK pooled rate, not the raw one: a member whose pooled evidence is
//--- thin must not hand its tiers a confident prior it does not have itself.
w = ShrunkRatePct(effHits, effN, trustPct, TIER_PRIOR_EFF_N);
feat(rank): AI models rank their own confidence tiers from held-out outcomes Closes the caveat 4858507 shipped with: the vote is a confidence percentage, but only to the extent the pattern weights are measured. AI tier weights sat at their designed defaults (25/50/75/100) because AI rows only ever arrive from LIVE journaling, of which a training run produces almost none. AND A STALE-TIER BUG THAT MADE THE EVIDENCE MEANINGLESS. The OOS scan bucketed every scanned bar by ConfidenceTier(), which reads dPrevSignal - and dPrevSignal is assigned in PASS 1 only, never anywhere in the OOS scan. So an entire era's fires were bucketed by one stale, unrelated bar's confidence and landed in a SINGLE tier. That is the "tier prec T0:72%(828) T1:n/a(0) T2:n/a(0) T3:n/a(0)" symptom recorded on 2026-08-16 and attributed to the calibration clamp. The clamp was real and was fixed then; this is a second, independent cause of the identical output that survived that fix untouched - which is why the log kept reading the same afterwards. Two causes, one symptom. Now ConfidenceTierFor(adjSig): the bar this iteration actually scored. WHY THIS DOES NOT WRITE ROWS TO THE SIGNAL DB, which was the obvious reading of "fill the database during training". The user's own observation is the reason: a classic Pattern_2 is a fixed geometric condition, so its win rate is legitimately accumulated over years, but an AI Pattern_2 means "confidence landed in tier 2" and tier 2 under era 100's weights is a different statement from tier 2 under era 500's. The DB's value is ACCUMULATION, and accumulation is exactly what is wrong here - it would average together models that no longer exist, while colliding with the per-table row cap and mixing measured-on-holdout outcomes into the live ledger's own tables. What the DB actually supplies is a measured win rate per pattern, and pass 3 already computes that on held-out bars, thousands at a time. So the model ranks itself once per era, REPLACING rather than accumulating, which makes the weights describe the current weights by construction. ESTIMATOR. Not WinRateFromCounts(): it returns NO_DATA below 100 raw trades BEFORE shrinking, which here would fire on every tier every era and hand all four the pooled rate - the tiers could never separate and the mechanism would be inert. Shrinkage is the answer to a small sample; a floor in front of it means the shrinkage never runs. Instead: a Beta prior of TIER_PRIOR_EFF_N pseudo-observations centred on the model's pooled holdout rate, counted in EFFECTIVE observations, because overlapping triple-barrier labels mean 800 raw fires can be worth ~12 independent ones. Rounded to the integer, not to the decade NormalizeWinRate() uses, which would collapse the shrunk tiers back into one number. NO SAME-ERA CIRCULARITY, and it falls out of the ordering rather than a guard: weights are computed at the END of era N, so the vote scored during era N was cast with era N-1's weights. The deploy gate never grades a vote whose weights were fitted on the bars it is scoring. Residual leakage remains - the same OOS bars each era under a different model - and is stated in the code rather than papered over. Both DB clobber paths are closed: ApplyPatternWeight() declines once self-ranked, and UpdateSignalsWeights()' filter.Weight() call is guarded by SelfRanked() - guarding only the tiers would have let the hourly ranking pass undo half the self-ranking. NOT COMPILED - user compiles in MetaEditor. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-18 16:00:32 -04:00
}
//--- Rounded to the nearest INTEGER, not to the nearest 10 as NormalizeWinRate() does. After
//--- shrinkage the tiers legitimately sit within a few points of each other, and decade rounding
//--- would collapse them back into one number - undoing the separation this exists to produce.
int wi = (int)MathMax(0, MathMin(100, MathRound(w)));
ApplyTierWeight(t, wi);
line += StringFormat(" T%d=%d(%d fires, %.1f eff)", t, wi, fired,
(fired > 0 ? EffectiveSampleSize((double)fired) : 0.0));
}
//--- MODULE WEIGHT = how much this model's opinion COUNTS in the weighted mean, which since the
//--- 2026-08-18 currency change is a trust weight and no longer a discount on the estimate. The
//--- pooled holdout win rate is the honest measure of that trust.
fix(geometry): a free zero made "never resolve" the winning geometry The CANDIDATE GEOMETRY line shipped in 05f1a53 said per-candidate geometry beats the global pair on every SP500 member at 2-3 sigma. It does not. It said so because a bar that reached neither barrier scored 0 R, and the incumbent's mean is NEGATIVE (-0.07 to -0.21 R). Against a losing baseline a free zero is a win, so the widest candidate always came out ahead - and the reported gain ordered itself by timeout share, not by skill: PAI 95.1% timed out -> +0.189 R (head measured -2.42 sigma, HARMFUL) HYB 73.8% -> +0.182 R (head at chance, +0.68 sigma) CONV 61.8% -> +0.163 R (head measured -2.47 sigma, HARMFUL) LSTM 27.1% -> +0.158 R (head +1.67 sigma) Monotone in the timeout share and inverted against the sigma gate. The acceptance test written when this was built - "the sigma gate predicts LSTM helps and CONV hurts; if the R difference does not reproduce that ordering, something is wrong" - is what caught it. A trade that reaches neither barrier is not worth zero. It is closed at the horizon, which is what the scheduled close-all does live and what SimulateTradeOutcome's timeout path already charges. So mark it there: TripleBarrierLabel now publishes the signed close-to-close travel at the last bar it actually visited (m_termTravelCache, same validity flag as the excursion and ladder caches), and LadderOutcomeR prices a timeout off it instead of returning false. A bar that cannot be evaluated under BOTH pairs is now dropped whole - scoring one leg and defaulting the other is the same bug in a smaller costume. Second defect, same function: CandidateGeometryFor applied neither of the floors the global derivation applies, so on USDJPY it chose stop 2.00 / target 1.00 - a 67% break-even, forbidden by the 1:2 policy floor. c3daded in miniature: a selector optimising its own criterion with no reference to the decision criterion. Both floors now apply, and the ratio is re-checked AFTER the per-leg rung snap, which can lose it. Also: the module weight was an unshrunk pooled win rate. USDJPY ConvLSTM fired 19 times (2.0 effective), won 36.8%, and took module weight 0.37 - 41% of the ensemble's capable weight and the loudest voice on the chart, off two effective observations. It also lifted the computed vote ceiling to 26.3 against a 25 threshold, which is why THRESHOLD UNREACHABLE never printed on a chart whose peak vote is 14 and whose practical ceiling without that member is 18.8. The pooled rate is now shrunk toward the coin-flip rate on the era's own OOS bars over 30 prior-equivalent calls, and the tiers shrink toward the shrunk value rather than the raw one. A member with ~300 effective calls moves by ~0.4pp; the 19-fire member goes 0.37 -> ~0.15. MEASUREMENT ONLY still - no order reads any of this. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-22 11:30:54 -04:00
Weight(MathMax(0.0, MathMin(1.0, trustPct / 100.0)));
feat(rank): AI models rank their own confidence tiers from held-out outcomes Closes the caveat 4858507 shipped with: the vote is a confidence percentage, but only to the extent the pattern weights are measured. AI tier weights sat at their designed defaults (25/50/75/100) because AI rows only ever arrive from LIVE journaling, of which a training run produces almost none. AND A STALE-TIER BUG THAT MADE THE EVIDENCE MEANINGLESS. The OOS scan bucketed every scanned bar by ConfidenceTier(), which reads dPrevSignal - and dPrevSignal is assigned in PASS 1 only, never anywhere in the OOS scan. So an entire era's fires were bucketed by one stale, unrelated bar's confidence and landed in a SINGLE tier. That is the "tier prec T0:72%(828) T1:n/a(0) T2:n/a(0) T3:n/a(0)" symptom recorded on 2026-08-16 and attributed to the calibration clamp. The clamp was real and was fixed then; this is a second, independent cause of the identical output that survived that fix untouched - which is why the log kept reading the same afterwards. Two causes, one symptom. Now ConfidenceTierFor(adjSig): the bar this iteration actually scored. WHY THIS DOES NOT WRITE ROWS TO THE SIGNAL DB, which was the obvious reading of "fill the database during training". The user's own observation is the reason: a classic Pattern_2 is a fixed geometric condition, so its win rate is legitimately accumulated over years, but an AI Pattern_2 means "confidence landed in tier 2" and tier 2 under era 100's weights is a different statement from tier 2 under era 500's. The DB's value is ACCUMULATION, and accumulation is exactly what is wrong here - it would average together models that no longer exist, while colliding with the per-table row cap and mixing measured-on-holdout outcomes into the live ledger's own tables. What the DB actually supplies is a measured win rate per pattern, and pass 3 already computes that on held-out bars, thousands at a time. So the model ranks itself once per era, REPLACING rather than accumulating, which makes the weights describe the current weights by construction. ESTIMATOR. Not WinRateFromCounts(): it returns NO_DATA below 100 raw trades BEFORE shrinking, which here would fire on every tier every era and hand all four the pooled rate - the tiers could never separate and the mechanism would be inert. Shrinkage is the answer to a small sample; a floor in front of it means the shrinkage never runs. Instead: a Beta prior of TIER_PRIOR_EFF_N pseudo-observations centred on the model's pooled holdout rate, counted in EFFECTIVE observations, because overlapping triple-barrier labels mean 800 raw fires can be worth ~12 independent ones. Rounded to the integer, not to the decade NormalizeWinRate() uses, which would collapse the shrunk tiers back into one number. NO SAME-ERA CIRCULARITY, and it falls out of the ordering rather than a guard: weights are computed at the END of era N, so the vote scored during era N was cast with era N-1's weights. The deploy gate never grades a vote whose weights were fitted on the bars it is scoring. Residual leakage remains - the same OOS bars each era under a different model - and is stated in the code rather than papered over. Both DB clobber paths are closed: ApplyPatternWeight() declines once self-ranked, and UpdateSignalsWeights()' filter.Weight() call is guarded by SelfRanked() - guarding only the tiers would have let the hourly ranking pass undo half the self-ranking. NOT COMPILED - user compiles in MetaEditor. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-18 16:00:32 -04:00
m_tiersSelfRanked = true;
fix(vote): persist the member's skill verdict - a converged model was ruled no-skill on every restart SP500 resumed converged at era 136 with its tier ladder correctly restored and still swept 4999 bars reporting "0 had a snapshot, drew 0 arrow(s)" while the other five charts drew 221-312. HasDemonstratedEdge() - added with the no-skill exclusion - compares m_eraStatPrecPct against m_eraStatChancePct. Both are written once per era by EnsembleStashEraStats. A converged model runs no eras, so after a restart both sat at their -1 ctor defaults, every member was ruled no-skill, ReconstructionWeight() returned 0 for all four, and the overlay divisor was zero on every bar. Exactly the failure the WST7 ladder persistence fixed one level down: the ladder says how much a member votes, this says whether it may. RankTiersFromOos already computes the pair (pooled holdout precision and the zero-skill reference rate) and now records it as the CERTIFIED edge. That path is reached by the era end AND by the deployed replay, which is the only measurement a converged model will ever make. Persisted as WST8; HasDemonstratedEdge() prefers the era pair and falls back to it. The census line also had to be fixed: it reported "NOT ONE of those bars had a single member snapshot ... no enrolled member has published m_overlaySigSnap" for a condition that was purely a skill verdict. The snapshots were there. It now counts the two causes separately and names the one that fired. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-26 16:53:16 -04:00
//--- ...AND WHETHER IT MAY VOTE AT ALL, from the same two numbers. The ladder says how much a
//--- member's opinion counts; this says whether it is admitted to the divisor (HasDemonstratedEdge
//--- -> VoteCapableWeight/ReconstructionWeight). Recorded here rather than at the era end because
//--- the DEPLOYED REPLAY path arrives here too, and that path is the only measurement a converged
//--- model will ever make.
m_certifiedPrecPct = pooledPct;
m_certifiedChancePct = chancePct;
feat(logs): throttle the settled per-era diagnostics - measured 22MB/9.5h of confirmed-working systems Measured from the journal (2026-08-19): the era deep-dive line (~2KB) plus the excursion verdict, tier re-rank, calibration move, barrier hold and selection-regressed note each printed EVERY era for EVERY member - ~940 eras/member/day - long after the systems they watch were confirmed working. Yesterday's file was 1.3GB (70% of it the news-filter calendar spam the sweep fix already removed). VerboseMode returns as an INPUT (demoted 2026-08-01 for the marketplace; that track is dead since the 2026-08-16 pivot) and gains a second job: false throttles each settled per-era print to eras 0-3 plus every TRAIN_LOG_EVERY_ERAS-th (25 ~= one deep-dive per ~15min per member); true restores the per-era firehose, flippable live. Never throttled: anything that marks a CHANGE - new bests, restores + eta decays, plateau stage transitions, deploy approvals, warnings, errors, the label-cache/adoption one-shots, and the combined-vote gate line (the active system's primary telemetry, still every era). Semantic fixes over blanket gating: - barrier hold now ARMS silently and prints only when the hold outlasts the 2-min report interval - a brief hold every era is the design, the long hold is the watchdog case the line exists for; - the ensemble deploy REFUSAL prints immediately when its reason changes (that is a finding), on cadence when unchanged; - the filtered-view census prints when its RESULT moves (drawn count, or strongest vote by >=2pp) and at least every 10th sweep. NOT COMPILED - user compiles in MetaEditor. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-19 09:36:15 -04:00
//--- THROTTLED (2026-08-19): the re-rank happens (and must happen) every era, but saying so
//--- every era was ~950 near-identical lines/member/day once the system was confirmed working.
//--- The weights it prints are visible live on the member HUD lines anyway.
if(TrainLogDue())
fix(geometry): a free zero made "never resolve" the winning geometry The CANDIDATE GEOMETRY line shipped in 05f1a53 said per-candidate geometry beats the global pair on every SP500 member at 2-3 sigma. It does not. It said so because a bar that reached neither barrier scored 0 R, and the incumbent's mean is NEGATIVE (-0.07 to -0.21 R). Against a losing baseline a free zero is a win, so the widest candidate always came out ahead - and the reported gain ordered itself by timeout share, not by skill: PAI 95.1% timed out -> +0.189 R (head measured -2.42 sigma, HARMFUL) HYB 73.8% -> +0.182 R (head at chance, +0.68 sigma) CONV 61.8% -> +0.163 R (head measured -2.47 sigma, HARMFUL) LSTM 27.1% -> +0.158 R (head +1.67 sigma) Monotone in the timeout share and inverted against the sigma gate. The acceptance test written when this was built - "the sigma gate predicts LSTM helps and CONV hurts; if the R difference does not reproduce that ordering, something is wrong" - is what caught it. A trade that reaches neither barrier is not worth zero. It is closed at the horizon, which is what the scheduled close-all does live and what SimulateTradeOutcome's timeout path already charges. So mark it there: TripleBarrierLabel now publishes the signed close-to-close travel at the last bar it actually visited (m_termTravelCache, same validity flag as the excursion and ladder caches), and LadderOutcomeR prices a timeout off it instead of returning false. A bar that cannot be evaluated under BOTH pairs is now dropped whole - scoring one leg and defaulting the other is the same bug in a smaller costume. Second defect, same function: CandidateGeometryFor applied neither of the floors the global derivation applies, so on USDJPY it chose stop 2.00 / target 1.00 - a 67% break-even, forbidden by the 1:2 policy floor. c3daded in miniature: a selector optimising its own criterion with no reference to the decision criterion. Both floors now apply, and the ratio is re-checked AFTER the per-leg rung snap, which can lose it. Also: the module weight was an unshrunk pooled win rate. USDJPY ConvLSTM fired 19 times (2.0 effective), won 36.8%, and took module weight 0.37 - 41% of the ensemble's capable weight and the loudest voice on the chart, off two effective observations. It also lifted the computed vote ceiling to 26.3 against a 25 threshold, which is why THRESHOLD UNREACHABLE never printed on a chart whose peak vote is 14 and whose practical ceiling without that member is 18.8. The pooled rate is now shrunk toward the coin-flip rate on the era's own OOS bars over 30 prior-equivalent calls, and the tiers shrink toward the shrunk value rather than the raw one. A member with ~300 effective calls moves by ~0.4pp; the 19-fire member goes 0.37 -> ~0.15. MEASUREMENT ONLY still - no order reads any of this. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-22 11:30:54 -04:00
Print(ID + StringFormat(": tier weights re-ranked from %d held-out fires (%.1f effective,"
" pooled %.1f%% raw -> %.1f%% shrunk toward the %.1f%% coin-flip rate on"
" %.0f prior-equivalent calls) ->%s | module weight %.2f. These are the"
" weights the NEXT era votes with.",
pooledFired, pooledEffN, pooledPct, trustPct, chancePct,
MODULE_PRIOR_EFF_N, line, ModuleWeight()));
feat(rank): AI models rank their own confidence tiers from held-out outcomes Closes the caveat 4858507 shipped with: the vote is a confidence percentage, but only to the extent the pattern weights are measured. AI tier weights sat at their designed defaults (25/50/75/100) because AI rows only ever arrive from LIVE journaling, of which a training run produces almost none. AND A STALE-TIER BUG THAT MADE THE EVIDENCE MEANINGLESS. The OOS scan bucketed every scanned bar by ConfidenceTier(), which reads dPrevSignal - and dPrevSignal is assigned in PASS 1 only, never anywhere in the OOS scan. So an entire era's fires were bucketed by one stale, unrelated bar's confidence and landed in a SINGLE tier. That is the "tier prec T0:72%(828) T1:n/a(0) T2:n/a(0) T3:n/a(0)" symptom recorded on 2026-08-16 and attributed to the calibration clamp. The clamp was real and was fixed then; this is a second, independent cause of the identical output that survived that fix untouched - which is why the log kept reading the same afterwards. Two causes, one symptom. Now ConfidenceTierFor(adjSig): the bar this iteration actually scored. WHY THIS DOES NOT WRITE ROWS TO THE SIGNAL DB, which was the obvious reading of "fill the database during training". The user's own observation is the reason: a classic Pattern_2 is a fixed geometric condition, so its win rate is legitimately accumulated over years, but an AI Pattern_2 means "confidence landed in tier 2" and tier 2 under era 100's weights is a different statement from tier 2 under era 500's. The DB's value is ACCUMULATION, and accumulation is exactly what is wrong here - it would average together models that no longer exist, while colliding with the per-table row cap and mixing measured-on-holdout outcomes into the live ledger's own tables. What the DB actually supplies is a measured win rate per pattern, and pass 3 already computes that on held-out bars, thousands at a time. So the model ranks itself once per era, REPLACING rather than accumulating, which makes the weights describe the current weights by construction. ESTIMATOR. Not WinRateFromCounts(): it returns NO_DATA below 100 raw trades BEFORE shrinking, which here would fire on every tier every era and hand all four the pooled rate - the tiers could never separate and the mechanism would be inert. Shrinkage is the answer to a small sample; a floor in front of it means the shrinkage never runs. Instead: a Beta prior of TIER_PRIOR_EFF_N pseudo-observations centred on the model's pooled holdout rate, counted in EFFECTIVE observations, because overlapping triple-barrier labels mean 800 raw fires can be worth ~12 independent ones. Rounded to the integer, not to the decade NormalizeWinRate() uses, which would collapse the shrunk tiers back into one number. NO SAME-ERA CIRCULARITY, and it falls out of the ordering rather than a guard: weights are computed at the END of era N, so the vote scored during era N was cast with era N-1's weights. The deploy gate never grades a vote whose weights were fitted on the bars it is scoring. Residual leakage remains - the same OOS bars each era under a different model - and is stated in the code rather than papered over. Both DB clobber paths are closed: ApplyPatternWeight() declines once self-ranked, and UpdateSignalsWeights()' filter.Weight() call is guarded by SelfRanked() - guarding only the tiers would have let the hourly ranking pass undo half the self-ranking. NOT COMPILED - user compiles in MetaEditor. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-18 16:00:32 -04:00
}
//+------------------------------------------------------------------+
//| Set the specified pattern's weight to the specified value |
//+------------------------------------------------------------------+
void CExpertSignalAIBase::ApplyPatternWeight(int patternNumber, int weight)
{
//--- THE SIGNAL DB DOES NOT OUTRANK THE HOLDOUT. UpdateSignalsWeights() calls this hourly for
//--- every filter it can find rows for, and for an AI filter those rows are LIVE-journaled fires
//--- accumulated across eras - i.e. across models. Without this the ranking pass would silently
//--- undo every era's self-ranking within the hour.
feat(rank): AI models rank their own confidence tiers from held-out outcomes Closes the caveat 4858507 shipped with: the vote is a confidence percentage, but only to the extent the pattern weights are measured. AI tier weights sat at their designed defaults (25/50/75/100) because AI rows only ever arrive from LIVE journaling, of which a training run produces almost none. AND A STALE-TIER BUG THAT MADE THE EVIDENCE MEANINGLESS. The OOS scan bucketed every scanned bar by ConfidenceTier(), which reads dPrevSignal - and dPrevSignal is assigned in PASS 1 only, never anywhere in the OOS scan. So an entire era's fires were bucketed by one stale, unrelated bar's confidence and landed in a SINGLE tier. That is the "tier prec T0:72%(828) T1:n/a(0) T2:n/a(0) T3:n/a(0)" symptom recorded on 2026-08-16 and attributed to the calibration clamp. The clamp was real and was fixed then; this is a second, independent cause of the identical output that survived that fix untouched - which is why the log kept reading the same afterwards. Two causes, one symptom. Now ConfidenceTierFor(adjSig): the bar this iteration actually scored. WHY THIS DOES NOT WRITE ROWS TO THE SIGNAL DB, which was the obvious reading of "fill the database during training". The user's own observation is the reason: a classic Pattern_2 is a fixed geometric condition, so its win rate is legitimately accumulated over years, but an AI Pattern_2 means "confidence landed in tier 2" and tier 2 under era 100's weights is a different statement from tier 2 under era 500's. The DB's value is ACCUMULATION, and accumulation is exactly what is wrong here - it would average together models that no longer exist, while colliding with the per-table row cap and mixing measured-on-holdout outcomes into the live ledger's own tables. What the DB actually supplies is a measured win rate per pattern, and pass 3 already computes that on held-out bars, thousands at a time. So the model ranks itself once per era, REPLACING rather than accumulating, which makes the weights describe the current weights by construction. ESTIMATOR. Not WinRateFromCounts(): it returns NO_DATA below 100 raw trades BEFORE shrinking, which here would fire on every tier every era and hand all four the pooled rate - the tiers could never separate and the mechanism would be inert. Shrinkage is the answer to a small sample; a floor in front of it means the shrinkage never runs. Instead: a Beta prior of TIER_PRIOR_EFF_N pseudo-observations centred on the model's pooled holdout rate, counted in EFFECTIVE observations, because overlapping triple-barrier labels mean 800 raw fires can be worth ~12 independent ones. Rounded to the integer, not to the decade NormalizeWinRate() uses, which would collapse the shrunk tiers back into one number. NO SAME-ERA CIRCULARITY, and it falls out of the ordering rather than a guard: weights are computed at the END of era N, so the vote scored during era N was cast with era N-1's weights. The deploy gate never grades a vote whose weights were fitted on the bars it is scoring. Residual leakage remains - the same OOS bars each era under a different model - and is stated in the code rather than papered over. Both DB clobber paths are closed: ApplyPatternWeight() declines once self-ranked, and UpdateSignalsWeights()' filter.Weight() call is guarded by SelfRanked() - guarding only the tiers would have let the hourly ranking pass undo half the self-ranking. NOT COMPILED - user compiles in MetaEditor. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-18 16:00:32 -04:00
if(m_tiersSelfRanked)
return;
switch(patternNumber)
{
case 0:
Pattern_0(weight);
break;
case 1:
Pattern_1(weight);
break;
case 2:
Pattern_2(weight);
break;
case 3:
Pattern_3(weight);
break;
default:
break;
}
}
//+------------------------------------------------------------------+
//| OnTick function |
//+------------------------------------------------------------------+
void CExpertSignalAIBase::OnTickHandler(void)
{
ScheduleTrainingIfNeeded();
}
//+------------------------------------------------------------------+
//| Schedules the next training pass (if one is due) and refreshes |
//| the per-tick status label. Factored out of OnTickHandler() so |
//| Warrior_EA.mq5's always-on timer (see PollTraining()) can drive |
//| this on a fixed wall-clock schedule too - training must not stall |
//| just because the market is closed and no ticks are arriving. |
//+------------------------------------------------------------------+
void CExpertSignalAIBase::ScheduleTrainingIfNeeded(void)
{
//--- stopped: no new training passes get scheduled at all (StartTraining() re-arms this).
//--- paused: still schedule so bEventStudy/dtStudied bookkeeping stays current, but Train() itself
//--- blocks at the next era boundary until resumed - keeps in-memory state coherent either way.
//--- complete: training already converged - a plain new bar must NOT re-enter Train()'s full era
//--- loop, which would otherwise reset the best-checkpoint/g_eta-decay tracking and run real
//--- Net.backProp() passes again, forever, once per bar, on an already-converged model (see
//--- RefreshConvergedSignal()'s declaration comment). Just keep the live signal current instead.
refactor(trade-mgmt): remove all confidence-scaled trade management Five modes went, all of them staking real risk on the model's confidence: Intelligent entry (ENTRY_INTELLIGENT), stop (SL_INTELLIGENT), target (TP_INTELLIGENT), trailing (CTrailingIntelligent) and lot size (CMoneyIntelligent's quarter-Kelly). With them, the Confidence_Source input and the CONFIDENCE_SOURCE enum, whose only job was choosing which number those five read. The reason is calibration, not correctness: the confidence magnitude is known to be miscalibrated against the label prior, so every one of these modes multiplied money by a quantity whose units were never established. The DB arm had a second, independent defect - since the tester DB guard (SignalDatabaseActive) it reads 0 in tester and optimizer but non-zero live, so any backtest of CONF_DB/CONF_BLENDED could not reproduce live trading. And what the DB produces is a filter-RANKING win rate, not a per-trade win probability. Both confidence numbers are still recorded per trade (aiConfidence / dbConfidence) and still bucketed against outcome in TradeJournalReport. Recording is what keeps the question answerable; acting on it was the part with no evidence behind it. ConfidenceBridge.mqh now carries an explicit telemetry-only rule at the top. ENUM ORDINALS PINNED. Removing a member vacated a value in four enums at once and MT5 does not validate an enum input replayed from a saved .set or a stored optimization pass. TRAILING_STRATEGY and MONEY_MANAGEMENT_STRATEGY now carry explicit values so the survivors keep the numbers they were saved as, and ValidateBarrierInputs is widened into ValidateTradeManagementInputs covering SL_Mode, TP_Mode, Entry_Multiplier, TrailingStrategy and MM_STRATEGY. Without that gate a chart saved with the Intelligent stop would feed SL_Mode = -1 into a multiplier now used verbatim, placing the stop on the wrong side of entry. RETRAIN-NEUTRAL: neither SL_Mode nor TP_Mode appears in BuildModelFingerprint() or ComputeDbConfigFingerprint() since the swing-pivot target replaced the barrier labels. No .nnw, .cfg or .db re-keys. Also drops the now-dead g_TradeRewardRiskRatio bridge, the CMoneyRiskBase::AdjustRiskAmount hook and the unsigned AIConfidence(). Compile-verified in _claude_stage: 0 errors, 0 warnings. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-25 10:10:20 -04:00
//--- publish this signal's current signed confidence - see g_LiveAISignedConfidence in
//--- Variables\ConfidenceBridge.mqh. Cheap: SignedAIConfidence() just reads the already-computed
//--- dPrevSignal.
//--- TELEMETRY SINCE 2026-08-25. The last consumer that could act on this - the confidence-adaptive
//--- trailing stop - was removed with the rest of the confidence-scaled trade management, so what
//--- this feeds now is the per-trade journal columns and the confidence-vs-outcome buckets in
//--- Database\TradeJournalReport.mqh. It is kept running rather than deleted precisely because
//--- those buckets are the only way the question "is this number worth anything?" ever gets an
//--- answer, and a trade cannot be scored against a conviction nobody recorded.
feat(gates): derive the recall floor instead of configuring it, publish what the configuration can PROVE, and stop an arbitrary member driving live exits Three changes, all from the same principle: measure what is there before aiming at it, and never certify a number you do not trade. 1. THE RECALL FLOOR IS DERIVED, AND IT MOVES BELOW CHANCE. MinRecall=40 was a constant doing a statistical job. Its reference point is the 33.3% recall a zero-skill 3-class model gets on EVERY class, and against that the constant was accidentally calibrated for exactly one sample size: on USDJPY CONV (n_eff 195) 40% is chance + 2.0 SE; on SP500 PAI (n_eff 42) the same 40% is chance + 0.9 SE. One chart was being held to a bar twice as strict as the other, for no reason anyone chose. CollapseRecallFloorPct() computes it per class from that class's own effective sample - EffectiveSampleSize(), so the overlap deflation the rest of the gates use applies here too - as chance - EDGE_MIN_SIGMAS x SE. 26.5% at n_eff 195, 18.8% at n_eff 42. BELOW chance, deliberately, and this is the substantive change rather than the arithmetic. This gate's only job is refusing to call a COLLAPSED model converged. It is not a quality bar; the deploy gate is the quality bar and it is already rigorous (chance + 2 SE on the deflated sample, Sidak over candidate eras, then the cross-instrument pooled certificate). A convergence gate that ALSO demands provably-above-chance recall on all three classes double-counts that job, and it has failed that way twice here: MinRecall=60 blocked every SP500 H1 run in 2026-07, and the 40 that replaced it made Neutral structurally unreachable once first-touch resolution cut Neutral to a 0.65% residue. A floor nothing can reach does not make a funded account safer, it stops the run converging at all. Testing significantly BELOW chance instead catches what a fixed 40 was actually catching - a model that has stopped emitting a class - and cannot become unreachable by construction. It also fixes the direction the old constant scaled: it now widens on a thin OOS window, where low recall genuinely cannot be told from noise, and tightens on a rich one. Today's SP500 PAI (Buy 51 / Sell 18 / Neutral 30) is still correctly blocked on Sell. This also resolves a standing contradiction the code half-admitted at the isBetterEra comment: selection ranks on coverage-weighted PRECISION while convergence gated on RECALL, so a sparse high-precision abstainer - precisely the model that could clear the deploy bar - was blocked by the floor. The era line now PRINTS the derived floor. Anyone comparing these recalls against a remembered "40" is reading the wrong bar. 2. DETECTABILITY: WHAT THIS CONFIGURATION COULD PROVE, BEFORE IT TRAINS. The DEPLOY BAR line states the bar. It never said what reaching it would take, and that is the actionable direction. ReportDetectability() inverts the same identity - the gate passes when edge >= z x sqrt(p(1-p)/n_eff), so certifying an edge d needs n_eff >= z^2 p(1-p)/d^2 independent calls, hence L times as many raw ones - and prints a +2 / +5 / +10pp ladder as required independent calls, raw calls, and share of the OOS window, marking any rung that needs more than the window holds IMPOSSIBLE. Every term is a property of the CONFIGURATION - geometry via break-even, horizon via mean label lifespan, window via oosCutoff - so no amount of training moves any of them. It fires once, at the first healthy sweep, beside ReportFeatureHealth, for the same reason: that is the first moment the bar grid, the measured geometry and the lifespan are real numbers rather than defaults. It gates nothing. This is the EdgeFinder discipline applied to our own gate: establish what the market and the measurement design have to offer, then point the net at it - rather than spending a thousand eras chasing something this OOS window could never certify. 3. AN ARBITRARY MEMBER WAS DRIVING LIVE EXITS AND TRAILING (user-identified). Every ensemble member ran g_LiveAISignedConfidence = SignedAIConfidence(); unconditionally, every tick. Last writer wins. Its consumers are the AI early-exit route (CExpertSignalCustom::LiveSignedConfidence) and TrailingIntelligent - so on a four-model chart an LSTM entry could be closed, and its stop moved, on the Perceptron's opinion alone, decided by scheduling order. Not the vote, not a weighted blend. Now the mean across registered members, matching how the ensemble actually trades: the open decision is the weighted-average vote, and an abstaining member contributes 0 and dilutes exactly as it does there. Members still training read 0, so a half-trained ensemble reads WEAKER rather than louder - the safe direction for an exit trigger. Deployed and paused members are included, which is the opposite of the era barrier's exemption rule and correct for the opposite reason: that one asks who must be waited for, this asks who has an opinion. Latent today and staying that way for now by choice - Min_Vote_Close ships Disabled (101, unreachable on both scales it drives) and TrailingStrategy is off, so live exits are SL/TP only and the certified hold-to-barrier win rate is what actually gets traded. Fixed now precisely because the plan is to enable vote exits once the models are accurate, at which point a scheduling-order exit would be both harmful and very hard to see. STILL OPEN, and needs a decision before vote exits go on: the member gate and the ensemble vote gate both grade hold-to-barrier, so enabling vote exits makes the certified number stop describing the traded one. Warrior_EA.mq5 currently argues barrier models may keep vote exits because "their label IS the vote's own horizon" - that does not hold, since a vote flip at bar 5 of a 64-bar horizon is not the target-before-stop outcome the gate measured. Either grade the OOS call on the real exit rule (first of SL / TP / vote-flip / horizon) through the fill engine, or set HoldToBarrier for ensemble members so the policy cannot drift from the certificate. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-17 16:15:08 -04:00
//--- ENSEMBLE: the AVERAGE across members, not this member's own reading. Every member ran this line
//--- unconditionally, every tick, so the global was simply whichever member's OnTick happened to run
refactor(kiss): drop the AI sub-vote early-exit route; certified == traded First of the AI vote layers to go. CheckClosePosition had two exit routes: the stock blended vote, and an AI-only one reading the AI members' sub-vote undiluted. The second existed because an AI reversal averaged in with the classic filters could be diluted below the threshold before it could close a position. It is gone, and with it m_lastAiVote and the aiResult/aiWeightSum pair Direction() carried to feed it. This CLOSES the certified-vs-traded gap rather than widening it. The deploy gate certifies a win rate measured on hold-to-resolution outcomes, and CheckClosePosition already gated the blended route off whenever an AI model's derived geometry was on the order - so the AI route was the only vote exit an AI-certified trade could take, and the exit replay existed to reproduce it. With it removed, an AI-certified position holds to its barrier by construction instead of by reconstruction, so Warrior_EA.mq5 now pushes ExitPolicy(0.0, true) unconditionally. Previously it forwarded Min_Vote_Close and relied on Disabled arriving as 1.01 to switch the simulated exit off by arithmetic - correct at the shipped default, and one input change away from the simulation and the live path describing different games. Min_Vote_Close keeps its meaning for the classic route and is now documented as inert wherever an AI certificate governs, rather than appearing to drive an exit it can no longer reach. Comment debt cleared while here: a tombstone block for m_ai_exit_threshold (a member deleted 2026-08-18) still sat in the header, and four sites still named LiveSignedConfidence's "two consumers" - it had one, the intelligent trailing stop, since that same date. NOT touched, and deliberately: NMS declustering is NOT a quality layer. It gates the live signal at Inference.mqh:226 (NmsLiveAccept), and the undeclustered population is ~8x what the EA trades. Removing it would multiply live position count, not simplify a scoring path. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-19 20:27:28 -04:00
//--- last. So on a four-model chart an LSTM entry could have its stop moved on the Perceptron's
refactor(trade-mgmt): remove all confidence-scaled trade management Five modes went, all of them staking real risk on the model's confidence: Intelligent entry (ENTRY_INTELLIGENT), stop (SL_INTELLIGENT), target (TP_INTELLIGENT), trailing (CTrailingIntelligent) and lot size (CMoneyIntelligent's quarter-Kelly). With them, the Confidence_Source input and the CONFIDENCE_SOURCE enum, whose only job was choosing which number those five read. The reason is calibration, not correctness: the confidence magnitude is known to be miscalibrated against the label prior, so every one of these modes multiplied money by a quantity whose units were never established. The DB arm had a second, independent defect - since the tester DB guard (SignalDatabaseActive) it reads 0 in tester and optimizer but non-zero live, so any backtest of CONF_DB/CONF_BLENDED could not reproduce live trading. And what the DB produces is a filter-RANKING win rate, not a per-trade win probability. Both confidence numbers are still recorded per trade (aiConfidence / dbConfidence) and still bucketed against outcome in TradeJournalReport. Recording is what keeps the question answerable; acting on it was the part with no evidence behind it. ConfidenceBridge.mqh now carries an explicit telemetry-only rule at the top. ENUM ORDINALS PINNED. Removing a member vacated a value in four enums at once and MT5 does not validate an enum input replayed from a saved .set or a stored optimization pass. TRAILING_STRATEGY and MONEY_MANAGEMENT_STRATEGY now carry explicit values so the survivors keep the numbers they were saved as, and ValidateBarrierInputs is widened into ValidateTradeManagementInputs covering SL_Mode, TP_Mode, Entry_Multiplier, TrailingStrategy and MM_STRATEGY. Without that gate a chart saved with the Intelligent stop would feed SL_Mode = -1 into a multiplier now used verbatim, placing the stop on the wrong side of entry. RETRAIN-NEUTRAL: neither SL_Mode nor TP_Mode appears in BuildModelFingerprint() or ComputeDbConfigFingerprint() since the swing-pivot target replaced the barrier labels. No .nnw, .cfg or .db re-keys. Also drops the now-dead g_TradeRewardRiskRatio bridge, the CMoneyRiskBase::AdjustRiskAmount hook and the unsigned AIConfidence(). Compile-verified in _claude_stage: 0 errors, 0 warnings. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-25 10:10:20 -04:00
//--- opinion alone, purely by scheduling order. (User-identified 2026-08-17.) That bug is now
//--- unreachable twice over - the aggregate is a mean, AND nothing acts on it - but the shape stays
//--- correct so that a future consumer inherits a defined number rather than a race.
feat(gates): derive the recall floor instead of configuring it, publish what the configuration can PROVE, and stop an arbitrary member driving live exits Three changes, all from the same principle: measure what is there before aiming at it, and never certify a number you do not trade. 1. THE RECALL FLOOR IS DERIVED, AND IT MOVES BELOW CHANCE. MinRecall=40 was a constant doing a statistical job. Its reference point is the 33.3% recall a zero-skill 3-class model gets on EVERY class, and against that the constant was accidentally calibrated for exactly one sample size: on USDJPY CONV (n_eff 195) 40% is chance + 2.0 SE; on SP500 PAI (n_eff 42) the same 40% is chance + 0.9 SE. One chart was being held to a bar twice as strict as the other, for no reason anyone chose. CollapseRecallFloorPct() computes it per class from that class's own effective sample - EffectiveSampleSize(), so the overlap deflation the rest of the gates use applies here too - as chance - EDGE_MIN_SIGMAS x SE. 26.5% at n_eff 195, 18.8% at n_eff 42. BELOW chance, deliberately, and this is the substantive change rather than the arithmetic. This gate's only job is refusing to call a COLLAPSED model converged. It is not a quality bar; the deploy gate is the quality bar and it is already rigorous (chance + 2 SE on the deflated sample, Sidak over candidate eras, then the cross-instrument pooled certificate). A convergence gate that ALSO demands provably-above-chance recall on all three classes double-counts that job, and it has failed that way twice here: MinRecall=60 blocked every SP500 H1 run in 2026-07, and the 40 that replaced it made Neutral structurally unreachable once first-touch resolution cut Neutral to a 0.65% residue. A floor nothing can reach does not make a funded account safer, it stops the run converging at all. Testing significantly BELOW chance instead catches what a fixed 40 was actually catching - a model that has stopped emitting a class - and cannot become unreachable by construction. It also fixes the direction the old constant scaled: it now widens on a thin OOS window, where low recall genuinely cannot be told from noise, and tightens on a rich one. Today's SP500 PAI (Buy 51 / Sell 18 / Neutral 30) is still correctly blocked on Sell. This also resolves a standing contradiction the code half-admitted at the isBetterEra comment: selection ranks on coverage-weighted PRECISION while convergence gated on RECALL, so a sparse high-precision abstainer - precisely the model that could clear the deploy bar - was blocked by the floor. The era line now PRINTS the derived floor. Anyone comparing these recalls against a remembered "40" is reading the wrong bar. 2. DETECTABILITY: WHAT THIS CONFIGURATION COULD PROVE, BEFORE IT TRAINS. The DEPLOY BAR line states the bar. It never said what reaching it would take, and that is the actionable direction. ReportDetectability() inverts the same identity - the gate passes when edge >= z x sqrt(p(1-p)/n_eff), so certifying an edge d needs n_eff >= z^2 p(1-p)/d^2 independent calls, hence L times as many raw ones - and prints a +2 / +5 / +10pp ladder as required independent calls, raw calls, and share of the OOS window, marking any rung that needs more than the window holds IMPOSSIBLE. Every term is a property of the CONFIGURATION - geometry via break-even, horizon via mean label lifespan, window via oosCutoff - so no amount of training moves any of them. It fires once, at the first healthy sweep, beside ReportFeatureHealth, for the same reason: that is the first moment the bar grid, the measured geometry and the lifespan are real numbers rather than defaults. It gates nothing. This is the EdgeFinder discipline applied to our own gate: establish what the market and the measurement design have to offer, then point the net at it - rather than spending a thousand eras chasing something this OOS window could never certify. 3. AN ARBITRARY MEMBER WAS DRIVING LIVE EXITS AND TRAILING (user-identified). Every ensemble member ran g_LiveAISignedConfidence = SignedAIConfidence(); unconditionally, every tick. Last writer wins. Its consumers are the AI early-exit route (CExpertSignalCustom::LiveSignedConfidence) and TrailingIntelligent - so on a four-model chart an LSTM entry could be closed, and its stop moved, on the Perceptron's opinion alone, decided by scheduling order. Not the vote, not a weighted blend. Now the mean across registered members, matching how the ensemble actually trades: the open decision is the weighted-average vote, and an abstaining member contributes 0 and dilutes exactly as it does there. Members still training read 0, so a half-trained ensemble reads WEAKER rather than louder - the safe direction for an exit trigger. Deployed and paused members are included, which is the opposite of the era barrier's exemption rule and correct for the opposite reason: that one asks who must be waited for, this asks who has an opinion. Latent today and staying that way for now by choice - Min_Vote_Close ships Disabled (101, unreachable on both scales it drives) and TrailingStrategy is off, so live exits are SL/TP only and the certified hold-to-barrier win rate is what actually gets traded. Fixed now precisely because the plan is to enable vote exits once the models are accurate, at which point a scheduling-order exit would be both harmful and very hard to see. STILL OPEN, and needs a decision before vote exits go on: the member gate and the ensemble vote gate both grade hold-to-barrier, so enabling vote exits makes the certified number stop describing the traded one. Warrior_EA.mq5 currently argues barrier models may keep vote exits because "their label IS the vote's own horizon" - that does not hold, since a vote flip at bar 5 of a 64-bar horizon is not the target-before-stop outcome the gate measured. Either grade the OOS call on the real exit rule (first of SL / TP / vote-flip / horizon) through the fill engine, or set HoldToBarrier for ensemble members so the policy cannot drift from the certificate. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-17 16:15:08 -04:00
//--- Averaging matches how the ensemble actually trades: the open decision is the weighted-average
//--- vote, and a member that abstains contributes 0 and dilutes, exactly as it does there. Members
//--- still training read 0 from SignedAIConfidence(), so a half-trained ensemble reads WEAKER rather
refactor(trade-mgmt): remove all confidence-scaled trade management Five modes went, all of them staking real risk on the model's confidence: Intelligent entry (ENTRY_INTELLIGENT), stop (SL_INTELLIGENT), target (TP_INTELLIGENT), trailing (CTrailingIntelligent) and lot size (CMoneyIntelligent's quarter-Kelly). With them, the Confidence_Source input and the CONFIDENCE_SOURCE enum, whose only job was choosing which number those five read. The reason is calibration, not correctness: the confidence magnitude is known to be miscalibrated against the label prior, so every one of these modes multiplied money by a quantity whose units were never established. The DB arm had a second, independent defect - since the tester DB guard (SignalDatabaseActive) it reads 0 in tester and optimizer but non-zero live, so any backtest of CONF_DB/CONF_BLENDED could not reproduce live trading. And what the DB produces is a filter-RANKING win rate, not a per-trade win probability. Both confidence numbers are still recorded per trade (aiConfidence / dbConfidence) and still bucketed against outcome in TradeJournalReport. Recording is what keeps the question answerable; acting on it was the part with no evidence behind it. ConfidenceBridge.mqh now carries an explicit telemetry-only rule at the top. ENUM ORDINALS PINNED. Removing a member vacated a value in four enums at once and MT5 does not validate an enum input replayed from a saved .set or a stored optimization pass. TRAILING_STRATEGY and MONEY_MANAGEMENT_STRATEGY now carry explicit values so the survivors keep the numbers they were saved as, and ValidateBarrierInputs is widened into ValidateTradeManagementInputs covering SL_Mode, TP_Mode, Entry_Multiplier, TrailingStrategy and MM_STRATEGY. Without that gate a chart saved with the Intelligent stop would feed SL_Mode = -1 into a multiplier now used verbatim, placing the stop on the wrong side of entry. RETRAIN-NEUTRAL: neither SL_Mode nor TP_Mode appears in BuildModelFingerprint() or ComputeDbConfigFingerprint() since the swing-pivot target replaced the barrier labels. No .nnw, .cfg or .db re-keys. Also drops the now-dead g_TradeRewardRiskRatio bridge, the CMoneyRiskBase::AdjustRiskAmount hook and the unsigned AIConfidence(). Compile-verified in _claude_stage: 0 errors, 0 warnings. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-25 10:10:20 -04:00
//--- than louder, which is the safe direction for anything that could ever close a position.
feat(gate): grade OOS calls on the exit policy actually in force, and move vote combining out of the members and into the orchestrator Option (a) from the exit-policy question: the certified number must be the traded number. Plus the modularity correction the user called for on 778b6c0. 1. VOTE COMBINING BELONGS TO THE ORCHESTRATOR, NOT TO A MEMBER. 778b6c0 fixed the last-writer-wins bug on g_LiveAISignedConfidence by having a member average its siblings through g_warriorEnsemble. That trades a scheduling bug for a coupling bug, and it is the wrong shape for this EA: every signal runs in its own instance, minds its own state, and VOTES to the orchestrator, which is the only thing allowed to combine opinions. Replaced with a publish/aggregate pair in Variables\ConfidenceBridge.mqh whose split is enforced by shape rather than by convention: - PublishAIVote(slot, conf) - a member writes ONLY its own slot, reads nobody's; - AggregateAIVotes() - called by CExpertSignalCustom::LiveSignedConfidence. CExpertSignalAIBase::EnsembleLiveSignedConfidence is gone. The orchestrator also republishes the aggregate into g_LiveAISignedConfidence, because the intelligent trailing reads that global directly and must act on the same number the exit route does rather than on a leftover from whichever member ticked last. A solo AI signal owns slot 0, so the non-ensemble path is unchanged. 2. THE GATE NOW REPLAYS THE REAL EXIT RULE. SimulateTradeOutcome() walks the same price series with the same fill/barrier/spread convention as ComputeLabelForBar - deliberately by copy, so a disagreement between the two can only be a policy effect and never a discrepancy between two pieces of our own arithmetic - and terminates at the FIRST of stop / target / vote reversal / horizon. Barriers are tested before the vote on the same bar: intrabar we cannot know which came first, and the barrier is what the broker executes automatically, so checking the vote first would credit the exit policy with escapes a real stop would have taken out of its hands. It runs AFTER pass 3, not inside it. A vote-flip exit for a trade entered at bar r is decided by the model's output at bars r-1, r-2, ... - NEWER bars - and pass 3 walks oldest-to-newest, so at the moment r is graded its own exit does not exist yet. Only once m_oosDecisionSeries is complete over the whole OOS window can the replay run. In ensemble mode that series carries the member's adjusted decision and the live exit reads the ensemble aggregate, which is the coupling the user identified: an LSTM entry really can be closed by the ensemble turning against it. 3. THE STATISTIC HAS TO CHANGE WITH THE POLICY, AND THAT IS THE REAL FINDING. A barrier exit pays a fixed R. A vote exit pays whatever the close happens to be. So the moment vote exits are enabled the payoff is CONTINUOUS, and "win rate vs break-even" stops being a meaningful test - there is no fixed break-even for a variable payoff. SimulateTradeOutcome therefore returns R rather than a bool, and the replay reports expectancy in R with its SE taken from the R distribution (overlap- deflated on the same EffectiveSampleSize doctrine as every other SE here), not from a binomial. This is the same class of error as win-based scoring in 2026-08-09: measuring a variable-payoff process with a fixed-payoff statistic. Naming it now, while vote exits are still off, is much cheaper than discovering it after they go on. 4. WHY THIS IS SAFE TO SHIP TODAY. Min_Vote_Close ships Disabled (101, unreachable on both scales it drives) and reaches the AI signal through the new ExitPolicy() setter as 1.01, which the setter turns into 0.0 = no vote exit. Under that policy every replayed trade resolves at a barrier and the simulation is arithmetically the same trade the deploy gate already certifies - they cannot drift. The report says so explicitly, and prints ONCE per run in that state; when vote exits are on it prints every era, because then the divergence is the thing to watch. Nothing about today's numbers moves. The gate switchover is wired but dormant by construction: it becomes exit-aware the moment the input is enabled, which is exactly what "the certified number is the traded number" has to mean. KNOWN LIMIT, stated rather than hidden: only the AI early-exit route is replayed. The rule-based averaged-vote close (m_threshold_close) depends on every other filter's live vote, which pass 3 does not reproduce, so a position the classic filters would have closed is held to its barrier here. The replay therefore holds LONGER than live and overstates barrier-reached outcomes. Faithful only while the AI is the dominant vote - which is the configuration this is being built for. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-17 16:25:57 -04:00
//--- PUBLISH ONLY THIS SIGNAL'S OWN VOTE. Combining is the orchestrator's job, never a member's - see
//--- the vote board in Variables\ConfidenceBridge.mqh for the scheduling bug this replaces and for why
//--- "have a member average its siblings" was the wrong shape of fix in a codebase whose whole point is
//--- that signals do not reach into each other. A solo AI signal owns slot 0 and the aggregate is then
//--- just its own value, so the non-ensemble path is unchanged.
PublishAIVote(m_ensembleMember ? m_ensembleIndex : 0, SignedAIConfidence());
fix(panel,arrows): one deploy predicate, a deployed-only readout, and persist the vote arrows Four reported symptoms, three of them one root cause: the ensemble's certified record was session-scoped and written ONLY at pass-3 completion. A deployed ensemble runs no further eras, so every restart lost the aggregate win rate, the aggregate panel line and the overlay snapshots - and could never regenerate them, because regeneration only happens at an era end that will never come. THE SELF-CONTRADICTION. Member rows read "Live - learning from new bars" (from m_trainingComplete) while the line under them read "training, not tradable yet" (from `prospective`, which means "this number came from ProspectiveVote() rather than a real Direction() call" - what happens on any bar where every member abstains, and which says nothing whatever about training state). Both now resolve through one predicate: WarriorChartModelsDeployed(), fed by members publishing their own state on the same slot and cadence as their vote. Adds a third verdict word, "armed (bar still open)", for a deployed model on a prospective recompute - the case that used to claim it was training. DEPLOYED PANEL. Once every published model is converged the per-member rows are dropped: what ships is the aggregate vote win rate, the live vote, and the verdict. While training the rows stay - they are the only way a collapsed or lagging member is visible, since a collapsed member abstains and so is invisible in the aggregate by construction. ACCURACY NOW RESPECTS THE ENTRY THRESHOLD. The panel's "precision 65%" came from m_cumOosCorrect/m_cumOosTotal, which counts every bar a model called Buy or Sell - threshold-blind, and per-model rather than per-vote. The correct number already existed (votePrecPct: bars where |vote| >= threshold and the direction policy allows) and is now what the panel shows, with the threshold named in the text because the number is meaningless without it. VOTE ARROWS PERSIST. With DrawUnfilteredSignals off - the default - the chart shows SIG_VOTE_PREFIX arrows, and nothing saved them: CChartUI's .arrows sidecar is member-scoped and never saw that layer. New CVoteArrowStore mirrors them to a chart-keyed sidecar and restores them progressively at init, on the same budgeted non-blocking path. The header stores the open/close thresholds; a mismatch on load DISCARDS the arrows rather than redrawing a picture of a strategy no longer configured - stale arrows are worse than none, because none is visibly empty and stale is confidently wrong. Also: .stats bumped to WST7 carrying the ensemble record (guarded on threshold match, most-complete-copy-wins), and the loader's version tests collapsed from an or-chain to ">=" - the magics are ASCII 'WST1'.. 'WST7' so they are already ordered, and a missed arm in that chain reads the NEXT field's bytes into this one, which fails as plausible numbers rather than as an error. Compile-verified in _claude_stage: 0 errors, 0 warnings. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-25 13:00:15 -04:00
//--- ...and this member's own training state, on the same slot and the same cadence. Published
//--- HERE, beside the vote, so the number and the word describing it can never come from different
//--- moments - see WarriorChartModelsDeployed() in ExpertSignalCustom.mqh for the contradiction
//--- that produced.
PublishModelConverged(m_ensembleMember ? m_ensembleIndex : 0, m_trainingComplete);
datetime lastBarDate = (datetime)SeriesInfoInteger(m_symbol.Name(), m_period, SERIES_LASTBAR_DATE);
//--- A failed lookup (0) must not silently read as "dtStudied is already caught up, nothing
//--- pending" - that would freeze this function into never re-triggering training/signal refresh
//--- again until some other path happens to bump dtStudied.
bool newBarPending = (dPrevSignal == -2 || lastBarDate <= 0 || ((m_inferenceOnly ? m_lastBarTime : dtStudied) < lastBarDate));
//--- A MODEL THAT HAS NOT FINISHED TRAINING IS ALWAYS PENDING. Gating them on the watermark
//--- meant training could only advance when a new BAR closed. On H1 that is one 120ms chunk per
//--- hour. A POST-TRAINING WALK IS ALSO PENDING.
refactor(online-learning): OnlineLearning is a real collaborator, not a raw-include partial (S5) Expert\AIBase\OnlineLearning.mqh (595 lines) -> Expert\OnlineLearning\: IOnlineLearningView.mqh (abstract, ~50 accessors) + AIBaseOnlineLearningView.mqh/ AIBaseOnlineLearningViewImpl.mqh (the adapter) + OnlineLearning.mqh (COnlineLearning). STATEFUL, unlike CModelPersistence: grep-verified the shadow net, the OOS continual-learning simulation state and the pattern-database backfill state are genuinely exclusive to this file's own methods - Training.mqh/Topology.mqh/ Lifecycle.mqh/the signal's own header only ever CHECKED or RESET this state at era/lifecycle boundaries, never owned it, so it moved onto the collaborator as real members (same doctrine as Excursion). Those external touch points became consolidated view/forward calls instead of raw field pokes - AbortSimIfActive() replaces THREE separate copies of the same delete/null/false triple (Training.mqh's stop path, FlushTrainRun, ResetWeights), matching the geometry-scan duplicate-reset precedent in project memory. ResetForFreshTopology() replaces Topology.mqh's five- field reset block, DeployNet() replaces the shadow-preferred net selection duplicated in Inference.mqh and ChartScoreBarForRescan, and BlendTowardNet() replaces the era-end blend Training.mqh used to poke m_shadowNet for directly. Reused the signal's existing Data*()/Chart*()/Persist*() getters wherever one already answered the question (labels/outcome/history/horizon/priors/servable-bars/etc.); added ~30 new Online*() wrappers only for what nothing else exposed yet. The three PersistOnline*() get/set pairs (WST3 .stats fields) now forward through the owning member instead of touching the field directly - CModelPersistence is unaffected. Every method body is a pure relocation of the original's statements in original order; verified against `git show HEAD~1:Expert/AIBase/Excursion.mqh`-style diff against the pre-extraction file kept in the working tree until this commit. Compiled 0 errors, 0 warnings (stage mirror + MetaEditor64 /compile, ~91s). Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-08-23 22:59:02 -04:00
bool postTrainWalkPending = (m_onlineLearning.SimRunActive() || m_onlineLearning.BackfillActive());
fix: the DB backfill could never run, and HEAD did not compile Four defects in 64c5dd5/1a05e63, found by review + a baseline compile. Goals 1-8 of that session are unchanged; this makes 6 and 8 actually reachable. 1. HEAD DID NOT COMPILE - 6 errors. CControlPanel::Minimize/Maximize were declared `virtual bool ... override`, but CAppDialog declares both as `virtual void` (Controls\Dialog.mqh). errors 265 + 404 on each, plus 151 on `bool ok = CAppDialog::Minimize()`. Return type is void now; there was never a success flag to forward. Verified: 0 errors, 0 warnings. 2. THE BACKFILL COULD NEVER ADVANCE, and neither could the OOS continual simulation (that one has been dead since it was written). Both are armed at the instant convergence is declared, and both advance only from inside Train(), one chunk per call. But ScheduleTrainingIfNeeded's only per-tick ArmStudyEvent site sits in the `else` of a branch taken whenever m_trainingComplete is set and m_trainRunActive is clear - which is exactly the state FinalizeTrainRun() leaves behind one line before they are armed. Train() was never called again, so the walks sat at their start index forever: no "simulation complete" line, and not one row written to the DB this feature exists to fill. Only a manual Resume/Retrain unstuck them. Both flags now keep the model schedulable. 3. IN AI_HYBRID - the mode this ships in - the backfill was never even armed. Ensemble members deploy at Train() ENTRY and return immediately (so no era is wasted), which skips the era-end block the backfill was started from. All four members were a no-op for a second, independent reason. Armed on the ensemble deploy path too, from m_resumeBars/m_resumeOosCutoff. 4. RE-RUNS DUPLICATED ROWS. RegisterSignal inserts unconditionally - no key, no duplicate check - and m_dbBackfillDone is in-memory, so every later attach that retrained to convergence wrote a second full set of rows for the same bars. The ranking would count one bar once per model that ever deployed, weighting superseded opinions as heavily as the live one. A .dbfill marker stamps the deployed era; written only on completion (an interrupted walk redoes itself rather than ranking a partial window) and deleted with the other sidecars on reset-weights. Also: WarmBlocking's timeout was silent, which restored the exact silent pin failure it was added to prevent - it now says so in the journal, and returns true for "no reference pairs to wait for" so the warning stays rare enough to be read. Not addressed, needs a decision: the backfill scores the OOS window with the checkpoint that was SELECTED as best on that same window, then writes those win rates into the table filter weights rank on - the selection set consumed twice, undiscounted, while the deploy gate right next to it applies a family-wise correction for exactly that effect. The rows are also simulated triple-barrier outcomes at today's spread sharing a table with realised fills. The completion log line now states both plainly. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-16 21:25:51 -04:00
bool trainingPending = !(m_trainingComplete || m_inferenceOnly) || postTrainWalkPending;
//--- m_inferenceOnly (single backtest) takes the converged/inference branch even if the seeded model
//--- wasn't flagged complete, so a backtest never drops into Train()'s era loop - see m_inferenceOnly.
fix: the DB backfill could never run, and HEAD did not compile Four defects in 64c5dd5/1a05e63, found by review + a baseline compile. Goals 1-8 of that session are unchanged; this makes 6 and 8 actually reachable. 1. HEAD DID NOT COMPILE - 6 errors. CControlPanel::Minimize/Maximize were declared `virtual bool ... override`, but CAppDialog declares both as `virtual void` (Controls\Dialog.mqh). errors 265 + 404 on each, plus 151 on `bool ok = CAppDialog::Minimize()`. Return type is void now; there was never a success flag to forward. Verified: 0 errors, 0 warnings. 2. THE BACKFILL COULD NEVER ADVANCE, and neither could the OOS continual simulation (that one has been dead since it was written). Both are armed at the instant convergence is declared, and both advance only from inside Train(), one chunk per call. But ScheduleTrainingIfNeeded's only per-tick ArmStudyEvent site sits in the `else` of a branch taken whenever m_trainingComplete is set and m_trainRunActive is clear - which is exactly the state FinalizeTrainRun() leaves behind one line before they are armed. Train() was never called again, so the walks sat at their start index forever: no "simulation complete" line, and not one row written to the DB this feature exists to fill. Only a manual Resume/Retrain unstuck them. Both flags now keep the model schedulable. 3. IN AI_HYBRID - the mode this ships in - the backfill was never even armed. Ensemble members deploy at Train() ENTRY and return immediately (so no era is wasted), which skips the era-end block the backfill was started from. All four members were a no-op for a second, independent reason. Armed on the ensemble deploy path too, from m_resumeBars/m_resumeOosCutoff. 4. RE-RUNS DUPLICATED ROWS. RegisterSignal inserts unconditionally - no key, no duplicate check - and m_dbBackfillDone is in-memory, so every later attach that retrained to convergence wrote a second full set of rows for the same bars. The ranking would count one bar once per model that ever deployed, weighting superseded opinions as heavily as the live one. A .dbfill marker stamps the deployed era; written only on completion (an interrupted walk redoes itself rather than ranking a partial window) and deleted with the other sidecars on reset-weights. Also: WarmBlocking's timeout was silent, which restored the exact silent pin failure it was added to prevent - it now says so in the journal, and returns true for "no reference pairs to wait for" so the warning stays rare enough to be read. Not addressed, needs a decision: the backfill scores the OOS window with the checkpoint that was SELECTED as best on that same window, then writes those win rates into the table filter weights rank on - the selection set consumed twice, undiscounted, while the deploy gate right next to it applies a family-wise correction for exactly that effect. The rows are also simulated triple-barrier outcomes at today's spread sharing a table with realised fills. The completion log line now states both plainly. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-16 21:25:51 -04:00
if((m_trainingComplete || m_inferenceOnly) && !m_trainingStopRequested && !m_trainRunActive &&
!postTrainWalkPending)
{
if(newBarPending)
RefreshConvergedSignal();
}
else
fix(ensemble): responsive panel + synchronized eras + combined-vote accuracy Four user-reported/requested items, one root cause chain: 1) DEAD CONTROL PANEL in AI_HYBRID mode. All members posted custom event id 1 and handled id 1001, and CExpertCustom broadcasts every chart event to every filter - so each posted event ran a train chunk in ALL N members (N*N chunks per round) and the chart thread never idled long enough to deliver clicks/drags. profiling.csv: 99.45% of time in OnChartEventHandler. Fix: per-instance study-event ids (STUDY_EVENT_ID_BASE + construction order, offset above the Controls library's ON_* codes - id 1 was also ON_DBL_CLICK, so panel double-clicks fired training chunks). ArmStudyEvent() is the single post site; lost-event watchdog replaces the accidental sibling-clears-my-flag rescue. 2) WARM-UP DUPLICATION. The auto-tune sweep is deterministic over identical features/labels, and it ends in the full MI diagnostic suite, which the MI-share gate never intercepted on the sweep path - four members ran four identical ~36s sweep+report blocks. First member publishes outcome (g_ensembleChartTuneDone/Installed/Settings); the rest apply it and skip both. 3) DEINIT STRANDED PANEL+ARROWS (user repro 18:52). Root cause from the log: the 4,500ms budget runs from MetaTrader's stop REQUEST - a heavy autosave in flight ate it, OnDeinit got ~430ms and died in the first member's arrow persist ("Abnormal termination" 432ms in). Fix: early visible-UI sweep (native prefix deletes for status/panel/dialog) right after ClearStatusLabel, and a fast path for still-training models - their arrows are re-rendered every era, so they get one bulk purge instead of scan+atomic-write in the death window. 4) ENSEMBLE FEATURES (user requests): era BARRIER - members advance era by era together; a member ahead of the slowest still-training member declines Train() calls and its chunk budget is donated (TRAIN_TIME_BUDGET_MS = 120/activeTrainers, UI headroom constant). COMBINED-VOTE OOS SCORE - each member's pass-3 scan contributes its adjusted per-bar decision (0.0 on abstain) to a shared row buffer; the last member to finish the era scores the averaged vote vs the mirrored Min_Vote_Open against the same target-before-stop outcomes members grade themselves on, publishing an "Ensemble vote" line on the aggregated panel. Member headlines now carry their lifetime win rate with break-even. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-16 19:06:04 -04:00
{
//--- Lost-event watchdog: an armed event that never arrived (chart-event queue overflow) would
//--- otherwise leave bEventStudy true forever and silently stall training - the accidental rescue
//--- (any sibling's event clearing this flag) went away with the per-instance ids. See
//--- STUDY_EVENT_LOST_MS for why a false trip is not a realistic concern.
if(bEventStudy && GetTickCount() - m_studyArmedTick > STUDY_EVENT_LOST_MS)
{
Print(ID + ": study event armed " + IntegerToString(STUDY_EVENT_LOST_MS / 1000) +
"s ago never arrived (chart-event queue overflow?) - re-arming.");
bEventStudy = false;
}
fix: training could only advance one 120ms chunk per bar ScheduleTrainingIfNeeded() armed the next Train() call only when dtStudied < lastBarDate. That watermark test is right for a CONVERGED model - one inference refresh per new bar - and wrong for a training run, because Train() is chunked: it does ~120ms of work and yields, needing thousands of calls to finish one era, and every one of those calls has to be armed from there. dtStudied is two incompatible things. Train() sets it to the training WINDOW START (~2008); FinalizeTrainRun() sets it to the last bar SCANNED (~now). So the moment any run finalized, the scheduler went silent until the next candle closed. On H1 that is one chunk per hour. The symptom was indistinguishable from a hang: no era lines, no heartbeats, not one of the six instrumented stall branches - because Train() was not being CALLED. The TRAIN STALL line that caught it reported runActive=Y only because m_trainRunActive had been set microseconds earlier in that same call, and eraResume=N proved no era was in flight. Two log bursts, 28 minutes apart, exactly one H1 bar. Before 0c85c54 this was survivable rather than correct: the saved watermark left almost no bars eligible per era, so eras were nearly free and one call per bar still looked like progress. An unconverged model is now always pending. Pause/stop are handled by m_trainingPaused/m_trainingStopRequested, which Train() checks itself. Also: the one Train() exit that tears down the whole run on a buffer failure was completely silent - it now says so. And the build tag moves to train-dispatch-v2; it had not moved since ce52654, which is why the running binary could not be identified from its own log. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-10 10:06:49 -04:00
if(!m_trainingStopRequested && !bEventStudy && (newBarPending || trainingPending))
fix(ensemble): responsive panel + synchronized eras + combined-vote accuracy Four user-reported/requested items, one root cause chain: 1) DEAD CONTROL PANEL in AI_HYBRID mode. All members posted custom event id 1 and handled id 1001, and CExpertCustom broadcasts every chart event to every filter - so each posted event ran a train chunk in ALL N members (N*N chunks per round) and the chart thread never idled long enough to deliver clicks/drags. profiling.csv: 99.45% of time in OnChartEventHandler. Fix: per-instance study-event ids (STUDY_EVENT_ID_BASE + construction order, offset above the Controls library's ON_* codes - id 1 was also ON_DBL_CLICK, so panel double-clicks fired training chunks). ArmStudyEvent() is the single post site; lost-event watchdog replaces the accidental sibling-clears-my-flag rescue. 2) WARM-UP DUPLICATION. The auto-tune sweep is deterministic over identical features/labels, and it ends in the full MI diagnostic suite, which the MI-share gate never intercepted on the sweep path - four members ran four identical ~36s sweep+report blocks. First member publishes outcome (g_ensembleChartTuneDone/Installed/Settings); the rest apply it and skip both. 3) DEINIT STRANDED PANEL+ARROWS (user repro 18:52). Root cause from the log: the 4,500ms budget runs from MetaTrader's stop REQUEST - a heavy autosave in flight ate it, OnDeinit got ~430ms and died in the first member's arrow persist ("Abnormal termination" 432ms in). Fix: early visible-UI sweep (native prefix deletes for status/panel/dialog) right after ClearStatusLabel, and a fast path for still-training models - their arrows are re-rendered every era, so they get one bulk purge instead of scan+atomic-write in the death window. 4) ENSEMBLE FEATURES (user requests): era BARRIER - members advance era by era together; a member ahead of the slowest still-training member declines Train() calls and its chunk budget is donated (TRAIN_TIME_BUDGET_MS = 120/activeTrainers, UI headroom constant). COMBINED-VOTE OOS SCORE - each member's pass-3 scan contributes its adjusted per-bar decision (0.0 on abstain) to a shared row buffer; the last member to finish the era scores the averaged vote vs the mirrored Min_Vote_Open against the same target-before-stop outcomes members grade themselves on, publishing an "Ensemble vote" line on the aggregated panel. Member headlines now carry their lifetime win rate with break-even. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-16 19:06:04 -04:00
ArmStudyEvent((long)MathMax(0, MathMin(iTime(m_symbol.Name(), PERIOD_CURRENT, (int)(100 * Net.recentAverageSmoothingFactor * (m_trainingComplete ? 1 : 10))), dtStudied)), "New Bar");
}
//--- Train() (see its declaration comment) now yields every ~TRAIN_TIME_BUDGET_MS instead of
//--- blocking for a whole era, so while a run is active this per-tick line would otherwise
//--- overwrite Train()'s own full-detail status label on every single tick between chunks -
//--- flickering between the two instead of showing one steady picture.
fix(ensemble+depth): the barrier had no liveness escape, and the depth gate could not report the one state the evidence pointed at Two charts (USDJPY 50,179 bars / XAUUSD 33,982) sat at era 0 for 38 minutes with four of their eight members completely silent. Nothing in this commit guesses at why the sweep fails - the last five guesses were all wrong. It makes the failure say what it is, and stops one broken member taking its whole chart down with it. WHAT THE LOG ACTUALLY SAYS, before any of this. - The running build IS d9f834d (pulled 14:18, compiled 14:19:01, 0 errors), so every depth instrument from 1dda479/7e63a8b/45c9e21 was live. - It printed NOTHING. Zero "PRIMING", zero "CAPPED", zero "Per-indicator depth" in 27 MB of journal. The instrument built to find the depth shortfall returned "not this". - On USDJPY at 14:24, CONV-cad8 completed eras 0 AND 1 across all 50,179 bars - same chart, same 832-value window, same indicators, byte-identical fingerprint - while LSTM-cad8 and HYB-cad8 reported ok=0 failed=50163. So it is not the symbol, the history, the bar count or the indicator depth. It is per-member. - ok=0 means the NEWEST anchors failed too, and a short indicator cannot do that. The depth reading in project_silent_block_failures is therefore retired by its own instrumentation. THE ROOT CAUSE IS STILL UNKNOWN and this commit does not claim one. 1. THE DEPTH GATE'S SILENT PATH WAS THE STATE IT WAS HUNTING. ServableBars() read `if(servable <= 0 || servable >= want) return want;` - one branch over three unrelated states, silent in all of them: enabled == 0 -> nothing tunable is on. No cap. Healthy. enabled > 0, servable == -1 -> a handle answered INVALID. enabled > 0, servable == 0 -> created, never calculated. BarsCalculated() returns -1 for a dead handle, so a dead MA is indistinguishable from "no tunable indicators enabled" - and both returned `want` without printing a character. That is exactly the state a per-member, every-index, depth-independent failure produces, and it is the single reason a build carrying full depth instrumentation logged nothing through the whole outage. TunableBarsCalculated() now also reports HOW MANY indicators it consulted, and the dead-handle case is reported (latched, with per-handle depths). The RETURN is deliberately unchanged - what to do about a dead handle is not yet known, and changing control flow on an unproven cause is how the last four fixes here went wrong. SettledBars() routes its three pass-through states via ServableBars() so the report is reachable from the training sweep, which is the only caller that hits it. 2. THE STALL REPORT NAMED A SLOT, NEVER A BLOCK. "lookback slot 0 REJECTED (window had 24 of 832 values)" plus a guess ("an indicator warm-up or a history-edge read"). Which guard fired was INFERRED by counting 4+5+4+4+6+1 = 24 and concluding feature 25 must be the MA. The arithmetic was right; every conclusion drawn from it was wrong, because a value count names a POSITION and a position cannot tell cold from capped from invalid from off-the-end. Every guard that can reject a bar now records itself - m_featureFailBlock - and the report carries it, the series index, IndicatorDepthReport()'s per-handle depths, and for each indicator whether the NEWEST bar reads. That last field is the whole diagnosis in one word: newest-also-EMPTY means the buffer is unreadable everywhere (cold or dead handle), newest-reads means a genuine history edge. Instrumented: open, ATR, MA, RSI, MACD, Ichimoku, and all five AD blocks via ADIndicatorCold(). 3. THE TOTAL-FAILURE BACKOFF WAS GATED ON THE WRONG QUESTION. It armed only when m_featureFailTransient was set. Keeping that flag correct across every guard is a list that has to stay right forever - the same shape of fix the feature cache abandoned for the same reason - and the gate is pointless anyway: a sweep where ZERO of 50,163 bars produced a window will produce zero again if it restarts a millisecond later, transient or not. Doing that at full speed is what starved six indicator threads on a six-core box. The backoff is now unconditional on a total failure. The flag keeps its real job, deciding whether a MISS may be cached, which is a per-bar question and not a scheduling one. 4. THE ERA BARRIER DEADLOCKED, AND SILENCED THE MEMBERS IT FROZE. EnsembleMinTrainingEra() exempted deployed, stopped and paused members and its comment concluded "so nothing deadlocks". Those three are all VOLUNTARY. A member that simply CANNOT finish an era is none of them, so it pinned the minimum at its own era with no time limit - and the hold branch's only action was `m_lastEraCompleteTick = GetTickCount()`, which silences the stall watchdog. So on USDJPY the two members that could not train reported, and the two healthy members frozen behind them wrote nothing anywhere. The outage was visible only through the members that were not suffering it. - BarrierEraHeartbeat() stamps a clock on real era CHANGE, kept separate from m_lastEraCompleteTick precisely because the barrier resets that one. Only a member AT the minimum can be a blocker; a member ahead is idle by design and is never counted as stuck. - After ENSEMBLE_BARRIER_STUCK_MS (12 min) a non-advancing member is dropped from the barrier minimum. It keeps training and rejoins the instant it completes an era - at which point, being behind, it legitimately becomes the minimum again, which is the documented resumed-laggard behaviour. - Both transitions say so loudly, and the release states plainly that the combined-vote score cannot be computed while the ensemble is desynchronised. - A held member now writes a rate-limited journal line naming WHICH members it is waiting on, so the blocker is read off one line. 5. THE PANEL FLICKER. OnTickHandler gates its terse writer on !m_trainRunActive, and a barrier-held member returns from Train() before ever setting it - so both writers thought they were the only one updating the label and fought every tick. That is the reported "Getting ready..." <-> "Waiting at era N for slower ensemble members" oscillation, and it hit Perceptron but not Convolutional purely because Convolutional had a run active from a completed era and Perceptron, resumed from disk, never did. Train()'s message is the specific one, so it wins. NEXT STEP once this is running: the stall line now ends in "REJECTED BY: ..." and the per-handle depths. Read it. Do not reason around it. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-17 15:21:15 -04:00
if(m_ensembleMember && EnsembleEraBarrierHolds())
return;
if(!m_trainRunActive)
{
//--- Compact, accurate end-state text.
refactor(online-learning): OnlineLearning is a real collaborator, not a raw-include partial (S5) Expert\AIBase\OnlineLearning.mqh (595 lines) -> Expert\OnlineLearning\: IOnlineLearningView.mqh (abstract, ~50 accessors) + AIBaseOnlineLearningView.mqh/ AIBaseOnlineLearningViewImpl.mqh (the adapter) + OnlineLearning.mqh (COnlineLearning). STATEFUL, unlike CModelPersistence: grep-verified the shadow net, the OOS continual-learning simulation state and the pattern-database backfill state are genuinely exclusive to this file's own methods - Training.mqh/Topology.mqh/ Lifecycle.mqh/the signal's own header only ever CHECKED or RESET this state at era/lifecycle boundaries, never owned it, so it moved onto the collaborator as real members (same doctrine as Excursion). Those external touch points became consolidated view/forward calls instead of raw field pokes - AbortSimIfActive() replaces THREE separate copies of the same delete/null/false triple (Training.mqh's stop path, FlushTrainRun, ResetWeights), matching the geometry-scan duplicate-reset precedent in project memory. ResetForFreshTopology() replaces Topology.mqh's five- field reset block, DeployNet() replaces the shadow-preferred net selection duplicated in Inference.mqh and ChartScoreBarForRescan, and BlendTowardNet() replaces the era-end blend Training.mqh used to poke m_shadowNet for directly. Reused the signal's existing Data*()/Chart*()/Persist*() getters wherever one already answered the question (labels/outcome/history/horizon/priors/servable-bars/etc.); added ~30 new Online*() wrappers only for what nothing else exposed yet. The three PersistOnline*() get/set pairs (WST3 .stats fields) now forward through the owning member instead of touching the field directly - CModelPersistence is unaffected. Every method body is a pure relocation of the original's statements in original order; verified against `git show HEAD~1:Expert/AIBase/Excursion.mqh`-style diff against the pre-extraction file kept in the working tree until this commit. Compiled 0 errors, 0 warnings (stage mirror + MetaEditor64 /compile, ~91s). Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-08-23 22:59:02 -04:00
bool onlineActive = m_onlineLearning.Enabled() && !m_inferenceOnly
&& !MQLInfoInteger(MQL_TESTER) && !MQLInfoInteger(MQL_OPTIMIZATION) && !MQLInfoInteger(MQL_FORWARD)
&& CheckPointer(Net) != POINTER_INVALID && !Net.CpuInference();
//--- Simple end-state panel (default, VerboseMode off): plain-language status + the model's
//--- compounded/persistent Buy/Sell win-rate (directional accuracy, Neutral excluded - persisted in
//--- .stats WST5, so it survives a fresh chart reload and is meaningful the moment a drop-and-go user
//--- attaches the EA) + the current call. The verbose era/forecast dump below stays for power users.
if(!VerboseMode)
{
string statusPlain;
if(m_trainingComplete)
statusPlain = onlineActive ? "Live - learning from new bars" : "Ready for live trading";
else if(m_trainingStopRequested)
statusPlain = "Paused - progress saved";
else if(m_trainingPaused)
statusPlain = "Paused";
else
statusPlain = "Getting ready...";
ENUM_SIGNAL liveSig = DoubleToSignal(dPrevSignal);
string liveSigPlain = (liveSig == Buy) ? "Buy" : (liveSig == Sell) ? "Sell" : "Neutral (no trade)";
string simpleLive;
//--- The ensemble headline is the FIRST line, so lead with the signal - on the combined
//--- panel each member's one line must answer "what is this model saying right now". Branched
//--- FIRST, not built-then-overwritten: ComputeCompoundedAccuracyLine() is a real string build
//--- (OosTally lookups, formatting) whose result the ensemble branch used to discard outright.
if(m_ensembleMember)
fix(ensemble): responsive panel + synchronized eras + combined-vote accuracy Four user-reported/requested items, one root cause chain: 1) DEAD CONTROL PANEL in AI_HYBRID mode. All members posted custom event id 1 and handled id 1001, and CExpertCustom broadcasts every chart event to every filter - so each posted event ran a train chunk in ALL N members (N*N chunks per round) and the chart thread never idled long enough to deliver clicks/drags. profiling.csv: 99.45% of time in OnChartEventHandler. Fix: per-instance study-event ids (STUDY_EVENT_ID_BASE + construction order, offset above the Controls library's ON_* codes - id 1 was also ON_DBL_CLICK, so panel double-clicks fired training chunks). ArmStudyEvent() is the single post site; lost-event watchdog replaces the accidental sibling-clears-my-flag rescue. 2) WARM-UP DUPLICATION. The auto-tune sweep is deterministic over identical features/labels, and it ends in the full MI diagnostic suite, which the MI-share gate never intercepted on the sweep path - four members ran four identical ~36s sweep+report blocks. First member publishes outcome (g_ensembleChartTuneDone/Installed/Settings); the rest apply it and skip both. 3) DEINIT STRANDED PANEL+ARROWS (user repro 18:52). Root cause from the log: the 4,500ms budget runs from MetaTrader's stop REQUEST - a heavy autosave in flight ate it, OnDeinit got ~430ms and died in the first member's arrow persist ("Abnormal termination" 432ms in). Fix: early visible-UI sweep (native prefix deletes for status/panel/dialog) right after ClearStatusLabel, and a fast path for still-training models - their arrows are re-rendered every era, so they get one bulk purge instead of scan+atomic-write in the death window. 4) ENSEMBLE FEATURES (user requests): era BARRIER - members advance era by era together; a member ahead of the slowest still-training member declines Train() calls and its chunk budget is donated (TRAIN_TIME_BUDGET_MS = 120/activeTrainers, UI headroom constant). COMBINED-VOTE OOS SCORE - each member's pass-3 scan contributes its adjusted per-bar decision (0.0 on abstain) to a shared row buffer; the last member to finish the era scores the averaged vote vs the mirrored Min_Vote_Open against the same target-before-stop outcomes members grade themselves on, publishing an "Ensemble vote" line on the aggregated panel. Member headlines now carry their lifetime win rate with break-even. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-16 19:06:04 -04:00
{
simpleLive = statusPlain + " -> " + liveSigPlain;
fix(ensemble): responsive panel + synchronized eras + combined-vote accuracy Four user-reported/requested items, one root cause chain: 1) DEAD CONTROL PANEL in AI_HYBRID mode. All members posted custom event id 1 and handled id 1001, and CExpertCustom broadcasts every chart event to every filter - so each posted event ran a train chunk in ALL N members (N*N chunks per round) and the chart thread never idled long enough to deliver clicks/drags. profiling.csv: 99.45% of time in OnChartEventHandler. Fix: per-instance study-event ids (STUDY_EVENT_ID_BASE + construction order, offset above the Controls library's ON_* codes - id 1 was also ON_DBL_CLICK, so panel double-clicks fired training chunks). ArmStudyEvent() is the single post site; lost-event watchdog replaces the accidental sibling-clears-my-flag rescue. 2) WARM-UP DUPLICATION. The auto-tune sweep is deterministic over identical features/labels, and it ends in the full MI diagnostic suite, which the MI-share gate never intercepted on the sweep path - four members ran four identical ~36s sweep+report blocks. First member publishes outcome (g_ensembleChartTuneDone/Installed/Settings); the rest apply it and skip both. 3) DEINIT STRANDED PANEL+ARROWS (user repro 18:52). Root cause from the log: the 4,500ms budget runs from MetaTrader's stop REQUEST - a heavy autosave in flight ate it, OnDeinit got ~430ms and died in the first member's arrow persist ("Abnormal termination" 432ms in). Fix: early visible-UI sweep (native prefix deletes for status/panel/dialog) right after ClearStatusLabel, and a fast path for still-training models - their arrows are re-rendered every era, so they get one bulk purge instead of scan+atomic-write in the death window. 4) ENSEMBLE FEATURES (user requests): era BARRIER - members advance era by era together; a member ahead of the slowest still-training member declines Train() calls and its chunk budget is donated (TRAIN_TIME_BUDGET_MS = 120/activeTrainers, UI headroom constant). COMBINED-VOTE OOS SCORE - each member's pass-3 scan contributes its adjusted per-bar decision (0.0 on abstain) to a shared row buffer; the last member to finish the era scores the averaged vote vs the mirrored Min_Vote_Open against the same target-before-stop outcomes members grade themselves on, publishing an "Ensemble vote" line on the aggregated panel. Member headlines now carry their lifetime win rate with break-even. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-16 19:06:04 -04:00
//--- one-line accuracy on the member headline (user request 2026-08-16: the solo panel
//--- shows accuracy, the ensemble panel did not).
fix(ensemble): responsive panel + synchronized eras + combined-vote accuracy Four user-reported/requested items, one root cause chain: 1) DEAD CONTROL PANEL in AI_HYBRID mode. All members posted custom event id 1 and handled id 1001, and CExpertCustom broadcasts every chart event to every filter - so each posted event ran a train chunk in ALL N members (N*N chunks per round) and the chart thread never idled long enough to deliver clicks/drags. profiling.csv: 99.45% of time in OnChartEventHandler. Fix: per-instance study-event ids (STUDY_EVENT_ID_BASE + construction order, offset above the Controls library's ON_* codes - id 1 was also ON_DBL_CLICK, so panel double-clicks fired training chunks). ArmStudyEvent() is the single post site; lost-event watchdog replaces the accidental sibling-clears-my-flag rescue. 2) WARM-UP DUPLICATION. The auto-tune sweep is deterministic over identical features/labels, and it ends in the full MI diagnostic suite, which the MI-share gate never intercepted on the sweep path - four members ran four identical ~36s sweep+report blocks. First member publishes outcome (g_ensembleChartTuneDone/Installed/Settings); the rest apply it and skip both. 3) DEINIT STRANDED PANEL+ARROWS (user repro 18:52). Root cause from the log: the 4,500ms budget runs from MetaTrader's stop REQUEST - a heavy autosave in flight ate it, OnDeinit got ~430ms and died in the first member's arrow persist ("Abnormal termination" 432ms in). Fix: early visible-UI sweep (native prefix deletes for status/panel/dialog) right after ClearStatusLabel, and a fast path for still-training models - their arrows are re-rendered every era, so they get one bulk purge instead of scan+atomic-write in the death window. 4) ENSEMBLE FEATURES (user requests): era BARRIER - members advance era by era together; a member ahead of the slowest still-training member declines Train() calls and its chunk budget is donated (TRAIN_TIME_BUDGET_MS = 120/activeTrainers, UI headroom constant). COMBINED-VOTE OOS SCORE - each member's pass-3 scan contributes its adjusted per-bar decision (0.0 on abstain) to a shared row buffer; the last member to finish the era scores the averaged vote vs the mirrored Min_Vote_Open against the same target-before-stop outcomes members grade themselves on, publishing an "Ensemble vote" line on the aggregated panel. Member headlines now carry their lifetime win rate with break-even. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-16 19:06:04 -04:00
if(m_cumOosTotal > 0)
feat(target): delete the barrier/geometry stack - the label is the verdict Step 3 of the swing-pivot plan, whole-hog. The swing label is now the ONE target and the era verdict is precision + recall per class against the label's own base rate - no win rate, no break-even, no expectancy, no geometry anywhere in training. DELETED - Expert/Excursion/ (4), Expert/BarrierHorizon/ (4), GeometrySweep, FirstPassageLadder, Labeling/TripleBarrier.mqh (CLabelOverlap survives in Labeling/LabelOverlap.mqh), 3 test EAs. - TripleBarrierLabel + walk, fractal label, geometry derivation/scan/ adoption, exit-policy replay, excursion MI targets, the drift verdict (DIRECTION_INTELLIGENT), the recall floor, balanced-accuracy telemetry, the barrier defines, the .cfg geometry adopt (slots kept as zeros for the positional layout), the derived-geometry live-order override. - TRAINING_TARGET input/enum: direction models are always swing; META2 re-keys the meta head onto label agreement (descriptor loses its two geometry slots). REWORKED - Labels.mqh (1795 -> ~370 lines): AdvanceSwingLabelState with FINALITY-GATED CACHING - an unresolved bar (pivot pair uncommitted) is never cached, so it can never freeze as a false Neutral; training, calibration, OOS scoring and online learning all skip unresolved bars. - SDeployVerdict: significance-only; SOosTally chance = larger directional class share; pooled gate poolability = timeframe (record v2). - Purge/embargo/declustering gaps: the measured mean label resolution lag (LabelResolutionBars), not a barrier horizon. - Pool purge key + backfill DB rows: marked at the bar the label resolved on (m_labelResolveAge), not a fabricated barrier touch. - Online learning frontier: finality, not a horizon delay. - m_bestBalancedOos -> m_bestSelectionScore, m_erasSinceBestBalanced -> m_erasSinceBest, ensemble vote outcome arrays -> label arrays. STEP 4 folded in: Entry_Multiplier / SL_Mode / TP_Mode / tradingdirection are inputs again - trade management is the tester GA's search space. Fingerprints: every direction model re-keys (TGT:SWG1 now unconditional, CUT token gone); META1 -> META2. Full retrain, as planned. Compile-verified in _claude_stage: Warrior_EA + both surviving test EAs, 0 errors, 0 warnings each. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-24 20:42:31 -04:00
simpleLive += StringFormat(" | precision %d%%", (int)MathRound(m_cumOosCorrect * 100.0 / m_cumOosTotal));
fix(ensemble): responsive panel + synchronized eras + combined-vote accuracy Four user-reported/requested items, one root cause chain: 1) DEAD CONTROL PANEL in AI_HYBRID mode. All members posted custom event id 1 and handled id 1001, and CExpertCustom broadcasts every chart event to every filter - so each posted event ran a train chunk in ALL N members (N*N chunks per round) and the chart thread never idled long enough to deliver clicks/drags. profiling.csv: 99.45% of time in OnChartEventHandler. Fix: per-instance study-event ids (STUDY_EVENT_ID_BASE + construction order, offset above the Controls library's ON_* codes - id 1 was also ON_DBL_CLICK, so panel double-clicks fired training chunks). ArmStudyEvent() is the single post site; lost-event watchdog replaces the accidental sibling-clears-my-flag rescue. 2) WARM-UP DUPLICATION. The auto-tune sweep is deterministic over identical features/labels, and it ends in the full MI diagnostic suite, which the MI-share gate never intercepted on the sweep path - four members ran four identical ~36s sweep+report blocks. First member publishes outcome (g_ensembleChartTuneDone/Installed/Settings); the rest apply it and skip both. 3) DEINIT STRANDED PANEL+ARROWS (user repro 18:52). Root cause from the log: the 4,500ms budget runs from MetaTrader's stop REQUEST - a heavy autosave in flight ate it, OnDeinit got ~430ms and died in the first member's arrow persist ("Abnormal termination" 432ms in). Fix: early visible-UI sweep (native prefix deletes for status/panel/dialog) right after ClearStatusLabel, and a fast path for still-training models - their arrows are re-rendered every era, so they get one bulk purge instead of scan+atomic-write in the death window. 4) ENSEMBLE FEATURES (user requests): era BARRIER - members advance era by era together; a member ahead of the slowest still-training member declines Train() calls and its chunk budget is donated (TRAIN_TIME_BUDGET_MS = 120/activeTrainers, UI headroom constant). COMBINED-VOTE OOS SCORE - each member's pass-3 scan contributes its adjusted per-bar decision (0.0 on abstain) to a shared row buffer; the last member to finish the era scores the averaged vote vs the mirrored Min_Vote_Open against the same target-before-stop outcomes members grade themselves on, publishing an "Ensemble vote" line on the aggregated panel. Member headlines now carry their lifetime win rate with break-even. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-16 19:06:04 -04:00
}
else
{
simpleLive = DisplayName() + " - " + statusPlain + "\n";
//--- Only show the accuracy line once at least one signal has been validated (compounded
//--- counts persist across restarts, so a deployed model shows real numbers immediately,
//--- not "measuring").
if(m_cumIsTotal > 0 || m_cumOosTotal > 0)
simpleLive += ComputeCompoundedAccuracyLine() + "\n";
}
PublishStatus(simpleLive);
return;
}
string completeText = onlineActive ? "Complete - live (adapting to new bars)" : "Complete - ready for live (inference)";
string trainingState = m_trainingStopRequested
? (m_trainingComplete ? completeText : "Stopped - resumable (weights kept)")
: (m_trainingPaused ? "Paused" : (m_trainingComplete ? completeText : "In progress"));
//--- same "Forecast: <signal> -> <value>" line the active training loop's status label ends on
//--- (see the classLine-terminated StringFormat below), instead of a raw bEventStudy/dPrevSignal/
//--- dtStudied debug dump - this is what stays on screen once training stops/pauses/completes.
PublishStatus(StringFormat(
ID + " : Era %d -> Training %s\n" +
"Forecast: %s -> %.2f",
m_eraCount, trainingState,
EnumToString(DoubleToSignal(dPrevSignal)), dPrevSignal));
}
}
//+------------------------------------------------------------------+
//| Timer-driven equivalent of OnTickHandler()'s scheduling, called |
//| from Warrior_EA.mq5's always-on OnTimer() so training keeps |
//| progressing purely on wall-clock time - no dependency on ticks, |
//| which simply don't arrive while the market is closed. |
//+------------------------------------------------------------------+
void CExpertSignalAIBase::PollTraining(void)
{
//--- Drain a slice of the queued chart-arrow restore FIRST, and skip training work on any tick
//--- where restoring is still in flight.
//--- STOP BEFORE ANY OF IT. Drawing arrows for a chart that is unloading is the worst case of the
//--- three: it lengthens the very sweep that has to finish.
fix(shutdown): make ExitPolicy public, and stop every long loop the moment MT5 asks Two things, one of which was a compile error. 1. ExitPolicy() was declared in the protected block but is pushed in from Warrior_EA.mq5:770. Moved to public beside the other EA-facing setters. 2. Chart objects surviving OnDeinit. The 4,500 ms teardown budget is measured from the STOP REQUEST, not from OnDeinit's first line, and OnDeinit cannot begin until whatever is in flight returns - so a scan still running after _StopFlag is raised does not delay the cleanup, it SPENDS it, and the purge never gets its turn. New CExpertSignalAIBase::ShutdownRequested() = IsStopped() || m_shutdownInProgress. Deliberately NOT m_trainingStopRequested: that latches, and a latched flag would permanently disable scans that must run again on the next Start. Guarded, longest first: - TuneIndicatorsByFilter - per candidate, restoring the OPERATOR's settings on the way out (best[] is mutated in place; the tuner otherwise keeps the last trial's parameters, which nothing chose). - ReportBarrierGeometryScan - per pairing, breaking to ONE restore point so m_barrierScanLiveLabels can never be left true (that makes ComputeLabelForBar read the last candidate's multiples as the configured geometry). - ReportFeatureLabelInformation / ReportExcursionInformation / lag profile - nulls ABANDON rather than truncate: fewer draws is not a smaller null, it is a wrong one, and p shifts toward significance. m_dirEvidence staying false is the safe direction. - SimulateExitPolicyOutcomes - zeroes its accumulators so the divergence line is dropped instead of latching a partial expectancy as the run's only report. - ReportGeometryExpectancyScan - per ladder rung. - HttpGet - one choke point for up to a dozen blocking WebRequests per first-pass Update(). An in-flight request cannot be cancelled; refusing to start another is the whole remedy. - PollTraining, OnChartEventHandler's study event, TuneIndicatorsAndTrain - entry points, so a queued event cannot open an era during teardown. TuneIndicatorsAndTrain's guard is the first statement, ahead of the m_tuneFilterDone / g_ensembleChartTuneDone latches. - OnTick / OnTimer / OnChartEvent. Training's own bar loops already honoured this (pass 1 per bar, passes 2/2.5/3 yield on a 120 ms budget); the warm-up scans did not, and they are the longest uninterruptible stretches the EA has. StopTraining() is unchanged: the operator's Stop still finalises synchronously. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-17 17:03:11 -04:00
if(ShutdownRequested())
return;
refactor(chart): ChartUI is a real collaborator, not a raw-include partial (S2) Expert/AIBase/ChartUI.mqh was 869 lines of method bodies of CExpertSignalAIBase, #included after the class declaration - free to touch any of its ~500 members. First of the eleven AIBase/*.mqh partials to come out (fewest inbound edges - see the SOLID campaign session order), using the same view+adapter shape already proven for CTrainingDataView. CChartView (Expert/Chart/IChartView.mqh) is the abstract read/behaviour surface a chart-rendering collaborator needs - identity, bar/model access, the prediction cache, and the training/vote/meta scalars the panel and HUD line summarise. CAIBaseChartView is the adapter the signal owns and binds to itself (MQL5 gives a class exactly one base, so CExpertSignalAIBase cannot implement the view directly). CChartUI is the real collaborator: it owns the arrow-restore queue, the rescan queue/tally, the last-arrows-saved count and the purge-mismatch latch as its own fields (verified via grep to be touched nowhere else in Expert/), and reaches everything else - including StartChartSignalRescan, moved in from its old inline home in the header since it drives the exact same rescan state machine AdvanceChartSignalRescan drains - through the view. m_arrowSignalCache and m_signalClusterWindow stay on the signal: Training.mqh writes the cache directly every era and the training-data view already reads it, so moving it would mean rewriting Training.mqh's write sites too - out of scope here. CChartUI reaches it through four bounds-checked accessors instead of a raw member poke. All 10 public methods keep their exact signatures and become one-line forwards on the signal, so no other file's call sites change except Training.mqh's one era-end status refresh, which now reads RefreshStatusLabel() rather than reaching into CChartUI's now-private last-displayed-neuron cache directly. Verified structurally, not compiled (never compile - the operator does, in MetaEditor): brace balance checked on every touched/new file against HEAD, and the view/adapter/impl method lists cross-diffed to confirm all 59 accessors match 1:1 across the interface, the adapter declaration and the adapter body. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-08-23 20:03:52 -04:00
if(m_chartUI.ArrowRestorePending())
{
AdvanceChartSignalRestore();
return;
}
//--- Same one-thread reasoning as the arrow restore above: a manual rescan (Show Signals) also competes
//--- for the single MQL5 thread, and its per-bar feedForward is real compute rather than a cheap object
//--- write, so it must finish its own slices before training resumes rather than interleaving with it.
refactor(chart): ChartUI is a real collaborator, not a raw-include partial (S2) Expert/AIBase/ChartUI.mqh was 869 lines of method bodies of CExpertSignalAIBase, #included after the class declaration - free to touch any of its ~500 members. First of the eleven AIBase/*.mqh partials to come out (fewest inbound edges - see the SOLID campaign session order), using the same view+adapter shape already proven for CTrainingDataView. CChartView (Expert/Chart/IChartView.mqh) is the abstract read/behaviour surface a chart-rendering collaborator needs - identity, bar/model access, the prediction cache, and the training/vote/meta scalars the panel and HUD line summarise. CAIBaseChartView is the adapter the signal owns and binds to itself (MQL5 gives a class exactly one base, so CExpertSignalAIBase cannot implement the view directly). CChartUI is the real collaborator: it owns the arrow-restore queue, the rescan queue/tally, the last-arrows-saved count and the purge-mismatch latch as its own fields (verified via grep to be touched nowhere else in Expert/), and reaches everything else - including StartChartSignalRescan, moved in from its old inline home in the header since it drives the exact same rescan state machine AdvanceChartSignalRescan drains - through the view. m_arrowSignalCache and m_signalClusterWindow stay on the signal: Training.mqh writes the cache directly every era and the training-data view already reads it, so moving it would mean rewriting Training.mqh's write sites too - out of scope here. CChartUI reaches it through four bounds-checked accessors instead of a raw member poke. All 10 public methods keep their exact signatures and become one-line forwards on the signal, so no other file's call sites change except Training.mqh's one era-end status refresh, which now reads RefreshStatusLabel() rather than reaching into CChartUI's now-private last-displayed-neuron cache directly. Verified structurally, not compiled (never compile - the operator does, in MetaEditor): brace balance checked on every touched/new file against HEAD, and the view/adapter/impl method lists cross-diffed to confirm all 59 accessors match 1:1 across the interface, the adapter declaration and the adapter body. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-08-23 20:03:52 -04:00
if(m_chartUI.RescanPending())
{
AdvanceChartSignalRescan();
return;
}
feat(vote): replay pass rebuilds a deployed model's ladder without retraining The previous commit persisted the tier ladder, which fixes this going forward but did nothing for models whose .stats predates WST7 - they still had to retrain to mint one. They never did. Every number a converged model needs in order to vote is a pure function of weights already on disk plus labels derivable from the chart, so replay them: stage 1 build the label cache (existing chunked prebuild) stage 2 rescan history (existing chunked rescan, deployed net) stage 3 score + rank + persist (one walk over two arrays) ScoreReplayFromCache() walks m_arrowSignalCache against m_labelCacheBuy/Sell, fills the same m_oosTierFired/Hits and per-class totals pass 3 fills, and hands them to RankTiersFromOos() - deliberately feeding the existing ranker rather than reimplementing it. The shrinkage, the chance reference and the module trust weight are subtle enough that a second copy would drift, and a ladder measured by a slightly different rule would be silently incomparable with every ladder training produced. AdvanceDeployedRebuild() sequences the three stages off the timer. It has to be a sequence: stages 1 and 2 are each minutes of work draining in time-boxed slices, and stage 2's output is meaningless until stage 1 has labels to score against. The previous version ran the rescan with no labels at all, which is why it could only ever rebuild arrows and never the ladder - the thing actually blocking the vote. The result is written to .stats immediately. The failure being repaired is state that lived in memory and was never written down; recomputing it and not saving it would repeat that exactly. Also routes every rescan completion through one hook, so there is a single place that knows what a finished rescan means - republish for a manual one, score and rank for a rebuild. Compile-verified in _claude_stage: 0 errors, 0 warnings. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-25 15:01:24 -04:00
if(m_isInitialized && AdvanceDeployedRebuild())
return;
if(m_isInitialized)
ScheduleTrainingIfNeeded();
}
//+------------------------------------------------------------------+
feat(vote): replay pass rebuilds a deployed model's ladder without retraining The previous commit persisted the tier ladder, which fixes this going forward but did nothing for models whose .stats predates WST7 - they still had to retrain to mint one. They never did. Every number a converged model needs in order to vote is a pure function of weights already on disk plus labels derivable from the chart, so replay them: stage 1 build the label cache (existing chunked prebuild) stage 2 rescan history (existing chunked rescan, deployed net) stage 3 score + rank + persist (one walk over two arrays) ScoreReplayFromCache() walks m_arrowSignalCache against m_labelCacheBuy/Sell, fills the same m_oosTierFired/Hits and per-class totals pass 3 fills, and hands them to RankTiersFromOos() - deliberately feeding the existing ranker rather than reimplementing it. The shrinkage, the chance reference and the module trust weight are subtle enough that a second copy would drift, and a ladder measured by a slightly different rule would be silently incomparable with every ladder training produced. AdvanceDeployedRebuild() sequences the three stages off the timer. It has to be a sequence: stages 1 and 2 are each minutes of work draining in time-boxed slices, and stage 2's output is meaningless until stage 1 has labels to score against. The previous version ran the rescan with no labels at all, which is why it could only ever rebuild arrows and never the ladder - the thing actually blocking the vote. The result is written to .stats immediately. The failure being repaired is state that lived in memory and was never written down; recomputing it and not saving it would repeat that exactly. Also routes every rescan completion through one hook, so there is a single place that knows what a finished rescan means - republish for a manual one, score and rank for a rebuild. Compile-verified in _claude_stage: 0 errors, 0 warnings. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-25 15:01:24 -04:00
//| A DEPLOYED MODEL REBUILDS ITS OWN VOTE, WITHOUT RETRAINING. |
//| |
//| Everything a converged model needs in order to vote - the tier |
//| ladder, the module trust weight, the overlay snapshot the arrows |
//| are drawn from - is produced by a completed pass 3 and by nothing |
//| else. A converged model runs no passes. So a model that lost those |
//| (a .stats predating WST7, a wipe, a fresh deploy from a build that |
//| never stored them) was permanently mute: no vote, no trades, no |
//| arrows, and a win rate stuck on "measuring...". |
//| |
//| It never needed a retrain. Every one of those numbers is a pure |
fix(replay): resolve labels inline - the prebuilt cache's window never overlapped the rescan The 15:13 session proved the replay pass ran end-to-end on all 24 models and scored ZERO labelled bars on every one of them, while each rescan sat on ~5000 scored predictions (~2755 Buy / ~2232 Sell). The two windows never overlapped: StartLabelCachePrebuild deliberately keeps a CONVERGED model's dtStudied watermark (it gates inference recency and must not move), so the prebuild's window was the handful of bars since the last studied bar - all with uncommitted pivots, hence "label cache pre-built - Buy: 0 | Sell: 0 | Neutral: 0" on every member. The label never needed a cache. SwingPivotDirectionLabel(idx) is a pure function of the ZigZag/Close/ATR buffers the rescan itself refreshes over exactly the scoring window, and m_lastLabelLifespan == 0 is its own unresolved flag - the same finality gate the cache applies, applied directly. ScoreReplayFromCache now resolves each bar's label inline and the label-prebuild stage is deleted from the rebuild state machine outright; going through a cache built for a different window was indirection that changed the answer. Also splits the empty-result diagnostics: "no resolved labels" (a windowing/data fault) is now distinguished from "labels present, every call Neutral" (a calibration verdict). The first version reported the second message for both, which mislabelled this very bug as a calibration outcome in the same breath as reporting scored=0. Honest limitation, stated in the code too: the replay window includes bars the model trained on, so a replay-minted ladder is measured partly in-sample and will read stronger than a holdout-measured one. It is replaced by the genuine article at the next completed scoring pass; until then it is what makes a restarted deployed model able to vote at all. Compile-verified in _claude_stage: 0 errors, 0 warnings. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-25 15:22:01 -04:00
//| function of weights already on disk plus labels derivable from the |
//| chart, so this replays them: run the deployed net over history |
//| (the existing chunked rescan), then score each prediction against |
//| the swing label RESOLVED INLINE for the same bar and rank the |
//| ladder from the result (ScoreReplayFromCache, via the rescan's |
//| completion hook). |
feat(vote): replay pass rebuilds a deployed model's ladder without retraining The previous commit persisted the tier ladder, which fixes this going forward but did nothing for models whose .stats predates WST7 - they still had to retrain to mint one. They never did. Every number a converged model needs in order to vote is a pure function of weights already on disk plus labels derivable from the chart, so replay them: stage 1 build the label cache (existing chunked prebuild) stage 2 rescan history (existing chunked rescan, deployed net) stage 3 score + rank + persist (one walk over two arrays) ScoreReplayFromCache() walks m_arrowSignalCache against m_labelCacheBuy/Sell, fills the same m_oosTierFired/Hits and per-class totals pass 3 fills, and hands them to RankTiersFromOos() - deliberately feeding the existing ranker rather than reimplementing it. The shrinkage, the chance reference and the module trust weight are subtle enough that a second copy would drift, and a ladder measured by a slightly different rule would be silently incomparable with every ladder training produced. AdvanceDeployedRebuild() sequences the three stages off the timer. It has to be a sequence: stages 1 and 2 are each minutes of work draining in time-boxed slices, and stage 2's output is meaningless until stage 1 has labels to score against. The previous version ran the rescan with no labels at all, which is why it could only ever rebuild arrows and never the ladder - the thing actually blocking the vote. The result is written to .stats immediately. The failure being repaired is state that lived in memory and was never written down; recomputing it and not saving it would repeat that exactly. Also routes every rescan completion through one hook, so there is a single place that knows what a finished rescan means - republish for a manual one, score and rank for a rebuild. Compile-verified in _claude_stage: 0 errors, 0 warnings. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-25 15:01:24 -04:00
//| |
fix(replay): resolve labels inline - the prebuilt cache's window never overlapped the rescan The 15:13 session proved the replay pass ran end-to-end on all 24 models and scored ZERO labelled bars on every one of them, while each rescan sat on ~5000 scored predictions (~2755 Buy / ~2232 Sell). The two windows never overlapped: StartLabelCachePrebuild deliberately keeps a CONVERGED model's dtStudied watermark (it gates inference recency and must not move), so the prebuild's window was the handful of bars since the last studied bar - all with uncommitted pivots, hence "label cache pre-built - Buy: 0 | Sell: 0 | Neutral: 0" on every member. The label never needed a cache. SwingPivotDirectionLabel(idx) is a pure function of the ZigZag/Close/ATR buffers the rescan itself refreshes over exactly the scoring window, and m_lastLabelLifespan == 0 is its own unresolved flag - the same finality gate the cache applies, applied directly. ScoreReplayFromCache now resolves each bar's label inline and the label-prebuild stage is deleted from the rebuild state machine outright; going through a cache built for a different window was indirection that changed the answer. Also splits the empty-result diagnostics: "no resolved labels" (a windowing/data fault) is now distinguished from "labels present, every call Neutral" (a calibration verdict). The first version reported the second message for both, which mislabelled this very bug as a calibration outcome in the same breath as reporting scored=0. Honest limitation, stated in the code too: the replay window includes bars the model trained on, so a replay-minted ladder is measured partly in-sample and will read stronger than a holdout-measured one. It is replaced by the genuine article at the next completed scoring pass; until then it is what makes a restarted deployed model able to vote at all. Compile-verified in _claude_stage: 0 errors, 0 warnings. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-25 15:22:01 -04:00
//| There is deliberately NO label-prebuild stage any more. The first |
//| version had one, and it is exactly why that version scored zero |
//| bars on 24/24 models: the prebuild's window is anchored at |
//| dtStudied, which a converged model keeps at its recency watermark |
//| - so the "label cache" covered a handful of just-closed bars whose |
//| pivots cannot have committed yet, while the rescan sat on five |
//| thousand scored predictions it could never be matched against. |
//| The label is a pure function of buffers the rescan itself |
//| refreshes; going through a cache built for a different window was |
//| indirection that changed the answer. |
feat(vote): replay pass rebuilds a deployed model's ladder without retraining The previous commit persisted the tier ladder, which fixes this going forward but did nothing for models whose .stats predates WST7 - they still had to retrain to mint one. They never did. Every number a converged model needs in order to vote is a pure function of weights already on disk plus labels derivable from the chart, so replay them: stage 1 build the label cache (existing chunked prebuild) stage 2 rescan history (existing chunked rescan, deployed net) stage 3 score + rank + persist (one walk over two arrays) ScoreReplayFromCache() walks m_arrowSignalCache against m_labelCacheBuy/Sell, fills the same m_oosTierFired/Hits and per-class totals pass 3 fills, and hands them to RankTiersFromOos() - deliberately feeding the existing ranker rather than reimplementing it. The shrinkage, the chance reference and the module trust weight are subtle enough that a second copy would drift, and a ladder measured by a slightly different rule would be silently incomparable with every ladder training produced. AdvanceDeployedRebuild() sequences the three stages off the timer. It has to be a sequence: stages 1 and 2 are each minutes of work draining in time-boxed slices, and stage 2's output is meaningless until stage 1 has labels to score against. The previous version ran the rescan with no labels at all, which is why it could only ever rebuild arrows and never the ladder - the thing actually blocking the vote. The result is written to .stats immediately. The failure being repaired is state that lived in memory and was never written down; recomputing it and not saving it would repeat that exactly. Also routes every rescan completion through one hook, so there is a single place that knows what a finished rescan means - republish for a manual one, score and rank for a rebuild. Compile-verified in _claude_stage: 0 errors, 0 warnings. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-25 15:01:24 -04:00
//+------------------------------------------------------------------+
bool CExpertSignalAIBase::AdvanceDeployedRebuild(void)
{
fix(replay): resolve labels inline - the prebuilt cache's window never overlapped the rescan The 15:13 session proved the replay pass ran end-to-end on all 24 models and scored ZERO labelled bars on every one of them, while each rescan sat on ~5000 scored predictions (~2755 Buy / ~2232 Sell). The two windows never overlapped: StartLabelCachePrebuild deliberately keeps a CONVERGED model's dtStudied watermark (it gates inference recency and must not move), so the prebuild's window was the handful of bars since the last studied bar - all with uncommitted pivots, hence "label cache pre-built - Buy: 0 | Sell: 0 | Neutral: 0" on every member. The label never needed a cache. SwingPivotDirectionLabel(idx) is a pure function of the ZigZag/Close/ATR buffers the rescan itself refreshes over exactly the scoring window, and m_lastLabelLifespan == 0 is its own unresolved flag - the same finality gate the cache applies, applied directly. ScoreReplayFromCache now resolves each bar's label inline and the label-prebuild stage is deleted from the rebuild state machine outright; going through a cache built for a different window was indirection that changed the answer. Also splits the empty-result diagnostics: "no resolved labels" (a windowing/data fault) is now distinguished from "labels present, every call Neutral" (a calibration verdict). The first version reported the second message for both, which mislabelled this very bug as a calibration outcome in the same breath as reporting scored=0. Honest limitation, stated in the code too: the replay window includes bars the model trained on, so a replay-minted ladder is measured partly in-sample and will read stronger than a holdout-measured one. It is replaced by the genuine article at the next completed scoring pass; until then it is what makes a restarted deployed model able to vote at all. Compile-verified in _claude_stage: 0 errors, 0 warnings. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-25 15:22:01 -04:00
if(m_deployedRebuildStage >= 2)
feat(vote): replay pass rebuilds a deployed model's ladder without retraining The previous commit persisted the tier ladder, which fixes this going forward but did nothing for models whose .stats predates WST7 - they still had to retrain to mint one. They never did. Every number a converged model needs in order to vote is a pure function of weights already on disk plus labels derivable from the chart, so replay them: stage 1 build the label cache (existing chunked prebuild) stage 2 rescan history (existing chunked rescan, deployed net) stage 3 score + rank + persist (one walk over two arrays) ScoreReplayFromCache() walks m_arrowSignalCache against m_labelCacheBuy/Sell, fills the same m_oosTierFired/Hits and per-class totals pass 3 fills, and hands them to RankTiersFromOos() - deliberately feeding the existing ranker rather than reimplementing it. The shrinkage, the chance reference and the module trust weight are subtle enough that a second copy would drift, and a ladder measured by a slightly different rule would be silently incomparable with every ladder training produced. AdvanceDeployedRebuild() sequences the three stages off the timer. It has to be a sequence: stages 1 and 2 are each minutes of work draining in time-boxed slices, and stage 2's output is meaningless until stage 1 has labels to score against. The previous version ran the rescan with no labels at all, which is why it could only ever rebuild arrows and never the ladder - the thing actually blocking the vote. The result is written to .stats immediately. The failure being repaired is state that lived in memory and was never written down; recomputing it and not saving it would repeat that exactly. Also routes every rescan completion through one hook, so there is a single place that knows what a finished rescan means - republish for a manual one, score and rank for a rebuild. Compile-verified in _claude_stage: 0 errors, 0 warnings. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-25 15:01:24 -04:00
return false;
//--- WHO NEEDS THIS: a converged model that cannot currently vote (no ladder) or cannot currently
//--- be drawn (no snapshot). Never in the tester, never for an inference-only run, and never while
//--- a training run is live - a training pass produces all of this itself, correctly, and racing it
//--- would have two writers on the same counters.
if(m_deployedRebuildStage == 0)
{
if(!m_trainingComplete || m_inferenceOnly || m_trainRunActive || m_trainingPaused
|| MQLInfoInteger(MQL_TESTER) || MQLInfoInteger(MQL_OPTIMIZATION) || MQLInfoInteger(MQL_FORWARD))
return false;
if(m_tiersSelfRanked && m_overlaySnapBars > 0)
{
fix(replay): resolve labels inline - the prebuilt cache's window never overlapped the rescan The 15:13 session proved the replay pass ran end-to-end on all 24 models and scored ZERO labelled bars on every one of them, while each rescan sat on ~5000 scored predictions (~2755 Buy / ~2232 Sell). The two windows never overlapped: StartLabelCachePrebuild deliberately keeps a CONVERGED model's dtStudied watermark (it gates inference recency and must not move), so the prebuild's window was the handful of bars since the last studied bar - all with uncommitted pivots, hence "label cache pre-built - Buy: 0 | Sell: 0 | Neutral: 0" on every member. The label never needed a cache. SwingPivotDirectionLabel(idx) is a pure function of the ZigZag/Close/ATR buffers the rescan itself refreshes over exactly the scoring window, and m_lastLabelLifespan == 0 is its own unresolved flag - the same finality gate the cache applies, applied directly. ScoreReplayFromCache now resolves each bar's label inline and the label-prebuild stage is deleted from the rebuild state machine outright; going through a cache built for a different window was indirection that changed the answer. Also splits the empty-result diagnostics: "no resolved labels" (a windowing/data fault) is now distinguished from "labels present, every call Neutral" (a calibration verdict). The first version reported the second message for both, which mislabelled this very bug as a calibration outcome in the same breath as reporting scored=0. Honest limitation, stated in the code too: the replay window includes bars the model trained on, so a replay-minted ladder is measured partly in-sample and will read stronger than a holdout-measured one. It is replaced by the genuine article at the next completed scoring pass; until then it is what makes a restarted deployed model able to vote at all. Compile-verified in _claude_stage: 0 errors, 0 warnings. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-25 15:22:01 -04:00
m_deployedRebuildStage = 2; // nothing missing - never look again
feat(vote): replay pass rebuilds a deployed model's ladder without retraining The previous commit persisted the tier ladder, which fixes this going forward but did nothing for models whose .stats predates WST7 - they still had to retrain to mint one. They never did. Every number a converged model needs in order to vote is a pure function of weights already on disk plus labels derivable from the chart, so replay them: stage 1 build the label cache (existing chunked prebuild) stage 2 rescan history (existing chunked rescan, deployed net) stage 3 score + rank + persist (one walk over two arrays) ScoreReplayFromCache() walks m_arrowSignalCache against m_labelCacheBuy/Sell, fills the same m_oosTierFired/Hits and per-class totals pass 3 fills, and hands them to RankTiersFromOos() - deliberately feeding the existing ranker rather than reimplementing it. The shrinkage, the chance reference and the module trust weight are subtle enough that a second copy would drift, and a ladder measured by a slightly different rule would be silently incomparable with every ladder training produced. AdvanceDeployedRebuild() sequences the three stages off the timer. It has to be a sequence: stages 1 and 2 are each minutes of work draining in time-boxed slices, and stage 2's output is meaningless until stage 1 has labels to score against. The previous version ran the rescan with no labels at all, which is why it could only ever rebuild arrows and never the ladder - the thing actually blocking the vote. The result is written to .stats immediately. The failure being repaired is state that lived in memory and was never written down; recomputing it and not saving it would repeat that exactly. Also routes every rescan completion through one hook, so there is a single place that knows what a finished rescan means - republish for a manual one, score and rank for a rebuild. Compile-verified in _claude_stage: 0 errors, 0 warnings. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-25 15:01:24 -04:00
return false;
}
Print(ID + ": deployed but " + (!m_tiersSelfRanked ? "WITHOUT A TIER LADDER (so it cannot vote)"
: "without a historical vote snapshot (so it cannot draw arrows)") +
" - replaying history against the deployed weights to rebuild it. No training will run;"
" these numbers are a function of the weights already on disk.");
m_deployedRebuildStage = 1;
}
fix(replay): resolve labels inline - the prebuilt cache's window never overlapped the rescan The 15:13 session proved the replay pass ran end-to-end on all 24 models and scored ZERO labelled bars on every one of them, while each rescan sat on ~5000 scored predictions (~2755 Buy / ~2232 Sell). The two windows never overlapped: StartLabelCachePrebuild deliberately keeps a CONVERGED model's dtStudied watermark (it gates inference recency and must not move), so the prebuild's window was the handful of bars since the last studied bar - all with uncommitted pivots, hence "label cache pre-built - Buy: 0 | Sell: 0 | Neutral: 0" on every member. The label never needed a cache. SwingPivotDirectionLabel(idx) is a pure function of the ZigZag/Close/ATR buffers the rescan itself refreshes over exactly the scoring window, and m_lastLabelLifespan == 0 is its own unresolved flag - the same finality gate the cache applies, applied directly. ScoreReplayFromCache now resolves each bar's label inline and the label-prebuild stage is deleted from the rebuild state machine outright; going through a cache built for a different window was indirection that changed the answer. Also splits the empty-result diagnostics: "no resolved labels" (a windowing/data fault) is now distinguished from "labels present, every call Neutral" (a calibration verdict). The first version reported the second message for both, which mislabelled this very bug as a calibration outcome in the same breath as reporting scored=0. Honest limitation, stated in the code too: the replay window includes bars the model trained on, so a replay-minted ladder is measured partly in-sample and will read stronger than a holdout-measured one. It is replaced by the genuine article at the next completed scoring pass; until then it is what makes a restarted deployed model able to vote at all. Compile-verified in _claude_stage: 0 errors, 0 warnings. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-25 15:22:01 -04:00
//--- Run the deployed net over the window. StartChartSignalRescan queues; the drain is handled by
//--- the RescanPending() branch above this in PollTraining, which is why returning true here is
//--- correct - the next slices go there, and completion arrives via OnChartRescanComplete.
if(m_deployedRebuildStage == 1 && !m_chartUI.RescanPending())
feat(vote): replay pass rebuilds a deployed model's ladder without retraining The previous commit persisted the tier ladder, which fixes this going forward but did nothing for models whose .stats predates WST7 - they still had to retrain to mint one. They never did. Every number a converged model needs in order to vote is a pure function of weights already on disk plus labels derivable from the chart, so replay them: stage 1 build the label cache (existing chunked prebuild) stage 2 rescan history (existing chunked rescan, deployed net) stage 3 score + rank + persist (one walk over two arrays) ScoreReplayFromCache() walks m_arrowSignalCache against m_labelCacheBuy/Sell, fills the same m_oosTierFired/Hits and per-class totals pass 3 fills, and hands them to RankTiersFromOos() - deliberately feeding the existing ranker rather than reimplementing it. The shrinkage, the chance reference and the module trust weight are subtle enough that a second copy would drift, and a ladder measured by a slightly different rule would be silently incomparable with every ladder training produced. AdvanceDeployedRebuild() sequences the three stages off the timer. It has to be a sequence: stages 1 and 2 are each minutes of work draining in time-boxed slices, and stage 2's output is meaningless until stage 1 has labels to score against. The previous version ran the rescan with no labels at all, which is why it could only ever rebuild arrows and never the ladder - the thing actually blocking the vote. The result is written to .stats immediately. The failure being repaired is state that lived in memory and was never written down; recomputing it and not saving it would repeat that exactly. Also routes every rescan completion through one hook, so there is a single place that knows what a finished rescan means - republish for a manual one, score and rank for a rebuild. Compile-verified in _claude_stage: 0 errors, 0 warnings. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-25 15:01:24 -04:00
{
fix(replay): resolve labels inline - the prebuilt cache's window never overlapped the rescan The 15:13 session proved the replay pass ran end-to-end on all 24 models and scored ZERO labelled bars on every one of them, while each rescan sat on ~5000 scored predictions (~2755 Buy / ~2232 Sell). The two windows never overlapped: StartLabelCachePrebuild deliberately keeps a CONVERGED model's dtStudied watermark (it gates inference recency and must not move), so the prebuild's window was the handful of bars since the last studied bar - all with uncommitted pivots, hence "label cache pre-built - Buy: 0 | Sell: 0 | Neutral: 0" on every member. The label never needed a cache. SwingPivotDirectionLabel(idx) is a pure function of the ZigZag/Close/ATR buffers the rescan itself refreshes over exactly the scoring window, and m_lastLabelLifespan == 0 is its own unresolved flag - the same finality gate the cache applies, applied directly. ScoreReplayFromCache now resolves each bar's label inline and the label-prebuild stage is deleted from the rebuild state machine outright; going through a cache built for a different window was indirection that changed the answer. Also splits the empty-result diagnostics: "no resolved labels" (a windowing/data fault) is now distinguished from "labels present, every call Neutral" (a calibration verdict). The first version reported the second message for both, which mislabelled this very bug as a calibration outcome in the same breath as reporting scored=0. Honest limitation, stated in the code too: the replay window includes bars the model trained on, so a replay-minted ladder is measured partly in-sample and will read stronger than a holdout-measured one. It is replaced by the genuine article at the next completed scoring pass; until then it is what makes a restarted deployed model able to vote at all. Compile-verified in _claude_stage: 0 errors, 0 warnings. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-25 15:22:01 -04:00
if(!StartChartSignalRescan())
feat(vote): replay pass rebuilds a deployed model's ladder without retraining The previous commit persisted the tier ladder, which fixes this going forward but did nothing for models whose .stats predates WST7 - they still had to retrain to mint one. They never did. Every number a converged model needs in order to vote is a pure function of weights already on disk plus labels derivable from the chart, so replay them: stage 1 build the label cache (existing chunked prebuild) stage 2 rescan history (existing chunked rescan, deployed net) stage 3 score + rank + persist (one walk over two arrays) ScoreReplayFromCache() walks m_arrowSignalCache against m_labelCacheBuy/Sell, fills the same m_oosTierFired/Hits and per-class totals pass 3 fills, and hands them to RankTiersFromOos() - deliberately feeding the existing ranker rather than reimplementing it. The shrinkage, the chance reference and the module trust weight are subtle enough that a second copy would drift, and a ladder measured by a slightly different rule would be silently incomparable with every ladder training produced. AdvanceDeployedRebuild() sequences the three stages off the timer. It has to be a sequence: stages 1 and 2 are each minutes of work draining in time-boxed slices, and stage 2's output is meaningless until stage 1 has labels to score against. The previous version ran the rescan with no labels at all, which is why it could only ever rebuild arrows and never the ladder - the thing actually blocking the vote. The result is written to .stats immediately. The failure being repaired is state that lived in memory and was never written down; recomputing it and not saving it would repeat that exactly. Also routes every rescan completion through one hook, so there is a single place that knows what a finished rescan means - republish for a manual one, score and rank for a rebuild. Compile-verified in _claude_stage: 0 errors, 0 warnings. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-25 15:01:24 -04:00
{
fix(replay): resolve labels inline - the prebuilt cache's window never overlapped the rescan The 15:13 session proved the replay pass ran end-to-end on all 24 models and scored ZERO labelled bars on every one of them, while each rescan sat on ~5000 scored predictions (~2755 Buy / ~2232 Sell). The two windows never overlapped: StartLabelCachePrebuild deliberately keeps a CONVERGED model's dtStudied watermark (it gates inference recency and must not move), so the prebuild's window was the handful of bars since the last studied bar - all with uncommitted pivots, hence "label cache pre-built - Buy: 0 | Sell: 0 | Neutral: 0" on every member. The label never needed a cache. SwingPivotDirectionLabel(idx) is a pure function of the ZigZag/Close/ATR buffers the rescan itself refreshes over exactly the scoring window, and m_lastLabelLifespan == 0 is its own unresolved flag - the same finality gate the cache applies, applied directly. ScoreReplayFromCache now resolves each bar's label inline and the label-prebuild stage is deleted from the rebuild state machine outright; going through a cache built for a different window was indirection that changed the answer. Also splits the empty-result diagnostics: "no resolved labels" (a windowing/data fault) is now distinguished from "labels present, every call Neutral" (a calibration verdict). The first version reported the second message for both, which mislabelled this very bug as a calibration outcome in the same breath as reporting scored=0. Honest limitation, stated in the code too: the replay window includes bars the model trained on, so a replay-minted ladder is measured partly in-sample and will read stronger than a holdout-measured one. It is replaced by the genuine article at the next completed scoring pass; until then it is what makes a restarted deployed model able to vote at all. Compile-verified in _claude_stage: 0 errors, 0 warnings. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-25 15:22:01 -04:00
Print(ID + ": WARNING - could not start the replay rescan (no servable history, or the"
" model is not ready to infer). This member stays silent; it will rebuild on the"
" next attach or the next training pass.");
m_deployedRebuildStage = 2;
return false;
feat(vote): replay pass rebuilds a deployed model's ladder without retraining The previous commit persisted the tier ladder, which fixes this going forward but did nothing for models whose .stats predates WST7 - they still had to retrain to mint one. They never did. Every number a converged model needs in order to vote is a pure function of weights already on disk plus labels derivable from the chart, so replay them: stage 1 build the label cache (existing chunked prebuild) stage 2 rescan history (existing chunked rescan, deployed net) stage 3 score + rank + persist (one walk over two arrays) ScoreReplayFromCache() walks m_arrowSignalCache against m_labelCacheBuy/Sell, fills the same m_oosTierFired/Hits and per-class totals pass 3 fills, and hands them to RankTiersFromOos() - deliberately feeding the existing ranker rather than reimplementing it. The shrinkage, the chance reference and the module trust weight are subtle enough that a second copy would drift, and a ladder measured by a slightly different rule would be silently incomparable with every ladder training produced. AdvanceDeployedRebuild() sequences the three stages off the timer. It has to be a sequence: stages 1 and 2 are each minutes of work draining in time-boxed slices, and stage 2's output is meaningless until stage 1 has labels to score against. The previous version ran the rescan with no labels at all, which is why it could only ever rebuild arrows and never the ladder - the thing actually blocking the vote. The result is written to .stats immediately. The failure being repaired is state that lived in memory and was never written down; recomputing it and not saving it would repeat that exactly. Also routes every rescan completion through one hook, so there is a single place that knows what a finished rescan means - republish for a manual one, score and rank for a rebuild. Compile-verified in _claude_stage: 0 errors, 0 warnings. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-25 15:01:24 -04:00
}
}
fix(replay): resolve labels inline - the prebuilt cache's window never overlapped the rescan The 15:13 session proved the replay pass ran end-to-end on all 24 models and scored ZERO labelled bars on every one of them, while each rescan sat on ~5000 scored predictions (~2755 Buy / ~2232 Sell). The two windows never overlapped: StartLabelCachePrebuild deliberately keeps a CONVERGED model's dtStudied watermark (it gates inference recency and must not move), so the prebuild's window was the handful of bars since the last studied bar - all with uncommitted pivots, hence "label cache pre-built - Buy: 0 | Sell: 0 | Neutral: 0" on every member. The label never needed a cache. SwingPivotDirectionLabel(idx) is a pure function of the ZigZag/Close/ATR buffers the rescan itself refreshes over exactly the scoring window, and m_lastLabelLifespan == 0 is its own unresolved flag - the same finality gate the cache applies, applied directly. ScoreReplayFromCache now resolves each bar's label inline and the label-prebuild stage is deleted from the rebuild state machine outright; going through a cache built for a different window was indirection that changed the answer. Also splits the empty-result diagnostics: "no resolved labels" (a windowing/data fault) is now distinguished from "labels present, every call Neutral" (a calibration verdict). The first version reported the second message for both, which mislabelled this very bug as a calibration outcome in the same breath as reporting scored=0. Honest limitation, stated in the code too: the replay window includes bars the model trained on, so a replay-minted ladder is measured partly in-sample and will read stronger than a holdout-measured one. It is replaced by the genuine article at the next completed scoring pass; until then it is what makes a restarted deployed model able to vote at all. Compile-verified in _claude_stage: 0 errors, 0 warnings. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-25 15:22:01 -04:00
return true;
feat(vote): replay pass rebuilds a deployed model's ladder without retraining The previous commit persisted the tier ladder, which fixes this going forward but did nothing for models whose .stats predates WST7 - they still had to retrain to mint one. They never did. Every number a converged model needs in order to vote is a pure function of weights already on disk plus labels derivable from the chart, so replay them: stage 1 build the label cache (existing chunked prebuild) stage 2 rescan history (existing chunked rescan, deployed net) stage 3 score + rank + persist (one walk over two arrays) ScoreReplayFromCache() walks m_arrowSignalCache against m_labelCacheBuy/Sell, fills the same m_oosTierFired/Hits and per-class totals pass 3 fills, and hands them to RankTiersFromOos() - deliberately feeding the existing ranker rather than reimplementing it. The shrinkage, the chance reference and the module trust weight are subtle enough that a second copy would drift, and a ladder measured by a slightly different rule would be silently incomparable with every ladder training produced. AdvanceDeployedRebuild() sequences the three stages off the timer. It has to be a sequence: stages 1 and 2 are each minutes of work draining in time-boxed slices, and stage 2's output is meaningless until stage 1 has labels to score against. The previous version ran the rescan with no labels at all, which is why it could only ever rebuild arrows and never the ladder - the thing actually blocking the vote. The result is written to .stats immediately. The failure being repaired is state that lived in memory and was never written down; recomputing it and not saving it would repeat that exactly. Also routes every rescan completion through one hook, so there is a single place that knows what a finished rescan means - republish for a manual one, score and rank for a rebuild. Compile-verified in _claude_stage: 0 errors, 0 warnings. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-25 15:01:24 -04:00
}
//+------------------------------------------------------------------+
//| |
//+------------------------------------------------------------------+
void CExpertSignalAIBase::OnChartEventHandler(const int id,
const long &lparam,
const double &dparam,
const string &sparam)
{
//--- Match THIS instance's id only. See STUDY_EVENT_ID_BASE (ExpertSignalAIBase.mqh) for the
//--- full failure shape.
fix(ensemble): responsive panel + synchronized eras + combined-vote accuracy Four user-reported/requested items, one root cause chain: 1) DEAD CONTROL PANEL in AI_HYBRID mode. All members posted custom event id 1 and handled id 1001, and CExpertCustom broadcasts every chart event to every filter - so each posted event ran a train chunk in ALL N members (N*N chunks per round) and the chart thread never idled long enough to deliver clicks/drags. profiling.csv: 99.45% of time in OnChartEventHandler. Fix: per-instance study-event ids (STUDY_EVENT_ID_BASE + construction order, offset above the Controls library's ON_* codes - id 1 was also ON_DBL_CLICK, so panel double-clicks fired training chunks). ArmStudyEvent() is the single post site; lost-event watchdog replaces the accidental sibling-clears-my-flag rescue. 2) WARM-UP DUPLICATION. The auto-tune sweep is deterministic over identical features/labels, and it ends in the full MI diagnostic suite, which the MI-share gate never intercepted on the sweep path - four members ran four identical ~36s sweep+report blocks. First member publishes outcome (g_ensembleChartTuneDone/Installed/Settings); the rest apply it and skip both. 3) DEINIT STRANDED PANEL+ARROWS (user repro 18:52). Root cause from the log: the 4,500ms budget runs from MetaTrader's stop REQUEST - a heavy autosave in flight ate it, OnDeinit got ~430ms and died in the first member's arrow persist ("Abnormal termination" 432ms in). Fix: early visible-UI sweep (native prefix deletes for status/panel/dialog) right after ClearStatusLabel, and a fast path for still-training models - their arrows are re-rendered every era, so they get one bulk purge instead of scan+atomic-write in the death window. 4) ENSEMBLE FEATURES (user requests): era BARRIER - members advance era by era together; a member ahead of the slowest still-training member declines Train() calls and its chunk budget is donated (TRAIN_TIME_BUDGET_MS = 120/activeTrainers, UI headroom constant). COMBINED-VOTE OOS SCORE - each member's pass-3 scan contributes its adjusted per-bar decision (0.0 on abstain) to a shared row buffer; the last member to finish the era scores the averaged vote vs the mirrored Min_Vote_Open against the same target-before-stop outcomes members grade themselves on, publishing an "Ensemble vote" line on the aggregated panel. Member headlines now carry their lifetime win rate with break-even. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-16 19:06:04 -04:00
if(id == CHARTEVENT_CUSTOM + m_studyEventId)
{
fix(shutdown): make ExitPolicy public, and stop every long loop the moment MT5 asks Two things, one of which was a compile error. 1. ExitPolicy() was declared in the protected block but is pushed in from Warrior_EA.mq5:770. Moved to public beside the other EA-facing setters. 2. Chart objects surviving OnDeinit. The 4,500 ms teardown budget is measured from the STOP REQUEST, not from OnDeinit's first line, and OnDeinit cannot begin until whatever is in flight returns - so a scan still running after _StopFlag is raised does not delay the cleanup, it SPENDS it, and the purge never gets its turn. New CExpertSignalAIBase::ShutdownRequested() = IsStopped() || m_shutdownInProgress. Deliberately NOT m_trainingStopRequested: that latches, and a latched flag would permanently disable scans that must run again on the next Start. Guarded, longest first: - TuneIndicatorsByFilter - per candidate, restoring the OPERATOR's settings on the way out (best[] is mutated in place; the tuner otherwise keeps the last trial's parameters, which nothing chose). - ReportBarrierGeometryScan - per pairing, breaking to ONE restore point so m_barrierScanLiveLabels can never be left true (that makes ComputeLabelForBar read the last candidate's multiples as the configured geometry). - ReportFeatureLabelInformation / ReportExcursionInformation / lag profile - nulls ABANDON rather than truncate: fewer draws is not a smaller null, it is a wrong one, and p shifts toward significance. m_dirEvidence staying false is the safe direction. - SimulateExitPolicyOutcomes - zeroes its accumulators so the divergence line is dropped instead of latching a partial expectancy as the run's only report. - ReportGeometryExpectancyScan - per ladder rung. - HttpGet - one choke point for up to a dozen blocking WebRequests per first-pass Update(). An in-flight request cannot be cancelled; refusing to start another is the whole remedy. - PollTraining, OnChartEventHandler's study event, TuneIndicatorsAndTrain - entry points, so a queued event cannot open an era during teardown. TuneIndicatorsAndTrain's guard is the first statement, ahead of the m_tuneFilterDone / g_ensembleChartTuneDone latches. - OnTick / OnTimer / OnChartEvent. Training's own bar loops already honoured this (pass 1 per bar, passes 2/2.5/3 yield on a 120 ms budget); the warm-up scans did not, and they are the longest uninterruptible stretches the EA has. StopTraining() is unchanged: the operator's Stop still finalises synchronously. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-17 17:03:11 -04:00
//--- The study event IS the training driver, so a queued one landing after the stop request would
//--- open a whole era inside the teardown window. Checked here as well as in Warrior_EA.mq5's
//--- OnChartEvent because CExpertCustom re-posts these between members.
if(ShutdownRequested())
return;
TuneIndicatorsAndTrain(lparam);
bEventStudy = false;
OnTickHandler();
}
}
//--- AcquireConfigLock/ReleaseConfigLock bodies now live on CConfigLock (m_configLock) - see
//--- Expert\ConfigLock\ConfigLock.mqh's class comment, including g_initFatalReason's declaration.
#endif