feat(target): delete the barrier/geometry stack - the label is the verdict
Step 3 of the swing-pivot plan, whole-hog. The swing label is now the ONE
target and the era verdict is precision + recall per class against the
label's own base rate - no win rate, no break-even, no expectancy, no
geometry anywhere in training.
DELETED
- Expert/Excursion/ (4), Expert/BarrierHorizon/ (4), GeometrySweep,
FirstPassageLadder, Labeling/TripleBarrier.mqh (CLabelOverlap survives
in Labeling/LabelOverlap.mqh), 3 test EAs.
- TripleBarrierLabel + walk, fractal label, geometry derivation/scan/
adoption, exit-policy replay, excursion MI targets, the drift verdict
(DIRECTION_INTELLIGENT), the recall floor, balanced-accuracy telemetry,
the barrier defines, the .cfg geometry adopt (slots kept as zeros for
the positional layout), the derived-geometry live-order override.
- TRAINING_TARGET input/enum: direction models are always swing; META2
re-keys the meta head onto label agreement (descriptor loses its two
geometry slots).
REWORKED
- Labels.mqh (1795 -> ~370 lines): AdvanceSwingLabelState with
FINALITY-GATED CACHING - an unresolved bar (pivot pair uncommitted) is
never cached, so it can never freeze as a false Neutral; training,
calibration, OOS scoring and online learning all skip unresolved bars.
- SDeployVerdict: significance-only; SOosTally chance = larger
directional class share; pooled gate poolability = timeframe (record v2).
- Purge/embargo/declustering gaps: the measured mean label resolution
lag (LabelResolutionBars), not a barrier horizon.
- Pool purge key + backfill DB rows: marked at the bar the label
resolved on (m_labelResolveAge), not a fabricated barrier touch.
- Online learning frontier: finality, not a horizon delay.
- m_bestBalancedOos -> m_bestSelectionScore, m_erasSinceBestBalanced ->
m_erasSinceBest, ensemble vote outcome arrays -> label arrays.
STEP 4 folded in: Entry_Multiplier / SL_Mode / TP_Mode / tradingdirection
are inputs again - trade management is the tester GA's search space.
Fingerprints: every direction model re-keys (TGT:SWG1 now unconditional,
CUT token gone); META1 -> META2. Full retrain, as planned.
Compile-verified in _claude_stage: Warrior_EA + both surviving test EAs,
0 errors, 0 warnings each.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-24 20:42:31 -04:00
|
|
|
//+------------------------------------------------------------------+
|
2026-08-22 00:30:14 -04:00
|
|
|
//+------------------------------------------------------------------+
|
2026-08-01 11:27:28 -04:00
|
|
|
//| Lifecycle.mqh |
|
|
|
|
|
//| |
|
2026-08-24 04:39:17 -04:00
|
|
|
//| Construction/destruction (the composition root - every |
|
|
|
|
|
//| collaborator's Bind() call lives here), the CExpertSignal vote |
|
|
|
|
|
//| API (LongCondition/ShortCondition/ConfidenceTier/pattern |
|
|
|
|
|
//| weights), and tick + chart-event dispatch. The per-config chart |
|
|
|
|
|
//| lock now lives on CConfigLock - see Expert\ConfigLock\ConfigLock.mqh. |
|
2026-08-01 11:27:28 -04:00
|
|
|
//+------------------------------------------------------------------+
|
|
|
|
|
#ifndef WARRIOR_AIBASE_LIFECYCLE_MQH
|
|
|
|
|
#define WARRIOR_AIBASE_LIFECYCLE_MQH
|
|
|
|
|
//+------------------------------------------------------------------+
|
|
|
|
|
//| Constructor |
|
|
|
|
|
//+------------------------------------------------------------------+
|
|
|
|
|
//--- These are only fallback defaults for a fresh object before the EA's OnInit() applies the
|
|
|
|
|
//--- active input values via the public setters in Warrior_EA.mq5. The input-driven values are the
|
|
|
|
|
//--- source of truth for the actual run configuration.
|
|
|
|
|
CExpertSignalAIBase::CExpertSignalAIBase(void) :
|
|
|
|
|
ID("NULL"),
|
|
|
|
|
m_neuronsCount(0),
|
|
|
|
|
m_minTrainYear(1970),
|
|
|
|
|
m_optimizationAlgo(TrainingOptimizer), // see the member declaration comment
|
|
|
|
|
//--- Placeholder only; InitNeuralNetwork() replaces it with ComputeFirstLayerWidth() before anything
|
|
|
|
|
//--- reads it. Deliberately the floor rather than 0, so a hypothetical path that built a topology
|
|
|
|
|
//--- without going through init would produce a small usable net instead of a zero-width layer.
|
|
|
|
|
m_initialNeuronsCount(FIRST_LAYER_MIN_WIDTH),
|
|
|
|
|
m_outputNeuronsCount(OUTPUT_CLASSIFICATION),
|
2026-08-22 00:25:52 -04:00
|
|
|
//--- Frozen. Nothing reads these to build a topology any more - the taper derives its own
|
|
|
|
|
//--- endpoints (BuildFreshTopology) - but they still occupy positional slots in the .cfg sidecar
|
|
|
|
|
//--- and the weights fingerprint.
|
2026-08-01 11:27:28 -04:00
|
|
|
m_minNeuronsCount(MIN_NEURONS_20),
|
|
|
|
|
m_neuronsReduction(RF_70),
|
|
|
|
|
m_hiddenLayersCount(3),
|
|
|
|
|
m_lstmHiddenSize(32),
|
|
|
|
|
m_convFilterCount(16),
|
|
|
|
|
m_historyBars(14),
|
|
|
|
|
m_fractalPeriods(5),
|
|
|
|
|
m_pattern_0(25),
|
|
|
|
|
m_pattern_1(50),
|
|
|
|
|
m_pattern_2(75),
|
|
|
|
|
m_pattern_3(100),
|
2026-08-15 16:50:36 -04:00
|
|
|
m_ensembleMember(false),
|
2026-08-15 16:54:43 -04:00
|
|
|
m_ensemblePanelSlot(-1),
|
2026-08-01 11:27:28 -04:00
|
|
|
m_useVolumes(true),
|
|
|
|
|
m_useTime(true),
|
|
|
|
|
m_useATR(true),
|
|
|
|
|
m_useMA(false),
|
|
|
|
|
m_useSwingContext(false),
|
|
|
|
|
m_useNews(false),
|
fix(signals): revive a dead MA model, and demote Sanyaku from state to event
Two defects surfaced by research/test_classic.py, both verified fixed by re-running the
transcription against 178k bars of EURUSD H1.
CSignalMA model 1 could never fire. For any recursive average - and MA_TYPE_EMA is the
shipped default - MA(i) = a*Close(i) + (1-a)*MA(i+1), so
DiffMA(i) = a * (Close(i) - MA(i+1))
DiffCloseMA(i) = (1-a) * (Close(i) - MA(i+1))
are positive multiples of one quantity and always share a sign. Model 1 asks for a close
BELOW a RISING average, which is precisely the combination that identity forbids: 0.000%
of bars, either direction, any symbol. The MQL5 standard library this was ported from
defaults to MODE_SMA, where the two are merely correlated - the bug arrived with the EMA
default, not with the port. Reading the slope one bar back (DiffMAPrev) breaks the tie for
every MA type while keeping the model's stated meaning. Now fires on 7.92% of bars.
CSignalIchimoku model 11 fired on 27% of bars at weight 100. Sanyaku is three standing
STATES conjoined with no transition term, so it held across long stretches - and being
last in the if-chain at the top weight, the module's highest-conviction reading was also
its most common one, overwriting all eight event models below it on a quarter of all bars.
The old comment rejected an event form because "demanding all three flip on the same bar
would fire almost never" - true, but that is not the alternative. Kouten is the TURN: the
ALIGNMENT transitions, and only one role need change for it to. Testing !Sanyaku(idx+1)
fires once per aligned stretch. Now 2.17%, in line with Kumo breakout (2.4%) and the
strong TK cross (1.1%). DataReady() extended one bar deeper to cover the lookback.
Neither pattern showed edge before or after; this is about the models meaning what they
say and the vote not being dominated by a constant.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-01 17:14:34 -04:00
|
|
|
m_useCrossAsset(false),
|
feat(ai): spread as a volatility-regime feature, and fix a stale-index cache in both new blocks
Adds spread/ATR and the spread change ratio as network inputs (EnableSpreadFeature,
default on). Spread is the one microstructure channel that is both FX-available and
genuinely historical in the Strategy Tester - "during testing, the spread is not modeled
but is taken from historical data" - so unlike swap, signed tick flow or depth of market it
is something a backtest can honestly validate.
What it encodes, stated precisely because the raw measurement overstates it.
research/test_spread.py found spr/atr the strongest single feature in this codebase, on 5
of 8 instrument/geometry cells at 2-4x any volume feature. But the barrier LABEL charges
the spread inside its own barriers, so a wide-spread bar is mechanically likelier to
resolve as a loss and the feature would partly be predicting its own cost model. Relabelling
at zero cost and re-measuring the identical feature showed 20-40% of it WAS that tautology
and the majority was not (XAUUSD retained 97%). What survives is a volatility-regime
reading: spread is near-fixed while ATR is not, so the ratio runs high exactly when
realised volatility is below its own ATR estimate, which genuinely predicts whether
ATR-scaled barriers get reached. It is UNSIGNED - Neutral-vs-directional only, never a side.
Also fixes a stale-index bug I introduced with the cross-asset panel and had just repeated
in the spread series. Both cached on length alone:
if(m_crossAsset.Bars() >= bars) return true;
MQL5 series indices are relative to NOW, so one new closed candle shifts every index by
one. Keyed only on length, the panel keeps serving its index 0 as a bar that is no longer
the newest, and every cross-asset value is read one bar out of step with the price features
sitting beside it in the same vector - silently, with no error and no shape change. This is
the same class of defect as the dtStudied watermark behind the zero-direction backtests.
Both now carry a datetime anchor on m_Time.GetData(0), the same invalidation key the
label/feature bar caches already use.
And a performance fix that fell out of it: with correct invalidation the panel rebuilds on
every new bar, and RefreshConvergedSignal runs per bar - which in the tester would mean one
full multi-symbol resample per simulated bar at training depth. Inference only reads bars
0..m_historyBars-1 plus the panel's own slow window, so it now requests exactly that. The
cache check is >=, so a deeper panel left from training still satisfies it.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-01 17:42:40 -04:00
|
|
|
m_useSpreadFeature(false),
|
2026-08-11 21:29:14 -04:00
|
|
|
m_crossAssetPairsPinned(""),
|
|
|
|
|
m_crossAssetCfgSaved(false),
|
2026-08-16 13:39:00 -04:00
|
|
|
m_useAltData(false),
|
2026-08-16 15:12:54 -04:00
|
|
|
m_altDataEnabled(true),
|
2026-08-16 20:04:13 -04:00
|
|
|
m_altDataLateWarned(false),
|
2026-08-16 13:39:00 -04:00
|
|
|
m_altDataNamesPinned(""),
|
2026-08-01 11:27:28 -04:00
|
|
|
m_newsFeatureWindowMinutes(60),
|
|
|
|
|
m_autoTuneIndicators(false),
|
|
|
|
|
m_indicatorsPtr(NULL),
|
|
|
|
|
Net(NULL),
|
|
|
|
|
TempData(NULL),
|
|
|
|
|
dError(-1),
|
|
|
|
|
dUndefine(0),
|
|
|
|
|
dForecast(0),
|
|
|
|
|
dPrevSignal(0),
|
|
|
|
|
m_refreshOk(0),
|
|
|
|
|
m_refreshFailFeatures(0),
|
|
|
|
|
m_refreshFailShort(0),
|
|
|
|
|
m_refreshBuy(0),
|
|
|
|
|
m_refreshSell(0),
|
|
|
|
|
m_refreshNeutral(0),
|
|
|
|
|
m_voteGateBlocked(0),
|
|
|
|
|
m_voteGatePassed(0),
|
|
|
|
|
m_voteGateCompleteAtFirst(-1),
|
|
|
|
|
m_voteGateLoadedAtFirst(-1),
|
|
|
|
|
m_signalClusterWindow(6),
|
|
|
|
|
m_nmsLiveBuyTime(0),
|
|
|
|
|
m_nmsLiveSellTime(0),
|
|
|
|
|
m_nmsLiveBuyAccept(false),
|
|
|
|
|
m_nmsLiveSellAccept(false),
|
|
|
|
|
m_nmsLiveKeptTime(0),
|
|
|
|
|
m_nmsLiveKeptDir(Neutral),
|
|
|
|
|
m_nmsLiveKeptConf(0),
|
|
|
|
|
dtStudied(0),
|
perf(deinit): I/O-free chart cleanup, dead-panel purge, skip clean weight saves
The 18:23 terminal close (20260825.log) killed two of six charts inside
OnDeinit: they printed "shutting down" then nothing for 5.9 s until
"Abnormal termination", stranding ~700 objects each - including the one
family no prefix sweep can reach, the control panel (CAppDialog names
its 15 objects <numeric instance id><control>, and a re-attach mints a
new id, so a killed panel is a permanent ghost; XTIUSD carried one
across sessions). The stall sat in the two file writes that preceded
all visible cleanup while the four sibling charts flooded the same
2013-era disk - the ~4x18MB-per-chart shutdown weight saves.
Three changes:
1. OnDeinit touches no file until the chart is clean. CVoteArrowStore
splits Save() into Snapshot() (the chart scan, in memory) and
WriteSnapshot() (the disk half, consuming). New order: status label,
vote-arrow snapshot, prefix sweep, panel destroy - all object ops -
then member sidecars, final sweep, timings, and only then the
visibility file, the vote-arrow write and the weight saves.
2. PurgeOrphanedPanelObjects() at OnInit: deletes numeric-prefix
CAppDialog ghosts by name (6 chrome + 9 buttons), qualifying a
prefix only when >=4 of OUR button names carry it, so a foreign
dialog sharing stock chrome names is never touched.
3. m_netDirty: set by every net mutation (both backProp sites, both
RestoreWeights sites, online learning conservatively, panel reset),
cleared only on a successful Net.Save. Shutdown AND the per-bar
autosave now skip the ~18MB write when the net is provably unchanged
- for converged ensembles that is every save - which removes the
very flood that starved the sibling charts. .stats still writes
every time (small; carries the vote record and calibration). A
skipped save leaves the .nnw header dtStudied stale, which is the
already-handled attach-after-offline-gap case.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-25 18:46:51 -04:00
|
|
|
m_netDirty(true),
|
2026-08-01 11:27:28 -04:00
|
|
|
m_eraCount(0),
|
|
|
|
|
m_trainingComplete(false),
|
|
|
|
|
m_inferenceOnly(false),
|
|
|
|
|
m_modelLoadedFromDisk(false),
|
|
|
|
|
m_topologySuperseded(false),
|
|
|
|
|
m_mqlInferenceValidated(false),
|
|
|
|
|
m_freezePriorCalibration(false),
|
|
|
|
|
bEventStudy(false),
|
|
|
|
|
m_oosSplitPct(30),
|
|
|
|
|
dOosError(-1),
|
|
|
|
|
dOosForecast(0),
|
|
|
|
|
m_oosSamples(0),
|
|
|
|
|
m_cumIsCorrect(0),
|
|
|
|
|
m_cumIsTotal(0),
|
|
|
|
|
m_cumOosCorrect(0),
|
|
|
|
|
m_cumOosTotal(0),
|
|
|
|
|
m_oosOutSpreadSum(0),
|
|
|
|
|
m_oosOutCount(0),
|
feat(diagnostics): split a reported "Neutral" into CHOSE vs TIED - they need opposite fixes
ApplyClassificationSoftmax() requires a STRICT majority over both rivals and
sends every tie, 2-way or 3-way, to Neutral. So "OOS recall Neutral:100%" is
two completely different events sharing one label:
CHOSE - the net genuinely ranks Neutral highest. A class-prior/label problem.
TIED - the top two are EXACTLY equal, so the net expressed no preference and
the tie-break reported Neutral. A SATURATION problem: the head is
SIGMOID, and a saturated sigmoid returns exactly 0.0f or 1.0f in the
DLL's float32, so two classes pinned to the same rail compare equal
and the bar is silently discarded.
Nothing in the logs could tell them apart, and the fixes point opposite ways.
Eras 1-25 of the 2026-08-17 solo PAI run read "Neutral 100%" at spread avg 0.99
- fully saturated - and broke out at era 27 as the spread fell to 0.75. That is
consistent with EITHER story. The user reports the Neutral phase on most runs,
so it is worth four longs to stop guessing.
Four per-era counters on the pass 3 OOS walk, reported as:
| Neutral CHOSE 12.4% / TIED 38.1% (of which B=S 1204) | rail 61.2%
m_oosNeutralStrict - Neutral strictly highest
m_oosNeutralTie - no strict winner; the tie-break produced Neutral
m_oosTieBuySell - the costly subset: Buy and Sell tied AT the top, i.e. a
DIRECTIONAL reading thrown away by float equality
m_oosRailBars - any raw output sitting on a sigmoid asymptote, the
saturation that makes exact ties possible at all
Read on the RAW logits, before ApplyClassificationSoftmax() overwrites TempData
in place. Legitimate because softmax is strictly monotone: it cannot change the
ordering and cannot break a tie either, so the raw reading and the decision
always agree. Placed alongside the existing min/max/spread capture so all the
output diagnostics describe the same values.
Measurement only - no decision path reads these.
NOT COMPILED - user compiles.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-17 00:06:54 -04:00
|
|
|
m_oosNeutralStrict(0),
|
|
|
|
|
m_oosNeutralTie(0),
|
|
|
|
|
m_oosTieBuySell(0),
|
|
|
|
|
m_oosRailBars(0),
|
2026-08-01 11:27:28 -04:00
|
|
|
m_countBuySignals(0),
|
|
|
|
|
m_countSellSignals(0),
|
|
|
|
|
m_countNeutralSignals(0),
|
|
|
|
|
m_trueBuyCount(0),
|
|
|
|
|
m_trueSellCount(0),
|
|
|
|
|
m_trueNeutralCount(0),
|
|
|
|
|
m_logitAdjustLogged(false),
|
|
|
|
|
m_logitAdjustSkipWarned(false),
|
|
|
|
|
m_prevEraTrueBuyCount(0),
|
|
|
|
|
m_prevEraTrueSellCount(0),
|
|
|
|
|
m_prevEraTrueNeutralCount(0),
|
|
|
|
|
m_confidenceCalScale(1.0),
|
fix(telemetry): one ensemble member had never printed a single era line, in any run on record
The era-progress rate limit was a function-scope `static`:
static uint lastProgressLogTick = 0;
shouldLogProgress = (nowTick - lastProgressLogTick >= 5000);
In MQL5 that is ONE variable for the whole build, not one per object. A 5s
limit meant to keep a single model's console readable was therefore a limit
across the WHOLE ENSEMBLE, and it did not distribute fairly - it starved
whichever member finishes last, every era, deterministically.
Measured on today's run: the era barrier releases the members together and
LSTM landed 2.06s, 2.43s and 2.28s behind ConvLSTM on eras 1-3, against a 5s
window it could never reach. LSTM printed zero era lines. PAI, CONV and HYB
printed all of theirs - 3 each this run, 17 each in the 10:00 run, LSTM 0 in
both, and 0 again in the 08:32 run.
So one model in four has been training with NO per-era telemetry: no
per-class recall, no dW/W ratios, no zero-skill comparison, no deploy-bar
line. It was still doing the work - tier re-ranks, threshold fits and
exit-policy replays all appear on cadence - which is what made the hole look
like a grep that kept missing the line rather than a line that was never
written. It cost me the LSTM half of a gradient check earlier today.
Now a member, so each model rate-limits its own console output.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-20 14:48:56 -04:00
|
|
|
m_lastProgressLogTick(0),
|
2026-08-01 11:27:28 -04:00
|
|
|
m_priorBuy(0.0),
|
|
|
|
|
m_priorSell(0.0),
|
|
|
|
|
m_priorNeutral(0.0),
|
fix: NMS gates the TRADE, not just the arrow - one arrow is now one trade
NmsLiveAccept() appeared in exactly one place: wrapped around DrawObject().
It never touched dPrevSignal, and dPrevSignal is what LongCondition() /
ShortCondition() / SignedAIConfidence() read. So a declustered bar lost its
arrow and still opened a position.
Measured on SP500 H1 2026-08-09: CONV called a direction on 64% of bars,
so the ~500 bars visible on screen held ~320 decisions - and ~40 arrows
were drawn. Roughly one arrow per eight positions the EA would take.
And the survivors are not a random eighth. Rule 2 of the declustering
keeps the HIGHER-CONFIDENCE side of a cluster, so the visible set is
systematically the best member of each run. A chart showing the best of
every eight decisions and hiding the rest reads far better than the model
is - the same best-of-N selection error already corrected in the geometry
scan, the indicator tuner, the lag profile and the deploy gate, this time
on the display layer, where it is most likely to mislead the person
deciding whether to trade.
Fixed by neutralising dPrevSignal when NMS rejects, rather than adding a
"may trade" flag consulted at each read site: that leaves exactly ONE
definition of what the model decided this bar, so the arrow, the panel's
"Current signal", the confidence feeding sizing/SL/TP/trailing, the
refresh tally and the order itself cannot drift apart again.
Also reports the consequence instead of hiding it. Every OOS counter on
the era line still scores every directional call - a population ~8x larger
than what now trades - so the line carries a second figure:
| TRADED (declustered) NN% on N calls (edge +Npp)
replaying the identical rule over pass 3 (which walks OOS bars oldest to
newest, the same order the live sweep sees). Its cursors are separate
members from the live ones so a training pass can never disturb the live
chart's declustering.
Deliberately NOT switched into selectionScore yet. Declustering cuts
coverage from ~64% of bars to ~8%, well under
MIN_COVERAGE_FRACTION_OF_BASE_RATE, which would make every checkpoint
undeployable overnight - the minRR collision and the recall-floor catch-22
twice over. The floor gets re-derived from these measurements first.
Compiles clean: 0 errors, 0 warnings.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-09 10:22:31 -04:00
|
|
|
m_oosNmsFired(0),
|
|
|
|
|
m_oosNmsHits(0),
|
|
|
|
|
m_oosNmsLastBuyIdx(-1),
|
|
|
|
|
m_oosNmsLastSellIdx(-1),
|
|
|
|
|
m_oosNmsKeptIdx(-1),
|
|
|
|
|
m_oosNmsKeptConf(0.0),
|
|
|
|
|
m_oosNmsKeptDir(Neutral),
|
2026-08-01 11:27:28 -04:00
|
|
|
m_lastBuyFiredPrecPct(-1),
|
|
|
|
|
m_lastSellFiredPrecPct(-1),
|
|
|
|
|
m_lastBuyFired(0),
|
|
|
|
|
m_lastSellFired(0),
|
|
|
|
|
m_swingConfirmationBars(100),
|
|
|
|
|
m_maxErasPerRun(300),
|
refactor(chart): ChartUI is a real collaborator, not a raw-include partial (S2)
Expert/AIBase/ChartUI.mqh was 869 lines of method bodies of
CExpertSignalAIBase, #included after the class declaration - free to touch
any of its ~500 members. First of the eleven AIBase/*.mqh partials to come
out (fewest inbound edges - see the SOLID campaign session order), using
the same view+adapter shape already proven for CTrainingDataView.
CChartView (Expert/Chart/IChartView.mqh) is the abstract read/behaviour
surface a chart-rendering collaborator needs - identity, bar/model access,
the prediction cache, and the training/vote/meta scalars the panel and HUD
line summarise. CAIBaseChartView is the adapter the signal owns and binds
to itself (MQL5 gives a class exactly one base, so CExpertSignalAIBase
cannot implement the view directly). CChartUI is the real collaborator: it
owns the arrow-restore queue, the rescan queue/tally, the last-arrows-saved
count and the purge-mismatch latch as its own fields (verified via grep to
be touched nowhere else in Expert/), and reaches everything else - including
StartChartSignalRescan, moved in from its old inline home in the header
since it drives the exact same rescan state machine AdvanceChartSignalRescan
drains - through the view.
m_arrowSignalCache and m_signalClusterWindow stay on the signal: Training.mqh
writes the cache directly every era and the training-data view already reads
it, so moving it would mean rewriting Training.mqh's write sites too - out of
scope here. CChartUI reaches it through four bounds-checked accessors instead
of a raw member poke. All 10 public methods keep their exact signatures and
become one-line forwards on the signal, so no other file's call sites change
except Training.mqh's one era-end status refresh, which now reads
RefreshStatusLabel() rather than reaching into CChartUI's now-private
last-displayed-neuron cache directly.
Verified structurally, not compiled (never compile - the operator does, in
MetaEditor): brace balance checked on every touched/new file against HEAD,
and the view/adapter/impl method lists cross-diffed to confirm all 59
accessors match 1:1 across the interface, the adapter declaration and the
adapter body.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-08-23 20:03:52 -04:00
|
|
|
//--- The arrow-restore queue and the rescan queue/tally now default-construct on CChartUI itself
|
|
|
|
|
//--- (m_chartUI, declared below) - see its own constructor.
|
2026-08-01 11:27:28 -04:00
|
|
|
m_trainRunActive(false),
|
|
|
|
|
m_eraResumePending(false),
|
|
|
|
|
m_resumeBars(0),
|
|
|
|
|
m_resumeTotalIter(0),
|
|
|
|
|
m_resumeOosCutoff(0),
|
|
|
|
|
m_resumeBarIndex(0),
|
|
|
|
|
m_resumeAddLoop(false),
|
|
|
|
|
m_isTrainQueueCount(0),
|
|
|
|
|
m_isTrainCursor(0),
|
|
|
|
|
m_isPass2Active(false),
|
|
|
|
|
m_isPass2Done(false),
|
|
|
|
|
m_isPass3Active(false),
|
|
|
|
|
m_oosScoreIndex(0),
|
|
|
|
|
m_oosScoreStartIndex(0),
|
fix: the operating point was fitted on bars the net had memorized
FitDirConfThreshold harvested its margin histogram from pass 2's own
backprop samples. Pairing every fit against the same era's OOS result
shows what that measured:
PAI era 1 IS 25% cov @ 66.1% (-0.8pp) -> OOS 64% (-3pp) gap +2.1pp
PAI era 76 IS 90% cov @ 79.6% (+12.7pp) -> OOS 65% (-2pp) gap +14.6pp
LSTM era 9 IS 77% cov @ 81.6% (+14.6pp) -> OOS 63% (-4pp) gap +18.6pp
The gap grows monotonically while OOS stays flat, so within a handful of
eras the curve stops describing behaviour on unseen bars. That is fatal
here specifically, because the objective branches on the SIGN of
(p - break-even): the memorized curve reads +12pp at 95% coverage, so
coverage x (p - p0) correctly maximises coverage and returns ~0.02 - fire
on every bar. The "p < p0 -> get more selective" branch, which is the
actual regime and the entire point of 983a6a3, could never fire because IS
never showed p < p0.
Carve a calibration slice out of the IS span - DIR_CONF_CALIB_PCT_OF_IS,
purged from backprop by one label horizon on BOTH sides (the far-side
purge is not optional: without it the newest training bars carry labels
partly decided by price action inside the slice, putting the memorization
straight back into the curve). Score it in a new chunked pass 2.5, after
pass 2 has trained and before pass 3 grades - the only position where the
histogram is simultaneously not-trained-on, not-graded, and current with
the weights it will be applied to.
Costs 15% of the training data. Worth it beyond honesty: the deploy gate
needs dirPrecPct > chance + EDGE_MIN_SIGMAS*SE, and a threshold pinned
near zero dilutes any edge concentrated in the confident bars across every
bar the model calls, driving dirPrecPct toward chance by construction. A
threshold that can be selective is the only mechanism by which a small,
concentrated edge could ever clear that gate.
Also: a sparse histogram now KEEPS the previous threshold instead of
resetting to 0.0. A failed measurement must not decay to the most exposed
setting in the range.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-10 15:58:18 -04:00
|
|
|
m_isCalibActive(false),
|
|
|
|
|
m_isCalibDone(false),
|
|
|
|
|
m_calibIndex(0),
|
|
|
|
|
m_calibStartIndex(0),
|
2026-08-23 22:28:31 -04:00
|
|
|
//--- The excursion head's own state (m_excNet and its accumulators) now default-constructs on
|
|
|
|
|
//--- CExcursionHead (S4 - see its own constructor), same doctrine as CChartUI below.
|
refactor(chart): ChartUI is a real collaborator, not a raw-include partial (S2)
Expert/AIBase/ChartUI.mqh was 869 lines of method bodies of
CExpertSignalAIBase, #included after the class declaration - free to touch
any of its ~500 members. First of the eleven AIBase/*.mqh partials to come
out (fewest inbound edges - see the SOLID campaign session order), using
the same view+adapter shape already proven for CTrainingDataView.
CChartView (Expert/Chart/IChartView.mqh) is the abstract read/behaviour
surface a chart-rendering collaborator needs - identity, bar/model access,
the prediction cache, and the training/vote/meta scalars the panel and HUD
line summarise. CAIBaseChartView is the adapter the signal owns and binds
to itself (MQL5 gives a class exactly one base, so CExpertSignalAIBase
cannot implement the view directly). CChartUI is the real collaborator: it
owns the arrow-restore queue, the rescan queue/tally, the last-arrows-saved
count and the purge-mismatch latch as its own fields (verified via grep to
be touched nowhere else in Expert/), and reaches everything else - including
StartChartSignalRescan, moved in from its old inline home in the header
since it drives the exact same rescan state machine AdvanceChartSignalRescan
drains - through the view.
m_arrowSignalCache and m_signalClusterWindow stay on the signal: Training.mqh
writes the cache directly every era and the training-data view already reads
it, so moving it would mean rewriting Training.mqh's write sites too - out of
scope here. CChartUI reaches it through four bounds-checked accessors instead
of a raw member poke. All 10 public methods keep their exact signatures and
become one-line forwards on the signal, so no other file's call sites change
except Training.mqh's one era-end status refresh, which now reads
RefreshStatusLabel() rather than reaching into CChartUI's now-private
last-displayed-neuron cache directly.
Verified structurally, not compiled (never compile - the operator does, in
MetaEditor): brace balance checked on every touched/new file against HEAD,
and the view/adapter/impl method lists cross-diffed to confirm all 59
accessors match 1:1 across the interface, the adapter declaration and the
adapter body.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-08-23 20:03:52 -04:00
|
|
|
//--- m_lastStatusLabelUpdateTick and the four m_lastDisplay* fields now default-construct on
|
|
|
|
|
//--- CChartUI (see its own constructor); m_lastBuyRecallPct/m_lastSellRecallPct stay here - they
|
|
|
|
|
//--- are read by other subsystems too, not exclusive to the panel.
|
2026-08-01 11:27:28 -04:00
|
|
|
m_lastBuyRecallPct(-1),
|
|
|
|
|
m_lastSellRecallPct(-1),
|
|
|
|
|
m_lastBarTime(0),
|
|
|
|
|
m_modelEta(InitialEtaForOptimizer()),
|
|
|
|
|
m_etaCeiling(InitialEtaForOptimizer()),
|
|
|
|
|
m_erasSinceCooldown(0),
|
|
|
|
|
m_bestOosForecast(-1),
|
feat(target): delete the barrier/geometry stack - the label is the verdict
Step 3 of the swing-pivot plan, whole-hog. The swing label is now the ONE
target and the era verdict is precision + recall per class against the
label's own base rate - no win rate, no break-even, no expectancy, no
geometry anywhere in training.
DELETED
- Expert/Excursion/ (4), Expert/BarrierHorizon/ (4), GeometrySweep,
FirstPassageLadder, Labeling/TripleBarrier.mqh (CLabelOverlap survives
in Labeling/LabelOverlap.mqh), 3 test EAs.
- TripleBarrierLabel + walk, fractal label, geometry derivation/scan/
adoption, exit-policy replay, excursion MI targets, the drift verdict
(DIRECTION_INTELLIGENT), the recall floor, balanced-accuracy telemetry,
the barrier defines, the .cfg geometry adopt (slots kept as zeros for
the positional layout), the derived-geometry live-order override.
- TRAINING_TARGET input/enum: direction models are always swing; META2
re-keys the meta head onto label agreement (descriptor loses its two
geometry slots).
REWORKED
- Labels.mqh (1795 -> ~370 lines): AdvanceSwingLabelState with
FINALITY-GATED CACHING - an unresolved bar (pivot pair uncommitted) is
never cached, so it can never freeze as a false Neutral; training,
calibration, OOS scoring and online learning all skip unresolved bars.
- SDeployVerdict: significance-only; SOosTally chance = larger
directional class share; pooled gate poolability = timeframe (record v2).
- Purge/embargo/declustering gaps: the measured mean label resolution
lag (LabelResolutionBars), not a barrier horizon.
- Pool purge key + backfill DB rows: marked at the bar the label
resolved on (m_labelResolveAge), not a fabricated barrier touch.
- Online learning frontier: finality, not a horizon delay.
- m_bestBalancedOos -> m_bestSelectionScore, m_erasSinceBestBalanced ->
m_erasSinceBest, ensemble vote outcome arrays -> label arrays.
STEP 4 folded in: Entry_Multiplier / SL_Mode / TP_Mode / tradingdirection
are inputs again - trade management is the tester GA's search space.
Fingerprints: every direction model re-keys (TGT:SWG1 now unconditional,
CUT token gone); META1 -> META2. Full retrain, as planned.
Compile-verified in _claude_stage: Warrior_EA + both surviving test EAs,
0 errors, 0 warnings each.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-24 20:42:31 -04:00
|
|
|
m_bestSelectionScore(-1),
|
2026-08-01 11:27:28 -04:00
|
|
|
m_bestPassedRecall(false),
|
fix: a one-sided era can no longer become the best checkpoint
Measured on HYBRID, era 29 of the first win-scored run: the model collapsed
to always-Buy and was crowned "new best selection score 67.1%". Under
win-based scoring that is not a coincidence - the always-call-the-drift-side
model IS the chance reference, so it scores exactly chance (P(winLong) ~ 67%
on SP500), while every honest two-sided era scores 63-66% because shorts win
less often against the drift. Raw score ranking therefore actively prefers
the degenerate model, every regression restores back to it, and live NMS
collapses its near-constant signal to ~25 trades per era - observed as
"hybrid barely trades".
bothSidesLive already blocked one-sided eras from DEPLOYING (tradeableOK,
371f8aa), but among not-yet-deployable eras the score alone ranked - the same
early phase the coverage credit was added for, failing the same way through a
different door.
The ranking key is now three lexicographic tiers: deployable > two-sided >
score. A one-sided era cannot displace a two-sided best regardless of score -
by construction its score is a property of the data's drift, not the model -
and a two-sided era displaces a one-sided best no matter how much lower it
scores. m_bestBothSidesLive is snapshotted with the checkpoint and reset with
the rest of the best-tracking state.
Both build variants compile 0 errors, 0 warnings.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-09 21:19:44 -04:00
|
|
|
m_bestBothSidesLive(false),
|
diag: slow eras must explain themselves - heartbeat + era time split + pass-1 paint
The 23:42 restart left all four charts grinding ~25x slower than the 18:01
baseline (era lines in 86 seconds there; 20+ minutes of nothing here), and
NOTHING could say why from outside: pass 1 logs nothing, its status paint sat
inside the !wouldQueue branch so the IS sweep - 80% of the pass, processed
FIRST - painted nothing either, the VPS has no debugger for a thread stack,
and the hourly new-bar cache invalidation cancels and restarts an unfinished
era, so a slow era can stay invisible FOREVER. Externals gave: four chart
threads at ~95% pure user-mode compute, DLL pool idle, no file writes. That
narrows it to "MQL5-side per-item work in the era passes" and no further.
So training now explains itself:
- TrainHeartbeat: one line per 4096 processed items, only after an era has
already run 60s, at most 6 lines per era - a healthy era stays exactly as
quiet as before. Reports position and the cumulative split: feature-window
builds vs net forward/backprop vs everything else. Hooked into all three
passes.
- The era summary line gains "| ERA TOOK Ns (feature windows X, net fwd/back
Y, other Z)" whenever an era exceeded 120s.
- Pass 1 paints its progress for QUEUED bars too, not just the OOS slice, so
the panel shows "learning (era N)" instead of sitting on the idle writer's
"Getting ready..." for the entire IS sweep. The label is throttled
internally; painting per bar costs nothing.
Both build variants compile 0 errors, 0 warnings.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-10 00:10:45 -04:00
|
|
|
m_eraStartTick(0),
|
|
|
|
|
m_passFeatUs(0),
|
|
|
|
|
m_passNetUs(0),
|
|
|
|
|
m_passHeartbeatPrints(0),
|
diag: an era that discards itself now says so instead of scanning forever
add_loop is exactly "at least one bar produced a usable feature
window". When it stays false, pass 2, pass 3, the era counter, the
checkpoint and every log line in the era-end block are ALL skipped:
Train() returns having done nothing, m_eraResumePending is still false,
and the next call restarts the SAME era from bar 0. That is an
infinite 0->100% "scan" loop that prints absolutely nothing - the only
remaining silent restart path in Train(), and it matches the reported
symptom exactly.
Pass 1 now counts usable vs unusable windows and reports at the pass
boundary, which demonstrably executes:
- total failure routes through ReportTrainStall (already capped at
one line a minute, and carries the run-state flags) naming the
counts, the required window width and the bar count
- success prints how long the scan took and how many samples it
handed to pass 2, but only once the era has passed 10s - a fast
era stays as quiet as before, a slow one distinguishes "advancing"
from "sweeping the same bars forever"
A PARTIAL failure is normal and deliberately does not shout: pass 1
walks oldest-to-newest and the deepest bars predate the indicators'
warm-up, so those windows fail and are cached as misses.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-10 11:29:33 -04:00
|
|
|
m_passWindowOk(0),
|
|
|
|
|
m_passWindowFail(0),
|
fix: the recall gate was unsatisfiable and the LR decay was a spiral
Both made the run structurally unable to succeed, independently of any
signal in the data. Found by reading the 13:01 log.
RECALL GATE. m_objectiveMet required Buy, Sell AND Neutral OOS recall
each >= 40%. First-touch resolution (ce52654) collapsed Neutral from
the ~94% majority it was under exact-pivot labels to a same-bar-tie
residue - 250 of 38,261 bars, 0.65% - so the floor was asking the model
to identify 40% of coin-flip ties before it could converge. Measured:
CONV, LSTM and HYBRID all logged "Neutral:0% (need >=40% each)" on
every era. No model could ever satisfy it; every run was destined for
the plateau ladder or the era cap.
Only the DIRECTIONAL floors are load-bearing for the anti-collapse job
the gate exists to do: an all-Neutral model shows Buy and Sell recall
at 0% and is blocked by them. Neutral's own floor guarded the mirror
bias (over-calling Buy/Sell at Neutral's expense), which was real at
94% prevalence and is not at 0.65% - there, almost never calling
Neutral is correct rather than biased.
Prevalence-guarded rather than hardcoded off, so it returns by itself
if a future label rule makes Neutral substantial again. Deliberately
NOT extended to Buy/Sell: exempting a thin directional class reopens
the era-44-46 hole, which directionalRecallMeasured only half-covers -
it checks those classes were MEASURED, not that they passed.
ETA DECAY. A regressing era restored the checkpoint, reset the
optimizer and cut eta - all on the FIRST regression. The next era then
started from an identical state with a smaller step, regressed again,
and got the same treatment. The loop is self-sustaining and cannot
discover anything, because rolling the weights back is exactly what
removes the exploration that would end it.
Measured on PAI: eras 2-11 every one a regression against era 1, eta
0.000594 -> 0.000024, dW/W 0.000%/0.000% from era 2 onward. Ten eras,
~45s each, reproducing era 1 exactly and unable to do anything else.
Now requires ETA_DECAY_PATIENCE_ERAS consecutive regressions - the
standard ReduceLROnPlateau formulation. A single bad era is noise, and
an improving era clears the counter so alternating runs never
accumulate into a decay.
Build tag -> gate-patience-v3. It had not moved in six commits, which
is why the running binary could not be identified from its own log.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-10 13:28:58 -04:00
|
|
|
m_consecutiveRegressions(0),
|
fix(ensemble+depth): the barrier had no liveness escape, and the depth gate could not report the one state the evidence pointed at
Two charts (USDJPY 50,179 bars / XAUUSD 33,982) sat at era 0 for 38 minutes with
four of their eight members completely silent. Nothing in this commit guesses at
why the sweep fails - the last five guesses were all wrong. It makes the failure
say what it is, and stops one broken member taking its whole chart down with it.
WHAT THE LOG ACTUALLY SAYS, before any of this.
- The running build IS d9f834d (pulled 14:18, compiled 14:19:01, 0 errors), so
every depth instrument from 1dda479/7e63a8b/45c9e21 was live.
- It printed NOTHING. Zero "PRIMING", zero "CAPPED", zero "Per-indicator depth"
in 27 MB of journal. The instrument built to find the depth shortfall returned
"not this".
- On USDJPY at 14:24, CONV-cad8 completed eras 0 AND 1 across all 50,179 bars -
same chart, same 832-value window, same indicators, byte-identical fingerprint -
while LSTM-cad8 and HYB-cad8 reported ok=0 failed=50163. So it is not the
symbol, the history, the bar count or the indicator depth. It is per-member.
- ok=0 means the NEWEST anchors failed too, and a short indicator cannot do that.
The depth reading in project_silent_block_failures is therefore retired by its own
instrumentation. THE ROOT CAUSE IS STILL UNKNOWN and this commit does not claim one.
1. THE DEPTH GATE'S SILENT PATH WAS THE STATE IT WAS HUNTING.
ServableBars() read `if(servable <= 0 || servable >= want) return want;` - one
branch over three unrelated states, silent in all of them:
enabled == 0 -> nothing tunable is on. No cap. Healthy.
enabled > 0, servable == -1 -> a handle answered INVALID.
enabled > 0, servable == 0 -> created, never calculated.
BarsCalculated() returns -1 for a dead handle, so a dead MA is indistinguishable
from "no tunable indicators enabled" - and both returned `want` without printing a
character. That is exactly the state a per-member, every-index, depth-independent
failure produces, and it is the single reason a build carrying full depth
instrumentation logged nothing through the whole outage.
TunableBarsCalculated() now also reports HOW MANY indicators it consulted, and the
dead-handle case is reported (latched, with per-handle depths). The RETURN is
deliberately unchanged - what to do about a dead handle is not yet known, and
changing control flow on an unproven cause is how the last four fixes here went
wrong. SettledBars() routes its three pass-through states via ServableBars() so the
report is reachable from the training sweep, which is the only caller that hits it.
2. THE STALL REPORT NAMED A SLOT, NEVER A BLOCK.
"lookback slot 0 REJECTED (window had 24 of 832 values)" plus a guess ("an
indicator warm-up or a history-edge read"). Which guard fired was INFERRED by
counting 4+5+4+4+6+1 = 24 and concluding feature 25 must be the MA. The arithmetic
was right; every conclusion drawn from it was wrong, because a value count names a
POSITION and a position cannot tell cold from capped from invalid from off-the-end.
Every guard that can reject a bar now records itself - m_featureFailBlock - and the
report carries it, the series index, IndicatorDepthReport()'s per-handle depths,
and for each indicator whether the NEWEST bar reads. That last field is the whole
diagnosis in one word: newest-also-EMPTY means the buffer is unreadable everywhere
(cold or dead handle), newest-reads means a genuine history edge. Instrumented:
open, ATR, MA, RSI, MACD, Ichimoku, and all five AD blocks via ADIndicatorCold().
3. THE TOTAL-FAILURE BACKOFF WAS GATED ON THE WRONG QUESTION.
It armed only when m_featureFailTransient was set. Keeping that flag correct across
every guard is a list that has to stay right forever - the same shape of fix the
feature cache abandoned for the same reason - and the gate is pointless anyway: a
sweep where ZERO of 50,163 bars produced a window will produce zero again if it
restarts a millisecond later, transient or not. Doing that at full speed is what
starved six indicator threads on a six-core box. The backoff is now unconditional
on a total failure. The flag keeps its real job, deciding whether a MISS may be
cached, which is a per-bar question and not a scheduling one.
4. THE ERA BARRIER DEADLOCKED, AND SILENCED THE MEMBERS IT FROZE.
EnsembleMinTrainingEra() exempted deployed, stopped and paused members and its
comment concluded "so nothing deadlocks". Those three are all VOLUNTARY. A member
that simply CANNOT finish an era is none of them, so it pinned the minimum at its
own era with no time limit - and the hold branch's only action was
`m_lastEraCompleteTick = GetTickCount()`, which silences the stall watchdog. So on
USDJPY the two members that could not train reported, and the two healthy members
frozen behind them wrote nothing anywhere. The outage was visible only through the
members that were not suffering it.
- BarrierEraHeartbeat() stamps a clock on real era CHANGE, kept separate from
m_lastEraCompleteTick precisely because the barrier resets that one. Only a
member AT the minimum can be a blocker; a member ahead is idle by design and is
never counted as stuck.
- After ENSEMBLE_BARRIER_STUCK_MS (12 min) a non-advancing member is dropped from
the barrier minimum. It keeps training and rejoins the instant it completes an
era - at which point, being behind, it legitimately becomes the minimum again,
which is the documented resumed-laggard behaviour.
- Both transitions say so loudly, and the release states plainly that the
combined-vote score cannot be computed while the ensemble is desynchronised.
- A held member now writes a rate-limited journal line naming WHICH members it is
waiting on, so the blocker is read off one line.
5. THE PANEL FLICKER.
OnTickHandler gates its terse writer on !m_trainRunActive, and a barrier-held member
returns from Train() before ever setting it - so both writers thought they were the
only one updating the label and fought every tick. That is the reported "Getting
ready..." <-> "Waiting at era N for slower ensemble members" oscillation, and it hit
Perceptron but not Convolutional purely because Convolutional had a run active from
a completed era and Perceptron, resumed from disk, never did. Train()'s message is
the specific one, so it wins.
NEXT STEP once this is running: the stall line now ends in "REJECTED BY: ..." and
the per-handle depths. Read it. Do not reason around it.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-17 15:21:15 -04:00
|
|
|
m_featureFailBlock(""),
|
|
|
|
|
m_featureFailIdx(-1),
|
diag: name the cause when every feature window fails, and enforce the width contract
Era 0 stalls with "NOT ONE of 54681 scanned bars produced a usable
feature window, windows ok=0 failed=54681" and nothing else. That line
reads identically for a cold ATR, a conditionally-missing optional
feature block and an out-of-range index, so it cannot be diagnosed
without one restart per hypothesis.
Two changes:
1. WIDTH CONTRACT in BufferTempData. Every enabled block must emit
exactly m_neuronsCount values on EVERY bar. A block that emits its
values on some bars and skips them on others (indicator, panel or
series unavailable for that bar) does not merely shorten the window -
it SHIFTS every feature after it into the wrong slot, and the net
then trains on silently misaligned inputs that still look like a
valid window to everything downstream. Now rejected, rolled back and
reported once, naming the optional blocks (XA / SPR / swing context)
as the ones carrying an availability test. Worth having independently
of the current stall.
2. BuildFeatureWindow records WHICH lookback slot rejected and how much
of the window was assembled, and the pass-1 stall report renders it:
"slot 0 of 20 REJECTED (window had 0 of 760)" is an indicator warm-up
or history-edge read; "every lookback bar ACCEPTED and the window was
still short: 640 of 760" is a missing 6-value block.
No behaviour change on a healthy run: the width check is an equality
that already holds, and the diagnostics render only inside the
total-failure branch.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-11 10:31:56 -04:00
|
|
|
m_windowFailSlot(-2),
|
|
|
|
|
m_windowFailTotal(0),
|
2026-08-10 07:44:03 -04:00
|
|
|
m_lastHeartbeatTick(0),
|
2026-08-10 07:22:29 -04:00
|
|
|
m_passProgressPct(0),
|
|
|
|
|
m_passLabel("starting"),
|
fix: prebuild and era sized different windows; diag: Train() names its branch
TWO things, one incident.
1) THE BUG I SHIPPED IN 0c85c54. m_tuneStartTrainBar is declared, initialised
to 0, and NEVER ASSIGNED - the assignment existed before the God-class split
and the split dropped it, leaving a dead member. Harmless while nothing read
it; a real defect the moment 0c85c54 made StartLabelCachePrebuild() reset
dtStudied from it. Train() then computed the window as
max(StartTrainBar, floor) while the prebuild computed max(0, floor), where
StartTrainBar is the non-zero datetime OnChartEventHandler passes through from
the "New Bar" event. The two therefore disagreed about `bars`, so
EnsureBarCachesCapacity() saw a changed size at era start, wiped the caches,
and re-armed a full 38k-bar prebuild - instead of training. Restored the
assignment so both sides evaluate the identical expression.
2) THE REASON IT TOOK ALL NIGHT TO FIND. Train() is a state machine with six
early-return branches above the era loop and every one of them is silent. Four
charts burned a core each for 15 minutes with an empty journal: the pass
heartbeats (694b756) proved the era loop was never reached, no prebuild
completion line appeared either, and nothing external can see inside a single
MQL5 thread - per-thread CPU says "busy", file writes say nothing, and the VPS
has no debugger. That is an undiagnosable state, and it is the thing to fix,
not just the bug of the day.
ReportTrainStall() now names the branch Train() is taking whenever no era has
completed for 3 minutes, at most once a minute per signal, with the state that
decides the branch: run/prebuild/simOos/resume flags, era, dtStudied, and -
for the cache-invalidation branch specifically - BOTH bar counts, since two
sizings disagreeing is exactly what re-arms the prebuild forever. Silent on a
healthy run: an era completing resets the clock.
Both build variants compile 0 errors, 0 warnings.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-10 07:32:08 -04:00
|
|
|
m_lastEraCompleteTick(0),
|
|
|
|
|
m_lastStallReportTick(0),
|
diag(training): report the window an era ACTUALLY trains on
With VerboseMode on, pass 1 reported eras of 422 / 949 / 1358 / 2562 bars on
SP500 H4 - four models, same chart, same second - against a series holding
~16,264 bars, and the number moved every era (CONV: 2562, 3671, 3405, 3532,
2830). Nothing in the journal said so. ReportDetectability and the CAPACITY
line both quote EstimatedInSampleBars, which is derived from the configuration
and not from the era, so they kept reporting "11385 in-sample rows / OOS window
4874 bars" for a window that was a tenth of that.
era.bars is MathMin(Bars(symbol, PERIOD_CURRENT, dtStudied, now) + historyBars,
Bars(symbol, PERIOD_CURRENT)). A short era is therefore either a dtStudied that
is too recent or a short price series, and those need opposite fixes - so the
new line carries all three quantities plus the resolved dtStudied and
SERIES_FIRSTDATE, not just the result.
Reported on change only: an era over a warm feature cache runs in a fraction of
a second here, and a per-era line would bury the journal.
Diagnostic only - no training behaviour is changed by this commit.
Compile-verified in the staging copy: 0 errors, 0 warnings.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-24 15:12:05 -04:00
|
|
|
m_lastEraWindowBars(-1),
|
2026-08-01 11:27:28 -04:00
|
|
|
m_haveOosCheckpoint(false),
|
feat: gate deployment on the null of the MAXIMUM, not the per-era null
EDGE_MIN_SIGMAS is a PER-ERA test and the deployed model is the MAXIMUM
over every era a run ranks. A 2-sigma one-sided test passes on noise with
probability 0.0228 per era, so over N eras the chance at least one clears
it is 1-(1-0.0228)^N: 34% by era 18, 80% by era 70, 93% by era 112. The
gate was near-certain to open on a long run whatever the data held.
It did. HYBRID deployed 2026-08-08 at dir-precision 35.5% vs 34% chance -
+1.5pp, best of 112 eras whose per-era values wandered 30%..35.5%. At the
call counts these runs produce that is p_family 0.92..0.9999.
Every OTHER best-of-N decision here already carries this correction, and
every one REJECTS on this data: the barrier-geometry winner (null of the
maximum over 6, p=0.3902), the indicator tuner (Sidak, p=1.0000), the MI
lag profile (null of the maximum over 21 lags). The one decision that
ships a model to a live account had none.
BestCheckpointSurvivesSelection() re-tests the checkpoint that is about to
deploy:
z = (precision - chance)/SE, SE = sqrt(p0(1-p0)/n)
p_single = P(Z >= z)
p_family = 1 - (1-p_single)^N
against DEPLOY_FAMILY_WISE_ALPHA. It uses the checkpoint's OWN
snapshotted precision/chance/call-count, not the latest era's, because
the model that ships is the one that has to clear the bar.
N counts CANDIDATE eras (coverage measurable, at least one directional
call) - an era that called nothing directional could never have become
the best, so counting it would make the gate stricter than the search
that actually happened.
Conservative on purpose: consecutive eras share OOS bars and differ by
one gradient step, so they are nowhere near N independent draws and the
true family-wise error is below this bound. This gate decides what trades
real money and the house posture is reject-unless-demonstrated.
Effect at 2900 directional calls / N=112: required edge goes 1.76pp ->
2.92pp. A real edge clears it; +1.5pp does not.
Applied to BOTH automatic paths - the plateau ladder's stage-3 deploy and
the m_trainingComplete assignment - which must stay identical or the flag
persisted into the .nnw disagrees with the decision to stop, and a reload
runs inference on a model the ladder refused.
NOT applied to the two operator paths (era-cap deploy, panel Deploy
button). Those stay the operator's call; ReportSelectionGateVerdict()
logs the verdict beside them so an authorised deploy can never later be
misread as a validated one.
NormalUpperTail() is A&S 26.2.17 (|err| < 7.5e-8), self-contained rather
than pulling in Math\Stat. Verified against reference values to 6dp:
Q(1.645)=0.049985, Q(1.96)=0.024998, Q(3.0)=0.001350. Its locals are
ntB1..ntB5 because AI\Network.mqh line 79 does "#define b1 AdamBeta1".
Compiles clean: 0 errors, 0 warnings.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-08 18:01:04 -04:00
|
|
|
m_bestDirPrecPct(-1.0),
|
|
|
|
|
m_bestChancePrecPct(-1.0),
|
|
|
|
|
m_bestDirCalls(0),
|
|
|
|
|
m_deployCandidateEras(0),
|
2026-08-01 11:27:28 -04:00
|
|
|
m_oosStable(false),
|
|
|
|
|
m_objectiveMet(false),
|
feat(target): delete the barrier/geometry stack - the label is the verdict
Step 3 of the swing-pivot plan, whole-hog. The swing label is now the ONE
target and the era verdict is precision + recall per class against the
label's own base rate - no win rate, no break-even, no expectancy, no
geometry anywhere in training.
DELETED
- Expert/Excursion/ (4), Expert/BarrierHorizon/ (4), GeometrySweep,
FirstPassageLadder, Labeling/TripleBarrier.mqh (CLabelOverlap survives
in Labeling/LabelOverlap.mqh), 3 test EAs.
- TripleBarrierLabel + walk, fractal label, geometry derivation/scan/
adoption, exit-policy replay, excursion MI targets, the drift verdict
(DIRECTION_INTELLIGENT), the recall floor, balanced-accuracy telemetry,
the barrier defines, the .cfg geometry adopt (slots kept as zeros for
the positional layout), the derived-geometry live-order override.
- TRAINING_TARGET input/enum: direction models are always swing; META2
re-keys the meta head onto label agreement (descriptor loses its two
geometry slots).
REWORKED
- Labels.mqh (1795 -> ~370 lines): AdvanceSwingLabelState with
FINALITY-GATED CACHING - an unresolved bar (pivot pair uncommitted) is
never cached, so it can never freeze as a false Neutral; training,
calibration, OOS scoring and online learning all skip unresolved bars.
- SDeployVerdict: significance-only; SOosTally chance = larger
directional class share; pooled gate poolability = timeframe (record v2).
- Purge/embargo/declustering gaps: the measured mean label resolution
lag (LabelResolutionBars), not a barrier horizon.
- Pool purge key + backfill DB rows: marked at the bar the label
resolved on (m_labelResolveAge), not a fabricated barrier touch.
- Online learning frontier: finality, not a horizon delay.
- m_bestBalancedOos -> m_bestSelectionScore, m_erasSinceBestBalanced ->
m_erasSinceBest, ensemble vote outcome arrays -> label arrays.
STEP 4 folded in: Entry_Multiplier / SL_Mode / TP_Mode / tradingdirection
are inputs again - trade management is the tester GA's search space.
Fingerprints: every direction model re-keys (TGT:SWG1 now unconditional,
CUT token gone); META1 -> META2. Full retrain, as planned.
Compile-verified in _claude_stage: Warrior_EA + both surviving test EAs,
0 errors, 0 warnings each.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-24 20:42:31 -04:00
|
|
|
m_erasSinceBest(0),
|
2026-08-01 11:27:28 -04:00
|
|
|
m_plateauStage(0),
|
feat(search): stop on the IN-SAMPLE plateau, and shrink every best-of-K effect before quoting it
Points 3 and 4 of the four-point plan.
1. IN-SAMPLE EARLY STOP - and the reason it is worth having is not compute.
The plateau ladder stops on the OOS SELECTION score. That is a peek: by the time it
fires, every one of those eras has been evaluated out of sample, so all of them sit
in the family the deploy gate corrects over (g_ensCandidateEras, Sidak). Training
longer therefore does not merely cost time - it RAISES the bar the eventual winner
has to clear.
The new stop reads the TRAINING error, which the gate never looks at. When the
optimiser has stopped improving on data it can see, more eras will not find a better
model; they will only enlarge the OOS family. Ending there shrinks the correction,
and the shrinkage is legitimate precisely BECAUSE the stopping rule never consulted
an out-of-sample number.
That distinction is the whole point and it is the one this project has got wrong four
times: stop on IS and the family really is smaller; stop on OOS and those eras were
searched and still count. Both stops now exist; only this one buys a lower bar.
Deliberately more patient than the OOS ladder (IS_ERROR_PATIENCE_MULT = 3x): training
error is noisy per era - mini-batch order alone moves it - and ending a run that is
still learning costs far more than a few wasted eras. Improvement is RELATIVE
(IS_ERROR_IMPROVE_FRAC = 1%), so it does not depend on the loss's absolute scale, and
it only acts when a checkpoint exists, since otherwise it would end a run with
nothing to deploy. Reset per RUN alongside the ladder, so a resumed run cannot
early-stop on its first era against a previous run's best.
2. WINNER'S-CURSE SHRINKAGE ON THE BARRIER-GEOMETRY WINNER.
The family-wise permutation gate already establishes that the RANKING is not noise.
It says nothing about the SIZE of the winner's effect - and a best-of-K maximum is
biased upward by construction, being the largest of K noisy draws. The adoption
message quotes that raw maximum and compares it against the incumbent, so the number
a reader plans on is the inflated one.
The penalty is now measured, not assumed: the same permutation draws that produce the
p-value also produce, per draw, the MAXIMUM excess across all candidates under pure
noise. The mean of those maxima is exactly what a best-of-K selection is expected to
report when there is nothing there. This is the empirical form of the sqrt(2 ln K) x SE
penalty the SQX EdgeFinder plugin applies to every maximum it reports (Stats.java:79-88),
and it needs no normality assumption because the draws ARE the null distribution.
Applied in James-Stein form - effect x max(0, 1 - penalty^2/effect^2) - so a large
effect is nearly untouched and a marginal one collapses toward zero.
Reported, not gated. The adoption decision still turns on the permutation p-value,
which is the right test for "is the ranking real"; the shrunk number is there so the
magnitude quoted beside it is one worth planning on. Closes the first of the two
EdgeFinder ports identified on 2026-08-12.
NOTE on the second EdgeFinder port, deliberately not done here: "let the measurement
steer the target" is already true where it matters most - ReportGeometryExpectancyScan
ADOPTS the winning barrier geometry under the family-wise gate rather than advising
it, and the MI excursion suite publishes a verdict per instrument per config. What is
still missing is steering the TRAINING TARGET itself (direction vs excursion) off
those verdicts, and that is a design change rather than a surgical one - direction is
a closed verdict while excursion SIZE keeps clearing, so the honest version of that
change is a target-selection policy, not a flag.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-17 16:29:00 -04:00
|
|
|
m_bestIsError(-1.0),
|
|
|
|
|
m_erasSinceBestIsError(0),
|
fix(plateau): the IS-error early stop was inert for every ensemble member
15 hours of training, and the stop that exists to END a run announced itself
1,299 consecutive times without ending anything:
SP500 ConvLSTM IN-SAMPLE ERROR PLATEAU - not improved in 1297 / 1298 / 1299
eras (best 0.2689, now 0.3269) ... era 1396, 1397, 1398
SP500 LSTM 536 eras SP500 CONV 442 eras SP500 PAI 150 eras
XAUUSD HYB 478 eras XAUUSD LSTM 296 eras XAUUSD CONV 366 eras
CAUSE: it wrote its decision into m_plateauStage, and EnsembleEraVerdict mirrors
the shared ladder onto every member - `mm.m_plateauStage = g_ensPlateauStage` -
on EVERY era, purely so each member's status line shows the collective stage. A
display mirror was silently overwriting a decision, so the stop re-armed and
re-fired the next era, forever.
This is the worst possible direction for this particular bug. Every one of those
1,299 eras was scored out of sample and joined the family the deploy gate
corrects over (Sidak, g_ensCandidateEras). The stop's entire purpose is to make
that family SMALLER; instead the run spent fifteen hours raising its own bar.
- m_isErrorPlateaued: a one-way per-member latch, cleared only by a fresh run.
Nothing in the ladder may reset it. The stop condition and the two solo deploy
conditions read the latch, not the mirrored stage.
- The orchestrator combines: EnsembleEraVerdict requires UNANIMITY across
participating members (same participation test the era barrier uses, so an
excluded or finished member cannot veto). One member still learning can still
move the combined vote, and the vote is what the gate certifies.
- Fed in as `dueStage = PLATEAU_STAGE_DEPLOY`, NOT written to g_ensPlateauStage.
The block that actually ends the run sits under `dueStage > g_ensPlateauStage`,
so assigning the stage directly makes that test false and the deploy never
happens - the same inert-write shape as the bug being fixed. Caught before
committing; raising dueStage carries it through the ladder's own path (warm
restarts skipped, family-wise vote test, measurement screen, joint checkpoint)
unchanged.
- g_ensIsPlateauAnnounced: announce once per run, not once per era.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-18 13:09:26 -04:00
|
|
|
m_isErrorPlateaued(false),
|
fix: training-stability audit fixes F1/F2/F3/F5 - unbiased shuffle, real plateau escapes, fresh optimizer state on restore, pure OOS metric
Four of the six findings from research/training_pipeline_audit_2026-08-09.md
(F4 mini-batching and F6 feature re-encode deliberately deferred - see the
report's implementation-status section for why):
- F1: pass-2 Fisher-Yates (and AutoTune's MI block shuffle) used MathRand()%,
which is 15-bit - provably non-uniform on every full-history era over 32,768
queued samples. New 30-bit ShuffleRandomIndex().
- F2: plateau warm restarts were a no-op whenever eta already sat at its
ceiling (the normal state of a non-regressing plateau) - the ladder was just
a 24-era countdown. Restarts now overshoot to 5x the ceiling
(PLATEAU_RESTART_BOOST) and anneal geometrically back over the patience
window, SGDR-style; ETA_MIN widened 1e-4 -> 1e-5 so the decay schedule has
real range.
- F3: checkpoint restores put weights back but kept the rejected trajectory's
Adam moments, so the optimizer immediately pushed back toward the rolled-back
state (the restore->regress->restore oscillation). CNet::ResetOptimizerState()
zeroes moments/momentum/step counters (weights, BN statistics, gamma/beta
untouched) on every mid-run restore, every boosted restart, and the
deploy-time restore that online learning continues from.
- F5: batch-norm running statistics now freeze for the pass-3 OOS scoring walk,
so the selection metric the checkpoint ranking and deploy gate read is a pure
function of the checkpoint instead of partly measuring BN drift. Defensive
unfreeze in FinalizeTrainRun covers stop-mid-pass; live/online adaptation and
the OOS continual-learning simulation stay adaptive by design.
Compiled clean (0 errors, 0 warnings) via the staged-tree recipe.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-09 10:54:09 -04:00
|
|
|
m_restartBoostErasLeft(0),
|
2026-08-01 11:27:28 -04:00
|
|
|
m_syncWaitStartTick(0),
|
|
|
|
|
m_warmupPassesRemaining(0),
|
2026-08-13 10:23:11 -04:00
|
|
|
m_coldSweepTick(0),
|
fix(ensemble+depth): the barrier had no liveness escape, and the depth gate could not report the one state the evidence pointed at
Two charts (USDJPY 50,179 bars / XAUUSD 33,982) sat at era 0 for 38 minutes with
four of their eight members completely silent. Nothing in this commit guesses at
why the sweep fails - the last five guesses were all wrong. It makes the failure
say what it is, and stops one broken member taking its whole chart down with it.
WHAT THE LOG ACTUALLY SAYS, before any of this.
- The running build IS d9f834d (pulled 14:18, compiled 14:19:01, 0 errors), so
every depth instrument from 1dda479/7e63a8b/45c9e21 was live.
- It printed NOTHING. Zero "PRIMING", zero "CAPPED", zero "Per-indicator depth"
in 27 MB of journal. The instrument built to find the depth shortfall returned
"not this".
- On USDJPY at 14:24, CONV-cad8 completed eras 0 AND 1 across all 50,179 bars -
same chart, same 832-value window, same indicators, byte-identical fingerprint -
while LSTM-cad8 and HYB-cad8 reported ok=0 failed=50163. So it is not the
symbol, the history, the bar count or the indicator depth. It is per-member.
- ok=0 means the NEWEST anchors failed too, and a short indicator cannot do that.
The depth reading in project_silent_block_failures is therefore retired by its own
instrumentation. THE ROOT CAUSE IS STILL UNKNOWN and this commit does not claim one.
1. THE DEPTH GATE'S SILENT PATH WAS THE STATE IT WAS HUNTING.
ServableBars() read `if(servable <= 0 || servable >= want) return want;` - one
branch over three unrelated states, silent in all of them:
enabled == 0 -> nothing tunable is on. No cap. Healthy.
enabled > 0, servable == -1 -> a handle answered INVALID.
enabled > 0, servable == 0 -> created, never calculated.
BarsCalculated() returns -1 for a dead handle, so a dead MA is indistinguishable
from "no tunable indicators enabled" - and both returned `want` without printing a
character. That is exactly the state a per-member, every-index, depth-independent
failure produces, and it is the single reason a build carrying full depth
instrumentation logged nothing through the whole outage.
TunableBarsCalculated() now also reports HOW MANY indicators it consulted, and the
dead-handle case is reported (latched, with per-handle depths). The RETURN is
deliberately unchanged - what to do about a dead handle is not yet known, and
changing control flow on an unproven cause is how the last four fixes here went
wrong. SettledBars() routes its three pass-through states via ServableBars() so the
report is reachable from the training sweep, which is the only caller that hits it.
2. THE STALL REPORT NAMED A SLOT, NEVER A BLOCK.
"lookback slot 0 REJECTED (window had 24 of 832 values)" plus a guess ("an
indicator warm-up or a history-edge read"). Which guard fired was INFERRED by
counting 4+5+4+4+6+1 = 24 and concluding feature 25 must be the MA. The arithmetic
was right; every conclusion drawn from it was wrong, because a value count names a
POSITION and a position cannot tell cold from capped from invalid from off-the-end.
Every guard that can reject a bar now records itself - m_featureFailBlock - and the
report carries it, the series index, IndicatorDepthReport()'s per-handle depths,
and for each indicator whether the NEWEST bar reads. That last field is the whole
diagnosis in one word: newest-also-EMPTY means the buffer is unreadable everywhere
(cold or dead handle), newest-reads means a genuine history edge. Instrumented:
open, ATR, MA, RSI, MACD, Ichimoku, and all five AD blocks via ADIndicatorCold().
3. THE TOTAL-FAILURE BACKOFF WAS GATED ON THE WRONG QUESTION.
It armed only when m_featureFailTransient was set. Keeping that flag correct across
every guard is a list that has to stay right forever - the same shape of fix the
feature cache abandoned for the same reason - and the gate is pointless anyway: a
sweep where ZERO of 50,163 bars produced a window will produce zero again if it
restarts a millisecond later, transient or not. Doing that at full speed is what
starved six indicator threads on a six-core box. The backoff is now unconditional
on a total failure. The flag keeps its real job, deciding whether a MISS may be
cached, which is a per-bar question and not a scheduling one.
4. THE ERA BARRIER DEADLOCKED, AND SILENCED THE MEMBERS IT FROZE.
EnsembleMinTrainingEra() exempted deployed, stopped and paused members and its
comment concluded "so nothing deadlocks". Those three are all VOLUNTARY. A member
that simply CANNOT finish an era is none of them, so it pinned the minimum at its
own era with no time limit - and the hold branch's only action was
`m_lastEraCompleteTick = GetTickCount()`, which silences the stall watchdog. So on
USDJPY the two members that could not train reported, and the two healthy members
frozen behind them wrote nothing anywhere. The outage was visible only through the
members that were not suffering it.
- BarrierEraHeartbeat() stamps a clock on real era CHANGE, kept separate from
m_lastEraCompleteTick precisely because the barrier resets that one. Only a
member AT the minimum can be a blocker; a member ahead is idle by design and is
never counted as stuck.
- After ENSEMBLE_BARRIER_STUCK_MS (12 min) a non-advancing member is dropped from
the barrier minimum. It keeps training and rejoins the instant it completes an
era - at which point, being behind, it legitimately becomes the minimum again,
which is the documented resumed-laggard behaviour.
- Both transitions say so loudly, and the release states plainly that the
combined-vote score cannot be computed while the ensemble is desynchronised.
- A held member now writes a rate-limited journal line naming WHICH members it is
waiting on, so the blocker is read off one line.
5. THE PANEL FLICKER.
OnTickHandler gates its terse writer on !m_trainRunActive, and a barrier-held member
returns from Train() before ever setting it - so both writers thought they were the
only one updating the label and fought every tick. That is the reported "Getting
ready..." <-> "Waiting at era N for slower ensemble members" oscillation, and it hit
Perceptron but not Convolutional purely because Convolutional had a run active from
a completed era and Perceptron, resumed from disk, never did. Train()'s message is
the specific one, so it wins.
NEXT STEP once this is running: the stall line now ends in "REJECTED BY: ..." and
the per-handle depths. Read it. Do not reason around it.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-17 15:21:15 -04:00
|
|
|
m_barrierEraSeen(-1),
|
|
|
|
|
m_barrierEraTick(0),
|
|
|
|
|
m_barrierExcluded(false),
|
fix(ensemble): the era barrier read healthy startup work as a dead member
Reported symptom: one member at era 17 while the rest sat at era 2, with
the combined vote never scoring. Two faults compound to produce exactly
that, and neither needs a broken model to trigger.
FIRST - BUSY WAS READ AS STUCK. BarrierEraHeartbeat() decides liveness
from one signal: has m_eraCount changed in the last 12 minutes. But
Train() returns early, before the era loop, for three ONE-TIME phases
that never touch m_eraCount - the label-cache prebuild, the pattern-DB
backfill and the OOS simulation walk - and those are precisely what a
slow topology spends its first many minutes doing. A member grinding
steadily through a prebuild therefore looked identical to a dead one and
was dropped from the barrier at startup, before it had trained a single
era. The constant's own comment states the flawed premise: "comfortably
past the slowest healthy ERA on the deepest chart" - true, and not the
question being asked. Those three branches now call NoteBarrierProgress()
and a chunk of phase work re-arms the watchdog exactly as an era does.
SECOND - EXCLUSION HAD NO BOUND. Once dropped, a member is skipped by
EnsembleMinTrainingEra(). Drop every OTHER member and that loop finds
nothing to take a minimum over, falls through to its `return m_eraCount`
fallback - the CALLER'S own era - and EnsembleEraBarrierHolds() evaluates
`era > era`, false, for everybody. The barrier silently becomes a no-op
and the fastest member runs away unbounded. EnsembleMinEraAnyMember()
now measures against every still-training member, excluded or not, and a
member may lead it by at most ENSEMBLE_MAX_ERA_LEAD eras.
The cap is deliberately a real stop rather than a warning. A
desynchronised ensemble is not a degraded one: the combined-vote score
and the joint checkpoint both require every member on the same era, so
weights trained past the cap can never be certified by any gate. The
hold reports which of the two it is, because the operator's next move
differs - an ordinary barrier hold resolves itself, a lead-cap hold names
a member that needs diagnosing and will not resolve on its own.
Not yet explained: "only one NN listened to the stop command". The panel
now dispatches down the filter tree and reports the count it reached
("training stopped (N model(s))"), so the next run answers that
definitively instead of leaving it to inference.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-20 09:35:25 -04:00
|
|
|
m_barrierPhaseProgress(false),
|
fix(ensemble+depth): the barrier had no liveness escape, and the depth gate could not report the one state the evidence pointed at
Two charts (USDJPY 50,179 bars / XAUUSD 33,982) sat at era 0 for 38 minutes with
four of their eight members completely silent. Nothing in this commit guesses at
why the sweep fails - the last five guesses were all wrong. It makes the failure
say what it is, and stops one broken member taking its whole chart down with it.
WHAT THE LOG ACTUALLY SAYS, before any of this.
- The running build IS d9f834d (pulled 14:18, compiled 14:19:01, 0 errors), so
every depth instrument from 1dda479/7e63a8b/45c9e21 was live.
- It printed NOTHING. Zero "PRIMING", zero "CAPPED", zero "Per-indicator depth"
in 27 MB of journal. The instrument built to find the depth shortfall returned
"not this".
- On USDJPY at 14:24, CONV-cad8 completed eras 0 AND 1 across all 50,179 bars -
same chart, same 832-value window, same indicators, byte-identical fingerprint -
while LSTM-cad8 and HYB-cad8 reported ok=0 failed=50163. So it is not the
symbol, the history, the bar count or the indicator depth. It is per-member.
- ok=0 means the NEWEST anchors failed too, and a short indicator cannot do that.
The depth reading in project_silent_block_failures is therefore retired by its own
instrumentation. THE ROOT CAUSE IS STILL UNKNOWN and this commit does not claim one.
1. THE DEPTH GATE'S SILENT PATH WAS THE STATE IT WAS HUNTING.
ServableBars() read `if(servable <= 0 || servable >= want) return want;` - one
branch over three unrelated states, silent in all of them:
enabled == 0 -> nothing tunable is on. No cap. Healthy.
enabled > 0, servable == -1 -> a handle answered INVALID.
enabled > 0, servable == 0 -> created, never calculated.
BarsCalculated() returns -1 for a dead handle, so a dead MA is indistinguishable
from "no tunable indicators enabled" - and both returned `want` without printing a
character. That is exactly the state a per-member, every-index, depth-independent
failure produces, and it is the single reason a build carrying full depth
instrumentation logged nothing through the whole outage.
TunableBarsCalculated() now also reports HOW MANY indicators it consulted, and the
dead-handle case is reported (latched, with per-handle depths). The RETURN is
deliberately unchanged - what to do about a dead handle is not yet known, and
changing control flow on an unproven cause is how the last four fixes here went
wrong. SettledBars() routes its three pass-through states via ServableBars() so the
report is reachable from the training sweep, which is the only caller that hits it.
2. THE STALL REPORT NAMED A SLOT, NEVER A BLOCK.
"lookback slot 0 REJECTED (window had 24 of 832 values)" plus a guess ("an
indicator warm-up or a history-edge read"). Which guard fired was INFERRED by
counting 4+5+4+4+6+1 = 24 and concluding feature 25 must be the MA. The arithmetic
was right; every conclusion drawn from it was wrong, because a value count names a
POSITION and a position cannot tell cold from capped from invalid from off-the-end.
Every guard that can reject a bar now records itself - m_featureFailBlock - and the
report carries it, the series index, IndicatorDepthReport()'s per-handle depths,
and for each indicator whether the NEWEST bar reads. That last field is the whole
diagnosis in one word: newest-also-EMPTY means the buffer is unreadable everywhere
(cold or dead handle), newest-reads means a genuine history edge. Instrumented:
open, ATR, MA, RSI, MACD, Ichimoku, and all five AD blocks via ADIndicatorCold().
3. THE TOTAL-FAILURE BACKOFF WAS GATED ON THE WRONG QUESTION.
It armed only when m_featureFailTransient was set. Keeping that flag correct across
every guard is a list that has to stay right forever - the same shape of fix the
feature cache abandoned for the same reason - and the gate is pointless anyway: a
sweep where ZERO of 50,163 bars produced a window will produce zero again if it
restarts a millisecond later, transient or not. Doing that at full speed is what
starved six indicator threads on a six-core box. The backoff is now unconditional
on a total failure. The flag keeps its real job, deciding whether a MISS may be
cached, which is a per-bar question and not a scheduling one.
4. THE ERA BARRIER DEADLOCKED, AND SILENCED THE MEMBERS IT FROZE.
EnsembleMinTrainingEra() exempted deployed, stopped and paused members and its
comment concluded "so nothing deadlocks". Those three are all VOLUNTARY. A member
that simply CANNOT finish an era is none of them, so it pinned the minimum at its
own era with no time limit - and the hold branch's only action was
`m_lastEraCompleteTick = GetTickCount()`, which silences the stall watchdog. So on
USDJPY the two members that could not train reported, and the two healthy members
frozen behind them wrote nothing anywhere. The outage was visible only through the
members that were not suffering it.
- BarrierEraHeartbeat() stamps a clock on real era CHANGE, kept separate from
m_lastEraCompleteTick precisely because the barrier resets that one. Only a
member AT the minimum can be a blocker; a member ahead is idle by design and is
never counted as stuck.
- After ENSEMBLE_BARRIER_STUCK_MS (12 min) a non-advancing member is dropped from
the barrier minimum. It keeps training and rejoins the instant it completes an
era - at which point, being behind, it legitimately becomes the minimum again,
which is the documented resumed-laggard behaviour.
- Both transitions say so loudly, and the release states plainly that the
combined-vote score cannot be computed while the ensemble is desynchronised.
- A held member now writes a rate-limited journal line naming WHICH members it is
waiting on, so the blocker is read off one line.
5. THE PANEL FLICKER.
OnTickHandler gates its terse writer on !m_trainRunActive, and a barrier-held member
returns from Train() before ever setting it - so both writers thought they were the
only one updating the label and fought every tick. That is the reported "Getting
ready..." <-> "Waiting at era N for slower ensemble members" oscillation, and it hit
Perceptron but not Convolutional purely because Convolutional had a run active from
a completed era and Perceptron, resumed from disk, never did. Train()'s message is
the specific one, so it wins.
NEXT STEP once this is running: the stall line now ends in "REJECTED BY: ..." and
the per-handle depths. Read it. Do not reason around it.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-17 15:21:15 -04:00
|
|
|
m_barrierHoldReportTick(0),
|
fix(depth): route EVERY ResizeBuffers call site through one indicator-depth gate
1dda479 clamped the training sweep. It left five other paths asking the indicators
for a depth they cannot serve, and on a live account the quiet ones are worse than
the stall was - a stalled chart is visible, a chart trading on a degraded feature
window is not.
ServableBars(want, context) is now the single gate, and all six go through it:
training sweep clamp, floored at TRAIN_MIN_CLAMPED_BARS (below that a small
positive BarsCalculated is warm-up, which m_coldSweepTick owns)
label prebuild clamp - labels come from price/ADZigZag and would survive a
capped MA, but ResizeBuffers sizes EVERY buffer and a failed
CopyBuffer leaves m_MA EMPTY for the next reader, so this path
could silently re-break the block Train()'s clamp just fixed
live inference HOLD. Below `need` the swing block takes its degraded path and
inference runs on a different feature distribution than the model
was fitted on. This EA sizes real positions off that output, so
no signal beats a mismatched one
online learning HOLD, same reason and worse - this path WRITES to a live trading
model, so a mismatched (features, label) pair is not a wrong arrow,
it is a wrong weight update that compounds every bar
chart rescan clamp - SIGNAL_RESCAN_LOOKBACK_BARS is 5000 and MT5's smallest
"Max bars in chart" is also 5000, so this one is genuinely
reachable; uncapped it repaints the window all-Neutral
research export clamp before the emptiness test, so a capped symbol exports the
depth it has rather than writing a CSV with a dead feature block -
an artefact that looks complete and is silently wrong
Both HOLDs are insurance, not expected states: `need` tops out near 1,152 bars
(16 + 750 + 384 + 2) against a 5,000 floor on the terminal setting. They exist so
the failure mode is unreachable rather than merely unlikely.
Not changed: a genuinely SHORT price history still takes the old degraded path at
every site. That is pre-existing behaviour and narrowing it would mute charts that
trade today, so it stays a separate decision rather than a side effect of this fix.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-17 13:33:16 -04:00
|
|
|
m_inferenceDepthRefusalWarned(false),
|
fix(buffers): revert the MA +1 - it asked for a bar that does not exist and stopped every chart
REGRESSION I INTRODUCED IN 1dda479, live for ~15 minutes.
CSeries::BufferResize -> CheckLoadHistory -> CheckTerminalHistory succeeds only when
Bars() >= size. Train() calls ResizeBuffers with barIndex == Bars(), so sizing the MA
buffer to barIndex + 1 asks for one bar more than the symbol has and fails the WHOLE
ResizeBuffers call. The log named it exactly:
failed to get 50180 bars for USDJPY,PERIOD_H4 (Bars() = 50,179)
failed to get 33983 bars for XAUUSD,PERIOD_H4 (Bars() = 33,982)
StartLabelCachePrebuild() then bailed on the false return and stayed silent, so the
only symptom was Train() reporting "arming the first label-cache prebuild" forever
with labelCacheBars=0 - the panel's "getting ready".
The premise was wrong, not just the arithmetic. The MA block reads GetData(idx) AND
GetData(idx + 1), and at the OLDEST bar that second read is SUPPOSED to fail - there
is no older bar to difference against. Rejecting that one bar is correct behaviour;
buying it cost the entire history.
Two more things, since the same defect had a second instance and no alarm:
- The Ichimoku pair (closeBars and m_Ichimoku, both barIndex + ichiKijun) is the same
bug with a far larger constant, latent only because the feature is off. Both are now
clamped to Bars(). The oldest ichiKijun bars then have no cloud, which that block's
EMPTY_VALUE guard already handles per-bar - the right outcome.
- The prebuild's bare `return` on a false ResizeBuffers now says so once, naming the
depth and Bars(). MQL5's own "failed to get N bars" was in the log the entire time,
from a stack frame nothing connected to the prebuild.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-17 14:12:47 -04:00
|
|
|
m_prebuildBlockWarned(false),
|
2026-08-01 11:27:28 -04:00
|
|
|
m_labelCacheBars(0),
|
|
|
|
|
m_labelCacheAnchorTime(0),
|
|
|
|
|
m_labelCachePrebuilt(false),
|
fix(labels): overlapping-label sample correction + horizon cap on the scale ladder
Three defects, all surfaced by the 2026-08-17 SP500 H4 run that shipped
stop 4.86 / target 9.71 (width 14.57*ATR, horizon 384).
1. EVERY STANDARD ERROR ASSUMED INDEPENDENT SAMPLES. Triple-barrier labels
started one per bar overlap by the label's lifespan, so n calls are worth
~n/L independent observations (Lopez de Prado, AFML ch. 4 - sample
uniqueness). All three sqrt(p(1-p)/n) sites divided by the RAW count.
The tell: the operating point's null-of-the-maximum gate is family-wise and
should fire on ~5% of eras under the null. Measured fire rates - PAI 47/73
(64%), ConvLSTM 9/24, LSTM 8/21 (38%), CONV 4/62 (6%). CONV, the only model
whose margin distribution admits few bins, sat on the null; the rest cleared
a bar that was too low by ~sqrt(L). PAI's deployed threshold consequently
alternated between the ENDS of its own range era to era (0.10 -> 0.88 ->
0.86 -> 0.66; coverage 16% <-> 73%).
TripleBarrierLabel now records when each label became KNOWABLE - the first
winning touch, or both stops, or the timeout - and the prebuild accumulates
the mean. EffectiveSampleSize() feeds the operating point, the member deploy
gate and the ensemble vote gate. Conservative by construction (n/L is an
upper bound on the damage); gates get harder, never easier.
2. THE SCALE LADDER RAN AWAY, again. Horizon scales as swingMedian*sl*tp, and
since 4d8cb08 reachability is measured OVER that horizon - so a wider rung
buys itself the time that makes it look reachable. Same target -> horizon ->
reach -> target loop the excursion window is kept short to avoid; fixing the
window confusion reopened it through the other door. It walked 128 -> 256 ->
384 bars and stopped at q90, the widest rung there is, with every rung
reading 39-48% against a 20% floor. A floor nothing fails selects nothing.
Rungs whose required horizon exceeds BARRIER_HORIZON_MAX are now rejected -
the same rule ReportGeometryExpectancyScan already applied. It was printing
the shipped pair as CLAMPED and disqualified ('h384!') two lines under the
deriver that chose it: two subsystems, one geometry, opposite verdicts.
3. THE RUNG SNAP DESTROYED THE RATIO IT WAS COMPARING. Both legs snapped
independently to the coarse first-passage grid, re-rating each candidate:
q90 4.86/9.71 -> 5.00/10.00 (2.00), q85 4.07/8.14 -> 5.00/10.00 (IDENTICAL
measurement), q75 3.07/6.13 -> 4.00/6.50 (1.63 - a nearer target). So the
ladder compared win shares taken at ratios from 1.63 to 2.17 and read the
differences as scale. It is why the reach column came out non-monotone in
width (q75 48.5% above q90 42.9%). The stop now snaps to its nearest rung in
log space and the target follows the ratio off it; the pair actually measured
is returned and logged, so a collision reads as a collision.
Also: LadderWinShare guarded against the conditional (fractal) geometry path,
which fills n from m_fracLegCount while leaving idxList empty - a latent
out-of-bounds on a currently-dead path.
New log lines: mean label lifespan and effective n on the label-cache line, the
required-vs-available horizon per rung, and the grid pair the reconciliation
actually measured (its tolerance now scales with the grid skew instead of a flat
5pp).
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-17 01:12:05 -04:00
|
|
|
//--- 0 samples => MeanLabelLifespan() returns 1.0 => EffectiveSampleSize() is the identity, so an
|
|
|
|
|
//--- un-prebuilt model behaves exactly as it did before the overlap correction existed rather than
|
refactor(labeling): CTripleBarrier - one copy of the fill/barrier arithmetic
Session B of the feature-selection/labeling refactor track. Extracts the two
pieces of triple-barrier arithmetic that were genuinely duplicated or
scattered, taking price/ATR/geometry as plain arguments - no chart, no
indicator handle - so it is testable with synthetic numbers.
CTripleBarrier::ComputeLevels() replaces the fill/barrier level arithmetic
that TripleBarrierLabel() and SimulateTradeOutcome() each spelled out by
hand; their own comments already called it "IDENTICAL... deliberately and by
copy." One caller resolves both sides at once (the both-won tie-break needs
both); the other selects the side its isLong argument names. Same for
ApplyMinStopWidening(), the broker-minimum-stop floor both walks applied.
Fuzzed 200k random (entry, spread, risk, reward, minStop, isLong) tuples
against both original hand-written forms: 0 mismatches.
CLabelOverlap replaces m_labelLifespanSum/m_labelLifespanCount - two members
reset from three separate call sites (constructor, label-cache rebuild), the
exact "N loose members cleared in more than one place" shape a candidate-
geometry incident (7452bd1) turned into a live bug. One object, one Reset(),
default-constructed like every other object member. MeanLabelLifespan() and
EffectiveSampleSize() on the signal become thin forwarders with an unchanged
signature - every one of their ~15 existing callers, direct and through the
CAIBaseTrainingData adapter, is unaffected.
SnapHorizonToLadder() forwards to CTripleBarrier::SnapToLadder(), the ladder
array's one remaining copy; EffectiveHorizonMax() (the close-all cache) stays
on the signal since that state has no clean argument form.
NOT extracted: TripleBarrierLabel()'s ~200-line walk itself. It resolves both
sides simultaneously, tracks the first-passage ladder, and feeds the label
every live order is sized from; a rewrite of it cannot be checked without a
compiler, so only the two pieces provably identical to their originals moved.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-08-23 18:57:17 -04:00
|
|
|
//--- shrinking its own samples on a guess. See m_lastLabelLifespan. m_labelOverlap is a class
|
|
|
|
|
//--- member and default-constructs itself (CLabelOverlap::CLabelOverlap() calls Reset()) - it has
|
|
|
|
|
//--- no init-list form here, same as m_ladder and every other object member below.
|
fix(labels): overlapping-label sample correction + horizon cap on the scale ladder
Three defects, all surfaced by the 2026-08-17 SP500 H4 run that shipped
stop 4.86 / target 9.71 (width 14.57*ATR, horizon 384).
1. EVERY STANDARD ERROR ASSUMED INDEPENDENT SAMPLES. Triple-barrier labels
started one per bar overlap by the label's lifespan, so n calls are worth
~n/L independent observations (Lopez de Prado, AFML ch. 4 - sample
uniqueness). All three sqrt(p(1-p)/n) sites divided by the RAW count.
The tell: the operating point's null-of-the-maximum gate is family-wise and
should fire on ~5% of eras under the null. Measured fire rates - PAI 47/73
(64%), ConvLSTM 9/24, LSTM 8/21 (38%), CONV 4/62 (6%). CONV, the only model
whose margin distribution admits few bins, sat on the null; the rest cleared
a bar that was too low by ~sqrt(L). PAI's deployed threshold consequently
alternated between the ENDS of its own range era to era (0.10 -> 0.88 ->
0.86 -> 0.66; coverage 16% <-> 73%).
TripleBarrierLabel now records when each label became KNOWABLE - the first
winning touch, or both stops, or the timeout - and the prebuild accumulates
the mean. EffectiveSampleSize() feeds the operating point, the member deploy
gate and the ensemble vote gate. Conservative by construction (n/L is an
upper bound on the damage); gates get harder, never easier.
2. THE SCALE LADDER RAN AWAY, again. Horizon scales as swingMedian*sl*tp, and
since 4d8cb08 reachability is measured OVER that horizon - so a wider rung
buys itself the time that makes it look reachable. Same target -> horizon ->
reach -> target loop the excursion window is kept short to avoid; fixing the
window confusion reopened it through the other door. It walked 128 -> 256 ->
384 bars and stopped at q90, the widest rung there is, with every rung
reading 39-48% against a 20% floor. A floor nothing fails selects nothing.
Rungs whose required horizon exceeds BARRIER_HORIZON_MAX are now rejected -
the same rule ReportGeometryExpectancyScan already applied. It was printing
the shipped pair as CLAMPED and disqualified ('h384!') two lines under the
deriver that chose it: two subsystems, one geometry, opposite verdicts.
3. THE RUNG SNAP DESTROYED THE RATIO IT WAS COMPARING. Both legs snapped
independently to the coarse first-passage grid, re-rating each candidate:
q90 4.86/9.71 -> 5.00/10.00 (2.00), q85 4.07/8.14 -> 5.00/10.00 (IDENTICAL
measurement), q75 3.07/6.13 -> 4.00/6.50 (1.63 - a nearer target). So the
ladder compared win shares taken at ratios from 1.63 to 2.17 and read the
differences as scale. It is why the reach column came out non-monotone in
width (q75 48.5% above q90 42.9%). The stop now snaps to its nearest rung in
log space and the target follows the ratio off it; the pair actually measured
is returned and logged, so a collision reads as a collision.
Also: LadderWinShare guarded against the conditional (fractal) geometry path,
which fills n from m_fracLegCount while leaving idxList empty - a latent
out-of-bounds on a currently-dead path.
New log lines: mean label lifespan and effective n on the label-cache line, the
required-vs-available horizon per rung, and the grid pair the reconciliation
actually measured (its tolerance now scales with the grid skew instead of a flat
5pp).
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-17 01:12:05 -04:00
|
|
|
m_lastLabelLifespan(0),
|
feat(measurement): fix zero-skill denominator, publish the deploy bar, measure lifespan per rung, add a MEASURE scale objective
The last run could not have demonstrated an edge either way, and nothing in the
log said so. Four changes so it does.
1. THE ZERO-SKILL LINE DIVIDED BY THE WRONG DENOMINATOR. m_oosWinLongTotal resets
every era; m_oosSamples only resets on a full model reset. So 'always-long %'
decayed as ~1/era: a run whose true rate is 37% printed 1.2% at era 33 and
0.0% at era 2219. This is the SAME bug already found and fixed for
logBuyPredPct thirty lines above ('era-15 Buy:2% that was really ~30%'), left
in the one line whose whole job is to be the reference every other number is
read against. Correct at era 1, wrong everywhere after - including the '62%
zero-skill' figure in the 2026-08-16 notes. Now per-era, and always-short is
finally readable.
2. THE DEPLOY GATE STATES ITS OWN BAR. 'edge -1pp' era after era cannot separate
'short by a hair' from 'short by an amount no strategy could cover'. The era
line now prints the required win rate, the SE, the effective n and the
lifespan it was deflated by; above 100% it says UNREACHABLE. At 4,738 OOS bars
and L=75.6 there are ~63 independent observations, putting the bar near 66% at
typical coverage.
3. LIFESPAN MEASURED PER RUNG. The first-passage cache already stores touch ages
at every ladder level, so each candidate geometry's resolution time is
readable without training on it - L-vs-width becomes a measurement across the
whole ladder in ONE run rather than a second chart. Each rung reports L,
n_eff, min provable edge and min provable EV.
4. SCALE OBJECTIVE IS PHASE-AWARE, defaulting to MEASURE. Width and detectability
are opposed: labels overlap by L, L grows like m*k = width^2 at fixed ratio,
so min provable EV ~ width^2 while the cost saving from width is only linear.
Doubling width quadruples the smallest EV you can prove. DEPLOY (widest that
clears reachability) is right once an edge is known; MEASURE (narrowest that
keeps round-trip spread under BARRIER_MAX_COST_FRACTION_PCT) is right while it
still has to be shown. The direction does not depend on the exponent, and
item 3 makes the exponent checkable.
Fixed in review: m_lastRungLifespan is cleared on every LadderWinShare entry or a
rejected rung reports the previous rung's lifespan as its own; per-rung
detectability is labelled IS-sample based (the deriver may not see the holdout),
so absolute figures are optimistic while the ranking is unaffected.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-17 10:37:05 -04:00
|
|
|
//--- -1 = the deploy gate has not run its arithmetic yet this era; the era line then omits the bar
|
|
|
|
|
//--- rather than printing a stale one from a previous era.
|
|
|
|
|
m_lastEdgeFloorPct(-1.0),
|
|
|
|
|
m_lastPrecSE(-1.0),
|
|
|
|
|
m_lastEffN(-1.0),
|
feat(gate): cross-instrument pooled certification
The deploy bottleneck is CERTIFICATION, not training. A 4,738-bar OOS window at
L=75.6 holds ~63 independent observations; certifying a 3pp edge at 2 sigma needs
~1,036. More bars of the same symbol barely help - they overlap. Other symbols do
not.
WHAT POOLS. Not win rates: symbols have different derived geometries, different
break-evens and different drifts, so averaging raw rates across them is
meaningless. What pools is each symbol's EXCESS OVER ITS OWN CHANCE RATE,
combined by inverse-variance weighting (fixed-effects meta-analysis). Each symbol
keeps its own model, geometry and chance rate; only the evidence is combined.
THE CORRELATION PROBLEM, bracketed rather than assumed away. SP500 and NAS100 are
~0.9 correlated and pooling them as independent inflates the evidence. Nothing
here can measure that without sharing return series, so instead of guessing a
correction the gate reports both ends:
SE_INDEP = sqrt(1/SUM(1/var_i)) all members independent
SE_CORR = SUM(w_i * sqrt(var_i)) all members perfectly correlated
The truth is always between. THE GATE USES SE_CORR, so a pass cannot be an
artifact of correlated instruments - that bound already assumes the worst. The
ratio is logged as the diversification credit the gate declines to claim, so the
cost of that conservatism is visible instead of hidden.
SCOPE, deliberately limited: the pooled result is REPORTED, never folded into
tradeableOK. The local gate certifies the model that actually trades this symbol;
the pool answers the different question of whether the strategy has an edge at
all. Letting a cross-symbol result license a local deploy would ship a model that
never cleared its own bar - so it cannot.
Mechanics: one file per instrument (no concurrent-write path to get wrong), every
FileOpen carrying FILE_SHARE_READ|FILE_SHARE_WRITE, records skipped rather than
reinterpreted on a version mismatch, 12h staleness cutoff so a stopped chart
cannot vote, and pooling refused below 3 instruments. Poolability requires
matching timeframe and ratio; differing SYMBOL is the entire point. Publishing is
unconditional - a pool that only hears from winners is a selection effect, not a
meta-analysis.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-17 10:45:37 -04:00
|
|
|
m_lastPoolPasses(false),
|
|
|
|
|
m_lastPoolReport(""),
|
feat: fitted directional confidence threshold - selectivity gets a mechanism
The training loss and the selection metric wanted different things and only
the second one knew it. Logit-adjusted cross-entropy has no term for "how
often should I trade", so the head calls a direction on 87-91% of bars. The
selection metric is precision x coverage credit, saturating at the coverage
floor - above the floor extra calls earn NOTHING and only precision counts.
So selection wanted few good calls, the loss produced many mediocre ones, and
all selection could do was pick the least-bad era out of what it was handed.
Nothing pushed the model toward selectivity.
This gives the decision RULE the policy instead of distorting the loss (which
is estimating class probabilities correctly, and a probability estimate should
not be bent to encode a trading policy - Elkan 2001: estimate, then choose the
operating point separately). AdjustedSignalFromSoftmax now abstains unless the
winning direction's softmax margin over its best rival clears a fitted
threshold. Margin, not the winning probability: the latter moves with overall
calibration rather than with how close the decision actually was.
Fitted on IS, applied to OOS and live. Pass 2 already forward-passes every IS
sample, so the margin histogram is harvested there for free (primary
occurrences only, so the oversampled replay queue cannot skew the operating
point); the fit runs at the end of pass 2, BEFORE pass 3, so the deploy gate
grades the thresholded model on bars the threshold never saw. Fitting on
pass 3's own predictions would be choosing the operating point on the data
being graded - the best-of-N error corrected in five other places here.
Objective: maximise IS directional precision subject to still clearing the
SAME coverage floor the deploy gate uses (base rate x 0.25, re-derived
locally so the two cannot drift apart). Swept top-down in one pass; ties go
to the LOWER threshold, since equal precision for less coverage is strictly
worse. Under DIR_CONF_MIN_FIT_CALLS (200) it runs unthresholded rather than
on a guess.
The threshold is part of the MODEL, not the run: captured with
Net.CaptureWeights(), restored with the weights at both restore sites, and
appended to the .cfg under the same length-guard convention so a deployed
model reloads at the operating point its gate actually cleared. A pre-2026-08-09
.cfg reads 0.0, which is exactly the behaviour it was trained under.
Per-era line now prints "@margin>=X.XX" next to coverage, so a coverage drop
can be attributed to the operating point rather than guessed at.
Both build variants compile 0 errors / 0 warnings.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-09 15:04:37 -04:00
|
|
|
//--- 0.0 = unthresholded until the first pass 2 fits it (see DIR_CONF_THRESHOLD_BINS). Deliberately
|
|
|
|
|
//--- the permissive value: a model that has not measured its own operating point must not silently
|
|
|
|
|
//--- abstain on everything.
|
|
|
|
|
m_dirConfThreshold(0.0),
|
|
|
|
|
m_bestDirConfThreshold(0.0),
|
|
|
|
|
m_dirConfPrimaryBars(0),
|
|
|
|
|
m_dirConfSparseWarned(false),
|
2026-08-01 11:27:28 -04:00
|
|
|
m_labelPrebuildActive(false),
|
|
|
|
|
m_prebuildSeedPending(false),
|
|
|
|
|
m_labelPrebuildBars(0),
|
|
|
|
|
m_labelPrebuildOosCutoff(0),
|
|
|
|
|
m_labelPrebuildIndex(-1),
|
|
|
|
|
m_labelPrebuildBuyCount(0),
|
|
|
|
|
m_labelPrebuildSellCount(0),
|
|
|
|
|
m_labelPrebuildNeutralCount(0),
|
refactor(online-learning): OnlineLearning is a real collaborator, not a raw-include partial (S5)
Expert\AIBase\OnlineLearning.mqh (595 lines) -> Expert\OnlineLearning\:
IOnlineLearningView.mqh (abstract, ~50 accessors) + AIBaseOnlineLearningView.mqh/
AIBaseOnlineLearningViewImpl.mqh (the adapter) + OnlineLearning.mqh (COnlineLearning).
STATEFUL, unlike CModelPersistence: grep-verified the shadow net, the OOS
continual-learning simulation state and the pattern-database backfill state are
genuinely exclusive to this file's own methods - Training.mqh/Topology.mqh/
Lifecycle.mqh/the signal's own header only ever CHECKED or RESET this state at
era/lifecycle boundaries, never owned it, so it moved onto the collaborator as
real members (same doctrine as Excursion). Those external touch points became
consolidated view/forward calls instead of raw field pokes - AbortSimIfActive()
replaces THREE separate copies of the same delete/null/false triple (Training.mqh's
stop path, FlushTrainRun, ResetWeights), matching the geometry-scan duplicate-reset
precedent in project memory. ResetForFreshTopology() replaces Topology.mqh's five-
field reset block, DeployNet() replaces the shadow-preferred net selection duplicated
in Inference.mqh and ChartScoreBarForRescan, and BlendTowardNet() replaces the era-end
blend Training.mqh used to poke m_shadowNet for directly.
Reused the signal's existing Data*()/Chart*()/Persist*() getters wherever one already
answered the question (labels/outcome/history/horizon/priors/servable-bars/etc.);
added ~30 new Online*() wrappers only for what nothing else exposed yet. The three
PersistOnline*() get/set pairs (WST3 .stats fields) now forward through the owning
member instead of touching the field directly - CModelPersistence is unaffected.
Every method body is a pure relocation of the original's statements in original
order; verified against `git show HEAD~1:Expert/AIBase/Excursion.mqh`-style diff
against the pre-extraction file kept in the working tree until this commit.
Compiled 0 errors, 0 warnings (stage mirror + MetaEditor64 /compile, ~91s).
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-08-23 22:59:02 -04:00
|
|
|
//--- The shadow net, the OOS continual-learning simulation state, the pattern-database backfill
|
|
|
|
|
//--- state, and the online-learning watermark/guardrail/counters now default-construct on
|
|
|
|
|
//--- COnlineLearning (m_onlineLearning, declared below) - see its own constructor.
|
2026-08-01 11:27:28 -04:00
|
|
|
m_tuneTrialIndex(-1),
|
|
|
|
|
m_tuneBestOosForecast(-1),
|
|
|
|
|
m_tuneLastTrialWasWin(true),
|
|
|
|
|
m_tuneHaveBestCheckpoint(false),
|
|
|
|
|
m_tuneStartTrainBar(0),
|
|
|
|
|
m_tuneFilterDone(false),
|
|
|
|
|
m_trainingPaused(false),
|
|
|
|
|
m_trainingStopRequested(false),
|
|
|
|
|
m_activeFileCommon(true),
|
2026-08-24 04:39:17 -04:00
|
|
|
//--- m_configLockName now default-constructs on CConfigLock (m_configLock, declared below) - see
|
|
|
|
|
//--- its own constructor.
|
2026-08-01 11:27:28 -04:00
|
|
|
m_isInitialized(false),
|
fix(deinit): a full model write was running ahead of the cheap cleanup
"Abnormal termination" is back, and this time it is not the arrows. The
timing names the culprit exactly:
16:02:31.547 OnDeinit: shutting down
16:02:36.003 Abnormal termination <- 4.46 s, MetaTrader gave up
16:02:36.226 chart signals - persisted <- cleanup finished 0.2 s LATE
OnDeinit called StopTraining() BEFORE the chart cleanup. StopTraining()
finalises an in-flight run, and FinalizeTrainRun() restores the best
checkpoint and then persists it - a full ~1MB model write per signal. So
the expensive step ran ahead of the cheap bounded one, which is precisely
the inversion the shutdown ordering exists to prevent. The previous fix
put PersistWeightsOnShutdown last and missed that StopTraining smuggles a
second save in at the front.
Two changes:
Cleanup now runs FIRST, then StopTraining, then the weight save. The
visible teardown is cheap and bounded, so it always completes even when
everything after it is killed.
And the deploy-persist inside FinalizeTrainRun is suppressed during
shutdown. RestoreWeights() is an in-MEMORY swap, so the best checkpoint
is already the live net by that line, and PersistWeightsOnShutdown writes
exactly those weights moments later. The old path wrote the same model
twice per signal - eight full writes across four charts - for no benefit.
A user-pressed Stop still persists immediately, because nothing else
would.
Compiles 0 errors / 0 warnings. Build tag deinit-order-v2.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-01 16:06:40 -04:00
|
|
|
m_shutdownInProgress(false),
|
refactor(chart): ChartUI is a real collaborator, not a raw-include partial (S2)
Expert/AIBase/ChartUI.mqh was 869 lines of method bodies of
CExpertSignalAIBase, #included after the class declaration - free to touch
any of its ~500 members. First of the eleven AIBase/*.mqh partials to come
out (fewest inbound edges - see the SOLID campaign session order), using
the same view+adapter shape already proven for CTrainingDataView.
CChartView (Expert/Chart/IChartView.mqh) is the abstract read/behaviour
surface a chart-rendering collaborator needs - identity, bar/model access,
the prediction cache, and the training/vote/meta scalars the panel and HUD
line summarise. CAIBaseChartView is the adapter the signal owns and binds
to itself (MQL5 gives a class exactly one base, so CExpertSignalAIBase
cannot implement the view directly). CChartUI is the real collaborator: it
owns the arrow-restore queue, the rescan queue/tally, the last-arrows-saved
count and the purge-mismatch latch as its own fields (verified via grep to
be touched nowhere else in Expert/), and reaches everything else - including
StartChartSignalRescan, moved in from its old inline home in the header
since it drives the exact same rescan state machine AdvanceChartSignalRescan
drains - through the view.
m_arrowSignalCache and m_signalClusterWindow stay on the signal: Training.mqh
writes the cache directly every era and the training-data view already reads
it, so moving it would mean rewriting Training.mqh's write sites too - out of
scope here. CChartUI reaches it through four bounds-checked accessors instead
of a raw member poke. All 10 public methods keep their exact signatures and
become one-line forwards on the signal, so no other file's call sites change
except Training.mqh's one era-end status refresh, which now reads
RefreshStatusLabel() rather than reaching into CChartUI's now-private
last-displayed-neuron cache directly.
Verified structurally, not compiled (never compile - the operator does, in
MetaEditor): brace balance checked on every touched/new file against HEAD,
and the view/adapter/impl method lists cross-diffed to confirm all 59
accessors match 1:1 across the interface, the adapter declaration and the
adapter body.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-08-23 20:03:52 -04:00
|
|
|
//--- m_lastArrowsSaved and the purge-mismatch latch now default-construct on CChartUI.
|
diag(autotune): one label shuffle cannot settle the no-edge question
The permutation baseline added in 018afb1 came back on all four charts as
0.00401 nats against floors of 0.00267 / 0.00298 / 0.00318 - three draws
whose spread is as wide as the excess being judged, because one shuffle
is one sample from the null, not the null. That is not enough to retire a
topology on.
Now MI_NOISE_PERMUTATIONS draws, reported as mean +/- sd with a z-score,
plus two numbers the mean over 26 columns cannot express:
- the STRONGEST single feature's MI, against its own shuffled value.
One informative column among 25 useless ones is precisely the case
the mean hides, and precisely the case worth finding.
- the excess as a percentage of H(Y). At these sample sizes a z-score
can be comfortably significant while the effect is worthless, so
"is it real" and "is it big enough to matter" are asked separately
and answered separately.
The verdict line also now states the measure's limit every time rather
than only when the news is bad: this is a MARGINAL, PER-BAR statistic and
the network reads m_historyBars bars jointly, so it can prove signal
exists but never that it does not. It rules out a per-feature edge - and
therefore any indicator retuning - not an edge that lives in a
combination or across time.
Compiles 0 errors / 0 warnings, standard and Market.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-01 13:32:12 -04:00
|
|
|
m_miBestColumn(0.0),
|
2026-08-01 14:01:32 -04:00
|
|
|
m_miLabelEntropy(0.0),
|
diag(autotune): a positive control, and a scan that separates "no signal"
from "signal knocked out of step"
Four architecturally different networks landed on the same precision -
Buy 23-25% against a 25.4% base rate, Sell 19-22% against 22.0% - while
making completely different calls (HYBRID votes Sell on 69% of bars, PAI
on 41%). Precision equal to the base rate is what INDEPENDENCE looks
like, and precision under independence is fixed by the label
distribution, not by the architecture, so all four converging on it is
arithmetic rather than coincidence. Accuracy meanwhile tracks coverage
exactly as independence predicts (31.1/30.3/25.0 predicted vs
31.8/28.9/24.6 observed for PAI/CONV/HYB).
But "no information in the data" and "information destroyed upstream of
every topology" produce that identical picture, and the MI test alone
cannot tell them apart either. Two additions:
POSITIVE CONTROL. Three "measurements" in this codebase have turned out
to be silent no-ops that produced plausible numbers - the MI scorer
reading an array nobody filled, the eval-mode guard that switched off the
imbalance correction, the alternation gate whose premise was never true.
So the estimator now has to prove it responds to a signal known to be
present before any floor reading is believed: the label of a neighbouring
sample row, ~19 bars away and far inside the 128-bar barrier horizon, so
the two outcome windows overlap heavily and MUST be associated. Same
binning, same estimator. Near the floor => every MI figure is void.
ALIGNMENT SCAN. Re-scores against the label taken from bar i+k for k in
-5..+5. A peak at k != 0 is a feature/label misalignment - an off-by-one
in the label index, a horizon applied to the wrong bar, a feature window
that lags what it claims - which would destroy the information before any
topology saw it and would look identical in every accuracy number this EA
prints. A flat profile says the features simply do not carry this target.
The sampled range is trimmed by |k| at both ends so a shift is measured
rather than an edge effect, and both bars must carry a real label.
Also: BuildMiSample publishes its stride instead of the report
recomputing that arithmetic (it would drift), and the control sizes its
buffers from its own sample count rather than the caller's.
Compiles 0 errors / 0 warnings, standard and Market.
Build tag mi-control-align-v1. Redeploy only - no retrain, no model
deletion; the diagnostic runs on resumed models.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-01 14:12:10 -04:00
|
|
|
m_miStrideBars(0),
|
fix(diag): the symbol sweep was measuring its own sampling, not the market
Twelve cells came back with higher-timeframe "signal" 5-9x anything on
H1, at p=0.005. It was an artifact, and the sweep's own columns gave it
away: excess tracked the sampling STRIDE almost monotonically, and the
three D1 cells - stride collapsed to 1-5 bars against a 128-bar horizon,
i.e. ~99% window overlap - were the three highest. Three flaws, all the
same family: comparing numbers without the spread that belongs to them.
1. THE NULL ASSUMED INDEPENDENCE THE LABELS DO NOT HAVE. Triple-barrier
labels overlap; two rows less than one horizon apart share most of their
outcome window. A free Fisher-Yates shuffle destroys that dependence
along with the association, making the null far narrower than the truth
and handing out significance that isn't there - Lopez de Prado ch. 4
arriving through the back door of the significance test. Now permutes
contiguous BLOCKS of at least one horizon, so the null keeps the
autocorrelation and the p-value means what it says. It degrades honestly:
severe overlap leaves few blocks, the null widens, nothing reaches
significance. The block count is now printed, because THAT - not the row
count - is the sample size a p-value rests on, and a warning fires under
30 blocks so "not significant" is not misread as "no signal" when it
means "not enough independent history to tell".
2. THE POSITIVE CONTROL'S STRENGTH DEPENDED ON THE DATASET. It paired
each row's label with the NEXT SAMPLE ROW's, whose distance is the
stride - so on M5, where stride ran 160-717 bars against a 128-bar
horizon, it was pairing two windows that never overlap. All three M5
cells duly reported a FAILED estimator and voided their own results with
nothing wrong. A control whose strength varies with the cell cannot
certify the cell. Now pinned to a quarter of the horizon, where ~75%
overlap is guaranteed by construction.
3. THE LOOKAHEAD VERDICT HAD NO MARGIN. It flagged 7 of 12 cells on gaps
of 0.00008-0.00040 nats against a measured null sd of ~0.00030 - noise,
every one. Now requires 3 sd, the same discipline the deploy floor
applies to precision.
Compiles 0 errors / 0 warnings, standard and Market. Build tag
blockperm-v1. Supersedes every number from the sweep.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-01 15:11:40 -04:00
|
|
|
m_miNullBlocks(0),
|
feat(labels): measure which barrier is predictable at entry, don't guess
The alignment scan settled the shape of the problem: 4.7x more is
knowable 5 bars into a 128-bar window than at the entry the model
actually trades. A 6xATR target reached over 128 bars is decided
overwhelmingly by what happens DURING the window, so whatever the entry
state knows is buried under 128 bars of later noise. That is a property
of the TARGET, and it is why four different architectures all landed on
precision exactly equal to the base rate - no topology can undo it.
So measure the target. For each SL/TP pairing a user can actually select,
relabel the same sampled bars and score how much the SAME features say
about THAT outcome at entry. Seconds, no training, no topology, and it
runs on the diagnostic path that already exists.
Ranked on excess over its OWN null as a share of its OWN H(Y), never on
raw nats: each geometry has a different class balance, hence a different
finite-sample bias and a different amount of information there to find,
so raw MI would rank the most BALANCED label rather than the most
PREDICTABLE one. The break-even win rate m/(m+k) is printed beside each
so the ranking is read next to the bar the model must clear.
Stated in the output because it is the easy thing to get wrong: chance
precision EQUALS break-even at every geometry, so a tighter target does
not hand you expectancy. It buys predictability - less noise piled on top
of what the entry state knows - which is the one thing changing topology
cannot do.
Read-only by construction: it relabels a sampled copy via
TripleBarrierLabel(), never writes the label cache (which belongs to the
configured geometry), and restores the horizon and overrides it borrowed.
The overrides apply only when BOTH are positive, so a half-set pair can
never silently relabel a live run.
Compiles 0 errors / 0 warnings, standard and Market. Build tag
geometry-scan-v1. Redeploy only - no retrain to READ the ranking.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-01 14:32:04 -04:00
|
|
|
m_miReportDone(false),
|
feat(rank): AI models rank their own confidence tiers from held-out outcomes
Closes the caveat 4858507 shipped with: the vote is a confidence percentage,
but only to the extent the pattern weights are measured. AI tier weights sat
at their designed defaults (25/50/75/100) because AI rows only ever arrive
from LIVE journaling, of which a training run produces almost none.
AND A STALE-TIER BUG THAT MADE THE EVIDENCE MEANINGLESS. The OOS scan bucketed
every scanned bar by ConfidenceTier(), which reads dPrevSignal - and
dPrevSignal is assigned in PASS 1 only, never anywhere in the OOS scan. So an
entire era's fires were bucketed by one stale, unrelated bar's confidence and
landed in a SINGLE tier. That is the "tier prec T0:72%(828) T1:n/a(0)
T2:n/a(0) T3:n/a(0)" symptom recorded on 2026-08-16 and attributed to the
calibration clamp. The clamp was real and was fixed then; this is a second,
independent cause of the identical output that survived that fix untouched -
which is why the log kept reading the same afterwards. Two causes, one symptom.
Now ConfidenceTierFor(adjSig): the bar this iteration actually scored.
WHY THIS DOES NOT WRITE ROWS TO THE SIGNAL DB, which was the obvious reading of
"fill the database during training". The user's own observation is the reason:
a classic Pattern_2 is a fixed geometric condition, so its win rate is
legitimately accumulated over years, but an AI Pattern_2 means "confidence
landed in tier 2" and tier 2 under era 100's weights is a different statement
from tier 2 under era 500's. The DB's value is ACCUMULATION, and accumulation
is exactly what is wrong here - it would average together models that no
longer exist, while colliding with the per-table row cap and mixing
measured-on-holdout outcomes into the live ledger's own tables. What the DB
actually supplies is a measured win rate per pattern, and pass 3 already
computes that on held-out bars, thousands at a time. So the model ranks itself
once per era, REPLACING rather than accumulating, which makes the weights
describe the current weights by construction.
ESTIMATOR. Not WinRateFromCounts(): it returns NO_DATA below 100 raw trades
BEFORE shrinking, which here would fire on every tier every era and hand all
four the pooled rate - the tiers could never separate and the mechanism would
be inert. Shrinkage is the answer to a small sample; a floor in front of it
means the shrinkage never runs. Instead: a Beta prior of TIER_PRIOR_EFF_N
pseudo-observations centred on the model's pooled holdout rate, counted in
EFFECTIVE observations, because overlapping triple-barrier labels mean 800 raw
fires can be worth ~12 independent ones. Rounded to the integer, not to the
decade NormalizeWinRate() uses, which would collapse the shrunk tiers back
into one number.
NO SAME-ERA CIRCULARITY, and it falls out of the ordering rather than a guard:
weights are computed at the END of era N, so the vote scored during era N was
cast with era N-1's weights. The deploy gate never grades a vote whose weights
were fitted on the bars it is scoring. Residual leakage remains - the same OOS
bars each era under a different model - and is stated in the code rather than
papered over.
Both DB clobber paths are closed: ApplyPatternWeight() declines once
self-ranked, and UpdateSignalsWeights()' filter.Weight() call is guarded by
SelfRanked() - guarding only the tiers would have let the hourly ranking pass
undo half the self-ranking.
NOT COMPILED - user compiles in MetaEditor.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-18 16:00:32 -04:00
|
|
|
m_tiersSelfRanked(false),
|
fix(chart): display now reads era-end SNAPSHOTS - the live cache is wiped mid-era
Full-pipeline analysis after "threshold 30, attained often, nothing drawn,
still glued to buy". The log falsified the premise before any code did:
21:40:43 swept 4999, 794 voters, drew 491. Strongest 43.0% vs 30.0%
21:42:07 swept 4999, 0 voters, drew 0
21:51:30 swept 4999, 0 voters, drew 0
21:56:30 swept 4999, 922 voters, drew 382. Strongest 44.0% vs 30.0%
The arrows WERE drawn - 491 of them, then 382 - and then erased. ONE root
cause, three symptoms: every display path read m_arrowSignalCache, which is
wiped to sentinel at each era start and only complete again when pass 3
finishes. With eras at ~30s and a sweep at ~17s:
* ARROW FLICKER: a sweep landing mid-era found no voters anywhere, and its
else-branch deleted the arrow on every voteless bar - erasing the previous
sweep's entire output. The chart cycled populated -> blank -> populated;
the user kept catching the blank phase.
* READOUT GLUE: the newest-cache walk found only sentinel for ~90% of every
era and fell through to dPrevSignal - the frozen purge-band edge bar that
reads Buy. 659638e fixed which bar was frozen, not the freezing.
* VOTER FLAP: 1299 -> 257 -> 1113 across back-to-back sweeps - each saw a
different fraction of half-rebuilt caches.
THE FIX, structural rather than another patch:
1. Era-end snapshots. RankTiersFromOos() runs at pass-3 completion - the one
moment the cache is complete - and now copies it (raw signals, newest
LOOKBACK+16 bars) into member-owned snapshot state, unconditionally,
BEFORE its early return: an all-Neutral era is a snapshot worth showing,
not an absence of one. Raw signals rather than votes, so a tier re-rank
between eras reprices them at read time via LiveVoteContribution for free.
2. The sweep (SnapshotVoteAt) and the prospective readout both read
snapshots; the readout's fallback chain is live-cache -> snapshot ->
dPrevSignal, and the snapshot leg is the one that fires most of the time.
3. NO DATA IS NOT A VERDICT: a den==0 bar no longer deletes - only an actual
sub-threshold vote takes an arrow down. This alone ends the wipe half of
the flicker even where snapshots are missing (before the first era).
4. Arming moved from an era-counter diff (which fires at era BOUNDARIES,
i.e. precisely when caches are about to be wiped) to
g_warriorOverlayArmRequest, set by each RankTiersFromOos - "a member's
snapshot just got fresher", the only event a redraw can act on. 60s rate
limit collapses the four members' burst into one sweep. Classic-only
charts arm once at start.
5. Census now reports the direction split - "922 had a voter (610 buy / 312
sell)" - so "the vote leans buy" is checkable from the log instead of
inferred from arrow colours.
Also visible in the log and worth knowing: the threshold flip-flopped
30 -> 40 -> 30 across the evening's re-inits (census lines at 21:42-21:51
ran at 40), so part of the observed blankness was configuration, not code.
NOT COMPILED - user compiles in MetaEditor.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-18 22:03:20 -04:00
|
|
|
m_overlaySnapBars(0),
|
|
|
|
|
m_prospectiveSigSnap(-2.0),
|
feat(vote): replay pass rebuilds a deployed model's ladder without retraining
The previous commit persisted the tier ladder, which fixes this going
forward but did nothing for models whose .stats predates WST7 - they
still had to retrain to mint one. They never did. Every number a
converged model needs in order to vote is a pure function of weights
already on disk plus labels derivable from the chart, so replay them:
stage 1 build the label cache (existing chunked prebuild)
stage 2 rescan history (existing chunked rescan, deployed net)
stage 3 score + rank + persist (one walk over two arrays)
ScoreReplayFromCache() walks m_arrowSignalCache against
m_labelCacheBuy/Sell, fills the same m_oosTierFired/Hits and per-class
totals pass 3 fills, and hands them to RankTiersFromOos() - deliberately
feeding the existing ranker rather than reimplementing it. The shrinkage,
the chance reference and the module trust weight are subtle enough that a
second copy would drift, and a ladder measured by a slightly different
rule would be silently incomparable with every ladder training produced.
AdvanceDeployedRebuild() sequences the three stages off the timer. It has
to be a sequence: stages 1 and 2 are each minutes of work draining in
time-boxed slices, and stage 2's output is meaningless until stage 1 has
labels to score against. The previous version ran the rescan with no
labels at all, which is why it could only ever rebuild arrows and never
the ladder - the thing actually blocking the vote.
The result is written to .stats immediately. The failure being repaired
is state that lived in memory and was never written down; recomputing it
and not saving it would repeat that exactly.
Also routes every rescan completion through one hook, so there is a
single place that knows what a finished rescan means - republish for a
manual one, score and rank for a rebuild.
Compile-verified in _claude_stage: 0 errors, 0 warnings.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-25 15:01:24 -04:00
|
|
|
m_deployedRebuildStage(0),
|
feat(hud): per-member neuron lines + a vote label that moves as the nets learn
Both 2026-08-19 reports were the same staleness: every source behind the
label was an ERA artifact (live cache refills at pass-3 completion, the
snapshot copies once per era, dPrevSignal is the frozen purge-band edge
bar) - so the readout stepped at era cadence at best, stayed glued to
one direction, and lagged the era counter.
DisplayInference(): throttled (4s, 1s across an era boundary),
SIDE-EFFECT-FREE forward of the current decision bar (window ending on
bar 1, same question the live path asks) through the LEARNER net.
Batch-norm running stats are bracketed frozen/RESTORED via the new
CNet::GetBatchNormFrozen() + CNeuronBatchNormOCL::StatsFrozen() - restore,
not unfreeze, because a display tick can land between pass-3 chunks whose
whole scan holds them frozen. Writes nothing a trading or training path
reads (dPrevSignal, NMS state, tallies, watermarks all untouched;
RefreshLatestSignal is not reusable here precisely because it writes all
of them). LSTM safe by construction: h/c zeroed per forward.
ProspectiveVote() reads the fresh forward as its FIRST source; the
era-artifact chain becomes the fallback (meta head, warm-up, window
holes).
DisplayHudLine(): the reference library's training label, per ensemble
member - name, output activations (softmax probs or raw scalar), the
decision, its weighted vote (the exact consensus numerator term), era,
recent average error, "(trn)" while not vote-capable. Rendered under the
vote line in RefreshVoteReadout BEFORE the live-vote defer (member lines
are telemetry, not tradable readings), coloured by the member's own
direction in muted tones - the vote line's strict
green-only-when-it-would-trade rule is untouched.
NOT COMPILED - user compiles in MetaEditor.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-19 08:48:34 -04:00
|
|
|
m_dispSignal(0.0),
|
|
|
|
|
m_dispValid(false),
|
|
|
|
|
m_dispStamp(0),
|
|
|
|
|
m_dispEra(-1),
|
feat(logs): throttle the settled per-era diagnostics - measured 22MB/9.5h of confirmed-working systems
Measured from the journal (2026-08-19): the era deep-dive line (~2KB) plus
the excursion verdict, tier re-rank, calibration move, barrier hold and
selection-regressed note each printed EVERY era for EVERY member - ~940
eras/member/day - long after the systems they watch were confirmed
working. Yesterday's file was 1.3GB (70% of it the news-filter calendar
spam the sweep fix already removed).
VerboseMode returns as an INPUT (demoted 2026-08-01 for the marketplace;
that track is dead since the 2026-08-16 pivot) and gains a second job:
false throttles each settled per-era print to eras 0-3 plus every
TRAIN_LOG_EVERY_ERAS-th (25 ~= one deep-dive per ~15min per member);
true restores the per-era firehose, flippable live.
Never throttled: anything that marks a CHANGE - new bests, restores +
eta decays, plateau stage transitions, deploy approvals, warnings,
errors, the label-cache/adoption one-shots, and the combined-vote gate
line (the active system's primary telemetry, still every era).
Semantic fixes over blanket gating:
- barrier hold now ARMS silently and prints only when the hold outlasts
the 2-min report interval - a brief hold every era is the design, the
long hold is the watchdog case the line exists for;
- the ensemble deploy REFUSAL prints immediately when its reason
changes (that is a finding), on cadence when unchanged;
- the filtered-view census prints when its RESULT moves (drawn count,
or strongest vote by >=2pp) and at least every 10th sweep.
NOT COMPILED - user compiles in MetaEditor.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-19 09:36:15 -04:00
|
|
|
m_lastEnsRefusalKey(0),
|
feat(pool,mi): one feature layout fleet-wide, and the keep-screen stops self-disabling on a cold start
TWO CHANGES, BOTH RETRAIN-FORCING BY INTENT.
1. SP500 was training alone, and one alt-data column was the reason.
The alt block's width joins the model fingerprint, and the pool reader only
adopts peer rows whose fingerprint and width match. The exporter gives each
instrument the series that apply to it - FX 15 columns, metals/oil 14, SP500
13 - so the fleet ran as three incompatible pools:
EURUSD/USDJPY/USDCAD adopt ~57-60k peer rows each
XAUUSD/XTIUSD adopt 6.4k / 20.3k
SP500 "EVERY peer file was REJECTED, so this chart is
training alone" - 0 rows
SP500 therefore trained on 2279 independent observations against a 600-wide
input with its first layer floored at 16, printing its own "expect
overfitting" warning. It is the one chart with no pool and the worst
capacity ratio in the fleet by a factor of three.
Fresh models now pin ALTDATA_FLEET_COLUMNS - the 12-column intersection -
instead of their own file header. An existing model still adopts its .cfg
pin, so this re-keys nothing that is already trained.
Intersection rather than union: filling an absent series with its median
makes that column constant per instrument, which lets a pooled model
identify the source instrument and stop learning the shared mechanism. It
is also 6 columns narrower. Cost is six columns whose retained information
is UNMEASURED - the keep-screen reports a bitmask nothing has mapped back
to names.
2. The MI keep-screen disabled itself for the whole run on any cold start.
ReportFeatureLabelInformation set m_miReportDone on ENTRY. On a cold start
the label cache is allocated before it is filled, so BuildMiSample finds no
row carrying a resolved label and returns 0 - a sixth exit, and the only
one the 8c1266d instrumentation did not cover, which is why it printed
nothing. observed then stayed -1, the permutation loop never iterated, and
the report emitted "-1.00000 nats over 0 permutations" beside a plausible
"strongest single feature 0.05979" that was a STALE m_miBestColumn from an
earlier scoring call. The first ensemble member propagated the latch to
g_ensembleChartMiReportDone and silenced every member on the chart.
The flag now latches only once a measurement exists. A short sample is
reported as a deferral naming the two numbers that identify it (cached bars
vs bars carrying a resolved label) and retried, up to
MI_REPORT_MAX_ATTEMPTS.
Build tag -> fleet-pool-v1.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-26 17:41:14 -04:00
|
|
|
m_miReportDeferrals(0),
|
|
|
|
|
m_miReportAttempts(0)
|
2026-08-01 11:27:28 -04:00
|
|
|
{
|
fix(ensemble): responsive panel + synchronized eras + combined-vote accuracy
Four user-reported/requested items, one root cause chain:
1) DEAD CONTROL PANEL in AI_HYBRID mode. All members posted custom event
id 1 and handled id 1001, and CExpertCustom broadcasts every chart
event to every filter - so each posted event ran a train chunk in ALL
N members (N*N chunks per round) and the chart thread never idled
long enough to deliver clicks/drags. profiling.csv: 99.45% of time in
OnChartEventHandler. Fix: per-instance study-event ids
(STUDY_EVENT_ID_BASE + construction order, offset above the Controls
library's ON_* codes - id 1 was also ON_DBL_CLICK, so panel
double-clicks fired training chunks). ArmStudyEvent() is the single
post site; lost-event watchdog replaces the accidental
sibling-clears-my-flag rescue.
2) WARM-UP DUPLICATION. The auto-tune sweep is deterministic over
identical features/labels, and it ends in the full MI diagnostic
suite, which the MI-share gate never intercepted on the sweep path -
four members ran four identical ~36s sweep+report blocks. First
member publishes outcome (g_ensembleChartTuneDone/Installed/Settings);
the rest apply it and skip both.
3) DEINIT STRANDED PANEL+ARROWS (user repro 18:52). Root cause from the
log: the 4,500ms budget runs from MetaTrader's stop REQUEST - a heavy
autosave in flight ate it, OnDeinit got ~430ms and died in the first
member's arrow persist ("Abnormal termination" 432ms in). Fix: early
visible-UI sweep (native prefix deletes for status/panel/dialog)
right after ClearStatusLabel, and a fast path for still-training
models - their arrows are re-rendered every era, so they get one bulk
purge instead of scan+atomic-write in the death window.
4) ENSEMBLE FEATURES (user requests): era BARRIER - members advance era
by era together; a member ahead of the slowest still-training member
declines Train() calls and its chunk budget is donated
(TRAIN_TIME_BUDGET_MS = 120/activeTrainers, UI headroom constant).
COMBINED-VOTE OOS SCORE - each member's pass-3 scan contributes its
adjusted per-bar decision (0.0 on abstain) to a shared row buffer;
the last member to finish the era scores the averaged vote vs the
mirrored Min_Vote_Open against the same target-before-stop outcomes
members grade themselves on, publishing an "Ensemble vote" line on
the aggregated panel. Member headlines now carry their lifetime win
rate with break-even.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-16 19:06:04 -04:00
|
|
|
//--- Claim this instance's study-event id - see STUDY_EVENT_ID_BASE (ExpertSignalAIBase.mqh) for why
|
|
|
|
|
//--- these are per instance and offset above the Controls library's event codes.
|
refactor(arch): a read-only training-data view, so modules stop being #included code
The AIBase\*.mqh files are not modules. They are method bodies of one
3,400-line class, textually #included after its declaration. Every one
of them can touch every member of every other, which is why "move this
out" has so far meant "move the whole class".
Introduce the seam that ends that:
CTrainingDataView abstract - the ONLY thing a training-side
collaborator may see: a feature row, a label, an
outcome, an excursion, the shape they share, and
the identity to log under.
CAIBaseTrainingData the adapter. MQL5 gives a class exactly one base
and CExpertSignalAIBase is already a
CExpertSignalCustom, so it cannot implement the
view itself. It owns one of these instead.
Data*() on the the published read API the adapter forwards to.
signal MQL5 has no `friend`, so reaching in from outside
was never an option - and making it explicit is
the point rather than a workaround.
Every row accessor OWNS ITS BOUNDS TEST and answers false for a bar it
has nothing for. Thirty-odd call sites currently carry their own
ArraySize() guard; one that forgets reads past a cache that is shorter
than the bar count for the whole warm-up. The -2.0 "never scored"
sentinel on the arrow cache is folded in the same way, so it can no
longer be mistaken for a small confidence.
Nothing uses it yet - this is the seam only, kept as its own commit so
the pattern compiles before 951 lines of Baselines move onto it. The
pattern is the stdlib's own: abstract base with =0 (Canvas\DX\DXObject),
concrete override, forward-declared owner pointer.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-23 11:53:00 -04:00
|
|
|
//--- BIND THE VIEW FIRST. Collaborators are handed TrainingData() and nothing else, so an unbound
|
|
|
|
|
//--- adapter would answer every question with a safe default and a diagnostic would quietly report
|
|
|
|
|
//--- nothing at all - which is worse than one that fails loudly.
|
|
|
|
|
m_trainingData.Bind(GetPointer(this));
|
refactor(baselines): the first real module - a class, not an #included partial
Baselines was 951 lines of CExpertSignalAIBase method bodies in a file
that only looked like a module. It is now CBaselineComparator: a class
the signal OWNS, which reads a CTrainingDataView and prints. It does
not name the signal anywhere in its code.
What the seam forced out into the open:
- Thirty-odd ArraySize() bounds tests, each carried by its caller, are
now one test per accessor next to the data. The two `hasValueN` and
one `arrowN` locals are gone with them.
- The -2.0 "never scored" sentinel on the arrow cache was tested at the
call site. It is now inside DataDirectionalCall, where it cannot be
read as a small confidence.
- DoubleToSignal needs m_outputNeuronsCount, so a raw double could not
be turned into a side by any reader. The view answers
DirectionalCall(bar, isBuy, magnitude) instead - the conversion
happens where the head width lives, and the module no longer needs
ENUM_SIGNAL at all.
- m_baselineDone was a latch on the signal for a decision only this
module makes. It is m_done, private, where it belongs.
Correction to my own earlier claim: I said Baselines had nine exclusive
members "polluting the signal class". It had none. m_x, m_f, m_ngrad,
m_AvgCE and the rest are FIELDS OF ALGLIB REPORT OBJECTS (state.m_x,
mrep.m_AvgCE) that my `\bm_\w+` scan matched after the dot. The module
needs no private state but its view pointer and that latch - which is
why it came out this cleanly.
The include sits below the g_ens* vote globals and the Alglib headers
it reads, because unlike the AIBase\*.mqh partials this is a real class
declaration compiled where it stands.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-23 12:00:18 -04:00
|
|
|
//--- ...and every collaborator gets the VIEW, never `this`. That is what stops a module from
|
|
|
|
|
//--- quietly growing a second dependency on the signal the way the AIBase\*.mqh files all did.
|
|
|
|
|
m_baselines.Bind(GetPointer(m_trainingData));
|
refactor(chart): ChartUI is a real collaborator, not a raw-include partial (S2)
Expert/AIBase/ChartUI.mqh was 869 lines of method bodies of
CExpertSignalAIBase, #included after the class declaration - free to touch
any of its ~500 members. First of the eleven AIBase/*.mqh partials to come
out (fewest inbound edges - see the SOLID campaign session order), using
the same view+adapter shape already proven for CTrainingDataView.
CChartView (Expert/Chart/IChartView.mqh) is the abstract read/behaviour
surface a chart-rendering collaborator needs - identity, bar/model access,
the prediction cache, and the training/vote/meta scalars the panel and HUD
line summarise. CAIBaseChartView is the adapter the signal owns and binds
to itself (MQL5 gives a class exactly one base, so CExpertSignalAIBase
cannot implement the view directly). CChartUI is the real collaborator: it
owns the arrow-restore queue, the rescan queue/tally, the last-arrows-saved
count and the purge-mismatch latch as its own fields (verified via grep to
be touched nowhere else in Expert/), and reaches everything else - including
StartChartSignalRescan, moved in from its old inline home in the header
since it drives the exact same rescan state machine AdvanceChartSignalRescan
drains - through the view.
m_arrowSignalCache and m_signalClusterWindow stay on the signal: Training.mqh
writes the cache directly every era and the training-data view already reads
it, so moving it would mean rewriting Training.mqh's write sites too - out of
scope here. CChartUI reaches it through four bounds-checked accessors instead
of a raw member poke. All 10 public methods keep their exact signatures and
become one-line forwards on the signal, so no other file's call sites change
except Training.mqh's one era-end status refresh, which now reads
RefreshStatusLabel() rather than reaching into CChartUI's now-private
last-displayed-neuron cache directly.
Verified structurally, not compiled (never compile - the operator does, in
MetaEditor): brace balance checked on every touched/new file against HEAD,
and the view/adapter/impl method lists cross-diffed to confirm all 59
accessors match 1:1 across the interface, the adapter declaration and the
adapter body.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-08-23 20:03:52 -04:00
|
|
|
m_chartView.Bind(GetPointer(this));
|
|
|
|
|
m_chartUI.Bind(GetPointer(m_chartView));
|
refactor(persistence): ModelPersistence is a real collaborator, not a raw-include partial (S3)
Expert/AIBase/Persistence.mqh (598 lines, 8 methods) -> Expert/Persistence/:
IPersistenceView.mqh (abstract, 68 read+write accessors) + AIBasePersistenceView.mqh/
AIBasePersistenceViewImpl.mqh (the adapter) + ModelPersistence.mqh (CModelPersistence,
the real collaborator - signal owns m_modelPersistence and binds it to m_persistenceView,
same shape as ChartUI's S2).
Grep-verified before starting: every field these 8 methods touch is ALSO touched
elsewhere in the class (Training/Lifecycle/OnlineLearning/Topology/FeatureScreen/
Labels.mqh) or already exposed via ChartView. Zero exclusive state, unlike ChartUI's
arrow-restore/rescan queues - CModelPersistence is stateless, holding only the
borrowed view pointer, operating entirely through 68 Persist*/PersistSet*() accessors
on the signal.
ValidateCpuInference's Net-pointer/throwaway-clone core is ONE consolidated view call
(PersistRunCpuInferenceSelfCheck) rather than field-by-field - irreducible pointer/
object work, not signal state, same doctrine as ChartScoreBarForRescan.
LoadNetWithRetry keeps its original CheckPointer(Net)-free Net.Load() call unchanged
(no guard added - would change failure behaviour on what must be a pure relocation).
This code writes the actual on-disk .cfg/.stats binary layouts every deployed model
depends on (explicit "DO NOT REORDER" comment in the original), so beyond compiling
clean (0 errors, 0 warnings) this was verified with a positional field-order diff:
every FileWrite*/FileRead* call's target field, extracted and normalized from both
the original and the new file, matches 1:1 in the same order (43/43 on the write
side covering SaveModelStats+SaveTopologyConfiguration, 17/17 on LoadModelStats'
read side; LoadAndCompareTopologyConfiguration's local-variable read block was
copied verbatim, untouched, so nothing to diff there). The magic-version
conditionals (WST2-6, haveDerivedStages/haveBarrierGeometry/etc.) moved unchanged.
All 8 methods keep their exact original signatures as one-line forwards - zero
external call sites changed.
2026-08-23 21:45:09 -04:00
|
|
|
m_persistenceView.Bind(GetPointer(this));
|
|
|
|
|
m_modelPersistence.Bind(GetPointer(m_persistenceView));
|
refactor(online-learning): OnlineLearning is a real collaborator, not a raw-include partial (S5)
Expert\AIBase\OnlineLearning.mqh (595 lines) -> Expert\OnlineLearning\:
IOnlineLearningView.mqh (abstract, ~50 accessors) + AIBaseOnlineLearningView.mqh/
AIBaseOnlineLearningViewImpl.mqh (the adapter) + OnlineLearning.mqh (COnlineLearning).
STATEFUL, unlike CModelPersistence: grep-verified the shadow net, the OOS
continual-learning simulation state and the pattern-database backfill state are
genuinely exclusive to this file's own methods - Training.mqh/Topology.mqh/
Lifecycle.mqh/the signal's own header only ever CHECKED or RESET this state at
era/lifecycle boundaries, never owned it, so it moved onto the collaborator as
real members (same doctrine as Excursion). Those external touch points became
consolidated view/forward calls instead of raw field pokes - AbortSimIfActive()
replaces THREE separate copies of the same delete/null/false triple (Training.mqh's
stop path, FlushTrainRun, ResetWeights), matching the geometry-scan duplicate-reset
precedent in project memory. ResetForFreshTopology() replaces Topology.mqh's five-
field reset block, DeployNet() replaces the shadow-preferred net selection duplicated
in Inference.mqh and ChartScoreBarForRescan, and BlendTowardNet() replaces the era-end
blend Training.mqh used to poke m_shadowNet for directly.
Reused the signal's existing Data*()/Chart*()/Persist*() getters wherever one already
answered the question (labels/outcome/history/horizon/priors/servable-bars/etc.);
added ~30 new Online*() wrappers only for what nothing else exposed yet. The three
PersistOnline*() get/set pairs (WST3 .stats fields) now forward through the owning
member instead of touching the field directly - CModelPersistence is unaffected.
Every method body is a pure relocation of the original's statements in original
order; verified against `git show HEAD~1:Expert/AIBase/Excursion.mqh`-style diff
against the pre-extraction file kept in the working tree until this commit.
Compiled 0 errors, 0 warnings (stage mirror + MetaEditor64 /compile, ~91s).
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-08-23 22:59:02 -04:00
|
|
|
m_onlineLearningView.Bind(GetPointer(this));
|
|
|
|
|
m_onlineLearning.Bind(GetPointer(m_onlineLearningView));
|
refactor(topology): split shape derivation into CTopology, leave the boot sequence in place
Expert/AIBase/Topology.mqh (1191 lines) held two genuinely different jobs: the
fingerprint/derived-shape/BuildFreshTopology math, and InitNeuralNetwork/
InitFeatureIndicators - the network boot sequence (config-lock, tester-cache
seeding, load/save the .cfg, net-load backend fallback, chart/persistence/
online-learning orchestration).
Extracted the first job to Expert/Topology/ as CTopology + CTopologyView/
CAIBaseTopologyView (20 methods: BuildModelFingerprint, the Estimated*/Compute*
budget math, the Conv*/Lstm* shape helpers, Add*Stage, BuildFreshTopology).
STATELESS, like ModelPersistence - grep-verified zero exclusive fields, every
member these methods touch is shared elsewhere in the signal. Reused ~15
existing Data*/Chart*/Persist*/Exc* getters per the established convention;
added ~20 new getter overloads next to their existing setters (UseVolumes(),
MinDirectionalRecall(), etc. - same pattern as SignalClusterWindow) and ~16
new Topology*() wrappers for fields with no prior accessor. The Net-pointer
swap in BuildFreshTopology is one consolidated view call
(TopologyReplaceNetFromTopology), same doctrine as Persistence's
RunCpuInferenceSelfCheck - irreducible pointer work, not signal state.
Deliberately did NOT extract InitNeuralNetwork/InitFeatureIndicators: they
orchestrate nearly every other collaborator (chart, persistence, online-
learning, cross-asset, config-lock) rather than deriving a shape, so moving
them would just relocate a hub, not reduce coupling - same judgment call as
Inference.mqh (assessed, not extracted). They stay in the AIBase/Topology.mqh
partial, byte-identical to before (diffed against git HEAD to confirm), and
now call the extracted math through the same public forwards every other
caller already used.
Verified: string- and numeric-literal diff of the old file's 20 method bodies
against the new CTopology methods (0 differences), InitNeuralNetwork/
InitFeatureIndicators byte-identical, self-compiled 0 errors/0 warnings.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-08-23 23:23:00 -04:00
|
|
|
m_topologyView.Bind(GetPointer(this));
|
|
|
|
|
m_topology.Bind(GetPointer(m_topologyView));
|
refactor(features): FeatureBuilder is a real collaborator, not a raw-include partial
Expert/AIBase/Features.mqh (2017 lines, 38 methods) split by exclusivity grep
(whole-repo, not just Expert/): 30 methods -> Expert/Features/FeatureBuilder.mqh
(CFeatureBuilder + CFeaturesView/CAIBaseFeaturesView), 8 stay behind as a much
smaller raw partial.
CFeatureBuilder is STATEFUL, same shape as Excursion/OnlineLearning: owns the
10 feature-only indicator handles (m_Volumes/m_MA/m_RSI/m_MACDFeature/
m_Ichimoku/5 AD* CiCustom indicators - grep-verified touched nowhere else in
the repo, only their bare declarations) plus the depth-probe/handle-repair/
spread-series/detectability-latch scalars (exclusive, Lifecycle.mqh ctor-init
only elsewhere). m_Open/m_Close/m_High/m_Low/m_Time/m_ATR/m_ADZigZag stay
signal-owned - Labels.mqh/AutoTune.mqh/Training.mqh read them directly - and
are reached read-only through the view (FeatureOpenAt/FeatureHighAt/
FeatureLowAt/ChartBarClose/ChartBarTime/OnlineAtrMain, all reused where a
forward already existed).
Deliberately did NOT move InitOpen/InitClose/InitHigh/InitLow/InitTime/
InitADZigZag/ResizeBuffers/RefreshData: they manage the 7 shared indicators'
Create/BufferResize/Refresh lifecycle, which would need a pure-relay wrapper
per operation per indicator for zero coupling benefit - same judgment as
Topology's boot sequence. They stay in Expert/AIBase/Features.mqh and reach
CFeatureBuilder's 10 owned indicators through 20 new Feature*BufferResize()/
Feature*Refresh() forwards (signal calling into its own owned collaborator
directly, no view needed in that direction).
Whole-repo grep (not just Expert/) caught a real external miss the campaign's
own doctrine warns about: Signals/SignalMETA.mqh read m_spreadSeries/
m_spreadSeriesBars directly as an inherited protected field (a subclass, not
an AIBase/*.mqh partial) - fixed with two new FeatureSpreadSeriesBars()/
FeatureSpreadSeriesAt() forwards.
Verified: if(/for(/while( counts identical between the original file and the
new split (269/20/1); return-count delta (+12) fully accounted for by the 12
new trivial one-line forwards added (10 indicator BufferResize + 2 spread-
series getters); quoted-string-literal diff empty except two doc-comment
paraphrases. Self-compiled 0 errors, 0 warnings.
2026-08-24 00:00:31 -04:00
|
|
|
m_featuresView.Bind(GetPointer(this));
|
|
|
|
|
m_featureBuilder.Bind(GetPointer(m_featuresView));
|
2026-08-24 04:39:17 -04:00
|
|
|
m_configLockView.Bind(GetPointer(this));
|
|
|
|
|
m_configLock.Bind(GetPointer(m_configLockView));
|
fix(ensemble): responsive panel + synchronized eras + combined-vote accuracy
Four user-reported/requested items, one root cause chain:
1) DEAD CONTROL PANEL in AI_HYBRID mode. All members posted custom event
id 1 and handled id 1001, and CExpertCustom broadcasts every chart
event to every filter - so each posted event ran a train chunk in ALL
N members (N*N chunks per round) and the chart thread never idled
long enough to deliver clicks/drags. profiling.csv: 99.45% of time in
OnChartEventHandler. Fix: per-instance study-event ids
(STUDY_EVENT_ID_BASE + construction order, offset above the Controls
library's ON_* codes - id 1 was also ON_DBL_CLICK, so panel
double-clicks fired training chunks). ArmStudyEvent() is the single
post site; lost-event watchdog replaces the accidental
sibling-clears-my-flag rescue.
2) WARM-UP DUPLICATION. The auto-tune sweep is deterministic over
identical features/labels, and it ends in the full MI diagnostic
suite, which the MI-share gate never intercepted on the sweep path -
four members ran four identical ~36s sweep+report blocks. First
member publishes outcome (g_ensembleChartTuneDone/Installed/Settings);
the rest apply it and skip both.
3) DEINIT STRANDED PANEL+ARROWS (user repro 18:52). Root cause from the
log: the 4,500ms budget runs from MetaTrader's stop REQUEST - a heavy
autosave in flight ate it, OnDeinit got ~430ms and died in the first
member's arrow persist ("Abnormal termination" 432ms in). Fix: early
visible-UI sweep (native prefix deletes for status/panel/dialog)
right after ClearStatusLabel, and a fast path for still-training
models - their arrows are re-rendered every era, so they get one bulk
purge instead of scan+atomic-write in the death window.
4) ENSEMBLE FEATURES (user requests): era BARRIER - members advance era
by era together; a member ahead of the slowest still-training member
declines Train() calls and its chunk budget is donated
(TRAIN_TIME_BUDGET_MS = 120/activeTrainers, UI headroom constant).
COMBINED-VOTE OOS SCORE - each member's pass-3 scan contributes its
adjusted per-bar decision (0.0 on abstain) to a shared row buffer;
the last member to finish the era scores the averaged vote vs the
mirrored Min_Vote_Open against the same target-before-stop outcomes
members grade themselves on, publishing an "Ensemble vote" line on
the aggregated panel. Member headlines now carry their lifetime win
rate with break-even.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-16 19:06:04 -04:00
|
|
|
m_studyEventId = (ushort)(STUDY_EVENT_ID_BASE + g_warriorStudyEventSeq++);
|
|
|
|
|
m_studyArmedTick = 0;
|
|
|
|
|
m_ensembleIndex = -1; // not an ensemble member until EnsembleMember(true) registers one (the flag itself is in the init list)
|
feat(ensemble): deploy gate on the COMBINED VOTE, with a joint checkpoint
The unit of evaluation in ensemble mode becomes the vote, because the
vote is what trades (user: "at the end of the day they will vote
together during live trading so that would make sense").
Four decisions move from the member to the ensemble:
* which era is "best" -> the era whose COMBINED VOTE scored best
* what is checkpointed -> a JOINT snapshot: every member's weights
at that one era
* when the run gives up -> one shared plateau ladder
* whether it may deploy -> family-wise gate on the vote
WHY THE JOINT CHECKPOINT IS THE POINT: per-member selection picks each
net's own best era, and those eras differ. The resulting quartet was
never measured together at any instant, so the vote it casts live is a
configuration no OOS number ever described. Capturing all four at the
era whose vote won makes the deployed ensemble exactly the measured one.
Correct because of the era barrier (b77e7b4): Train() runs at most one
era per call and a member that finished era N is held until every member
reaches N, so when the last member scores the vote no member's weights
have advanced past end-of-era-N. That makes the deferred simultaneous
capture a guarantee rather than a race. Each snapshot is era-STAMPED and
deploy requires every stamp to equal the winning era - otherwise a member
whose capture failed would still hold an older snapshot and the deployed
quartet would again be one nothing measured. Partial capture rolls the
era back out of "best" so the search continues instead of freezing
behind a checkpoint that does not exist.
Statistics mirror the per-member gate one for one - same coverage floor
(MIN_COVERAGE_FRACTION_OF_BASE_RATE), same always-call-one-direction
chance reference, same EDGE_MIN_SIGMAS margin, same Sidak correction over
the eras ranked (DEPLOY_FAMILY_WISE_ALPHA). Only the population differs:
the bars the VOTE fired on, at Min_Vote_Open, rather than the bars one
member called. Two-sidedness is required of the vote itself - a vote that
never goes short IS the always-long model the chance reference prices in.
Members keep their own per-era statistics and their own learning-rate
dynamics (regression restore, eta decay); those are per-net training
mechanics, not deployment decisions. The shared ladder is mirrored onto
each member so per-era log lines report the state that actually governs
them. Solo charts are untouched on every path.
Verified: full MetaEditor compile, 0 errors 0 warnings.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-16 19:25:25 -04:00
|
|
|
//--- per-era stash the ensemble verdict reads (see EnsembleStashEraStats); -1/false = "no era yet"
|
|
|
|
|
m_eraStatPrecPct = -1.0;
|
|
|
|
|
m_eraStatChancePct = -1.0;
|
2026-08-26 16:53:16 -04:00
|
|
|
m_certifiedPrecPct = -1.0;
|
|
|
|
|
m_certifiedChancePct = -1.0;
|
feat(ensemble): deploy gate on the COMBINED VOTE, with a joint checkpoint
The unit of evaluation in ensemble mode becomes the vote, because the
vote is what trades (user: "at the end of the day they will vote
together during live trading so that would make sense").
Four decisions move from the member to the ensemble:
* which era is "best" -> the era whose COMBINED VOTE scored best
* what is checkpointed -> a JOINT snapshot: every member's weights
at that one era
* when the run gives up -> one shared plateau ladder
* whether it may deploy -> family-wise gate on the vote
WHY THE JOINT CHECKPOINT IS THE POINT: per-member selection picks each
net's own best era, and those eras differ. The resulting quartet was
never measured together at any instant, so the vote it casts live is a
configuration no OOS number ever described. Capturing all four at the
era whose vote won makes the deployed ensemble exactly the measured one.
Correct because of the era barrier (b77e7b4): Train() runs at most one
era per call and a member that finished era N is held until every member
reaches N, so when the last member scores the vote no member's weights
have advanced past end-of-era-N. That makes the deferred simultaneous
capture a guarantee rather than a race. Each snapshot is era-STAMPED and
deploy requires every stamp to equal the winning era - otherwise a member
whose capture failed would still hold an older snapshot and the deployed
quartet would again be one nothing measured. Partial capture rolls the
era back out of "best" so the search continues instead of freezing
behind a checkpoint that does not exist.
Statistics mirror the per-member gate one for one - same coverage floor
(MIN_COVERAGE_FRACTION_OF_BASE_RATE), same always-call-one-direction
chance reference, same EDGE_MIN_SIGMAS margin, same Sidak correction over
the eras ranked (DEPLOY_FAMILY_WISE_ALPHA). Only the population differs:
the bars the VOTE fired on, at Min_Vote_Open, rather than the bars one
member called. Two-sidedness is required of the vote itself - a vote that
never goes short IS the always-long model the chance reference prices in.
Members keep their own per-era statistics and their own learning-rate
dynamics (regression restore, eta decay); those are per-net training
mechanics, not deployment decisions. The shared ladder is mirrored onto
each member so per-era log lines report the state that actually governs
them. Solo charts are untouched on every path.
Verified: full MetaEditor compile, 0 errors 0 warnings.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-16 19:25:25 -04:00
|
|
|
m_eraStatCalls = 0;
|
|
|
|
|
m_eraStatTradeable = false;
|
|
|
|
|
m_eraStatTwoSided = false;
|
|
|
|
|
m_eraStatScore = 0.0;
|
|
|
|
|
m_eraStatBlended = 0.0;
|
|
|
|
|
m_eraStatThreshold = 0.0;
|
|
|
|
|
m_checkpointEra = -1;
|
2026-08-01 11:27:28 -04:00
|
|
|
//--- indicator tuning defaults live in CADIndicatorTuner's own constructor (Expert\ADIndicatorTuner.mqh),
|
|
|
|
|
//--- which runs automatically for the m_indicatorTuner member above.
|
|
|
|
|
}
|
|
|
|
|
//+------------------------------------------------------------------+
|
|
|
|
|
//| Destructor |
|
|
|
|
|
//+------------------------------------------------------------------+
|
|
|
|
|
CExpertSignalAIBase::~CExpertSignalAIBase(void)
|
|
|
|
|
{
|
2026-08-22 00:25:52 -04:00
|
|
|
//--- deliberately NOT calling PersistOnShutdown() here: OnDeinit() (Warrior_EA.mq5) already
|
|
|
|
|
//--- calls it explicitly for every signal, one call stack frame shallower, BEFORE
|
|
|
|
|
//--- Expert.Deinit() tears these objects down.
|
2026-08-01 11:27:28 -04:00
|
|
|
if(CheckPointer(Net) != POINTER_INVALID)
|
|
|
|
|
delete Net;
|
|
|
|
|
if(CheckPointer(TempData) != POINTER_INVALID)
|
|
|
|
|
delete TempData;
|
2026-08-23 22:28:31 -04:00
|
|
|
//--- Excursion head teardown (m_excNet/m_excTgt/m_excOut) now happens in CExcursionHead's own
|
refactor(online-learning): OnlineLearning is a real collaborator, not a raw-include partial (S5)
Expert\AIBase\OnlineLearning.mqh (595 lines) -> Expert\OnlineLearning\:
IOnlineLearningView.mqh (abstract, ~50 accessors) + AIBaseOnlineLearningView.mqh/
AIBaseOnlineLearningViewImpl.mqh (the adapter) + OnlineLearning.mqh (COnlineLearning).
STATEFUL, unlike CModelPersistence: grep-verified the shadow net, the OOS
continual-learning simulation state and the pattern-database backfill state are
genuinely exclusive to this file's own methods - Training.mqh/Topology.mqh/
Lifecycle.mqh/the signal's own header only ever CHECKED or RESET this state at
era/lifecycle boundaries, never owned it, so it moved onto the collaborator as
real members (same doctrine as Excursion). Those external touch points became
consolidated view/forward calls instead of raw field pokes - AbortSimIfActive()
replaces THREE separate copies of the same delete/null/false triple (Training.mqh's
stop path, FlushTrainRun, ResetWeights), matching the geometry-scan duplicate-reset
precedent in project memory. ResetForFreshTopology() replaces Topology.mqh's five-
field reset block, DeployNet() replaces the shadow-preferred net selection duplicated
in Inference.mqh and ChartScoreBarForRescan, and BlendTowardNet() replaces the era-end
blend Training.mqh used to poke m_shadowNet for directly.
Reused the signal's existing Data*()/Chart*()/Persist*() getters wherever one already
answered the question (labels/outcome/history/horizon/priors/servable-bars/etc.);
added ~30 new Online*() wrappers only for what nothing else exposed yet. The three
PersistOnline*() get/set pairs (WST3 .stats fields) now forward through the owning
member instead of touching the field directly - CModelPersistence is unaffected.
Every method body is a pure relocation of the original's statements in original
order; verified against `git show HEAD~1:Expert/AIBase/Excursion.mqh`-style diff
against the pre-extraction file kept in the working tree until this commit.
Compiled 0 errors, 0 warnings (stage mirror + MetaEditor64 /compile, ~91s).
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-08-23 22:59:02 -04:00
|
|
|
//--- destructor (S4), same as CChartUI/CModelPersistence needing none here. The shadow net and the
|
|
|
|
|
//--- OOS-simulation net teardown now happens in COnlineLearning's own destructor, same doctrine.
|
fix(chart): arrows survived the EA that drew them - persist, then clear
Reported: on deinit the panel and status label go, the signal arrows stay.
Two independent causes, both fixed here.
1. It was partly deliberate. ShutdownChartCleanup carried a second
behaviour selected by a `preserveChartArrows` flag derived from the
deinit reason: on RECOMPILE / PARAMETERS / CHARTCHANGE / TEMPLATE the
arrows were left on the chart on purpose, to avoid a reload flicker.
That branch IS the reported symptom, an operator cannot tell it apart
from a cleanup that failed, and it was outright wrong whenever the
reload changed the config - REASON_PARAMETERS means exactly that, and
the preserved arrows then belonged to a model the chart no longer
runs, with nothing marking them stale. It is gone, along with the flag
and m_purgeChartOnDestruct. One path now: persist, clear, restore on
the next attach.
2. Whatever remains was unfalsifiable. PurgeChart was a single
ObjectsDeleteAll(prefix) whose return value was discarded, with no
caller ever looking at the chart again - so "the arrows are still
there" and "the arrows were never there" produced identical evidence,
which is why the report survived three sessions. It now verifies:
after the bulk delete it walks the OBJ_ARROW-typed list (a handful of
objects, not the whole chart), deletes any surviving WarSig_ by name,
and says so. Costs one typed scan when the bulk delete works, which is
the normal case; names the root cause when it does not.
Every failure mode of SaveChartSignals was also silent - it returned void
and had three bare early returns. It returns bool now, logs the open
error with the filename, and the shutdown purge is CONDITIONAL on it: for
a converged model the chart objects are the only copy of its signal
history (nothing redraws them - the renderer runs per training era and a
deployed model has none left), so a chart left littered because the disk
write failed beats a clean chart bought by destroying the history. Either
way the log now says which happened.
Also states the user's rule once, where arrows come back rather than
across InitNeuralNetwork's several exits: no weights loaded for this
config => clear the sidecar and start visually clean. A fresh run must
not inherit calls it never made, and the first save would otherwise adopt
them (the sidecar is rebuilt by scanning the chart).
Compiles 0 errors / 0 warnings, standard and Market. Needs redeploy.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-01 13:28:34 -04:00
|
|
|
//--- Unconditional now that the warm-reload "leave the arrows up" branch is gone (see
|
|
|
|
|
//--- ShutdownChartCleanup). Cheap and idempotent: OnDeinit already purged, so this normally deletes
|
|
|
|
|
//--- nothing - it exists for the teardown paths that never reach OnDeinit (a failed OnInit).
|
|
|
|
|
PurgeChart();
|
2026-08-01 11:27:28 -04:00
|
|
|
//--- Last, and cheap by design (one global-variable delete): the claim must outlive every save above
|
|
|
|
|
//--- it, or a chart re-attaching during this teardown could start writing the same files mid-save.
|
|
|
|
|
ReleaseConfigLock();
|
|
|
|
|
}
|
|
|
|
|
//+------------------------------------------------------------------+
|
|
|
|
|
//| Sets the file/id identity a subclass constructor would otherwise |
|
|
|
|
|
//| repeat verbatim (ID, m_id, m_folderPath, m_fileName, pattern count)|
|
|
|
|
|
//+------------------------------------------------------------------+
|
|
|
|
|
void CExpertSignalAIBase::SetIdentity(string id, string shortId, int patternCount = 4)
|
|
|
|
|
{
|
|
|
|
|
ID = id;
|
|
|
|
|
m_id = shortId;
|
|
|
|
|
m_folderPath = eaName + "\\" + "Neural Networks" + "\\" + "State" + "\\" + m_id + "\\";
|
|
|
|
|
m_fileName = m_folderPath + _Symbol + "_" + IntegerToString(_Period);
|
|
|
|
|
m_pattern_count = patternCount;
|
|
|
|
|
}
|
|
|
|
|
//+------------------------------------------------------------------+
|
feat(panel): commands reach signals down the filter tree, not through a registry
The control panel drove training by looping g_aiSignals[] - a
hand-maintained, MAX_AI_SIGNALS-capped, AI-only registry that had
already dropped an ensemble member on the floor once (609be10). A model
missing from it still trains and still votes, it just cannot be paused,
stopped, deployed or reset, and every button label is computed from the
same short list, so the panel described one set of models while acting
on another. Classic signals could not respond to a panel action at all.
Commands now walk the signal tree CExpert already owns:
Expert.DispatchSignalCommand(cmd) -> root signal -> every filter,
recursively, returning how many actually acted.
CExpertSignalCustom carries the seam (OnSignalCommand / HasSignalTrait,
both no-ops by default), so a classic signal opts in by overriding two
methods and needs no registration and no cap. CExpertSignalAIBase
implements the training commands over its existing Pause/Stop/Deploy/
Reset methods - the behaviour is unchanged, only its reach reported.
Button labels ask the same tree via CountSignalTrait, with
SIGTRAIT_TRAINABLE as an explicit denominator: "all paused" is
meaningless without knowing how many could be paused. Pause/Stop resolve
their toggle direction ONCE in the EA and hand every model the same
plain command, instead of each re-deriving the direction from its own
local state - which is how a mixed set ends up half paused. The alerts
now report the count acted on rather than assuming it.
Two dispatch bugs found on the way, both from a database guard copied
onto event delivery: CExpertSignalCustom::OnTickHandler and
::OnChartEventHandler each skipped any filter whose GetFilterID() is
"NULL". That id is a DB folder name, and CSignalNewsFilter,
CSignalSessionFilter and CSignalRiskGuard never set one - so all three
were silently receiving neither ticks nor chart events. The guard stays
where it belongs, on the paths that write pattern tables.
ENUM_CP_ACTION moves to Enumerations\GlobalEnums.mqh (now include-
guarded) because the Expert bases have to name it and the panel is
included long after them.
The AI-only lifecycle loops - PollTraining, the weight autosave,
AltDataReload, OnDeinit's shutdown cascade - still use g_aiSignals[] and
are untouched here.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-20 09:05:50 -04:00
|
|
|
//| CONTROL-PANEL COMMAND. One switch, so a button can never reach |
|
|
|
|
|
//| some models and miss others: the panel resolves the direction |
|
|
|
|
|
//| once and every model in the tree is told the same thing. |
|
|
|
|
|
//+------------------------------------------------------------------+
|
|
|
|
|
bool CExpertSignalAIBase::OnSignalCommand(const ENUM_SIGNAL_COMMAND cmd)
|
|
|
|
|
{
|
|
|
|
|
switch(cmd)
|
|
|
|
|
{
|
|
|
|
|
case SIGCMD_PAUSE_TRAINING:
|
|
|
|
|
PauseTraining();
|
|
|
|
|
return true;
|
|
|
|
|
case SIGCMD_RESUME_TRAINING:
|
|
|
|
|
ResumeTraining();
|
|
|
|
|
return true;
|
|
|
|
|
case SIGCMD_STOP_TRAINING:
|
|
|
|
|
StopTraining();
|
|
|
|
|
return true;
|
|
|
|
|
case SIGCMD_START_TRAINING:
|
|
|
|
|
StartTraining();
|
|
|
|
|
return true;
|
|
|
|
|
//--- These three report SUCCESS rather than "I understood the command", because the panel
|
|
|
|
|
//--- counts them back to the operator and a failed deploy or a failed rebuild is exactly what
|
|
|
|
|
//--- they need told about.
|
|
|
|
|
case SIGCMD_DEPLOY:
|
|
|
|
|
return DeployNow();
|
|
|
|
|
case SIGCMD_RESET_WEIGHTS:
|
|
|
|
|
return ResetWeights();
|
|
|
|
|
case SIGCMD_SAVE_WEIGHTS:
|
|
|
|
|
return SaveWeightsNow();
|
|
|
|
|
case SIGCMD_LOAD_WEIGHTS:
|
|
|
|
|
return LoadWeightsNow();
|
|
|
|
|
case SIGCMD_RETRAIN_DEPLOYED:
|
|
|
|
|
RetrainDeployed();
|
|
|
|
|
return true;
|
|
|
|
|
//--- Queues a rescan; returns whether one was actually queued, which is what tells the EA to
|
|
|
|
|
//--- defer the "arrows shown" alert until every queued scan has drained.
|
|
|
|
|
case SIGCMD_RESCAN_SIGNALS:
|
|
|
|
|
return StartChartSignalRescan();
|
|
|
|
|
case SIGCMD_REPORT_IDENTITY:
|
|
|
|
|
Print(" " + RegistryLine());
|
|
|
|
|
return true;
|
|
|
|
|
}
|
|
|
|
|
return false;
|
|
|
|
|
}
|
|
|
|
|
//+------------------------------------------------------------------+
|
|
|
|
|
//| Panel button labels ask these; see ENUM_SIGNAL_TRAIT. |
|
|
|
|
|
//+------------------------------------------------------------------+
|
|
|
|
|
bool CExpertSignalAIBase::HasSignalTrait(const ENUM_SIGNAL_TRAIT trait)
|
|
|
|
|
{
|
|
|
|
|
switch(trait)
|
|
|
|
|
{
|
|
|
|
|
case SIGTRAIT_TRAINABLE:
|
|
|
|
|
return true;
|
|
|
|
|
case SIGTRAIT_TRAINING_PAUSED:
|
|
|
|
|
return m_trainingPaused;
|
|
|
|
|
case SIGTRAIT_TRAINING_STOPPED:
|
|
|
|
|
return m_trainingStopRequested;
|
|
|
|
|
case SIGTRAIT_TRAINING_COMPLETE:
|
|
|
|
|
return m_trainingComplete;
|
|
|
|
|
//--- "still trainable AND has never checkpointed an era that cleared the per-class recall
|
|
|
|
|
//--- floor" - the same bar the plateau ladder refuses to cross on its own.
|
|
|
|
|
case SIGTRAIT_DEPLOY_SKIPS_RECALL:
|
|
|
|
|
return (!m_trainingComplete && !m_bestPassedRecall);
|
|
|
|
|
case SIGTRAIT_RESCAN_PENDING:
|
refactor(chart): ChartUI is a real collaborator, not a raw-include partial (S2)
Expert/AIBase/ChartUI.mqh was 869 lines of method bodies of
CExpertSignalAIBase, #included after the class declaration - free to touch
any of its ~500 members. First of the eleven AIBase/*.mqh partials to come
out (fewest inbound edges - see the SOLID campaign session order), using
the same view+adapter shape already proven for CTrainingDataView.
CChartView (Expert/Chart/IChartView.mqh) is the abstract read/behaviour
surface a chart-rendering collaborator needs - identity, bar/model access,
the prediction cache, and the training/vote/meta scalars the panel and HUD
line summarise. CAIBaseChartView is the adapter the signal owns and binds
to itself (MQL5 gives a class exactly one base, so CExpertSignalAIBase
cannot implement the view directly). CChartUI is the real collaborator: it
owns the arrow-restore queue, the rescan queue/tally, the last-arrows-saved
count and the purge-mismatch latch as its own fields (verified via grep to
be touched nowhere else in Expert/), and reaches everything else - including
StartChartSignalRescan, moved in from its old inline home in the header
since it drives the exact same rescan state machine AdvanceChartSignalRescan
drains - through the view.
m_arrowSignalCache and m_signalClusterWindow stay on the signal: Training.mqh
writes the cache directly every era and the training-data view already reads
it, so moving it would mean rewriting Training.mqh's write sites too - out of
scope here. CChartUI reaches it through four bounds-checked accessors instead
of a raw member poke. All 10 public methods keep their exact signatures and
become one-line forwards on the signal, so no other file's call sites change
except Training.mqh's one era-end status refresh, which now reads
RefreshStatusLabel() rather than reaching into CChartUI's now-private
last-displayed-neuron cache directly.
Verified structurally, not compiled (never compile - the operator does, in
MetaEditor): brace balance checked on every touched/new file against HEAD,
and the view/adapter/impl method lists cross-diffed to confirm all 59
accessors match 1:1 across the interface, the adapter declaration and the
adapter body.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-08-23 20:03:52 -04:00
|
|
|
return RescanPending();
|
feat(panel): commands reach signals down the filter tree, not through a registry
The control panel drove training by looping g_aiSignals[] - a
hand-maintained, MAX_AI_SIGNALS-capped, AI-only registry that had
already dropped an ensemble member on the floor once (609be10). A model
missing from it still trains and still votes, it just cannot be paused,
stopped, deployed or reset, and every button label is computed from the
same short list, so the panel described one set of models while acting
on another. Classic signals could not respond to a panel action at all.
Commands now walk the signal tree CExpert already owns:
Expert.DispatchSignalCommand(cmd) -> root signal -> every filter,
recursively, returning how many actually acted.
CExpertSignalCustom carries the seam (OnSignalCommand / HasSignalTrait,
both no-ops by default), so a classic signal opts in by overriding two
methods and needs no registration and no cap. CExpertSignalAIBase
implements the training commands over its existing Pause/Stop/Deploy/
Reset methods - the behaviour is unchanged, only its reach reported.
Button labels ask the same tree via CountSignalTrait, with
SIGTRAIT_TRAINABLE as an explicit denominator: "all paused" is
meaningless without knowing how many could be paused. Pause/Stop resolve
their toggle direction ONCE in the EA and hand every model the same
plain command, instead of each re-deriving the direction from its own
local state - which is how a mixed set ends up half paused. The alerts
now report the count acted on rather than assuming it.
Two dispatch bugs found on the way, both from a database guard copied
onto event delivery: CExpertSignalCustom::OnTickHandler and
::OnChartEventHandler each skipped any filter whose GetFilterID() is
"NULL". That id is a DB folder name, and CSignalNewsFilter,
CSignalSessionFilter and CSignalRiskGuard never set one - so all three
were silently receiving neither ticks nor chart events. The guard stays
where it belongs, on the paths that write pattern tables.
ENUM_CP_ACTION moves to Enumerations\GlobalEnums.mqh (now include-
guarded) because the Expert bases have to name it and the panel is
included long after them.
The AI-only lifecycle loops - PollTraining, the weight autosave,
AltDataReload, OnDeinit's shutdown cascade - still use g_aiSignals[] and
are untouched here.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-20 09:05:50 -04:00
|
|
|
}
|
|
|
|
|
return false;
|
|
|
|
|
}
|
|
|
|
|
//+------------------------------------------------------------------+
|
2026-08-01 11:27:28 -04:00
|
|
|
//| "Voting" that price will grow. |
|
|
|
|
|
//+------------------------------------------------------------------+
|
|
|
|
|
int CExpertSignalAIBase::LongCondition(void)
|
|
|
|
|
{
|
|
|
|
|
int result = 0;
|
2026-08-22 00:25:52 -04:00
|
|
|
//--- Readiness gate: live trading still requires a converged model, but an inference-only tester
|
|
|
|
|
//--- run may replay a model that was ACTUALLY loaded from disk even if its persisted
|
|
|
|
|
//--- trainingComplete flag is false.
|
2026-08-01 11:27:28 -04:00
|
|
|
NoteVoteGate(DoubleToSignal(dPrevSignal) == Buy);
|
|
|
|
|
if(!m_trainingComplete && !(m_inferenceOnly && m_modelLoadedFromDisk))
|
|
|
|
|
return 0;
|
2026-08-22 00:25:52 -04:00
|
|
|
//--- No alternation gate any more - see the removal note at m_voteGateBlocked's declaration.
|
|
|
|
|
//--- Under triple-barrier labels consecutive same-direction setups are ordinary and correct.
|
2026-08-01 11:27:28 -04:00
|
|
|
if(dPrevSignal == -2)
|
|
|
|
|
return 0;
|
|
|
|
|
//--- NO confidence floor here, by design - see m_minSignalConfidence's former declaration site. A
|
|
|
|
|
//--- weak call is not blocked at the AI's own boundary; it votes at its tier weight (as low as
|
|
|
|
|
//--- m_pattern_0) and is then filtered by Min vote to open, exactly like a weak classic vote.
|
|
|
|
|
if(DoubleToSignal(dPrevSignal) == Buy)
|
|
|
|
|
{
|
|
|
|
|
int tier = ConfidenceTier();
|
|
|
|
|
result = PatternWeightForTier(tier);
|
|
|
|
|
m_active_pattern = "Pattern_" + IntegerToString(tier);
|
|
|
|
|
m_active_direction = "Buy";
|
|
|
|
|
}
|
|
|
|
|
return(result);
|
|
|
|
|
}
|
|
|
|
|
//+------------------------------------------------------------------+
|
|
|
|
|
//| "Voting" that price will fall. |
|
|
|
|
|
//+------------------------------------------------------------------+
|
|
|
|
|
int CExpertSignalAIBase::ShortCondition(void)
|
|
|
|
|
{
|
|
|
|
|
int result = 0;
|
|
|
|
|
//--- Readiness gate - see LongCondition's matching comment.
|
|
|
|
|
NoteVoteGate(DoubleToSignal(dPrevSignal) == Sell);
|
|
|
|
|
if(!m_trainingComplete && !(m_inferenceOnly && m_modelLoadedFromDisk))
|
|
|
|
|
return 0;
|
|
|
|
|
//--- "not yet studied" sentinel, and no confidence floor - see LongCondition's matching comments.
|
|
|
|
|
if(dPrevSignal == -2)
|
|
|
|
|
return 0;
|
|
|
|
|
if(DoubleToSignal(dPrevSignal) == Sell)
|
|
|
|
|
{
|
|
|
|
|
int tier = ConfidenceTier();
|
|
|
|
|
result = PatternWeightForTier(tier);
|
|
|
|
|
m_active_pattern = "Pattern_" + IntegerToString(tier);
|
|
|
|
|
m_active_direction = "Sell";
|
|
|
|
|
}
|
|
|
|
|
return result;
|
|
|
|
|
}
|
|
|
|
|
//+------------------------------------------------------------------+
|
fix(gate): the ensemble gate certified a vote the EA never casts
g_ensembleVoteThreshold's comment claims the combined-vote scorer "fires on
the same criterion the live trade does". It did not. Two independent
mismatches, both silent:
CURRENCY. Each member contributed its raw signed confidence x100 - a 33..100
number straight off the softmax head. Live contributes m_weight x the tier's
pattern weight, and BOTH of those are rewritten from the signal DB by
UpdateSignalsWeights(). A head output and a DB-ranked win-rate weight share
an axis and nothing relates them, so the same bar was one number to the gate
and a different one to the order path. Same shape as the 2026-08-09 geometry
incident: certified on one game, paid on another.
DENOMINATOR. The gate divided by the member count, so an abstaining member
pulled the average toward zero. CExpertSignalCustom::Direction() skips a zero
contribution in BOTH the sum and the count (`if(direction == 0) continue;`
before `number++`) - live is a mean over VOTERS. The gate was therefore
scoring a strictly more agreement-heavy set of bars than the EA trades. The
contribution hook's own comment asserted the opposite ("abstentions dilute
the average exactly as they do in the live vote"), while the AI_CHOICE enum
20 lines away correctly documented union semantics.
LiveVoteContribution() is now the single definition of "what this member
votes", called from the gate; the live path reaches the same arithmetic
through LongCondition/ShortCondition. g_ensVoteVoterMask records who actually
voted, separately from who evaluated the bar, because those are the divisor
and the shared-population test respectively.
ConfidenceTier() is split into ConfidenceTierFor(signal) plus a thin live-bar
wrapper - the OOS scan holds the scanned bar's decision in a local, and
dPrevSignal is a different bar.
NOT changed, deliberately: the gate still does not model live NMS
declustering, and the per-member solo gate still scores every directional
call rather than threshold-clearing ones. Both are selection-metric changes
and this codebase has twice been bitten by switching one blind.
Also corrects a stale paragraph in m_pattern_0's declaration block quoting
80/87/93/100 as the tier defaults. The constructor is 25/50/75/100 and has
been since the confidence floor and alternation gate were removed; the block
carried both tables at once, and the dead one was quoted back as fact.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-18 15:23:33 -04:00
|
|
|
//| Buckets a confidence magnitude into one of 4 equal bands between |
|
|
|
|
|
//| the head's own structural floor and 1.0 - see m_pattern_0's |
|
2026-08-01 11:27:28 -04:00
|
|
|
//| declaration comment for the resulting tier/weight table. Neither |
|
|
|
|
|
//| head has a configurable floor: the boundary is 1/3 for the 3-class|
|
|
|
|
|
//| softmax and 0.5 for regression (DoubleToSignal's own threshold), |
|
|
|
|
|
//| both arithmetic properties of the head rather than settings. |
|
|
|
|
|
//+------------------------------------------------------------------+
|
fix(gate): the ensemble gate certified a vote the EA never casts
g_ensembleVoteThreshold's comment claims the combined-vote scorer "fires on
the same criterion the live trade does". It did not. Two independent
mismatches, both silent:
CURRENCY. Each member contributed its raw signed confidence x100 - a 33..100
number straight off the softmax head. Live contributes m_weight x the tier's
pattern weight, and BOTH of those are rewritten from the signal DB by
UpdateSignalsWeights(). A head output and a DB-ranked win-rate weight share
an axis and nothing relates them, so the same bar was one number to the gate
and a different one to the order path. Same shape as the 2026-08-09 geometry
incident: certified on one game, paid on another.
DENOMINATOR. The gate divided by the member count, so an abstaining member
pulled the average toward zero. CExpertSignalCustom::Direction() skips a zero
contribution in BOTH the sum and the count (`if(direction == 0) continue;`
before `number++`) - live is a mean over VOTERS. The gate was therefore
scoring a strictly more agreement-heavy set of bars than the EA trades. The
contribution hook's own comment asserted the opposite ("abstentions dilute
the average exactly as they do in the live vote"), while the AI_CHOICE enum
20 lines away correctly documented union semantics.
LiveVoteContribution() is now the single definition of "what this member
votes", called from the gate; the live path reaches the same arithmetic
through LongCondition/ShortCondition. g_ensVoteVoterMask records who actually
voted, separately from who evaluated the bar, because those are the divisor
and the shared-population test respectively.
ConfidenceTier() is split into ConfidenceTierFor(signal) plus a thin live-bar
wrapper - the OOS scan holds the scanned bar's decision in a local, and
dPrevSignal is a different bar.
NOT changed, deliberately: the gate still does not model live NMS
declustering, and the per-member solo gate still scores every directional
call rather than threshold-clearing ones. Both are selection-metric changes
and this codebase has twice been bitten by switching one blind.
Also corrects a stale paragraph in m_pattern_0's declaration block quoting
80/87/93/100 as the tier defaults. The constructor is 25/50/75/100 and has
been since the confidence floor and alternation gate were removed; the block
carried both tables at once, and the dead one was quoted back as fact.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-18 15:23:33 -04:00
|
|
|
int CExpertSignalAIBase::ConfidenceTierFor(const double signal)
|
2026-08-01 11:27:28 -04:00
|
|
|
{
|
|
|
|
|
//--- The head's own STRUCTURAL decision boundary - the lowest confidence magnitude that head can
|
2026-08-22 00:25:52 -04:00
|
|
|
//--- possibly report for a directional call - not a user setting: - 3-class classification: the
|
|
|
|
|
//--- winning class of a 3-way softmax is arithmetically >= 1/3, since three probabilities
|
|
|
|
|
//--- summing to 1 cannot all be below it.
|
2026-08-01 11:27:28 -04:00
|
|
|
double floorConf = (m_outputNeuronsCount == 3) ? (1.0 / 3.0) : 0.5;
|
|
|
|
|
double span = MathMax(1.0 - floorConf, 0.0001);
|
2026-08-22 00:25:52 -04:00
|
|
|
//--- RAW magnitude, not the calibrated one (fixed 2026-08-16).
|
fix(gate): the ensemble gate certified a vote the EA never casts
g_ensembleVoteThreshold's comment claims the combined-vote scorer "fires on
the same criterion the live trade does". It did not. Two independent
mismatches, both silent:
CURRENCY. Each member contributed its raw signed confidence x100 - a 33..100
number straight off the softmax head. Live contributes m_weight x the tier's
pattern weight, and BOTH of those are rewritten from the signal DB by
UpdateSignalsWeights(). A head output and a DB-ranked win-rate weight share
an axis and nothing relates them, so the same bar was one number to the gate
and a different one to the order path. Same shape as the 2026-08-09 geometry
incident: certified on one game, paid on another.
DENOMINATOR. The gate divided by the member count, so an abstaining member
pulled the average toward zero. CExpertSignalCustom::Direction() skips a zero
contribution in BOTH the sum and the count (`if(direction == 0) continue;`
before `number++`) - live is a mean over VOTERS. The gate was therefore
scoring a strictly more agreement-heavy set of bars than the EA trades. The
contribution hook's own comment asserted the opposite ("abstentions dilute
the average exactly as they do in the live vote"), while the AI_CHOICE enum
20 lines away correctly documented union semantics.
LiveVoteContribution() is now the single definition of "what this member
votes", called from the gate; the live path reaches the same arithmetic
through LongCondition/ShortCondition. g_ensVoteVoterMask records who actually
voted, separately from who evaluated the bar, because those are the divisor
and the shared-population test respectively.
ConfidenceTier() is split into ConfidenceTierFor(signal) plus a thin live-bar
wrapper - the OOS scan holds the scanned bar's decision in a local, and
dPrevSignal is a different bar.
NOT changed, deliberately: the gate still does not model live NMS
declustering, and the per-member solo gate still scores every directional
call rather than threshold-clearing ones. Both are selection-metric changes
and this codebase has twice been bitten by switching one blind.
Also corrects a stale paragraph in m_pattern_0's declaration block quoting
80/87/93/100 as the tier defaults. The constructor is 25/50/75/100 and has
been since the confidence floor and alternation gate were removed; the block
carried both tables at once, and the dead one was quoted back as fact.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-18 15:23:33 -04:00
|
|
|
double rawMag = MathAbs(signal);
|
fix: drop the ranking slice for the calibration band; un-collapse the tiers
NOT COMPILED - user compiles.
(1) THE RANKING SLICE IS GONE. It reserved 20% of the OOS window so the
pattern-DB backfill would read bars the deployed checkpoint was not SELECTED on.
That objection stands; carving a new region to answer it did not. The calibration
band already has every property the slice was buying:
never trained on | never graded by pass 3 (which walks [0, oosCutoff) and so
never reaches it) | never seen by the deploy gate | purged by a full label
horizon on BOTH sides | and larger besides - 1,684 bars vs the ~970 carved
So the backfill now walks [calibLo, calibHi) and pass 3 goes back to grading the
entire OOS window, exactly as before any of this. The gate gets its full sample
back (~10% of a sigma), the split loses a region, and the failure mode found an
hour ago - a reserved region silently blanking ~10 months of chart arrows,
because arrows are only drawn on bars pass 3 grades - becomes impossible.
One impurity, stated in the completion log rather than hidden:
m_dirConfThreshold is FITTED on that band and the walk applies it to decide which
bars fired, so coverage there is mildly optimistic. One scalar under a coverage
floor, against checkpoint selection over hundreds of eras.
This backfill IS the deploy-time warm-up: it runs right after FinalizeTrainRun()
restores the deployed weights, so it scores with exactly what is about to trade.
(2) EVERY CALL WAS TIER 0, AND IT WAS ARITHMETIC. ConfidenceTier() quartiles
[floorConf, 1] where floorConf = 1/3 - the lowest magnitude a 3-way softmax
winner can hold. But it was fed CalibratedConfidenceMagnitude(), which multiplies
by m_confidenceCalScale, clamped to [0.3, 1.5]. That lower clamp is BELOW 1/3.
Whenever calibration bottoms out, t goes negative and MathMax(0, ...) pins every
call to tier 0.
Which is what the live run does. m_confidenceCalScale is EMA'd toward
empiricalAccuracy / avgClaimedConfidence; with the model over-calling Neutral,
3-class agreement sits near 10% against a claimed confidence near 0.9, so the
ratio is ~0.11 and clamps to 0.3 every era. Logged:
tier prec T0:72%(828) T1:n/a(0) T2:n/a(0) T3:n/a(0)
828 calls, one bucket - the four tier weights and the entire per-tier pattern-DB
ranking reduced to a single number. The backfill was feeding a mechanism that
structurally could not rank.
Tiering now reads the RAW head magnitude, which genuinely lives on the
[1/3, 1] range these bounds were written for. Calibration keeps its real jobs -
AIConfidence() for MM sizing and SignedAIConfidence() for the vote are unchanged.
STILL OPEN, deliberately not touched here: the calibration TARGET itself.
empiricalAccuracy is 3-class agreement, which is the wrong quantity to scale a
DIRECTIONAL confidence against - it counts a Neutral class that is 0.19% of
labels. The honest target is the win rate on the calls the confidence describes
(directional precision), with the claimed-confidence average taken over those
same called bars. That needs a new accumulator and it interacts with the Neutral
over-calling being fixed elsewhere, so it wants one clean run first.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-16 23:06:11 -04:00
|
|
|
if(!MathIsValidNumber(rawMag))
|
|
|
|
|
return 0;
|
|
|
|
|
double t = (MathMin(1.0, rawMag) - floorConf) / span;
|
2026-08-01 11:27:28 -04:00
|
|
|
int tier = (int)MathFloor(t * 4.0);
|
|
|
|
|
return MathMax(0, MathMin(tier, 3));
|
|
|
|
|
}
|
|
|
|
|
//+------------------------------------------------------------------+
|
fix(gate): the ensemble gate certified a vote the EA never casts
g_ensembleVoteThreshold's comment claims the combined-vote scorer "fires on
the same criterion the live trade does". It did not. Two independent
mismatches, both silent:
CURRENCY. Each member contributed its raw signed confidence x100 - a 33..100
number straight off the softmax head. Live contributes m_weight x the tier's
pattern weight, and BOTH of those are rewritten from the signal DB by
UpdateSignalsWeights(). A head output and a DB-ranked win-rate weight share
an axis and nothing relates them, so the same bar was one number to the gate
and a different one to the order path. Same shape as the 2026-08-09 geometry
incident: certified on one game, paid on another.
DENOMINATOR. The gate divided by the member count, so an abstaining member
pulled the average toward zero. CExpertSignalCustom::Direction() skips a zero
contribution in BOTH the sum and the count (`if(direction == 0) continue;`
before `number++`) - live is a mean over VOTERS. The gate was therefore
scoring a strictly more agreement-heavy set of bars than the EA trades. The
contribution hook's own comment asserted the opposite ("abstentions dilute
the average exactly as they do in the live vote"), while the AI_CHOICE enum
20 lines away correctly documented union semantics.
LiveVoteContribution() is now the single definition of "what this member
votes", called from the gate; the live path reaches the same arithmetic
through LongCondition/ShortCondition. g_ensVoteVoterMask records who actually
voted, separately from who evaluated the bar, because those are the divisor
and the shared-population test respectively.
ConfidenceTier() is split into ConfidenceTierFor(signal) plus a thin live-bar
wrapper - the OOS scan holds the scanned bar's decision in a local, and
dPrevSignal is a different bar.
NOT changed, deliberately: the gate still does not model live NMS
declustering, and the per-member solo gate still scores every directional
call rather than threshold-clearing ones. Both are selection-metric changes
and this codebase has twice been bitten by switching one blind.
Also corrects a stale paragraph in m_pattern_0's declaration block quoting
80/87/93/100 as the tier defaults. The constructor is 25/50/75/100 and has
been since the confidence floor and alternation gate were removed; the block
carried both tables at once, and the dead one was quoted back as fact.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-18 15:23:33 -04:00
|
|
|
//| The live bar's tier - the only caller shape that existed before |
|
|
|
|
|
//| ConfidenceTierFor() was split out, kept so LongCondition()/ |
|
|
|
|
|
//| ShortCondition() read exactly as they did. |
|
|
|
|
|
//+------------------------------------------------------------------+
|
|
|
|
|
int CExpertSignalAIBase::ConfidenceTier(void)
|
|
|
|
|
{
|
|
|
|
|
return ConfidenceTierFor(dPrevSignal);
|
|
|
|
|
}
|
|
|
|
|
//+------------------------------------------------------------------+
|
2026-08-22 00:30:14 -04:00
|
|
|
//| The signed vote this member would cast for a given decision, in |
|
|
|
|
|
//| the units CExpertSignalCustom::Direction() sums - see the |
|
|
|
|
|
//| declaration comment for why this is a shared function and not |
|
|
|
|
|
//| two parallel expressions. |
|
fix(gate): the ensemble gate certified a vote the EA never casts
g_ensembleVoteThreshold's comment claims the combined-vote scorer "fires on
the same criterion the live trade does". It did not. Two independent
mismatches, both silent:
CURRENCY. Each member contributed its raw signed confidence x100 - a 33..100
number straight off the softmax head. Live contributes m_weight x the tier's
pattern weight, and BOTH of those are rewritten from the signal DB by
UpdateSignalsWeights(). A head output and a DB-ranked win-rate weight share
an axis and nothing relates them, so the same bar was one number to the gate
and a different one to the order path. Same shape as the 2026-08-09 geometry
incident: certified on one game, paid on another.
DENOMINATOR. The gate divided by the member count, so an abstaining member
pulled the average toward zero. CExpertSignalCustom::Direction() skips a zero
contribution in BOTH the sum and the count (`if(direction == 0) continue;`
before `number++`) - live is a mean over VOTERS. The gate was therefore
scoring a strictly more agreement-heavy set of bars than the EA trades. The
contribution hook's own comment asserted the opposite ("abstentions dilute
the average exactly as they do in the live vote"), while the AI_CHOICE enum
20 lines away correctly documented union semantics.
LiveVoteContribution() is now the single definition of "what this member
votes", called from the gate; the live path reaches the same arithmetic
through LongCondition/ShortCondition. g_ensVoteVoterMask records who actually
voted, separately from who evaluated the bar, because those are the divisor
and the shared-population test respectively.
ConfidenceTier() is split into ConfidenceTierFor(signal) plus a thin live-bar
wrapper - the OOS scan holds the scanned bar's decision in a local, and
dPrevSignal is a different bar.
NOT changed, deliberately: the gate still does not model live NMS
declustering, and the per-member solo gate still scores every directional
call rather than threshold-clearing ones. Both are selection-metric changes
and this codebase has twice been bitten by switching one blind.
Also corrects a stale paragraph in m_pattern_0's declaration block quoting
80/87/93/100 as the tier defaults. The constructor is 25/50/75/100 and has
been since the confidence floor and alternation gate were removed; the block
carried both tables at once, and the dead one was quoted back as fact.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-18 15:23:33 -04:00
|
|
|
//+------------------------------------------------------------------+
|
|
|
|
|
double CExpertSignalAIBase::LiveVoteContribution(const double signal)
|
|
|
|
|
{
|
fix(vote): unranked members voted with the stock 25/50/75/100 ladder
Two defects behind "arrows drawn while members are still mid-era".
1. THE DRAW. The filtered overlay armed on the FIRST member to finish
pass 3 and leaned on a 60 s rate limit to "collapse the burst",
assuming members finish seconds apart. They do not - on USDJPY one
member was at sample 10496 of pass 2 while another was at 2304,
minutes apart. A member with no era-end snapshot returns false from
SnapshotVoteAt, and the sweep's `if(!hasData) continue;` skips it
BEFORE `den += ModuleWeight()`, so the one finished model's tier
weight became the entire vote and was drawn as a consensus arrow.
An abstention is a member that looked at the bar and said nothing; a
missing snapshot is a member that has not looked. The first must
dilute the vote, the second must suppress the draw. The arm is now a
readiness MASK - one bit per m_ensembleIndex, set at that member's
pass-3 completion, cleared when a sweep arms - and a sweep waits for
every enrolled member. Bounded at 10 minutes so a member that stops
cannot freeze the chart, and the partial draw PRINTS which members
were missing: the be39674 lesson is that a hold must never silence
the thing that reports it.
2. THE VOTE ITSELF, which is the worse half and is not display-only.
Tier weights are not persisted in the .nnw - they exist only as the
output of a completed pass 3 - so before a member's first
RankTiersFromOos() it holds the constructor's stock 25/50/75/100.
Since 4858507 the vote currency is a WIN RATE, so an unranked tier-3
call enters the capability-weighted mean claiming a 100% win rate
beside ranked members contributing ~25. Not a strong opinion: the
wrong unit. One unranked member drags the ensemble over any
threshold, on every fresh deploy and every resume. USDJPY has a
measured ceiling of ~19 and was firing anyway.
LiveVoteContribution() now abstains until self-ranked, which drops
the member from the sum AND the divisor. One function, so live and
the gate move together (2c443ba).
Era 0 will therefore report 0 coverage until each member completes one
era. The ensemble line says so explicitly rather than leaving it to look
like the USDJPY unreachable-threshold case - the two are identical in
the coverage number and completely different problems.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-23 08:55:04 -04:00
|
|
|
//--- A MEMBER THAT HAS NEVER RANKED ITSELF DOES NOT VOTE. Until RankTiersFromOos() runs once,
|
|
|
|
|
//--- m_pattern_0..3 hold the constructor's stock 25/50/75/100 ladder - and since 4858507 the vote
|
|
|
|
|
//--- currency is a WIN RATE, so an unranked tier-3 call enters the mean claiming a 100% win rate
|
|
|
|
|
//--- against ranked members contributing ~25. That is not a strong opinion, it is the wrong unit:
|
|
|
|
|
//--- one unranked member drags the whole ensemble over any threshold. It bites on every fresh
|
|
|
|
|
//--- deploy AND every resume, because the tier weights are not persisted in the .nnw - they exist
|
|
|
|
|
//--- only as the output of a completed pass 3. Found 2026-08-23 on USDJPY, whose measured ceiling
|
|
|
|
|
//--- is ~19 and which was firing anyway.
|
|
|
|
|
if(!m_tiersSelfRanked)
|
|
|
|
|
return 0.0;
|
fix(gate): the ensemble gate certified a vote the EA never casts
g_ensembleVoteThreshold's comment claims the combined-vote scorer "fires on
the same criterion the live trade does". It did not. Two independent
mismatches, both silent:
CURRENCY. Each member contributed its raw signed confidence x100 - a 33..100
number straight off the softmax head. Live contributes m_weight x the tier's
pattern weight, and BOTH of those are rewritten from the signal DB by
UpdateSignalsWeights(). A head output and a DB-ranked win-rate weight share
an axis and nothing relates them, so the same bar was one number to the gate
and a different one to the order path. Same shape as the 2026-08-09 geometry
incident: certified on one game, paid on another.
DENOMINATOR. The gate divided by the member count, so an abstaining member
pulled the average toward zero. CExpertSignalCustom::Direction() skips a zero
contribution in BOTH the sum and the count (`if(direction == 0) continue;`
before `number++`) - live is a mean over VOTERS. The gate was therefore
scoring a strictly more agreement-heavy set of bars than the EA trades. The
contribution hook's own comment asserted the opposite ("abstentions dilute
the average exactly as they do in the live vote"), while the AI_CHOICE enum
20 lines away correctly documented union semantics.
LiveVoteContribution() is now the single definition of "what this member
votes", called from the gate; the live path reaches the same arithmetic
through LongCondition/ShortCondition. g_ensVoteVoterMask records who actually
voted, separately from who evaluated the bar, because those are the divisor
and the shared-population test respectively.
ConfidenceTier() is split into ConfidenceTierFor(signal) plus a thin live-bar
wrapper - the OOS scan holds the scanned bar's decision in a local, and
dPrevSignal is a different bar.
NOT changed, deliberately: the gate still does not model live NMS
declustering, and the per-member solo gate still scores every directional
call rather than threshold-clearing ones. Both are selection-metric changes
and this codebase has twice been bitten by switching one blind.
Also corrects a stale paragraph in m_pattern_0's declaration block quoting
80/87/93/100 as the tier defaults. The constructor is 25/50/75/100 and has
been since the confidence floor and alternation gate were removed; the block
carried both tables at once, and the dead one was quoted back as fact.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-18 15:23:33 -04:00
|
|
|
ENUM_SIGNAL s = DoubleToSignal(signal);
|
|
|
|
|
if(s != Buy && s != Sell)
|
|
|
|
|
return 0.0; // abstention - live drops it from the sum AND the divisor
|
|
|
|
|
double w = (double)PatternWeightForTier(ConfidenceTierFor(signal));
|
|
|
|
|
if(!MathIsValidNumber(w) || w <= 0.0)
|
|
|
|
|
return 0.0; // a pattern ranked to weight 0 contributes nothing and is not a voter
|
feat(vote): edge-over-chance currency, no-skill exclusion, checkpoint burn-in
RETRAIN-FORCING and deliberately so. Two independent fixes for the same symptom -
charts that go quiet while others overtrade.
1. THE VOTE CURRENCY IS NOW EDGE OVER CHANCE, not an absolute win rate.
A tier weight is a raw win rate and a raw win rate means nothing without the
chance rate behind it: 30% is strong under a 14% base rate and catastrophic under
50%, yet both entered the mean as "30". That is why the threshold needed
re-tuning every time the label changed - 25 was permissive at ~70% win rates
under the old direction label and a near-unanimity rule at ~30% under the
pivot-event one - and why one chart's 25% was never the same statement as
another's. Subtracting the member's own chance rate makes the units percentage
points of demonstrated edge, comparable across charts, labels and regimes.
Clamped at zero: a below-chance tier is anti-informative, and contributing
negatively would act on a broken model as an inverted oracle rather than
discarding it.
2. A NO-SKILL MEMBER IS NOW ABSENT, NOT ABSTAINING.
Measured on XTIUSD: a Perceptron collapsed to B97/S6/N3, pooled win rate 11.5%
against a 14% chance rate - worse than guessing - and still voting. Three healthy
members voting Sell scored -21.06/0.77 = -27.4 and cleared; with the dead one
voting Buy it became (-21.06+1.44)/0.89 = -22.0 and was BLOCKED. It vetoed its
own ensemble on ~95% of bars, and that WAS the chart's 3.3% coverage. Neither
existing guard caught it: it IS self-ranked and its tier weights were 11-14.
The fix has to remove it from the DIVISOR, not just the sum - an abstainer
contributes weight by design, so zeroing only the contribution makes the dilution
worse. VoteCapableWeight() already means exactly "may this member's weight sit in
the denominator", so the skill test belongs there. ReconstructionWeight() and the
OOS scorer's divisor move with it or the scorer certifies a vote live does not
cast. The skill test reads the PREVIOUS era's measurement - gating this era's
vote on this era's own outcome would be circular.
3. CHECKPOINT BURN-IN (ENSEMBLE_CHECKPOINT_MIN_ERA 20).
XAUUSD deployed the checkpoint from ERA 2, XTIUSD from ERA 4, each after 69 and
65 further eras failed to beat it. Ensemble coverage measures AGREEMENT, and four
models that have barely moved off their initialisation agree almost by
construction - so coverage is inflated exactly when the models know least and
decays as they differentiate (XAUUSD 6.6% at era 8 -> 0.4% at era 75). Since
selectionScore is precision discounted by coverage, an early era outscores every
mature one and the ladder freezes on it.
INTENDED CONSEQUENCE: a chart whose MATURE coverage cannot clear the floor now
refuses to deploy rather than shipping era-2 weights. Fewer deploys, honest ones.
Burn-in eras are also kept out of g_ensCandidateEras (they could not have won, so
counting them inflates the family-wise N and raises the bar for nothing) and out
of g_ensErasSinceBest (or the run reaches "no better vote for N eras" with no
best to beat, exhausting the escalation ladder before the first era may compete).
Every pinned threshold and .stats record is in the OLD currency and is now
meaningless - this forces a fresh start on its own. Nothing needs re-tuning
because the threshold is DERIVED: the sweep re-picks the rung by itself.
Compiled clean; NOT yet run.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-26 15:38:26 -04:00
|
|
|
//--- THE CURRENCY IS EDGE OVER CHANCE, NOT AN ABSOLUTE WIN RATE (2026-08-26).
|
|
|
|
|
//---
|
|
|
|
|
//--- A tier weight is a raw win rate, and a raw win rate means nothing without the chance rate it
|
|
|
|
|
//--- is measured against. 30% is a strong call under a 14% base rate and a catastrophic one under
|
|
|
|
|
//--- 50% - yet both entered the mean as "30". That is why the threshold had to be re-tuned every
|
|
|
|
|
//--- time the LABEL changed (25 was permissive at ~70% win rates under the old direction label and
|
|
|
|
|
//--- a near-unanimity rule at ~30% under the pivot-event one), and why one chart's 25% was never
|
|
|
|
|
//--- the same statement as another's.
|
|
|
|
|
//---
|
|
|
|
|
//--- Subtracting the member's own chance rate fixes both: a chance-level call contributes 0 on its
|
|
|
|
|
//--- own, the units become percentage points of demonstrated edge, and the number is comparable
|
|
|
|
|
//--- across charts, labels and regimes. The threshold no longer needs re-tuning when any of those
|
|
|
|
|
//--- move - and since it is DERIVED rather than configured, the sweep re-picks the rung by itself.
|
|
|
|
|
//---
|
|
|
|
|
//--- CLAMPED AT ZERO, deliberately. A below-chance tier is anti-informative, and treating it as an
|
|
|
|
|
//--- inverted oracle (contributing negatively, i.e. voting the other way) would be acting on a
|
|
|
|
|
//--- broken model's output rather than discarding it. Zero means "this call carries nothing"; the
|
|
|
|
|
//--- member stays in the divisor because it DID look - only a member with no demonstrated skill at
|
|
|
|
|
//--- all leaves the denominator, via VoteCapableWeight().
|
|
|
|
|
double chancePct = m_eraStatChancePct;
|
|
|
|
|
if(chancePct < 0.0)
|
|
|
|
|
return 0.0; // no reference rate yet: an unmeasurable edge is not a strong one
|
|
|
|
|
double edge = w - chancePct;
|
|
|
|
|
if(edge <= 0.0)
|
|
|
|
|
return 0.0;
|
|
|
|
|
double contribution = m_weight * edge;
|
fix(gate): the ensemble gate certified a vote the EA never casts
g_ensembleVoteThreshold's comment claims the combined-vote scorer "fires on
the same criterion the live trade does". It did not. Two independent
mismatches, both silent:
CURRENCY. Each member contributed its raw signed confidence x100 - a 33..100
number straight off the softmax head. Live contributes m_weight x the tier's
pattern weight, and BOTH of those are rewritten from the signal DB by
UpdateSignalsWeights(). A head output and a DB-ranked win-rate weight share
an axis and nothing relates them, so the same bar was one number to the gate
and a different one to the order path. Same shape as the 2026-08-09 geometry
incident: certified on one game, paid on another.
DENOMINATOR. The gate divided by the member count, so an abstaining member
pulled the average toward zero. CExpertSignalCustom::Direction() skips a zero
contribution in BOTH the sum and the count (`if(direction == 0) continue;`
before `number++`) - live is a mean over VOTERS. The gate was therefore
scoring a strictly more agreement-heavy set of bars than the EA trades. The
contribution hook's own comment asserted the opposite ("abstentions dilute
the average exactly as they do in the live vote"), while the AI_CHOICE enum
20 lines away correctly documented union semantics.
LiveVoteContribution() is now the single definition of "what this member
votes", called from the gate; the live path reaches the same arithmetic
through LongCondition/ShortCondition. g_ensVoteVoterMask records who actually
voted, separately from who evaluated the bar, because those are the divisor
and the shared-population test respectively.
ConfidenceTier() is split into ConfidenceTierFor(signal) plus a thin live-bar
wrapper - the OOS scan holds the scanned bar's decision in a local, and
dPrevSignal is a different bar.
NOT changed, deliberately: the gate still does not model live NMS
declustering, and the per-member solo gate still scores every directional
call rather than threshold-clearing ones. Both are selection-metric changes
and this codebase has twice been bitten by switching one blind.
Also corrects a stale paragraph in m_pattern_0's declaration block quoting
80/87/93/100 as the tier defaults. The constructor is 25/50/75/100 and has
been since the confidence floor and alternation gate were removed; the block
carried both tables at once, and the dead one was quoted back as fact.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-18 15:23:33 -04:00
|
|
|
if(!MathIsValidNumber(contribution))
|
|
|
|
|
return 0.0;
|
|
|
|
|
return (s == Buy) ? contribution : -contribution;
|
|
|
|
|
}
|
|
|
|
|
//+------------------------------------------------------------------+
|
2026-08-01 11:27:28 -04:00
|
|
|
//| Returns the given tier's current pattern weight (0-100) |
|
|
|
|
|
//+------------------------------------------------------------------+
|
|
|
|
|
int CExpertSignalAIBase::PatternWeightForTier(int tier)
|
|
|
|
|
{
|
|
|
|
|
switch(tier)
|
|
|
|
|
{
|
|
|
|
|
case 0:
|
|
|
|
|
return m_pattern_0;
|
|
|
|
|
case 1:
|
|
|
|
|
return m_pattern_1;
|
|
|
|
|
case 2:
|
|
|
|
|
return m_pattern_2;
|
|
|
|
|
default:
|
|
|
|
|
return m_pattern_3;
|
|
|
|
|
}
|
|
|
|
|
}
|
|
|
|
|
//+------------------------------------------------------------------+
|
fix(chart): a deployed model rescans history to rebuild its vote arrows
The sidecar added in 484a9d8 restores the vote arrows from the previous
session - but there was no previous session to restore from, and a
deployed ensemble could never produce one.
The overlay that draws the vote layer replays each member's
m_overlaySigSnap, published in exactly one place: RankTiersFromOos, at
pass-3 completion. A converged model runs no further eras. So after a
restart every member's snapshot was empty, would never fill, the sweep
had nothing to replay and the chart stayed blank permanently - no route
back by any path.
The chart rescan is the route: it runs the DEPLOYED net forward over
history and rebuilds the per-bar cache, which is the same quantity pass 3
produces, obtained without training. It already existed for the panel's
Show-Signals button; it just never handed its result to the overlay, so
on the default filtered view a rescan rebuilt only the RAW per-member
layer - the one that is hidden - and appeared to do nothing.
- PublishOverlaySnapshotFromCache() extracted from RankTiersFromOos, so
the era end and a completed rescan publish through one implementation.
- A completed rescan now calls it, which also arms the sweep.
- PollTraining auto-arms one rescan for a model that is converged, has no
snapshot, and is on the filtered view. One-shot: a model that
legitimately calls Neutral everywhere must not rescan forever chasing a
snapshot that is correctly empty. On the timer, not in OnInit - it is a
full feedForward per bar over up to 5000 bars and drains in the same
time-boxed slices as a manual rescan.
Together with the sidecar this closes both halves: the rescan covers the
first session and any chart whose file was lost or invalidated by a
threshold change; the sidecar covers every session after one is saved.
Compile-verified in _claude_stage: 0 errors, 0 warnings.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-25 13:11:36 -04:00
|
|
|
//| COPY THE PER-BAR CACHE INTO THE OVERLAY'S SNAPSHOT, and tell the |
|
|
|
|
|
//| EA this member is ready to be swept. |
|
|
|
|
|
//| |
|
|
|
|
|
//| Display-side code reads the SNAPSHOT rather than m_arrowSignalCache|
|
|
|
|
|
//| because the cache is wiped at every era start - a sweep reading it |
|
|
|
|
|
//| directly would draw from a half-rebuilt array. See |
|
|
|
|
|
//| project_chart_filtered_view: the display may forward the live net, |
|
|
|
|
|
//| but must never read a live TRAINING cache. |
|
|
|
|
|
//| |
|
|
|
|
|
//| TWO CALLERS, AND THE SECOND ONE IS THE POINT. RankTiersFromOos |
|
|
|
|
|
//| calls it at pass-3 completion, which covers a model that is still |
|
|
|
|
|
//| training. A DEPLOYED model runs no further eras, so on a restart |
|
|
|
|
|
//| its snapshot was empty and stayed empty forever - the sweep had |
|
|
|
|
|
//| nothing to replay, drew nothing, and the vote arrows could never |
|
|
|
|
|
//| come back (user report 2026-08-25, "still no signals drawn on |
|
|
|
|
|
//| chart"). A completed chart rescan now publishes here too: the |
|
|
|
|
|
//| rescan runs the DEPLOYED net forward over history, which is |
|
|
|
|
|
//| exactly the same quantity pass 3 would have produced, obtained |
|
|
|
|
|
//| without training. |
|
|
|
|
|
//+------------------------------------------------------------------+
|
|
|
|
|
void CExpertSignalAIBase::PublishOverlaySnapshotFromCache(void)
|
feat(rank): AI models rank their own confidence tiers from held-out outcomes
Closes the caveat 4858507 shipped with: the vote is a confidence percentage,
but only to the extent the pattern weights are measured. AI tier weights sat
at their designed defaults (25/50/75/100) because AI rows only ever arrive
from LIVE journaling, of which a training run produces almost none.
AND A STALE-TIER BUG THAT MADE THE EVIDENCE MEANINGLESS. The OOS scan bucketed
every scanned bar by ConfidenceTier(), which reads dPrevSignal - and
dPrevSignal is assigned in PASS 1 only, never anywhere in the OOS scan. So an
entire era's fires were bucketed by one stale, unrelated bar's confidence and
landed in a SINGLE tier. That is the "tier prec T0:72%(828) T1:n/a(0)
T2:n/a(0) T3:n/a(0)" symptom recorded on 2026-08-16 and attributed to the
calibration clamp. The clamp was real and was fixed then; this is a second,
independent cause of the identical output that survived that fix untouched -
which is why the log kept reading the same afterwards. Two causes, one symptom.
Now ConfidenceTierFor(adjSig): the bar this iteration actually scored.
WHY THIS DOES NOT WRITE ROWS TO THE SIGNAL DB, which was the obvious reading of
"fill the database during training". The user's own observation is the reason:
a classic Pattern_2 is a fixed geometric condition, so its win rate is
legitimately accumulated over years, but an AI Pattern_2 means "confidence
landed in tier 2" and tier 2 under era 100's weights is a different statement
from tier 2 under era 500's. The DB's value is ACCUMULATION, and accumulation
is exactly what is wrong here - it would average together models that no
longer exist, while colliding with the per-table row cap and mixing
measured-on-holdout outcomes into the live ledger's own tables. What the DB
actually supplies is a measured win rate per pattern, and pass 3 already
computes that on held-out bars, thousands at a time. So the model ranks itself
once per era, REPLACING rather than accumulating, which makes the weights
describe the current weights by construction.
ESTIMATOR. Not WinRateFromCounts(): it returns NO_DATA below 100 raw trades
BEFORE shrinking, which here would fire on every tier every era and hand all
four the pooled rate - the tiers could never separate and the mechanism would
be inert. Shrinkage is the answer to a small sample; a floor in front of it
means the shrinkage never runs. Instead: a Beta prior of TIER_PRIOR_EFF_N
pseudo-observations centred on the model's pooled holdout rate, counted in
EFFECTIVE observations, because overlapping triple-barrier labels mean 800 raw
fires can be worth ~12 independent ones. Rounded to the integer, not to the
decade NormalizeWinRate() uses, which would collapse the shrunk tiers back
into one number.
NO SAME-ERA CIRCULARITY, and it falls out of the ordering rather than a guard:
weights are computed at the END of era N, so the vote scored during era N was
cast with era N-1's weights. The deploy gate never grades a vote whose weights
were fitted on the bars it is scoring. Residual leakage remains - the same OOS
bars each era under a different model - and is stated in the code rather than
papered over.
Both DB clobber paths are closed: ApplyPatternWeight() declines once
self-ranked, and UpdateSignalsWeights()' filter.Weight() call is guarded by
SelfRanked() - guarding only the tiers would have let the hourly ranking pass
undo half the self-ranking.
NOT COMPILED - user compiles in MetaEditor.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-18 16:00:32 -04:00
|
|
|
{
|
fix(chart): display now reads era-end SNAPSHOTS - the live cache is wiped mid-era
Full-pipeline analysis after "threshold 30, attained often, nothing drawn,
still glued to buy". The log falsified the premise before any code did:
21:40:43 swept 4999, 794 voters, drew 491. Strongest 43.0% vs 30.0%
21:42:07 swept 4999, 0 voters, drew 0
21:51:30 swept 4999, 0 voters, drew 0
21:56:30 swept 4999, 922 voters, drew 382. Strongest 44.0% vs 30.0%
The arrows WERE drawn - 491 of them, then 382 - and then erased. ONE root
cause, three symptoms: every display path read m_arrowSignalCache, which is
wiped to sentinel at each era start and only complete again when pass 3
finishes. With eras at ~30s and a sweep at ~17s:
* ARROW FLICKER: a sweep landing mid-era found no voters anywhere, and its
else-branch deleted the arrow on every voteless bar - erasing the previous
sweep's entire output. The chart cycled populated -> blank -> populated;
the user kept catching the blank phase.
* READOUT GLUE: the newest-cache walk found only sentinel for ~90% of every
era and fell through to dPrevSignal - the frozen purge-band edge bar that
reads Buy. 659638e fixed which bar was frozen, not the freezing.
* VOTER FLAP: 1299 -> 257 -> 1113 across back-to-back sweeps - each saw a
different fraction of half-rebuilt caches.
THE FIX, structural rather than another patch:
1. Era-end snapshots. RankTiersFromOos() runs at pass-3 completion - the one
moment the cache is complete - and now copies it (raw signals, newest
LOOKBACK+16 bars) into member-owned snapshot state, unconditionally,
BEFORE its early return: an all-Neutral era is a snapshot worth showing,
not an absence of one. Raw signals rather than votes, so a tier re-rank
between eras reprices them at read time via LiveVoteContribution for free.
2. The sweep (SnapshotVoteAt) and the prospective readout both read
snapshots; the readout's fallback chain is live-cache -> snapshot ->
dPrevSignal, and the snapshot leg is the one that fires most of the time.
3. NO DATA IS NOT A VERDICT: a den==0 bar no longer deletes - only an actual
sub-threshold vote takes an arrow down. This alone ends the wipe half of
the flicker even where snapshots are missing (before the first era).
4. Arming moved from an era-counter diff (which fires at era BOUNDARIES,
i.e. precisely when caches are about to be wiped) to
g_warriorOverlayArmRequest, set by each RankTiersFromOos - "a member's
snapshot just got fresher", the only event a redraw can act on. 60s rate
limit collapses the four members' burst into one sweep. Classic-only
charts arm once at start.
5. Census now reports the direction split - "922 had a voter (610 buy / 312
sell)" - so "the vote leans buy" is checkable from the log instead of
inferred from arrow colours.
Also visible in the log and worth knowing: the threshold flip-flopped
30 -> 40 -> 30 across the evening's re-inits (census lines at 21:42-21:51
ran at 40), so part of the observed blankness was configuration, not code.
NOT COMPILED - user compiles in MetaEditor.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-18 22:03:20 -04:00
|
|
|
int snapN = MathMin(ArraySize(m_arrowSignalCache), SIGNAL_RESCAN_LOOKBACK_BARS + 16);
|
|
|
|
|
if(snapN > 0)
|
|
|
|
|
{
|
|
|
|
|
ArrayResize(m_overlaySigSnap, snapN);
|
|
|
|
|
ArrayCopy(m_overlaySigSnap, m_arrowSignalCache, 0, 0, snapN);
|
|
|
|
|
}
|
|
|
|
|
m_overlaySnapBars = snapN;
|
|
|
|
|
m_prospectiveSigSnap = -2.0;
|
|
|
|
|
for(int pi = 1; pi <= 16 && pi < snapN; pi++)
|
|
|
|
|
if(m_overlaySigSnap[pi] != -2.0 && MathIsValidNumber(m_overlaySigSnap[pi]))
|
|
|
|
|
{
|
|
|
|
|
m_prospectiveSigSnap = m_overlaySigSnap[pi];
|
|
|
|
|
break;
|
|
|
|
|
}
|
fix(vote): unranked members voted with the stock 25/50/75/100 ladder
Two defects behind "arrows drawn while members are still mid-era".
1. THE DRAW. The filtered overlay armed on the FIRST member to finish
pass 3 and leaned on a 60 s rate limit to "collapse the burst",
assuming members finish seconds apart. They do not - on USDJPY one
member was at sample 10496 of pass 2 while another was at 2304,
minutes apart. A member with no era-end snapshot returns false from
SnapshotVoteAt, and the sweep's `if(!hasData) continue;` skips it
BEFORE `den += ModuleWeight()`, so the one finished model's tier
weight became the entire vote and was drawn as a consensus arrow.
An abstention is a member that looked at the bar and said nothing; a
missing snapshot is a member that has not looked. The first must
dilute the vote, the second must suppress the draw. The arm is now a
readiness MASK - one bit per m_ensembleIndex, set at that member's
pass-3 completion, cleared when a sweep arms - and a sweep waits for
every enrolled member. Bounded at 10 minutes so a member that stops
cannot freeze the chart, and the partial draw PRINTS which members
were missing: the be39674 lesson is that a hold must never silence
the thing that reports it.
2. THE VOTE ITSELF, which is the worse half and is not display-only.
Tier weights are not persisted in the .nnw - they exist only as the
output of a completed pass 3 - so before a member's first
RankTiersFromOos() it holds the constructor's stock 25/50/75/100.
Since 4858507 the vote currency is a WIN RATE, so an unranked tier-3
call enters the capability-weighted mean claiming a 100% win rate
beside ranked members contributing ~25. Not a strong opinion: the
wrong unit. One unranked member drags the ensemble over any
threshold, on every fresh deploy and every resume. USDJPY has a
measured ceiling of ~19 and was firing anyway.
LiveVoteContribution() now abstains until self-ranked, which drops
the member from the sum AND the divisor. One function, so live and
the gate move together (2c443ba).
Era 0 will therefore report 0 coverage until each member completes one
era. The ensemble line says so explicitly rather than leaving it to look
like the USDJPY unreachable-threshold case - the two are identical in
the coverage number and completely different problems.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-23 08:55:04 -04:00
|
|
|
//--- ...and tell the EA THIS member's snapshot is fresh. The sweep waits for every enrolled
|
|
|
|
|
//--- member's bit - see g_warriorOverlayReadyMask for why a rate limit was not enough.
|
|
|
|
|
if(m_ensembleIndex >= 0 && m_ensembleIndex < ENS_MAX_MEMBERS)
|
|
|
|
|
g_warriorOverlayReadyMask |= (((uint)1) << m_ensembleIndex);
|
fix(chart): display now reads era-end SNAPSHOTS - the live cache is wiped mid-era
Full-pipeline analysis after "threshold 30, attained often, nothing drawn,
still glued to buy". The log falsified the premise before any code did:
21:40:43 swept 4999, 794 voters, drew 491. Strongest 43.0% vs 30.0%
21:42:07 swept 4999, 0 voters, drew 0
21:51:30 swept 4999, 0 voters, drew 0
21:56:30 swept 4999, 922 voters, drew 382. Strongest 44.0% vs 30.0%
The arrows WERE drawn - 491 of them, then 382 - and then erased. ONE root
cause, three symptoms: every display path read m_arrowSignalCache, which is
wiped to sentinel at each era start and only complete again when pass 3
finishes. With eras at ~30s and a sweep at ~17s:
* ARROW FLICKER: a sweep landing mid-era found no voters anywhere, and its
else-branch deleted the arrow on every voteless bar - erasing the previous
sweep's entire output. The chart cycled populated -> blank -> populated;
the user kept catching the blank phase.
* READOUT GLUE: the newest-cache walk found only sentinel for ~90% of every
era and fell through to dPrevSignal - the frozen purge-band edge bar that
reads Buy. 659638e fixed which bar was frozen, not the freezing.
* VOTER FLAP: 1299 -> 257 -> 1113 across back-to-back sweeps - each saw a
different fraction of half-rebuilt caches.
THE FIX, structural rather than another patch:
1. Era-end snapshots. RankTiersFromOos() runs at pass-3 completion - the one
moment the cache is complete - and now copies it (raw signals, newest
LOOKBACK+16 bars) into member-owned snapshot state, unconditionally,
BEFORE its early return: an all-Neutral era is a snapshot worth showing,
not an absence of one. Raw signals rather than votes, so a tier re-rank
between eras reprices them at read time via LiveVoteContribution for free.
2. The sweep (SnapshotVoteAt) and the prospective readout both read
snapshots; the readout's fallback chain is live-cache -> snapshot ->
dPrevSignal, and the snapshot leg is the one that fires most of the time.
3. NO DATA IS NOT A VERDICT: a den==0 bar no longer deletes - only an actual
sub-threshold vote takes an arrow down. This alone ends the wipe half of
the flicker even where snapshots are missing (before the first era).
4. Arming moved from an era-counter diff (which fires at era BOUNDARIES,
i.e. precisely when caches are about to be wiped) to
g_warriorOverlayArmRequest, set by each RankTiersFromOos - "a member's
snapshot just got fresher", the only event a redraw can act on. 60s rate
limit collapses the four members' burst into one sweep. Classic-only
charts arm once at start.
5. Census now reports the direction split - "922 had a voter (610 buy / 312
sell)" - so "the vote leans buy" is checkable from the log instead of
inferred from arrow colours.
Also visible in the log and worth knowing: the threshold flip-flopped
30 -> 40 -> 30 across the evening's re-inits (census lines at 21:42-21:51
ran at 40), so part of the observed blankness was configuration, not code.
NOT COMPILED - user compiles in MetaEditor.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-18 22:03:20 -04:00
|
|
|
g_warriorOverlayArmRequest = true;
|
fix(chart): a deployed model rescans history to rebuild its vote arrows
The sidecar added in 484a9d8 restores the vote arrows from the previous
session - but there was no previous session to restore from, and a
deployed ensemble could never produce one.
The overlay that draws the vote layer replays each member's
m_overlaySigSnap, published in exactly one place: RankTiersFromOos, at
pass-3 completion. A converged model runs no further eras. So after a
restart every member's snapshot was empty, would never fill, the sweep
had nothing to replay and the chart stayed blank permanently - no route
back by any path.
The chart rescan is the route: it runs the DEPLOYED net forward over
history and rebuilds the per-bar cache, which is the same quantity pass 3
produces, obtained without training. It already existed for the panel's
Show-Signals button; it just never handed its result to the overlay, so
on the default filtered view a rescan rebuilt only the RAW per-member
layer - the one that is hidden - and appeared to do nothing.
- PublishOverlaySnapshotFromCache() extracted from RankTiersFromOos, so
the era end and a completed rescan publish through one implementation.
- A completed rescan now calls it, which also arms the sweep.
- PollTraining auto-arms one rescan for a model that is converged, has no
snapshot, and is on the filtered view. One-shot: a model that
legitimately calls Neutral everywhere must not rescan forever chasing a
snapshot that is correctly empty. On the timer, not in OnInit - it is a
full feedForward per bar over up to 5000 bars and drains in the same
time-boxed slices as a manual rescan.
Together with the sidecar this closes both halves: the rescan covers the
first session and any chart whose file was lost or invalidated by a
threshold change; the sidecar covers every session after one is saved.
Compile-verified in _claude_stage: 0 errors, 0 warnings.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-25 13:11:36 -04:00
|
|
|
}
|
|
|
|
|
//+------------------------------------------------------------------+
|
feat(vote): replay pass rebuilds a deployed model's ladder without retraining
The previous commit persisted the tier ladder, which fixes this going
forward but did nothing for models whose .stats predates WST7 - they
still had to retrain to mint one. They never did. Every number a
converged model needs in order to vote is a pure function of weights
already on disk plus labels derivable from the chart, so replay them:
stage 1 build the label cache (existing chunked prebuild)
stage 2 rescan history (existing chunked rescan, deployed net)
stage 3 score + rank + persist (one walk over two arrays)
ScoreReplayFromCache() walks m_arrowSignalCache against
m_labelCacheBuy/Sell, fills the same m_oosTierFired/Hits and per-class
totals pass 3 fills, and hands them to RankTiersFromOos() - deliberately
feeding the existing ranker rather than reimplementing it. The shrinkage,
the chance reference and the module trust weight are subtle enough that a
second copy would drift, and a ladder measured by a slightly different
rule would be silently incomparable with every ladder training produced.
AdvanceDeployedRebuild() sequences the three stages off the timer. It has
to be a sequence: stages 1 and 2 are each minutes of work draining in
time-boxed slices, and stage 2's output is meaningless until stage 1 has
labels to score against. The previous version ran the rescan with no
labels at all, which is why it could only ever rebuild arrows and never
the ladder - the thing actually blocking the vote.
The result is written to .stats immediately. The failure being repaired
is state that lived in memory and was never written down; recomputing it
and not saving it would repeat that exactly.
Also routes every rescan completion through one hook, so there is a
single place that knows what a finished rescan means - republish for a
manual one, score and rank for a rebuild.
Compile-verified in _claude_stage: 0 errors, 0 warnings.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-25 15:01:24 -04:00
|
|
|
//| THE REPLAY PASS: score the deployed net's own history, then rank. |
|
|
|
|
|
//| |
|
|
|
|
|
//| A converged model's tier ladder is produced by pass 3 and by |
|
|
|
|
|
//| nothing else, so before this existed a deployed model that had |
|
|
|
|
|
//| lost its ladder (a .stats predating WST7, or a wipe) could only |
|
|
|
|
|
//| get one back by RETRAINING - hours of work to recompute numbers |
|
|
|
|
|
//| that are a pure function of weights already on disk. |
|
|
|
|
|
//| |
|
|
|
|
|
//| This is the same measurement without the training. The rescan has |
|
|
|
|
|
//| already run the DEPLOYED net over every bar in the window and left |
|
|
|
|
|
//| its prior-corrected decision in m_arrowSignalCache; the label |
|
|
|
|
|
//| prebuild has filled m_labelCacheBuy/Sell for the same bars. So the |
|
|
|
|
|
//| ladder is one walk over two arrays that already agree on indexing. |
|
|
|
|
|
//| |
|
|
|
|
|
//| It feeds RankTiersFromOos() rather than reimplementing it. The |
|
|
|
|
|
//| shrinkage, the chance reference and the module trust weight are |
|
|
|
|
|
//| subtle enough that a second copy would drift from the first, and |
|
|
|
|
|
//| a ladder measured by a slightly different rule is worse than no |
|
|
|
|
|
//| ladder - it would be silently incomparable with every ladder any |
|
|
|
|
|
//| training run ever produced. |
|
|
|
|
|
//| |
|
|
|
|
|
//| INDEXING: both arrays are series-indexed (0 = newest). Bar 0 is |
|
|
|
|
|
//| skipped because it is still forming. |
|
|
|
|
|
//+------------------------------------------------------------------+
|
feat(vote): backfill the ensemble win-rate record from the overlay sweep
"Vote win rate: measuring..." never resolved on a deployed chart whose
.stats predate the WST7 ensemble record: g_ensCumOosTotal is fed only by
the era-end combined-vote scorer (Training.mqh), and a deployed ensemble
runs no further eras. The replay pass rebuilt every MEMBER's ladder
(64-71% each, per the 16:12 log) but nothing ever scored the COMBINED
vote, so the aggregate line sat on "measuring" while 300+ arrows drew.
The overlay sweep already reconstructs the vote per bar with the live
threshold and direction policy - so it now also tallies, BEFORE
declustering (NMS thins arrows, not calls), each threshold-clearing bar
against the inline swing-pivot label (same resolution ScoreReplayFromCache
uses, same window-mismatch reason). On sweep completion Warrior_EA.mq5
harvests the tally through a consuming one-shot read and adopts it ONLY
when the record is empty and the models are deployed - a training-time
sweep can never pre-empt the era scorer, and a restored record always
wins. The result is persisted immediately into every member's .stats.
Also verified against the same log: the sweep does NOT ignore
DrawUnfilteredSignals - 4986 voter bars -> ~300 arrows, all gated on the
25% open threshold. The arrow increase vs the restored set (41-312 saved)
is the replay-minted ladder reading stronger (partly in-sample), plus the
reconstruction deliberately not replaying order validation/session hours
(tooltip says so); the backfilled record carries the same caveat and is
labelled so in the log.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-25 18:22:12 -04:00
|
|
|
//+------------------------------------------------------------------+
|
|
|
|
|
//| The resolved swing-pivot label at one bar, for the head's |
|
|
|
|
|
//| combined-vote scoring during the overlay sweep. Inline label |
|
|
|
|
|
//| resolution, NOT the prebuilt cache - identical reasoning to the |
|
|
|
|
|
//| comment inside ScoreReplayFromCache below: the cache's window is |
|
|
|
|
|
//| anchored at dtStudied and need not overlap the sweep's. |
|
|
|
|
|
//+------------------------------------------------------------------+
|
|
|
|
|
bool CExpertSignalAIBase::ReplayTruthAt(const int idx, ENUM_SIGNAL &truth)
|
|
|
|
|
{
|
|
|
|
|
truth = SwingPivotDirectionLabel(idx);
|
|
|
|
|
if(m_lastLabelLifespan <= 0)
|
|
|
|
|
{
|
|
|
|
|
truth = Neutral;
|
|
|
|
|
return false; // pivot pair not committed: no label exists for this bar
|
|
|
|
|
}
|
|
|
|
|
return true;
|
|
|
|
|
}
|
feat(vote): replay pass rebuilds a deployed model's ladder without retraining
The previous commit persisted the tier ladder, which fixes this going
forward but did nothing for models whose .stats predates WST7 - they
still had to retrain to mint one. They never did. Every number a
converged model needs in order to vote is a pure function of weights
already on disk plus labels derivable from the chart, so replay them:
stage 1 build the label cache (existing chunked prebuild)
stage 2 rescan history (existing chunked rescan, deployed net)
stage 3 score + rank + persist (one walk over two arrays)
ScoreReplayFromCache() walks m_arrowSignalCache against
m_labelCacheBuy/Sell, fills the same m_oosTierFired/Hits and per-class
totals pass 3 fills, and hands them to RankTiersFromOos() - deliberately
feeding the existing ranker rather than reimplementing it. The shrinkage,
the chance reference and the module trust weight are subtle enough that a
second copy would drift, and a ladder measured by a slightly different
rule would be silently incomparable with every ladder training produced.
AdvanceDeployedRebuild() sequences the three stages off the timer. It has
to be a sequence: stages 1 and 2 are each minutes of work draining in
time-boxed slices, and stage 2's output is meaningless until stage 1 has
labels to score against. The previous version ran the rescan with no
labels at all, which is why it could only ever rebuild arrows and never
the ladder - the thing actually blocking the vote.
The result is written to .stats immediately. The failure being repaired
is state that lived in memory and was never written down; recomputing it
and not saving it would repeat that exactly.
Also routes every rescan completion through one hook, so there is a
single place that knows what a finished rescan means - republish for a
manual one, score and rank for a rebuild.
Compile-verified in _claude_stage: 0 errors, 0 warnings.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-25 15:01:24 -04:00
|
|
|
void CExpertSignalAIBase::ScoreReplayFromCache(void)
|
|
|
|
|
{
|
fix(replay): resolve labels inline - the prebuilt cache's window never overlapped the rescan
The 15:13 session proved the replay pass ran end-to-end on all 24 models
and scored ZERO labelled bars on every one of them, while each rescan sat
on ~5000 scored predictions (~2755 Buy / ~2232 Sell). The two windows
never overlapped:
StartLabelCachePrebuild deliberately keeps a CONVERGED model's
dtStudied watermark (it gates inference recency and must not move), so
the prebuild's window was the handful of bars since the last studied
bar - all with uncommitted pivots, hence "label cache pre-built -
Buy: 0 | Sell: 0 | Neutral: 0" on every member.
The label never needed a cache. SwingPivotDirectionLabel(idx) is a pure
function of the ZigZag/Close/ATR buffers the rescan itself refreshes over
exactly the scoring window, and m_lastLabelLifespan == 0 is its own
unresolved flag - the same finality gate the cache applies, applied
directly. ScoreReplayFromCache now resolves each bar's label inline and
the label-prebuild stage is deleted from the rebuild state machine
outright; going through a cache built for a different window was
indirection that changed the answer.
Also splits the empty-result diagnostics: "no resolved labels" (a
windowing/data fault) is now distinguished from "labels present, every
call Neutral" (a calibration verdict). The first version reported the
second message for both, which mislabelled this very bug as a calibration
outcome in the same breath as reporting scored=0.
Honest limitation, stated in the code too: the replay window includes
bars the model trained on, so a replay-minted ladder is measured partly
in-sample and will read stronger than a holdout-measured one. It is
replaced by the genuine article at the next completed scoring pass; until
then it is what makes a restarted deployed model able to vote at all.
Compile-verified in _claude_stage: 0 errors, 0 warnings.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-25 15:22:01 -04:00
|
|
|
int n = ArraySize(m_arrowSignalCache);
|
feat(vote): replay pass rebuilds a deployed model's ladder without retraining
The previous commit persisted the tier ladder, which fixes this going
forward but did nothing for models whose .stats predates WST7 - they
still had to retrain to mint one. They never did. Every number a
converged model needs in order to vote is a pure function of weights
already on disk plus labels derivable from the chart, so replay them:
stage 1 build the label cache (existing chunked prebuild)
stage 2 rescan history (existing chunked rescan, deployed net)
stage 3 score + rank + persist (one walk over two arrays)
ScoreReplayFromCache() walks m_arrowSignalCache against
m_labelCacheBuy/Sell, fills the same m_oosTierFired/Hits and per-class
totals pass 3 fills, and hands them to RankTiersFromOos() - deliberately
feeding the existing ranker rather than reimplementing it. The shrinkage,
the chance reference and the module trust weight are subtle enough that a
second copy would drift, and a ladder measured by a slightly different
rule would be silently incomparable with every ladder training produced.
AdvanceDeployedRebuild() sequences the three stages off the timer. It has
to be a sequence: stages 1 and 2 are each minutes of work draining in
time-boxed slices, and stage 2's output is meaningless until stage 1 has
labels to score against. The previous version ran the rescan with no
labels at all, which is why it could only ever rebuild arrows and never
the ladder - the thing actually blocking the vote.
The result is written to .stats immediately. The failure being repaired
is state that lived in memory and was never written down; recomputing it
and not saving it would repeat that exactly.
Also routes every rescan completion through one hook, so there is a
single place that knows what a finished rescan means - republish for a
manual one, score and rank for a rebuild.
Compile-verified in _claude_stage: 0 errors, 0 warnings.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-25 15:01:24 -04:00
|
|
|
if(n <= 1)
|
|
|
|
|
{
|
fix(replay): resolve labels inline - the prebuilt cache's window never overlapped the rescan
The 15:13 session proved the replay pass ran end-to-end on all 24 models
and scored ZERO labelled bars on every one of them, while each rescan sat
on ~5000 scored predictions (~2755 Buy / ~2232 Sell). The two windows
never overlapped:
StartLabelCachePrebuild deliberately keeps a CONVERGED model's
dtStudied watermark (it gates inference recency and must not move), so
the prebuild's window was the handful of bars since the last studied
bar - all with uncommitted pivots, hence "label cache pre-built -
Buy: 0 | Sell: 0 | Neutral: 0" on every member.
The label never needed a cache. SwingPivotDirectionLabel(idx) is a pure
function of the ZigZag/Close/ATR buffers the rescan itself refreshes over
exactly the scoring window, and m_lastLabelLifespan == 0 is its own
unresolved flag - the same finality gate the cache applies, applied
directly. ScoreReplayFromCache now resolves each bar's label inline and
the label-prebuild stage is deleted from the rebuild state machine
outright; going through a cache built for a different window was
indirection that changed the answer.
Also splits the empty-result diagnostics: "no resolved labels" (a
windowing/data fault) is now distinguished from "labels present, every
call Neutral" (a calibration verdict). The first version reported the
second message for both, which mislabelled this very bug as a calibration
outcome in the same breath as reporting scored=0.
Honest limitation, stated in the code too: the replay window includes
bars the model trained on, so a replay-minted ladder is measured partly
in-sample and will read stronger than a holdout-measured one. It is
replaced by the genuine article at the next completed scoring pass; until
then it is what makes a restarted deployed model able to vote at all.
Compile-verified in _claude_stage: 0 errors, 0 warnings.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-25 15:22:01 -04:00
|
|
|
Print(ID + ": replay scoring skipped - the rescan left no per-bar signals to score.");
|
feat(vote): replay pass rebuilds a deployed model's ladder without retraining
The previous commit persisted the tier ladder, which fixes this going
forward but did nothing for models whose .stats predates WST7 - they
still had to retrain to mint one. They never did. Every number a
converged model needs in order to vote is a pure function of weights
already on disk plus labels derivable from the chart, so replay them:
stage 1 build the label cache (existing chunked prebuild)
stage 2 rescan history (existing chunked rescan, deployed net)
stage 3 score + rank + persist (one walk over two arrays)
ScoreReplayFromCache() walks m_arrowSignalCache against
m_labelCacheBuy/Sell, fills the same m_oosTierFired/Hits and per-class
totals pass 3 fills, and hands them to RankTiersFromOos() - deliberately
feeding the existing ranker rather than reimplementing it. The shrinkage,
the chance reference and the module trust weight are subtle enough that a
second copy would drift, and a ladder measured by a slightly different
rule would be silently incomparable with every ladder training produced.
AdvanceDeployedRebuild() sequences the three stages off the timer. It has
to be a sequence: stages 1 and 2 are each minutes of work draining in
time-boxed slices, and stage 2's output is meaningless until stage 1 has
labels to score against. The previous version ran the rescan with no
labels at all, which is why it could only ever rebuild arrows and never
the ladder - the thing actually blocking the vote.
The result is written to .stats immediately. The failure being repaired
is state that lived in memory and was never written down; recomputing it
and not saving it would repeat that exactly.
Also routes every rescan completion through one hook, so there is a
single place that knows what a finished rescan means - republish for a
manual one, score and rank for a rebuild.
Compile-verified in _claude_stage: 0 errors, 0 warnings.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-25 15:01:24 -04:00
|
|
|
return;
|
|
|
|
|
}
|
|
|
|
|
//--- Same counters pass 3 fills, cleared the same way. RankTiersFromOos reads ONLY these plus the
|
|
|
|
|
//--- per-class totals below, which is what makes this substitution exact.
|
|
|
|
|
ArrayInitialize(m_oosTierFired, 0);
|
|
|
|
|
ArrayInitialize(m_oosTierHits, 0);
|
|
|
|
|
m_oos.Reset();
|
fix(replay): resolve labels inline - the prebuilt cache's window never overlapped the rescan
The 15:13 session proved the replay pass ran end-to-end on all 24 models
and scored ZERO labelled bars on every one of them, while each rescan sat
on ~5000 scored predictions (~2755 Buy / ~2232 Sell). The two windows
never overlapped:
StartLabelCachePrebuild deliberately keeps a CONVERGED model's
dtStudied watermark (it gates inference recency and must not move), so
the prebuild's window was the handful of bars since the last studied
bar - all with uncommitted pivots, hence "label cache pre-built -
Buy: 0 | Sell: 0 | Neutral: 0" on every member.
The label never needed a cache. SwingPivotDirectionLabel(idx) is a pure
function of the ZigZag/Close/ATR buffers the rescan itself refreshes over
exactly the scoring window, and m_lastLabelLifespan == 0 is its own
unresolved flag - the same finality gate the cache applies, applied
directly. ScoreReplayFromCache now resolves each bar's label inline and
the label-prebuild stage is deleted from the rebuild state machine
outright; going through a cache built for a different window was
indirection that changed the answer.
Also splits the empty-result diagnostics: "no resolved labels" (a
windowing/data fault) is now distinguished from "labels present, every
call Neutral" (a calibration verdict). The first version reported the
second message for both, which mislabelled this very bug as a calibration
outcome in the same breath as reporting scored=0.
Honest limitation, stated in the code too: the replay window includes
bars the model trained on, so a replay-minted ladder is measured partly
in-sample and will read stronger than a holdout-measured one. It is
replaced by the genuine article at the next completed scoring pass; until
then it is what makes a restarted deployed model able to vote at all.
Compile-verified in _claude_stage: 0 errors, 0 warnings.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-25 15:22:01 -04:00
|
|
|
int scored = 0, unresolved = 0;
|
feat(vote): replay pass rebuilds a deployed model's ladder without retraining
The previous commit persisted the tier ladder, which fixes this going
forward but did nothing for models whose .stats predates WST7 - they
still had to retrain to mint one. They never did. Every number a
converged model needs in order to vote is a pure function of weights
already on disk plus labels derivable from the chart, so replay them:
stage 1 build the label cache (existing chunked prebuild)
stage 2 rescan history (existing chunked rescan, deployed net)
stage 3 score + rank + persist (one walk over two arrays)
ScoreReplayFromCache() walks m_arrowSignalCache against
m_labelCacheBuy/Sell, fills the same m_oosTierFired/Hits and per-class
totals pass 3 fills, and hands them to RankTiersFromOos() - deliberately
feeding the existing ranker rather than reimplementing it. The shrinkage,
the chance reference and the module trust weight are subtle enough that a
second copy would drift, and a ladder measured by a slightly different
rule would be silently incomparable with every ladder training produced.
AdvanceDeployedRebuild() sequences the three stages off the timer. It has
to be a sequence: stages 1 and 2 are each minutes of work draining in
time-boxed slices, and stage 2's output is meaningless until stage 1 has
labels to score against. The previous version ran the rescan with no
labels at all, which is why it could only ever rebuild arrows and never
the ladder - the thing actually blocking the vote.
The result is written to .stats immediately. The failure being repaired
is state that lived in memory and was never written down; recomputing it
and not saving it would repeat that exactly.
Also routes every rescan completion through one hook, so there is a
single place that knows what a finished rescan means - republish for a
manual one, score and rank for a rebuild.
Compile-verified in _claude_stage: 0 errors, 0 warnings.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-25 15:01:24 -04:00
|
|
|
for(int i = 1; i < n; i++)
|
|
|
|
|
{
|
|
|
|
|
double sig = m_arrowSignalCache[i];
|
|
|
|
|
if(sig == -2.0 || !MathIsValidNumber(sig))
|
|
|
|
|
continue; // the net never scored this bar (window could not be built)
|
fix(replay): resolve labels inline - the prebuilt cache's window never overlapped the rescan
The 15:13 session proved the replay pass ran end-to-end on all 24 models
and scored ZERO labelled bars on every one of them, while each rescan sat
on ~5000 scored predictions (~2755 Buy / ~2232 Sell). The two windows
never overlapped:
StartLabelCachePrebuild deliberately keeps a CONVERGED model's
dtStudied watermark (it gates inference recency and must not move), so
the prebuild's window was the handful of bars since the last studied
bar - all with uncommitted pivots, hence "label cache pre-built -
Buy: 0 | Sell: 0 | Neutral: 0" on every member.
The label never needed a cache. SwingPivotDirectionLabel(idx) is a pure
function of the ZigZag/Close/ATR buffers the rescan itself refreshes over
exactly the scoring window, and m_lastLabelLifespan == 0 is its own
unresolved flag - the same finality gate the cache applies, applied
directly. ScoreReplayFromCache now resolves each bar's label inline and
the label-prebuild stage is deleted from the rebuild state machine
outright; going through a cache built for a different window was
indirection that changed the answer.
Also splits the empty-result diagnostics: "no resolved labels" (a
windowing/data fault) is now distinguished from "labels present, every
call Neutral" (a calibration verdict). The first version reported the
second message for both, which mislabelled this very bug as a calibration
outcome in the same breath as reporting scored=0.
Honest limitation, stated in the code too: the replay window includes
bars the model trained on, so a replay-minted ladder is measured partly
in-sample and will read stronger than a holdout-measured one. It is
replaced by the genuine article at the next completed scoring pass; until
then it is what makes a restarted deployed model able to vote at all.
Compile-verified in _claude_stage: 0 errors, 0 warnings.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-25 15:22:01 -04:00
|
|
|
//--- THE LABEL IS RESOLVED INLINE, NOT READ FROM THE PREBUILT CACHE. The cache's window is
|
|
|
|
|
//--- anchored at dtStudied, which for a CONVERGED model is deliberately left at its watermark
|
|
|
|
|
//--- (inference recency) - a few bars ago. Scoring against it produced "0 labelled bars" on
|
|
|
|
|
//--- 24/24 models (2026-08-25 15:13 session) while the rescan sat on ~5000 scored predictions:
|
|
|
|
|
//--- the two windows simply never overlapped. SwingPivotDirectionLabel is a pure function of
|
|
|
|
|
//--- the ZigZag/Close/ATR buffers the rescan just refreshed over EXACTLY this window, and
|
|
|
|
|
//--- m_lastLabelLifespan == 0 is its own unresolved flag - the same finality gate the cache
|
|
|
|
|
//--- uses, applied directly. The cache itself is deliberately not touched: it belongs to the
|
|
|
|
|
//--- incremental training path, whose window this is not.
|
|
|
|
|
ENUM_SIGNAL truth = SwingPivotDirectionLabel(i);
|
|
|
|
|
if(m_lastLabelLifespan <= 0)
|
|
|
|
|
{
|
|
|
|
|
unresolved++;
|
|
|
|
|
continue; // pivot pair not committed: no label exists for this bar
|
|
|
|
|
}
|
feat(vote): replay pass rebuilds a deployed model's ladder without retraining
The previous commit persisted the tier ladder, which fixes this going
forward but did nothing for models whose .stats predates WST7 - they
still had to retrain to mint one. They never did. Every number a
converged model needs in order to vote is a pure function of weights
already on disk plus labels derivable from the chart, so replay them:
stage 1 build the label cache (existing chunked prebuild)
stage 2 rescan history (existing chunked rescan, deployed net)
stage 3 score + rank + persist (one walk over two arrays)
ScoreReplayFromCache() walks m_arrowSignalCache against
m_labelCacheBuy/Sell, fills the same m_oosTierFired/Hits and per-class
totals pass 3 fills, and hands them to RankTiersFromOos() - deliberately
feeding the existing ranker rather than reimplementing it. The shrinkage,
the chance reference and the module trust weight are subtle enough that a
second copy would drift, and a ladder measured by a slightly different
rule would be silently incomparable with every ladder training produced.
AdvanceDeployedRebuild() sequences the three stages off the timer. It has
to be a sequence: stages 1 and 2 are each minutes of work draining in
time-boxed slices, and stage 2's output is meaningless until stage 1 has
labels to score against. The previous version ran the rescan with no
labels at all, which is why it could only ever rebuild arrows and never
the ladder - the thing actually blocking the vote.
The result is written to .stats immediately. The failure being repaired
is state that lived in memory and was never written down; recomputing it
and not saving it would repeat that exactly.
Also routes every rescan completion through one hook, so there is a
single place that knows what a finished rescan means - republish for a
manual one, score and rank for a rebuild.
Compile-verified in _claude_stage: 0 errors, 0 warnings.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-25 15:01:24 -04:00
|
|
|
ENUM_SIGNAL pred = DoubleToSignal(sig);
|
|
|
|
|
scored++;
|
|
|
|
|
//--- Per-class totals: RankTiersFromOos derives its zero-skill reference from these
|
|
|
|
|
//--- (50% x the directional base rate), so they must be counted over the SAME population the
|
|
|
|
|
//--- tiers were counted over, not over a wider one.
|
|
|
|
|
switch(truth)
|
|
|
|
|
{
|
|
|
|
|
case Buy:
|
|
|
|
|
m_oos.buyTotal++;
|
|
|
|
|
break;
|
|
|
|
|
case Sell:
|
|
|
|
|
m_oos.sellTotal++;
|
|
|
|
|
break;
|
|
|
|
|
default:
|
|
|
|
|
m_oos.neutralTotal++;
|
|
|
|
|
break;
|
|
|
|
|
}
|
|
|
|
|
//--- FIRED = a directional call under the live decision rule, which is exactly the population
|
|
|
|
|
//--- the tier weights are meant to describe. A Neutral is an abstention, neither right nor wrong.
|
|
|
|
|
if(pred != Buy && pred != Sell)
|
|
|
|
|
continue;
|
|
|
|
|
int tier = ConfidenceTierFor(sig);
|
|
|
|
|
if(tier < 0 || tier > 3)
|
|
|
|
|
continue;
|
|
|
|
|
m_oosTierFired[tier]++;
|
|
|
|
|
if(pred == truth)
|
|
|
|
|
m_oosTierHits[tier]++;
|
|
|
|
|
}
|
|
|
|
|
int fired = 0, hits = 0;
|
|
|
|
|
for(int t = 0; t < 4; t++)
|
|
|
|
|
{
|
|
|
|
|
fired += m_oosTierFired[t];
|
|
|
|
|
hits += m_oosTierHits[t];
|
|
|
|
|
}
|
fix(replay): resolve labels inline - the prebuilt cache's window never overlapped the rescan
The 15:13 session proved the replay pass ran end-to-end on all 24 models
and scored ZERO labelled bars on every one of them, while each rescan sat
on ~5000 scored predictions (~2755 Buy / ~2232 Sell). The two windows
never overlapped:
StartLabelCachePrebuild deliberately keeps a CONVERGED model's
dtStudied watermark (it gates inference recency and must not move), so
the prebuild's window was the handful of bars since the last studied
bar - all with uncommitted pivots, hence "label cache pre-built -
Buy: 0 | Sell: 0 | Neutral: 0" on every member.
The label never needed a cache. SwingPivotDirectionLabel(idx) is a pure
function of the ZigZag/Close/ATR buffers the rescan itself refreshes over
exactly the scoring window, and m_lastLabelLifespan == 0 is its own
unresolved flag - the same finality gate the cache applies, applied
directly. ScoreReplayFromCache now resolves each bar's label inline and
the label-prebuild stage is deleted from the rebuild state machine
outright; going through a cache built for a different window was
indirection that changed the answer.
Also splits the empty-result diagnostics: "no resolved labels" (a
windowing/data fault) is now distinguished from "labels present, every
call Neutral" (a calibration verdict). The first version reported the
second message for both, which mislabelled this very bug as a calibration
outcome in the same breath as reporting scored=0.
Honest limitation, stated in the code too: the replay window includes
bars the model trained on, so a replay-minted ladder is measured partly
in-sample and will read stronger than a holdout-measured one. It is
replaced by the genuine article at the next completed scoring pass; until
then it is what makes a restarted deployed model able to vote at all.
Compile-verified in _claude_stage: 0 errors, 0 warnings.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-25 15:22:01 -04:00
|
|
|
Print(ID + StringFormat(": replay scored %d labelled bar(s) against the deployed weights"
|
|
|
|
|
" (%d unresolved bars excluded) - %d directional call(s), %d correct"
|
|
|
|
|
" (%.1f%%). No training was run.",
|
|
|
|
|
scored, unresolved, fired, hits, (fired > 0 ? 100.0 * hits / fired : 0.0)));
|
|
|
|
|
//--- TWO DISTINCT EMPTY OUTCOMES, and naming the wrong one cost a session: "no labels overlapped"
|
|
|
|
|
//--- is a windowing/data fault to be fixed, while "labels present, every call Neutral" is a
|
|
|
|
|
//--- calibration verdict to be respected. The first version of this reported the second for both.
|
|
|
|
|
if(scored <= 0)
|
|
|
|
|
{
|
|
|
|
|
Print(ID + ": WARNING - the replay found NO RESOLVED LABELS in the rescan window, so nothing"
|
|
|
|
|
" could be scored. That is a data/windowing fault (ZigZag pivots missing over the whole"
|
|
|
|
|
" window?), not a verdict on the model. No ladder was ranked.");
|
|
|
|
|
return;
|
|
|
|
|
}
|
feat(vote): replay pass rebuilds a deployed model's ladder without retraining
The previous commit persisted the tier ladder, which fixes this going
forward but did nothing for models whose .stats predates WST7 - they
still had to retrain to mint one. They never did. Every number a
converged model needs in order to vote is a pure function of weights
already on disk plus labels derivable from the chart, so replay them:
stage 1 build the label cache (existing chunked prebuild)
stage 2 rescan history (existing chunked rescan, deployed net)
stage 3 score + rank + persist (one walk over two arrays)
ScoreReplayFromCache() walks m_arrowSignalCache against
m_labelCacheBuy/Sell, fills the same m_oosTierFired/Hits and per-class
totals pass 3 fills, and hands them to RankTiersFromOos() - deliberately
feeding the existing ranker rather than reimplementing it. The shrinkage,
the chance reference and the module trust weight are subtle enough that a
second copy would drift, and a ladder measured by a slightly different
rule would be silently incomparable with every ladder training produced.
AdvanceDeployedRebuild() sequences the three stages off the timer. It has
to be a sequence: stages 1 and 2 are each minutes of work draining in
time-boxed slices, and stage 2's output is meaningless until stage 1 has
labels to score against. The previous version ran the rescan with no
labels at all, which is why it could only ever rebuild arrows and never
the ladder - the thing actually blocking the vote.
The result is written to .stats immediately. The failure being repaired
is state that lived in memory and was never written down; recomputing it
and not saving it would repeat that exactly.
Also routes every rescan completion through one hook, so there is a
single place that knows what a finished rescan means - republish for a
manual one, score and rank for a rebuild.
Compile-verified in _claude_stage: 0 errors, 0 warnings.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-25 15:01:24 -04:00
|
|
|
if(fired <= 0)
|
|
|
|
|
{
|
fix(replay): resolve labels inline - the prebuilt cache's window never overlapped the rescan
The 15:13 session proved the replay pass ran end-to-end on all 24 models
and scored ZERO labelled bars on every one of them, while each rescan sat
on ~5000 scored predictions (~2755 Buy / ~2232 Sell). The two windows
never overlapped:
StartLabelCachePrebuild deliberately keeps a CONVERGED model's
dtStudied watermark (it gates inference recency and must not move), so
the prebuild's window was the handful of bars since the last studied
bar - all with uncommitted pivots, hence "label cache pre-built -
Buy: 0 | Sell: 0 | Neutral: 0" on every member.
The label never needed a cache. SwingPivotDirectionLabel(idx) is a pure
function of the ZigZag/Close/ATR buffers the rescan itself refreshes over
exactly the scoring window, and m_lastLabelLifespan == 0 is its own
unresolved flag - the same finality gate the cache applies, applied
directly. ScoreReplayFromCache now resolves each bar's label inline and
the label-prebuild stage is deleted from the rebuild state machine
outright; going through a cache built for a different window was
indirection that changed the answer.
Also splits the empty-result diagnostics: "no resolved labels" (a
windowing/data fault) is now distinguished from "labels present, every
call Neutral" (a calibration verdict). The first version reported the
second message for both, which mislabelled this very bug as a calibration
outcome in the same breath as reporting scored=0.
Honest limitation, stated in the code too: the replay window includes
bars the model trained on, so a replay-minted ladder is measured partly
in-sample and will read stronger than a holdout-measured one. It is
replaced by the genuine article at the next completed scoring pass; until
then it is what makes a restarted deployed model able to vote at all.
Compile-verified in _claude_stage: 0 errors, 0 warnings.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-25 15:22:01 -04:00
|
|
|
Print(ID + ": WARNING - the replay produced NO directional calls on " + IntegerToString(scored) +
|
|
|
|
|
" labelled bar(s), so no ladder can be ranked. This model calls Neutral everywhere; that"
|
|
|
|
|
" is a calibration outcome, not a drawing or persistence fault, and it will stay silent"
|
|
|
|
|
" until retrained.");
|
feat(vote): replay pass rebuilds a deployed model's ladder without retraining
The previous commit persisted the tier ladder, which fixes this going
forward but did nothing for models whose .stats predates WST7 - they
still had to retrain to mint one. They never did. Every number a
converged model needs in order to vote is a pure function of weights
already on disk plus labels derivable from the chart, so replay them:
stage 1 build the label cache (existing chunked prebuild)
stage 2 rescan history (existing chunked rescan, deployed net)
stage 3 score + rank + persist (one walk over two arrays)
ScoreReplayFromCache() walks m_arrowSignalCache against
m_labelCacheBuy/Sell, fills the same m_oosTierFired/Hits and per-class
totals pass 3 fills, and hands them to RankTiersFromOos() - deliberately
feeding the existing ranker rather than reimplementing it. The shrinkage,
the chance reference and the module trust weight are subtle enough that a
second copy would drift, and a ladder measured by a slightly different
rule would be silently incomparable with every ladder training produced.
AdvanceDeployedRebuild() sequences the three stages off the timer. It has
to be a sequence: stages 1 and 2 are each minutes of work draining in
time-boxed slices, and stage 2's output is meaningless until stage 1 has
labels to score against. The previous version ran the rescan with no
labels at all, which is why it could only ever rebuild arrows and never
the ladder - the thing actually blocking the vote.
The result is written to .stats immediately. The failure being repaired
is state that lived in memory and was never written down; recomputing it
and not saving it would repeat that exactly.
Also routes every rescan completion through one hook, so there is a
single place that knows what a finished rescan means - republish for a
manual one, score and rank for a rebuild.
Compile-verified in _claude_stage: 0 errors, 0 warnings.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-25 15:01:24 -04:00
|
|
|
return;
|
|
|
|
|
}
|
|
|
|
|
//--- ...and rank exactly as an era end would, including publishing the overlay snapshot and
|
|
|
|
|
//--- arming the sweep, which is the whole reason this returns nothing.
|
|
|
|
|
RankTiersFromOos();
|
|
|
|
|
}
|
|
|
|
|
//+------------------------------------------------------------------+
|
fix(chart): a deployed model rescans history to rebuild its vote arrows
The sidecar added in 484a9d8 restores the vote arrows from the previous
session - but there was no previous session to restore from, and a
deployed ensemble could never produce one.
The overlay that draws the vote layer replays each member's
m_overlaySigSnap, published in exactly one place: RankTiersFromOos, at
pass-3 completion. A converged model runs no further eras. So after a
restart every member's snapshot was empty, would never fill, the sweep
had nothing to replay and the chart stayed blank permanently - no route
back by any path.
The chart rescan is the route: it runs the DEPLOYED net forward over
history and rebuilds the per-bar cache, which is the same quantity pass 3
produces, obtained without training. It already existed for the panel's
Show-Signals button; it just never handed its result to the overlay, so
on the default filtered view a rescan rebuilt only the RAW per-member
layer - the one that is hidden - and appeared to do nothing.
- PublishOverlaySnapshotFromCache() extracted from RankTiersFromOos, so
the era end and a completed rescan publish through one implementation.
- A completed rescan now calls it, which also arms the sweep.
- PollTraining auto-arms one rescan for a model that is converged, has no
snapshot, and is on the filtered view. One-shot: a model that
legitimately calls Neutral everywhere must not rescan forever chasing a
snapshot that is correctly empty. On the timer, not in OnInit - it is a
full feedForward per bar over up to 5000 bars and drains in the same
time-boxed slices as a manual rescan.
Together with the sidecar this closes both halves: the rescan covers the
first session and any chart whose file was lost or invalidated by a
threshold change; the sidecar covers every session after one is saved.
Compile-verified in _claude_stage: 0 errors, 0 warnings.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-25 13:11:36 -04:00
|
|
|
//| TURN THIS ERA'S HELD-OUT OUTCOMES INTO THE VOTE WEIGHTS. |
|
|
|
|
|
//+------------------------------------------------------------------+
|
|
|
|
|
void CExpertSignalAIBase::RankTiersFromOos(void)
|
|
|
|
|
{
|
|
|
|
|
//--- SNAPSHOT FIRST, unconditionally - this runs at pass-3 completion, the single moment the
|
|
|
|
|
//--- arrow cache is complete for the era.
|
|
|
|
|
PublishOverlaySnapshotFromCache();
|
feat(rank): AI models rank their own confidence tiers from held-out outcomes
Closes the caveat 4858507 shipped with: the vote is a confidence percentage,
but only to the extent the pattern weights are measured. AI tier weights sat
at their designed defaults (25/50/75/100) because AI rows only ever arrive
from LIVE journaling, of which a training run produces almost none.
AND A STALE-TIER BUG THAT MADE THE EVIDENCE MEANINGLESS. The OOS scan bucketed
every scanned bar by ConfidenceTier(), which reads dPrevSignal - and
dPrevSignal is assigned in PASS 1 only, never anywhere in the OOS scan. So an
entire era's fires were bucketed by one stale, unrelated bar's confidence and
landed in a SINGLE tier. That is the "tier prec T0:72%(828) T1:n/a(0)
T2:n/a(0) T3:n/a(0)" symptom recorded on 2026-08-16 and attributed to the
calibration clamp. The clamp was real and was fixed then; this is a second,
independent cause of the identical output that survived that fix untouched -
which is why the log kept reading the same afterwards. Two causes, one symptom.
Now ConfidenceTierFor(adjSig): the bar this iteration actually scored.
WHY THIS DOES NOT WRITE ROWS TO THE SIGNAL DB, which was the obvious reading of
"fill the database during training". The user's own observation is the reason:
a classic Pattern_2 is a fixed geometric condition, so its win rate is
legitimately accumulated over years, but an AI Pattern_2 means "confidence
landed in tier 2" and tier 2 under era 100's weights is a different statement
from tier 2 under era 500's. The DB's value is ACCUMULATION, and accumulation
is exactly what is wrong here - it would average together models that no
longer exist, while colliding with the per-table row cap and mixing
measured-on-holdout outcomes into the live ledger's own tables. What the DB
actually supplies is a measured win rate per pattern, and pass 3 already
computes that on held-out bars, thousands at a time. So the model ranks itself
once per era, REPLACING rather than accumulating, which makes the weights
describe the current weights by construction.
ESTIMATOR. Not WinRateFromCounts(): it returns NO_DATA below 100 raw trades
BEFORE shrinking, which here would fire on every tier every era and hand all
four the pooled rate - the tiers could never separate and the mechanism would
be inert. Shrinkage is the answer to a small sample; a floor in front of it
means the shrinkage never runs. Instead: a Beta prior of TIER_PRIOR_EFF_N
pseudo-observations centred on the model's pooled holdout rate, counted in
EFFECTIVE observations, because overlapping triple-barrier labels mean 800 raw
fires can be worth ~12 independent ones. Rounded to the integer, not to the
decade NormalizeWinRate() uses, which would collapse the shrunk tiers back
into one number.
NO SAME-ERA CIRCULARITY, and it falls out of the ordering rather than a guard:
weights are computed at the END of era N, so the vote scored during era N was
cast with era N-1's weights. The deploy gate never grades a vote whose weights
were fitted on the bars it is scoring. Residual leakage remains - the same OOS
bars each era under a different model - and is stated in the code rather than
papered over.
Both DB clobber paths are closed: ApplyPatternWeight() declines once
self-ranked, and UpdateSignalsWeights()' filter.Weight() call is guarded by
SelfRanked() - guarding only the tiers would have let the hourly ranking pass
undo half the self-ranking.
NOT COMPILED - user compiles in MetaEditor.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-18 16:00:32 -04:00
|
|
|
int pooledFired = 0, pooledHits = 0;
|
|
|
|
|
for(int t = 0; t < 4; t++)
|
|
|
|
|
{
|
|
|
|
|
pooledFired += m_oosTierFired[t];
|
|
|
|
|
pooledHits += m_oosTierHits[t];
|
|
|
|
|
}
|
|
|
|
|
//--- Nothing fired this era (all-Neutral, or a stopped/cap-hit era): leave the previous era's
|
|
|
|
|
//--- weights standing rather than collapsing every tier to a prior built on no evidence at all.
|
|
|
|
|
if(pooledFired <= 0)
|
|
|
|
|
return;
|
|
|
|
|
double pooledPct = 100.0 * pooledHits / pooledFired;
|
fix(geometry): a free zero made "never resolve" the winning geometry
The CANDIDATE GEOMETRY line shipped in 05f1a53 said per-candidate
geometry beats the global pair on every SP500 member at 2-3 sigma. It
does not. It said so because a bar that reached neither barrier scored
0 R, and the incumbent's mean is NEGATIVE (-0.07 to -0.21 R). Against a
losing baseline a free zero is a win, so the widest candidate always
came out ahead - and the reported gain ordered itself by timeout share,
not by skill:
PAI 95.1% timed out -> +0.189 R (head measured -2.42 sigma, HARMFUL)
HYB 73.8% -> +0.182 R (head at chance, +0.68 sigma)
CONV 61.8% -> +0.163 R (head measured -2.47 sigma, HARMFUL)
LSTM 27.1% -> +0.158 R (head +1.67 sigma)
Monotone in the timeout share and inverted against the sigma gate. The
acceptance test written when this was built - "the sigma gate predicts
LSTM helps and CONV hurts; if the R difference does not reproduce that
ordering, something is wrong" - is what caught it.
A trade that reaches neither barrier is not worth zero. It is closed at
the horizon, which is what the scheduled close-all does live and what
SimulateTradeOutcome's timeout path already charges. So mark it there:
TripleBarrierLabel now publishes the signed close-to-close travel at the
last bar it actually visited (m_termTravelCache, same validity flag as
the excursion and ladder caches), and LadderOutcomeR prices a timeout
off it instead of returning false. A bar that cannot be evaluated under
BOTH pairs is now dropped whole - scoring one leg and defaulting the
other is the same bug in a smaller costume.
Second defect, same function: CandidateGeometryFor applied neither of
the floors the global derivation applies, so on USDJPY it chose stop
2.00 / target 1.00 - a 67% break-even, forbidden by the 1:2 policy
floor. c3daded in miniature: a selector optimising its own criterion
with no reference to the decision criterion. Both floors now apply, and
the ratio is re-checked AFTER the per-leg rung snap, which can lose it.
Also: the module weight was an unshrunk pooled win rate. USDJPY ConvLSTM
fired 19 times (2.0 effective), won 36.8%, and took module weight 0.37 -
41% of the ensemble's capable weight and the loudest voice on the chart,
off two effective observations. It also lifted the computed vote ceiling
to 26.3 against a 25 threshold, which is why THRESHOLD UNREACHABLE never
printed on a chart whose peak vote is 14 and whose practical ceiling
without that member is 18.8. The pooled rate is now shrunk toward the
coin-flip rate on the era's own OOS bars over 30 prior-equivalent calls,
and the tiers shrink toward the shrunk value rather than the raw one. A
member with ~300 effective calls moves by ~0.4pp; the 19-fire member
goes 0.37 -> ~0.15.
MEASUREMENT ONLY still - no order reads any of this.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-22 11:30:54 -04:00
|
|
|
//--- WHAT A MEMBER WITH NO EVIDENCE IS WORTH: the coin-flip rate on this era's OOS bars, which is
|
|
|
|
|
//--- what the gate calls zero skill. Shrinking toward it means "few fires -> speaks at chance",
|
|
|
|
|
//--- where shrinking toward the member's own pooled rate would mean "few fires -> speaks at
|
|
|
|
|
//--- whatever those few fires said", which is no shrinkage at all.
|
refactor(oos): twenty-one counters with one lifetime become one object
SOosTally holds this era's OOS confusion counts and the rates they imply.
The signal keeps one member where it kept twenty-one, and the era-reset
block loses twenty of its twenty-one clearing lines.
THE SHAPE THIS ENDS is the one that produced 7452bd1: a group of tallies
read together but cleared one-per-line, so a second reset path could clear
a subset and leave stale numerators over restarted denominators. Reset()
is now the only way to clear them and it clears all of them.
The pair had already started to drift. m_oosBuyFired/m_oosBuyFiredHits sat
at line 1085 and their Sell twins at line 1140 - 55 lines and an unrelated
member apart, with the Buy comment still claiming to describe both.
DERIVED RATES MOVE WITH THE DATA. `(bars > 0) ? (int)MathRound(100.0 * x /
bars) : -1` was written out twelve times, and the "-1 means not measurable,
never 0" convention re-spelled at each - a convention the deploy gate
depends on, since every caller tests `< 0` to mean "this does not block".
One rounding rule and one sentinel now.
GROUPED BY LIFETIME, NOT BY NAME. m_oosSamples looks like it belongs here
and does not: it is RUN-level, reset only with the weights, and the status
panel prints it beside dOosError which is also a run-level EMA. That pairing
is correct and stays. But the confidence-calibration block divided per-era
numerators by it, naming the results `empiricalAccuracy` and
`avgClaimedConfidence` when neither is that - the run-level denominator
cancels in their ratio, so eraScale was right and the two named
intermediates were not. Now written as the ratio it actually is, with the
cancellation stated, so nobody logs or gates on a half that decays with era
count.
BEHAVIOUR UNCHANGED: every moved expression preserves its formula, its
denominator and its sentinel.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-23 14:15:22 -04:00
|
|
|
int zsBars = m_oos.Bars();
|
fix(geometry): a free zero made "never resolve" the winning geometry
The CANDIDATE GEOMETRY line shipped in 05f1a53 said per-candidate
geometry beats the global pair on every SP500 member at 2-3 sigma. It
does not. It said so because a bar that reached neither barrier scored
0 R, and the incumbent's mean is NEGATIVE (-0.07 to -0.21 R). Against a
losing baseline a free zero is a win, so the widest candidate always
came out ahead - and the reported gain ordered itself by timeout share,
not by skill:
PAI 95.1% timed out -> +0.189 R (head measured -2.42 sigma, HARMFUL)
HYB 73.8% -> +0.182 R (head at chance, +0.68 sigma)
CONV 61.8% -> +0.163 R (head measured -2.47 sigma, HARMFUL)
LSTM 27.1% -> +0.158 R (head +1.67 sigma)
Monotone in the timeout share and inverted against the sigma gate. The
acceptance test written when this was built - "the sigma gate predicts
LSTM helps and CONV hurts; if the R difference does not reproduce that
ordering, something is wrong" - is what caught it.
A trade that reaches neither barrier is not worth zero. It is closed at
the horizon, which is what the scheduled close-all does live and what
SimulateTradeOutcome's timeout path already charges. So mark it there:
TripleBarrierLabel now publishes the signed close-to-close travel at the
last bar it actually visited (m_termTravelCache, same validity flag as
the excursion and ladder caches), and LadderOutcomeR prices a timeout
off it instead of returning false. A bar that cannot be evaluated under
BOTH pairs is now dropped whole - scoring one leg and defaulting the
other is the same bug in a smaller costume.
Second defect, same function: CandidateGeometryFor applied neither of
the floors the global derivation applies, so on USDJPY it chose stop
2.00 / target 1.00 - a 67% break-even, forbidden by the 1:2 policy
floor. c3daded in miniature: a selector optimising its own criterion
with no reference to the decision criterion. Both floors now apply, and
the ratio is re-checked AFTER the per-leg rung snap, which can lose it.
Also: the module weight was an unshrunk pooled win rate. USDJPY ConvLSTM
fired 19 times (2.0 effective), won 36.8%, and took module weight 0.37 -
41% of the ensemble's capable weight and the loudest voice on the chart,
off two effective observations. It also lifted the computed vote ceiling
to 26.3 against a 25 threshold, which is why THRESHOLD UNREACHABLE never
printed on a chart whose peak vote is 14 and whose practical ceiling
without that member is 18.8. The pooled rate is now shrunk toward the
coin-flip rate on the era's own OOS bars over 30 prior-equivalent calls,
and the tiers shrink toward the shrunk value rather than the raw one. A
member with ~300 effective calls moves by ~0.4pp; the 19-fire member
goes 0.37 -> ~0.15.
MEASUREMENT ONLY still - no order reads any of this.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-22 11:30:54 -04:00
|
|
|
double chancePct = (zsBars > 0)
|
feat(target): delete the barrier/geometry stack - the label is the verdict
Step 3 of the swing-pivot plan, whole-hog. The swing label is now the ONE
target and the era verdict is precision + recall per class against the
label's own base rate - no win rate, no break-even, no expectancy, no
geometry anywhere in training.
DELETED
- Expert/Excursion/ (4), Expert/BarrierHorizon/ (4), GeometrySweep,
FirstPassageLadder, Labeling/TripleBarrier.mqh (CLabelOverlap survives
in Labeling/LabelOverlap.mqh), 3 test EAs.
- TripleBarrierLabel + walk, fractal label, geometry derivation/scan/
adoption, exit-policy replay, excursion MI targets, the drift verdict
(DIRECTION_INTELLIGENT), the recall floor, balanced-accuracy telemetry,
the barrier defines, the .cfg geometry adopt (slots kept as zeros for
the positional layout), the derived-geometry live-order override.
- TRAINING_TARGET input/enum: direction models are always swing; META2
re-keys the meta head onto label agreement (descriptor loses its two
geometry slots).
REWORKED
- Labels.mqh (1795 -> ~370 lines): AdvanceSwingLabelState with
FINALITY-GATED CACHING - an unresolved bar (pivot pair uncommitted) is
never cached, so it can never freeze as a false Neutral; training,
calibration, OOS scoring and online learning all skip unresolved bars.
- SDeployVerdict: significance-only; SOosTally chance = larger
directional class share; pooled gate poolability = timeframe (record v2).
- Purge/embargo/declustering gaps: the measured mean label resolution
lag (LabelResolutionBars), not a barrier horizon.
- Pool purge key + backfill DB rows: marked at the bar the label
resolved on (m_labelResolveAge), not a fabricated barrier touch.
- Online learning frontier: finality, not a horizon delay.
- m_bestBalancedOos -> m_bestSelectionScore, m_erasSinceBestBalanced ->
m_erasSinceBest, ensemble vote outcome arrays -> label arrays.
STEP 4 folded in: Entry_Multiplier / SL_Mode / TP_Mode / tradingdirection
are inputs again - trade management is the tester GA's search space.
Fingerprints: every direction model re-keys (TGT:SWG1 now unconditional,
CUT token gone); META1 -> META2. Full retrain, as planned.
Compile-verified in _claude_stage: Warrior_EA + both surviving test EAs,
0 errors, 0 warnings each.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-24 20:42:31 -04:00
|
|
|
? 50.0 * ((double)m_oos.buyTotal + (double)m_oos.sellTotal) / zsBars
|
fix(geometry): a free zero made "never resolve" the winning geometry
The CANDIDATE GEOMETRY line shipped in 05f1a53 said per-candidate
geometry beats the global pair on every SP500 member at 2-3 sigma. It
does not. It said so because a bar that reached neither barrier scored
0 R, and the incumbent's mean is NEGATIVE (-0.07 to -0.21 R). Against a
losing baseline a free zero is a win, so the widest candidate always
came out ahead - and the reported gain ordered itself by timeout share,
not by skill:
PAI 95.1% timed out -> +0.189 R (head measured -2.42 sigma, HARMFUL)
HYB 73.8% -> +0.182 R (head at chance, +0.68 sigma)
CONV 61.8% -> +0.163 R (head measured -2.47 sigma, HARMFUL)
LSTM 27.1% -> +0.158 R (head +1.67 sigma)
Monotone in the timeout share and inverted against the sigma gate. The
acceptance test written when this was built - "the sigma gate predicts
LSTM helps and CONV hurts; if the R difference does not reproduce that
ordering, something is wrong" - is what caught it.
A trade that reaches neither barrier is not worth zero. It is closed at
the horizon, which is what the scheduled close-all does live and what
SimulateTradeOutcome's timeout path already charges. So mark it there:
TripleBarrierLabel now publishes the signed close-to-close travel at the
last bar it actually visited (m_termTravelCache, same validity flag as
the excursion and ladder caches), and LadderOutcomeR prices a timeout
off it instead of returning false. A bar that cannot be evaluated under
BOTH pairs is now dropped whole - scoring one leg and defaulting the
other is the same bug in a smaller costume.
Second defect, same function: CandidateGeometryFor applied neither of
the floors the global derivation applies, so on USDJPY it chose stop
2.00 / target 1.00 - a 67% break-even, forbidden by the 1:2 policy
floor. c3daded in miniature: a selector optimising its own criterion
with no reference to the decision criterion. Both floors now apply, and
the ratio is re-checked AFTER the per-leg rung snap, which can lose it.
Also: the module weight was an unshrunk pooled win rate. USDJPY ConvLSTM
fired 19 times (2.0 effective), won 36.8%, and took module weight 0.37 -
41% of the ensemble's capable weight and the loudest voice on the chart,
off two effective observations. It also lifted the computed vote ceiling
to 26.3 against a 25 threshold, which is why THRESHOLD UNREACHABLE never
printed on a chart whose peak vote is 14 and whose practical ceiling
without that member is 18.8. The pooled rate is now shrunk toward the
coin-flip rate on the era's own OOS bars over 30 prior-equivalent calls,
and the tiers shrink toward the shrunk value rather than the raw one. A
member with ~300 effective calls moves by ~0.4pp; the 19-fire member
goes 0.37 -> ~0.15.
MEASUREMENT ONLY still - no order reads any of this.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-22 11:30:54 -04:00
|
|
|
: pooledPct;
|
|
|
|
|
double pooledEffN = MathMax(0.0, EffectiveSampleSize((double)pooledFired));
|
|
|
|
|
double trustPct = ShrunkRatePct(pooledEffN * ((double)pooledHits / pooledFired), pooledEffN,
|
|
|
|
|
chancePct, MODULE_PRIOR_EFF_N);
|
feat(rank): AI models rank their own confidence tiers from held-out outcomes
Closes the caveat 4858507 shipped with: the vote is a confidence percentage,
but only to the extent the pattern weights are measured. AI tier weights sat
at their designed defaults (25/50/75/100) because AI rows only ever arrive
from LIVE journaling, of which a training run produces almost none.
AND A STALE-TIER BUG THAT MADE THE EVIDENCE MEANINGLESS. The OOS scan bucketed
every scanned bar by ConfidenceTier(), which reads dPrevSignal - and
dPrevSignal is assigned in PASS 1 only, never anywhere in the OOS scan. So an
entire era's fires were bucketed by one stale, unrelated bar's confidence and
landed in a SINGLE tier. That is the "tier prec T0:72%(828) T1:n/a(0)
T2:n/a(0) T3:n/a(0)" symptom recorded on 2026-08-16 and attributed to the
calibration clamp. The clamp was real and was fixed then; this is a second,
independent cause of the identical output that survived that fix untouched -
which is why the log kept reading the same afterwards. Two causes, one symptom.
Now ConfidenceTierFor(adjSig): the bar this iteration actually scored.
WHY THIS DOES NOT WRITE ROWS TO THE SIGNAL DB, which was the obvious reading of
"fill the database during training". The user's own observation is the reason:
a classic Pattern_2 is a fixed geometric condition, so its win rate is
legitimately accumulated over years, but an AI Pattern_2 means "confidence
landed in tier 2" and tier 2 under era 100's weights is a different statement
from tier 2 under era 500's. The DB's value is ACCUMULATION, and accumulation
is exactly what is wrong here - it would average together models that no
longer exist, while colliding with the per-table row cap and mixing
measured-on-holdout outcomes into the live ledger's own tables. What the DB
actually supplies is a measured win rate per pattern, and pass 3 already
computes that on held-out bars, thousands at a time. So the model ranks itself
once per era, REPLACING rather than accumulating, which makes the weights
describe the current weights by construction.
ESTIMATOR. Not WinRateFromCounts(): it returns NO_DATA below 100 raw trades
BEFORE shrinking, which here would fire on every tier every era and hand all
four the pooled rate - the tiers could never separate and the mechanism would
be inert. Shrinkage is the answer to a small sample; a floor in front of it
means the shrinkage never runs. Instead: a Beta prior of TIER_PRIOR_EFF_N
pseudo-observations centred on the model's pooled holdout rate, counted in
EFFECTIVE observations, because overlapping triple-barrier labels mean 800 raw
fires can be worth ~12 independent ones. Rounded to the integer, not to the
decade NormalizeWinRate() uses, which would collapse the shrunk tiers back
into one number.
NO SAME-ERA CIRCULARITY, and it falls out of the ordering rather than a guard:
weights are computed at the END of era N, so the vote scored during era N was
cast with era N-1's weights. The deploy gate never grades a vote whose weights
were fitted on the bars it is scoring. Residual leakage remains - the same OOS
bars each era under a different model - and is stated in the code rather than
papered over.
Both DB clobber paths are closed: ApplyPatternWeight() declines once
self-ranked, and UpdateSignalsWeights()' filter.Weight() call is guarded by
SelfRanked() - guarding only the tiers would have let the hourly ranking pass
undo half the self-ranking.
NOT COMPILED - user compiles in MetaEditor.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-18 16:00:32 -04:00
|
|
|
//--- WHY NOT WinRateFromCounts(), which is the estimator the classic ladders use: it returns
|
2026-08-22 00:25:52 -04:00
|
|
|
//--- NO_DATA_WIN_RATE for anything under MIN_TRADES_FOR_WIN_RATE raw trades, BEFORE it shrinks.
|
feat(rank): AI models rank their own confidence tiers from held-out outcomes
Closes the caveat 4858507 shipped with: the vote is a confidence percentage,
but only to the extent the pattern weights are measured. AI tier weights sat
at their designed defaults (25/50/75/100) because AI rows only ever arrive
from LIVE journaling, of which a training run produces almost none.
AND A STALE-TIER BUG THAT MADE THE EVIDENCE MEANINGLESS. The OOS scan bucketed
every scanned bar by ConfidenceTier(), which reads dPrevSignal - and
dPrevSignal is assigned in PASS 1 only, never anywhere in the OOS scan. So an
entire era's fires were bucketed by one stale, unrelated bar's confidence and
landed in a SINGLE tier. That is the "tier prec T0:72%(828) T1:n/a(0)
T2:n/a(0) T3:n/a(0)" symptom recorded on 2026-08-16 and attributed to the
calibration clamp. The clamp was real and was fixed then; this is a second,
independent cause of the identical output that survived that fix untouched -
which is why the log kept reading the same afterwards. Two causes, one symptom.
Now ConfidenceTierFor(adjSig): the bar this iteration actually scored.
WHY THIS DOES NOT WRITE ROWS TO THE SIGNAL DB, which was the obvious reading of
"fill the database during training". The user's own observation is the reason:
a classic Pattern_2 is a fixed geometric condition, so its win rate is
legitimately accumulated over years, but an AI Pattern_2 means "confidence
landed in tier 2" and tier 2 under era 100's weights is a different statement
from tier 2 under era 500's. The DB's value is ACCUMULATION, and accumulation
is exactly what is wrong here - it would average together models that no
longer exist, while colliding with the per-table row cap and mixing
measured-on-holdout outcomes into the live ledger's own tables. What the DB
actually supplies is a measured win rate per pattern, and pass 3 already
computes that on held-out bars, thousands at a time. So the model ranks itself
once per era, REPLACING rather than accumulating, which makes the weights
describe the current weights by construction.
ESTIMATOR. Not WinRateFromCounts(): it returns NO_DATA below 100 raw trades
BEFORE shrinking, which here would fire on every tier every era and hand all
four the pooled rate - the tiers could never separate and the mechanism would
be inert. Shrinkage is the answer to a small sample; a floor in front of it
means the shrinkage never runs. Instead: a Beta prior of TIER_PRIOR_EFF_N
pseudo-observations centred on the model's pooled holdout rate, counted in
EFFECTIVE observations, because overlapping triple-barrier labels mean 800 raw
fires can be worth ~12 independent ones. Rounded to the integer, not to the
decade NormalizeWinRate() uses, which would collapse the shrunk tiers back
into one number.
NO SAME-ERA CIRCULARITY, and it falls out of the ordering rather than a guard:
weights are computed at the END of era N, so the vote scored during era N was
cast with era N-1's weights. The deploy gate never grades a vote whose weights
were fitted on the bars it is scoring. Residual leakage remains - the same OOS
bars each era under a different model - and is stated in the code rather than
papered over.
Both DB clobber paths are closed: ApplyPatternWeight() declines once
self-ranked, and UpdateSignalsWeights()' filter.Weight() call is guarded by
SelfRanked() - guarding only the tiers would have let the hourly ranking pass
undo half the self-ranking.
NOT COMPILED - user compiles in MetaEditor.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-18 16:00:32 -04:00
|
|
|
string line = "";
|
|
|
|
|
for(int t = 0; t < 4; t++)
|
|
|
|
|
{
|
|
|
|
|
int fired = m_oosTierFired[t];
|
|
|
|
|
int hits = m_oosTierHits[t];
|
fix(geometry): a free zero made "never resolve" the winning geometry
The CANDIDATE GEOMETRY line shipped in 05f1a53 said per-candidate
geometry beats the global pair on every SP500 member at 2-3 sigma. It
does not. It said so because a bar that reached neither barrier scored
0 R, and the incumbent's mean is NEGATIVE (-0.07 to -0.21 R). Against a
losing baseline a free zero is a win, so the widest candidate always
came out ahead - and the reported gain ordered itself by timeout share,
not by skill:
PAI 95.1% timed out -> +0.189 R (head measured -2.42 sigma, HARMFUL)
HYB 73.8% -> +0.182 R (head at chance, +0.68 sigma)
CONV 61.8% -> +0.163 R (head measured -2.47 sigma, HARMFUL)
LSTM 27.1% -> +0.158 R (head +1.67 sigma)
Monotone in the timeout share and inverted against the sigma gate. The
acceptance test written when this was built - "the sigma gate predicts
LSTM helps and CONV hurts; if the R difference does not reproduce that
ordering, something is wrong" - is what caught it.
A trade that reaches neither barrier is not worth zero. It is closed at
the horizon, which is what the scheduled close-all does live and what
SimulateTradeOutcome's timeout path already charges. So mark it there:
TripleBarrierLabel now publishes the signed close-to-close travel at the
last bar it actually visited (m_termTravelCache, same validity flag as
the excursion and ladder caches), and LadderOutcomeR prices a timeout
off it instead of returning false. A bar that cannot be evaluated under
BOTH pairs is now dropped whole - scoring one leg and defaulting the
other is the same bug in a smaller costume.
Second defect, same function: CandidateGeometryFor applied neither of
the floors the global derivation applies, so on USDJPY it chose stop
2.00 / target 1.00 - a 67% break-even, forbidden by the 1:2 policy
floor. c3daded in miniature: a selector optimising its own criterion
with no reference to the decision criterion. Both floors now apply, and
the ratio is re-checked AFTER the per-leg rung snap, which can lose it.
Also: the module weight was an unshrunk pooled win rate. USDJPY ConvLSTM
fired 19 times (2.0 effective), won 36.8%, and took module weight 0.37 -
41% of the ensemble's capable weight and the loudest voice on the chart,
off two effective observations. It also lifted the computed vote ceiling
to 26.3 against a 25 threshold, which is why THRESHOLD UNREACHABLE never
printed on a chart whose peak vote is 14 and whose practical ceiling
without that member is 18.8. The pooled rate is now shrunk toward the
coin-flip rate on the era's own OOS bars over 30 prior-equivalent calls,
and the tiers shrink toward the shrunk value rather than the raw one. A
member with ~300 effective calls moves by ~0.4pp; the 19-fire member
goes 0.37 -> ~0.15.
MEASUREMENT ONLY still - no order reads any of this.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-22 11:30:54 -04:00
|
|
|
double w = trustPct;
|
feat(rank): AI models rank their own confidence tiers from held-out outcomes
Closes the caveat 4858507 shipped with: the vote is a confidence percentage,
but only to the extent the pattern weights are measured. AI tier weights sat
at their designed defaults (25/50/75/100) because AI rows only ever arrive
from LIVE journaling, of which a training run produces almost none.
AND A STALE-TIER BUG THAT MADE THE EVIDENCE MEANINGLESS. The OOS scan bucketed
every scanned bar by ConfidenceTier(), which reads dPrevSignal - and
dPrevSignal is assigned in PASS 1 only, never anywhere in the OOS scan. So an
entire era's fires were bucketed by one stale, unrelated bar's confidence and
landed in a SINGLE tier. That is the "tier prec T0:72%(828) T1:n/a(0)
T2:n/a(0) T3:n/a(0)" symptom recorded on 2026-08-16 and attributed to the
calibration clamp. The clamp was real and was fixed then; this is a second,
independent cause of the identical output that survived that fix untouched -
which is why the log kept reading the same afterwards. Two causes, one symptom.
Now ConfidenceTierFor(adjSig): the bar this iteration actually scored.
WHY THIS DOES NOT WRITE ROWS TO THE SIGNAL DB, which was the obvious reading of
"fill the database during training". The user's own observation is the reason:
a classic Pattern_2 is a fixed geometric condition, so its win rate is
legitimately accumulated over years, but an AI Pattern_2 means "confidence
landed in tier 2" and tier 2 under era 100's weights is a different statement
from tier 2 under era 500's. The DB's value is ACCUMULATION, and accumulation
is exactly what is wrong here - it would average together models that no
longer exist, while colliding with the per-table row cap and mixing
measured-on-holdout outcomes into the live ledger's own tables. What the DB
actually supplies is a measured win rate per pattern, and pass 3 already
computes that on held-out bars, thousands at a time. So the model ranks itself
once per era, REPLACING rather than accumulating, which makes the weights
describe the current weights by construction.
ESTIMATOR. Not WinRateFromCounts(): it returns NO_DATA below 100 raw trades
BEFORE shrinking, which here would fire on every tier every era and hand all
four the pooled rate - the tiers could never separate and the mechanism would
be inert. Shrinkage is the answer to a small sample; a floor in front of it
means the shrinkage never runs. Instead: a Beta prior of TIER_PRIOR_EFF_N
pseudo-observations centred on the model's pooled holdout rate, counted in
EFFECTIVE observations, because overlapping triple-barrier labels mean 800 raw
fires can be worth ~12 independent ones. Rounded to the integer, not to the
decade NormalizeWinRate() uses, which would collapse the shrunk tiers back
into one number.
NO SAME-ERA CIRCULARITY, and it falls out of the ordering rather than a guard:
weights are computed at the END of era N, so the vote scored during era N was
cast with era N-1's weights. The deploy gate never grades a vote whose weights
were fitted on the bars it is scoring. Residual leakage remains - the same OOS
bars each era under a different model - and is stated in the code rather than
papered over.
Both DB clobber paths are closed: ApplyPatternWeight() declines once
self-ranked, and UpdateSignalsWeights()' filter.Weight() call is guarded by
SelfRanked() - guarding only the tiers would have let the hourly ranking pass
undo half the self-ranking.
NOT COMPILED - user compiles in MetaEditor.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-18 16:00:32 -04:00
|
|
|
if(fired > 0)
|
|
|
|
|
{
|
|
|
|
|
double effN = MathMax(0.0, EffectiveSampleSize((double)fired));
|
|
|
|
|
double effHits = effN * ((double)hits / fired);
|
2026-08-22 00:25:52 -04:00
|
|
|
//--- Same estimator the classic ladders use (ShrunkRatePct), with the prior deliberately
|
|
|
|
|
//--- far smaller: it is counted in the same EFFECTIVE units as the evidence, and a tier
|
|
|
|
|
//--- holds ~8-15 of those, so the classic path's 100 would drown every tier in the pool.
|
fix(geometry): a free zero made "never resolve" the winning geometry
The CANDIDATE GEOMETRY line shipped in 05f1a53 said per-candidate
geometry beats the global pair on every SP500 member at 2-3 sigma. It
does not. It said so because a bar that reached neither barrier scored
0 R, and the incumbent's mean is NEGATIVE (-0.07 to -0.21 R). Against a
losing baseline a free zero is a win, so the widest candidate always
came out ahead - and the reported gain ordered itself by timeout share,
not by skill:
PAI 95.1% timed out -> +0.189 R (head measured -2.42 sigma, HARMFUL)
HYB 73.8% -> +0.182 R (head at chance, +0.68 sigma)
CONV 61.8% -> +0.163 R (head measured -2.47 sigma, HARMFUL)
LSTM 27.1% -> +0.158 R (head +1.67 sigma)
Monotone in the timeout share and inverted against the sigma gate. The
acceptance test written when this was built - "the sigma gate predicts
LSTM helps and CONV hurts; if the R difference does not reproduce that
ordering, something is wrong" - is what caught it.
A trade that reaches neither barrier is not worth zero. It is closed at
the horizon, which is what the scheduled close-all does live and what
SimulateTradeOutcome's timeout path already charges. So mark it there:
TripleBarrierLabel now publishes the signed close-to-close travel at the
last bar it actually visited (m_termTravelCache, same validity flag as
the excursion and ladder caches), and LadderOutcomeR prices a timeout
off it instead of returning false. A bar that cannot be evaluated under
BOTH pairs is now dropped whole - scoring one leg and defaulting the
other is the same bug in a smaller costume.
Second defect, same function: CandidateGeometryFor applied neither of
the floors the global derivation applies, so on USDJPY it chose stop
2.00 / target 1.00 - a 67% break-even, forbidden by the 1:2 policy
floor. c3daded in miniature: a selector optimising its own criterion
with no reference to the decision criterion. Both floors now apply, and
the ratio is re-checked AFTER the per-leg rung snap, which can lose it.
Also: the module weight was an unshrunk pooled win rate. USDJPY ConvLSTM
fired 19 times (2.0 effective), won 36.8%, and took module weight 0.37 -
41% of the ensemble's capable weight and the loudest voice on the chart,
off two effective observations. It also lifted the computed vote ceiling
to 26.3 against a 25 threshold, which is why THRESHOLD UNREACHABLE never
printed on a chart whose peak vote is 14 and whose practical ceiling
without that member is 18.8. The pooled rate is now shrunk toward the
coin-flip rate on the era's own OOS bars over 30 prior-equivalent calls,
and the tiers shrink toward the shrunk value rather than the raw one. A
member with ~300 effective calls moves by ~0.4pp; the 19-fire member
goes 0.37 -> ~0.15.
MEASUREMENT ONLY still - no order reads any of this.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-22 11:30:54 -04:00
|
|
|
//--- Toward the SHRUNK pooled rate, not the raw one: a member whose pooled evidence is
|
|
|
|
|
//--- thin must not hand its tiers a confident prior it does not have itself.
|
|
|
|
|
w = ShrunkRatePct(effHits, effN, trustPct, TIER_PRIOR_EFF_N);
|
feat(rank): AI models rank their own confidence tiers from held-out outcomes
Closes the caveat 4858507 shipped with: the vote is a confidence percentage,
but only to the extent the pattern weights are measured. AI tier weights sat
at their designed defaults (25/50/75/100) because AI rows only ever arrive
from LIVE journaling, of which a training run produces almost none.
AND A STALE-TIER BUG THAT MADE THE EVIDENCE MEANINGLESS. The OOS scan bucketed
every scanned bar by ConfidenceTier(), which reads dPrevSignal - and
dPrevSignal is assigned in PASS 1 only, never anywhere in the OOS scan. So an
entire era's fires were bucketed by one stale, unrelated bar's confidence and
landed in a SINGLE tier. That is the "tier prec T0:72%(828) T1:n/a(0)
T2:n/a(0) T3:n/a(0)" symptom recorded on 2026-08-16 and attributed to the
calibration clamp. The clamp was real and was fixed then; this is a second,
independent cause of the identical output that survived that fix untouched -
which is why the log kept reading the same afterwards. Two causes, one symptom.
Now ConfidenceTierFor(adjSig): the bar this iteration actually scored.
WHY THIS DOES NOT WRITE ROWS TO THE SIGNAL DB, which was the obvious reading of
"fill the database during training". The user's own observation is the reason:
a classic Pattern_2 is a fixed geometric condition, so its win rate is
legitimately accumulated over years, but an AI Pattern_2 means "confidence
landed in tier 2" and tier 2 under era 100's weights is a different statement
from tier 2 under era 500's. The DB's value is ACCUMULATION, and accumulation
is exactly what is wrong here - it would average together models that no
longer exist, while colliding with the per-table row cap and mixing
measured-on-holdout outcomes into the live ledger's own tables. What the DB
actually supplies is a measured win rate per pattern, and pass 3 already
computes that on held-out bars, thousands at a time. So the model ranks itself
once per era, REPLACING rather than accumulating, which makes the weights
describe the current weights by construction.
ESTIMATOR. Not WinRateFromCounts(): it returns NO_DATA below 100 raw trades
BEFORE shrinking, which here would fire on every tier every era and hand all
four the pooled rate - the tiers could never separate and the mechanism would
be inert. Shrinkage is the answer to a small sample; a floor in front of it
means the shrinkage never runs. Instead: a Beta prior of TIER_PRIOR_EFF_N
pseudo-observations centred on the model's pooled holdout rate, counted in
EFFECTIVE observations, because overlapping triple-barrier labels mean 800 raw
fires can be worth ~12 independent ones. Rounded to the integer, not to the
decade NormalizeWinRate() uses, which would collapse the shrunk tiers back
into one number.
NO SAME-ERA CIRCULARITY, and it falls out of the ordering rather than a guard:
weights are computed at the END of era N, so the vote scored during era N was
cast with era N-1's weights. The deploy gate never grades a vote whose weights
were fitted on the bars it is scoring. Residual leakage remains - the same OOS
bars each era under a different model - and is stated in the code rather than
papered over.
Both DB clobber paths are closed: ApplyPatternWeight() declines once
self-ranked, and UpdateSignalsWeights()' filter.Weight() call is guarded by
SelfRanked() - guarding only the tiers would have let the hourly ranking pass
undo half the self-ranking.
NOT COMPILED - user compiles in MetaEditor.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-18 16:00:32 -04:00
|
|
|
}
|
|
|
|
|
//--- Rounded to the nearest INTEGER, not to the nearest 10 as NormalizeWinRate() does. After
|
|
|
|
|
//--- shrinkage the tiers legitimately sit within a few points of each other, and decade rounding
|
|
|
|
|
//--- would collapse them back into one number - undoing the separation this exists to produce.
|
|
|
|
|
int wi = (int)MathMax(0, MathMin(100, MathRound(w)));
|
|
|
|
|
ApplyTierWeight(t, wi);
|
|
|
|
|
line += StringFormat(" T%d=%d(%d fires, %.1f eff)", t, wi, fired,
|
|
|
|
|
(fired > 0 ? EffectiveSampleSize((double)fired) : 0.0));
|
|
|
|
|
}
|
|
|
|
|
//--- MODULE WEIGHT = how much this model's opinion COUNTS in the weighted mean, which since the
|
|
|
|
|
//--- 2026-08-18 currency change is a trust weight and no longer a discount on the estimate. The
|
|
|
|
|
//--- pooled holdout win rate is the honest measure of that trust.
|
fix(geometry): a free zero made "never resolve" the winning geometry
The CANDIDATE GEOMETRY line shipped in 05f1a53 said per-candidate
geometry beats the global pair on every SP500 member at 2-3 sigma. It
does not. It said so because a bar that reached neither barrier scored
0 R, and the incumbent's mean is NEGATIVE (-0.07 to -0.21 R). Against a
losing baseline a free zero is a win, so the widest candidate always
came out ahead - and the reported gain ordered itself by timeout share,
not by skill:
PAI 95.1% timed out -> +0.189 R (head measured -2.42 sigma, HARMFUL)
HYB 73.8% -> +0.182 R (head at chance, +0.68 sigma)
CONV 61.8% -> +0.163 R (head measured -2.47 sigma, HARMFUL)
LSTM 27.1% -> +0.158 R (head +1.67 sigma)
Monotone in the timeout share and inverted against the sigma gate. The
acceptance test written when this was built - "the sigma gate predicts
LSTM helps and CONV hurts; if the R difference does not reproduce that
ordering, something is wrong" - is what caught it.
A trade that reaches neither barrier is not worth zero. It is closed at
the horizon, which is what the scheduled close-all does live and what
SimulateTradeOutcome's timeout path already charges. So mark it there:
TripleBarrierLabel now publishes the signed close-to-close travel at the
last bar it actually visited (m_termTravelCache, same validity flag as
the excursion and ladder caches), and LadderOutcomeR prices a timeout
off it instead of returning false. A bar that cannot be evaluated under
BOTH pairs is now dropped whole - scoring one leg and defaulting the
other is the same bug in a smaller costume.
Second defect, same function: CandidateGeometryFor applied neither of
the floors the global derivation applies, so on USDJPY it chose stop
2.00 / target 1.00 - a 67% break-even, forbidden by the 1:2 policy
floor. c3daded in miniature: a selector optimising its own criterion
with no reference to the decision criterion. Both floors now apply, and
the ratio is re-checked AFTER the per-leg rung snap, which can lose it.
Also: the module weight was an unshrunk pooled win rate. USDJPY ConvLSTM
fired 19 times (2.0 effective), won 36.8%, and took module weight 0.37 -
41% of the ensemble's capable weight and the loudest voice on the chart,
off two effective observations. It also lifted the computed vote ceiling
to 26.3 against a 25 threshold, which is why THRESHOLD UNREACHABLE never
printed on a chart whose peak vote is 14 and whose practical ceiling
without that member is 18.8. The pooled rate is now shrunk toward the
coin-flip rate on the era's own OOS bars over 30 prior-equivalent calls,
and the tiers shrink toward the shrunk value rather than the raw one. A
member with ~300 effective calls moves by ~0.4pp; the 19-fire member
goes 0.37 -> ~0.15.
MEASUREMENT ONLY still - no order reads any of this.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-22 11:30:54 -04:00
|
|
|
Weight(MathMax(0.0, MathMin(1.0, trustPct / 100.0)));
|
feat(rank): AI models rank their own confidence tiers from held-out outcomes
Closes the caveat 4858507 shipped with: the vote is a confidence percentage,
but only to the extent the pattern weights are measured. AI tier weights sat
at their designed defaults (25/50/75/100) because AI rows only ever arrive
from LIVE journaling, of which a training run produces almost none.
AND A STALE-TIER BUG THAT MADE THE EVIDENCE MEANINGLESS. The OOS scan bucketed
every scanned bar by ConfidenceTier(), which reads dPrevSignal - and
dPrevSignal is assigned in PASS 1 only, never anywhere in the OOS scan. So an
entire era's fires were bucketed by one stale, unrelated bar's confidence and
landed in a SINGLE tier. That is the "tier prec T0:72%(828) T1:n/a(0)
T2:n/a(0) T3:n/a(0)" symptom recorded on 2026-08-16 and attributed to the
calibration clamp. The clamp was real and was fixed then; this is a second,
independent cause of the identical output that survived that fix untouched -
which is why the log kept reading the same afterwards. Two causes, one symptom.
Now ConfidenceTierFor(adjSig): the bar this iteration actually scored.
WHY THIS DOES NOT WRITE ROWS TO THE SIGNAL DB, which was the obvious reading of
"fill the database during training". The user's own observation is the reason:
a classic Pattern_2 is a fixed geometric condition, so its win rate is
legitimately accumulated over years, but an AI Pattern_2 means "confidence
landed in tier 2" and tier 2 under era 100's weights is a different statement
from tier 2 under era 500's. The DB's value is ACCUMULATION, and accumulation
is exactly what is wrong here - it would average together models that no
longer exist, while colliding with the per-table row cap and mixing
measured-on-holdout outcomes into the live ledger's own tables. What the DB
actually supplies is a measured win rate per pattern, and pass 3 already
computes that on held-out bars, thousands at a time. So the model ranks itself
once per era, REPLACING rather than accumulating, which makes the weights
describe the current weights by construction.
ESTIMATOR. Not WinRateFromCounts(): it returns NO_DATA below 100 raw trades
BEFORE shrinking, which here would fire on every tier every era and hand all
four the pooled rate - the tiers could never separate and the mechanism would
be inert. Shrinkage is the answer to a small sample; a floor in front of it
means the shrinkage never runs. Instead: a Beta prior of TIER_PRIOR_EFF_N
pseudo-observations centred on the model's pooled holdout rate, counted in
EFFECTIVE observations, because overlapping triple-barrier labels mean 800 raw
fires can be worth ~12 independent ones. Rounded to the integer, not to the
decade NormalizeWinRate() uses, which would collapse the shrunk tiers back
into one number.
NO SAME-ERA CIRCULARITY, and it falls out of the ordering rather than a guard:
weights are computed at the END of era N, so the vote scored during era N was
cast with era N-1's weights. The deploy gate never grades a vote whose weights
were fitted on the bars it is scoring. Residual leakage remains - the same OOS
bars each era under a different model - and is stated in the code rather than
papered over.
Both DB clobber paths are closed: ApplyPatternWeight() declines once
self-ranked, and UpdateSignalsWeights()' filter.Weight() call is guarded by
SelfRanked() - guarding only the tiers would have let the hourly ranking pass
undo half the self-ranking.
NOT COMPILED - user compiles in MetaEditor.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-18 16:00:32 -04:00
|
|
|
m_tiersSelfRanked = true;
|
2026-08-26 16:53:16 -04:00
|
|
|
//--- ...AND WHETHER IT MAY VOTE AT ALL, from the same two numbers. The ladder says how much a
|
|
|
|
|
//--- member's opinion counts; this says whether it is admitted to the divisor (HasDemonstratedEdge
|
|
|
|
|
//--- -> VoteCapableWeight/ReconstructionWeight). Recorded here rather than at the era end because
|
|
|
|
|
//--- the DEPLOYED REPLAY path arrives here too, and that path is the only measurement a converged
|
|
|
|
|
//--- model will ever make.
|
|
|
|
|
m_certifiedPrecPct = pooledPct;
|
|
|
|
|
m_certifiedChancePct = chancePct;
|
feat(logs): throttle the settled per-era diagnostics - measured 22MB/9.5h of confirmed-working systems
Measured from the journal (2026-08-19): the era deep-dive line (~2KB) plus
the excursion verdict, tier re-rank, calibration move, barrier hold and
selection-regressed note each printed EVERY era for EVERY member - ~940
eras/member/day - long after the systems they watch were confirmed
working. Yesterday's file was 1.3GB (70% of it the news-filter calendar
spam the sweep fix already removed).
VerboseMode returns as an INPUT (demoted 2026-08-01 for the marketplace;
that track is dead since the 2026-08-16 pivot) and gains a second job:
false throttles each settled per-era print to eras 0-3 plus every
TRAIN_LOG_EVERY_ERAS-th (25 ~= one deep-dive per ~15min per member);
true restores the per-era firehose, flippable live.
Never throttled: anything that marks a CHANGE - new bests, restores +
eta decays, plateau stage transitions, deploy approvals, warnings,
errors, the label-cache/adoption one-shots, and the combined-vote gate
line (the active system's primary telemetry, still every era).
Semantic fixes over blanket gating:
- barrier hold now ARMS silently and prints only when the hold outlasts
the 2-min report interval - a brief hold every era is the design, the
long hold is the watchdog case the line exists for;
- the ensemble deploy REFUSAL prints immediately when its reason
changes (that is a finding), on cadence when unchanged;
- the filtered-view census prints when its RESULT moves (drawn count,
or strongest vote by >=2pp) and at least every 10th sweep.
NOT COMPILED - user compiles in MetaEditor.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-19 09:36:15 -04:00
|
|
|
//--- THROTTLED (2026-08-19): the re-rank happens (and must happen) every era, but saying so
|
|
|
|
|
//--- every era was ~950 near-identical lines/member/day once the system was confirmed working.
|
|
|
|
|
//--- The weights it prints are visible live on the member HUD lines anyway.
|
|
|
|
|
if(TrainLogDue())
|
fix(geometry): a free zero made "never resolve" the winning geometry
The CANDIDATE GEOMETRY line shipped in 05f1a53 said per-candidate
geometry beats the global pair on every SP500 member at 2-3 sigma. It
does not. It said so because a bar that reached neither barrier scored
0 R, and the incumbent's mean is NEGATIVE (-0.07 to -0.21 R). Against a
losing baseline a free zero is a win, so the widest candidate always
came out ahead - and the reported gain ordered itself by timeout share,
not by skill:
PAI 95.1% timed out -> +0.189 R (head measured -2.42 sigma, HARMFUL)
HYB 73.8% -> +0.182 R (head at chance, +0.68 sigma)
CONV 61.8% -> +0.163 R (head measured -2.47 sigma, HARMFUL)
LSTM 27.1% -> +0.158 R (head +1.67 sigma)
Monotone in the timeout share and inverted against the sigma gate. The
acceptance test written when this was built - "the sigma gate predicts
LSTM helps and CONV hurts; if the R difference does not reproduce that
ordering, something is wrong" - is what caught it.
A trade that reaches neither barrier is not worth zero. It is closed at
the horizon, which is what the scheduled close-all does live and what
SimulateTradeOutcome's timeout path already charges. So mark it there:
TripleBarrierLabel now publishes the signed close-to-close travel at the
last bar it actually visited (m_termTravelCache, same validity flag as
the excursion and ladder caches), and LadderOutcomeR prices a timeout
off it instead of returning false. A bar that cannot be evaluated under
BOTH pairs is now dropped whole - scoring one leg and defaulting the
other is the same bug in a smaller costume.
Second defect, same function: CandidateGeometryFor applied neither of
the floors the global derivation applies, so on USDJPY it chose stop
2.00 / target 1.00 - a 67% break-even, forbidden by the 1:2 policy
floor. c3daded in miniature: a selector optimising its own criterion
with no reference to the decision criterion. Both floors now apply, and
the ratio is re-checked AFTER the per-leg rung snap, which can lose it.
Also: the module weight was an unshrunk pooled win rate. USDJPY ConvLSTM
fired 19 times (2.0 effective), won 36.8%, and took module weight 0.37 -
41% of the ensemble's capable weight and the loudest voice on the chart,
off two effective observations. It also lifted the computed vote ceiling
to 26.3 against a 25 threshold, which is why THRESHOLD UNREACHABLE never
printed on a chart whose peak vote is 14 and whose practical ceiling
without that member is 18.8. The pooled rate is now shrunk toward the
coin-flip rate on the era's own OOS bars over 30 prior-equivalent calls,
and the tiers shrink toward the shrunk value rather than the raw one. A
member with ~300 effective calls moves by ~0.4pp; the 19-fire member
goes 0.37 -> ~0.15.
MEASUREMENT ONLY still - no order reads any of this.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-22 11:30:54 -04:00
|
|
|
Print(ID + StringFormat(": tier weights re-ranked from %d held-out fires (%.1f effective,"
|
|
|
|
|
" pooled %.1f%% raw -> %.1f%% shrunk toward the %.1f%% coin-flip rate on"
|
|
|
|
|
" %.0f prior-equivalent calls) ->%s | module weight %.2f. These are the"
|
|
|
|
|
" weights the NEXT era votes with.",
|
|
|
|
|
pooledFired, pooledEffN, pooledPct, trustPct, chancePct,
|
|
|
|
|
MODULE_PRIOR_EFF_N, line, ModuleWeight()));
|
feat(rank): AI models rank their own confidence tiers from held-out outcomes
Closes the caveat 4858507 shipped with: the vote is a confidence percentage,
but only to the extent the pattern weights are measured. AI tier weights sat
at their designed defaults (25/50/75/100) because AI rows only ever arrive
from LIVE journaling, of which a training run produces almost none.
AND A STALE-TIER BUG THAT MADE THE EVIDENCE MEANINGLESS. The OOS scan bucketed
every scanned bar by ConfidenceTier(), which reads dPrevSignal - and
dPrevSignal is assigned in PASS 1 only, never anywhere in the OOS scan. So an
entire era's fires were bucketed by one stale, unrelated bar's confidence and
landed in a SINGLE tier. That is the "tier prec T0:72%(828) T1:n/a(0)
T2:n/a(0) T3:n/a(0)" symptom recorded on 2026-08-16 and attributed to the
calibration clamp. The clamp was real and was fixed then; this is a second,
independent cause of the identical output that survived that fix untouched -
which is why the log kept reading the same afterwards. Two causes, one symptom.
Now ConfidenceTierFor(adjSig): the bar this iteration actually scored.
WHY THIS DOES NOT WRITE ROWS TO THE SIGNAL DB, which was the obvious reading of
"fill the database during training". The user's own observation is the reason:
a classic Pattern_2 is a fixed geometric condition, so its win rate is
legitimately accumulated over years, but an AI Pattern_2 means "confidence
landed in tier 2" and tier 2 under era 100's weights is a different statement
from tier 2 under era 500's. The DB's value is ACCUMULATION, and accumulation
is exactly what is wrong here - it would average together models that no
longer exist, while colliding with the per-table row cap and mixing
measured-on-holdout outcomes into the live ledger's own tables. What the DB
actually supplies is a measured win rate per pattern, and pass 3 already
computes that on held-out bars, thousands at a time. So the model ranks itself
once per era, REPLACING rather than accumulating, which makes the weights
describe the current weights by construction.
ESTIMATOR. Not WinRateFromCounts(): it returns NO_DATA below 100 raw trades
BEFORE shrinking, which here would fire on every tier every era and hand all
four the pooled rate - the tiers could never separate and the mechanism would
be inert. Shrinkage is the answer to a small sample; a floor in front of it
means the shrinkage never runs. Instead: a Beta prior of TIER_PRIOR_EFF_N
pseudo-observations centred on the model's pooled holdout rate, counted in
EFFECTIVE observations, because overlapping triple-barrier labels mean 800 raw
fires can be worth ~12 independent ones. Rounded to the integer, not to the
decade NormalizeWinRate() uses, which would collapse the shrunk tiers back
into one number.
NO SAME-ERA CIRCULARITY, and it falls out of the ordering rather than a guard:
weights are computed at the END of era N, so the vote scored during era N was
cast with era N-1's weights. The deploy gate never grades a vote whose weights
were fitted on the bars it is scoring. Residual leakage remains - the same OOS
bars each era under a different model - and is stated in the code rather than
papered over.
Both DB clobber paths are closed: ApplyPatternWeight() declines once
self-ranked, and UpdateSignalsWeights()' filter.Weight() call is guarded by
SelfRanked() - guarding only the tiers would have let the hourly ranking pass
undo half the self-ranking.
NOT COMPILED - user compiles in MetaEditor.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-18 16:00:32 -04:00
|
|
|
}
|
|
|
|
|
//+------------------------------------------------------------------+
|
2026-08-01 11:27:28 -04:00
|
|
|
//| Set the specified pattern's weight to the specified value |
|
|
|
|
|
//+------------------------------------------------------------------+
|
|
|
|
|
void CExpertSignalAIBase::ApplyPatternWeight(int patternNumber, int weight)
|
|
|
|
|
{
|
2026-08-22 00:25:52 -04:00
|
|
|
//--- THE SIGNAL DB DOES NOT OUTRANK THE HOLDOUT. UpdateSignalsWeights() calls this hourly for
|
|
|
|
|
//--- every filter it can find rows for, and for an AI filter those rows are LIVE-journaled fires
|
|
|
|
|
//--- accumulated across eras - i.e. across models. Without this the ranking pass would silently
|
|
|
|
|
//--- undo every era's self-ranking within the hour.
|
feat(rank): AI models rank their own confidence tiers from held-out outcomes
Closes the caveat 4858507 shipped with: the vote is a confidence percentage,
but only to the extent the pattern weights are measured. AI tier weights sat
at their designed defaults (25/50/75/100) because AI rows only ever arrive
from LIVE journaling, of which a training run produces almost none.
AND A STALE-TIER BUG THAT MADE THE EVIDENCE MEANINGLESS. The OOS scan bucketed
every scanned bar by ConfidenceTier(), which reads dPrevSignal - and
dPrevSignal is assigned in PASS 1 only, never anywhere in the OOS scan. So an
entire era's fires were bucketed by one stale, unrelated bar's confidence and
landed in a SINGLE tier. That is the "tier prec T0:72%(828) T1:n/a(0)
T2:n/a(0) T3:n/a(0)" symptom recorded on 2026-08-16 and attributed to the
calibration clamp. The clamp was real and was fixed then; this is a second,
independent cause of the identical output that survived that fix untouched -
which is why the log kept reading the same afterwards. Two causes, one symptom.
Now ConfidenceTierFor(adjSig): the bar this iteration actually scored.
WHY THIS DOES NOT WRITE ROWS TO THE SIGNAL DB, which was the obvious reading of
"fill the database during training". The user's own observation is the reason:
a classic Pattern_2 is a fixed geometric condition, so its win rate is
legitimately accumulated over years, but an AI Pattern_2 means "confidence
landed in tier 2" and tier 2 under era 100's weights is a different statement
from tier 2 under era 500's. The DB's value is ACCUMULATION, and accumulation
is exactly what is wrong here - it would average together models that no
longer exist, while colliding with the per-table row cap and mixing
measured-on-holdout outcomes into the live ledger's own tables. What the DB
actually supplies is a measured win rate per pattern, and pass 3 already
computes that on held-out bars, thousands at a time. So the model ranks itself
once per era, REPLACING rather than accumulating, which makes the weights
describe the current weights by construction.
ESTIMATOR. Not WinRateFromCounts(): it returns NO_DATA below 100 raw trades
BEFORE shrinking, which here would fire on every tier every era and hand all
four the pooled rate - the tiers could never separate and the mechanism would
be inert. Shrinkage is the answer to a small sample; a floor in front of it
means the shrinkage never runs. Instead: a Beta prior of TIER_PRIOR_EFF_N
pseudo-observations centred on the model's pooled holdout rate, counted in
EFFECTIVE observations, because overlapping triple-barrier labels mean 800 raw
fires can be worth ~12 independent ones. Rounded to the integer, not to the
decade NormalizeWinRate() uses, which would collapse the shrunk tiers back
into one number.
NO SAME-ERA CIRCULARITY, and it falls out of the ordering rather than a guard:
weights are computed at the END of era N, so the vote scored during era N was
cast with era N-1's weights. The deploy gate never grades a vote whose weights
were fitted on the bars it is scoring. Residual leakage remains - the same OOS
bars each era under a different model - and is stated in the code rather than
papered over.
Both DB clobber paths are closed: ApplyPatternWeight() declines once
self-ranked, and UpdateSignalsWeights()' filter.Weight() call is guarded by
SelfRanked() - guarding only the tiers would have let the hourly ranking pass
undo half the self-ranking.
NOT COMPILED - user compiles in MetaEditor.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-18 16:00:32 -04:00
|
|
|
if(m_tiersSelfRanked)
|
|
|
|
|
return;
|
2026-08-01 11:27:28 -04:00
|
|
|
switch(patternNumber)
|
|
|
|
|
{
|
|
|
|
|
case 0:
|
|
|
|
|
Pattern_0(weight);
|
|
|
|
|
break;
|
|
|
|
|
case 1:
|
|
|
|
|
Pattern_1(weight);
|
|
|
|
|
break;
|
|
|
|
|
case 2:
|
|
|
|
|
Pattern_2(weight);
|
|
|
|
|
break;
|
|
|
|
|
case 3:
|
|
|
|
|
Pattern_3(weight);
|
|
|
|
|
break;
|
|
|
|
|
default:
|
|
|
|
|
break;
|
|
|
|
|
}
|
|
|
|
|
}
|
|
|
|
|
//+------------------------------------------------------------------+
|
|
|
|
|
//| OnTick function |
|
|
|
|
|
//+------------------------------------------------------------------+
|
|
|
|
|
void CExpertSignalAIBase::OnTickHandler(void)
|
|
|
|
|
{
|
|
|
|
|
ScheduleTrainingIfNeeded();
|
|
|
|
|
}
|
|
|
|
|
//+------------------------------------------------------------------+
|
|
|
|
|
//| Schedules the next training pass (if one is due) and refreshes |
|
|
|
|
|
//| the per-tick status label. Factored out of OnTickHandler() so |
|
|
|
|
|
//| Warrior_EA.mq5's always-on timer (see PollTraining()) can drive |
|
|
|
|
|
//| this on a fixed wall-clock schedule too - training must not stall |
|
|
|
|
|
//| just because the market is closed and no ticks are arriving. |
|
|
|
|
|
//+------------------------------------------------------------------+
|
|
|
|
|
void CExpertSignalAIBase::ScheduleTrainingIfNeeded(void)
|
|
|
|
|
{
|
|
|
|
|
//--- stopped: no new training passes get scheduled at all (StartTraining() re-arms this).
|
|
|
|
|
//--- paused: still schedule so bEventStudy/dtStudied bookkeeping stays current, but Train() itself
|
|
|
|
|
//--- blocks at the next era boundary until resumed - keeps in-memory state coherent either way.
|
|
|
|
|
//--- complete: training already converged - a plain new bar must NOT re-enter Train()'s full era
|
2026-08-20 07:00:09 -04:00
|
|
|
//--- loop, which would otherwise reset the best-checkpoint/g_eta-decay tracking and run real
|
2026-08-01 11:27:28 -04:00
|
|
|
//--- Net.backProp() passes again, forever, once per bar, on an already-converged model (see
|
|
|
|
|
//--- RefreshConvergedSignal()'s declaration comment). Just keep the live signal current instead.
|
refactor(trade-mgmt): remove all confidence-scaled trade management
Five modes went, all of them staking real risk on the model's confidence:
Intelligent entry (ENTRY_INTELLIGENT), stop (SL_INTELLIGENT), target
(TP_INTELLIGENT), trailing (CTrailingIntelligent) and lot size
(CMoneyIntelligent's quarter-Kelly). With them, the Confidence_Source
input and the CONFIDENCE_SOURCE enum, whose only job was choosing which
number those five read.
The reason is calibration, not correctness: the confidence magnitude is
known to be miscalibrated against the label prior, so every one of these
modes multiplied money by a quantity whose units were never established.
The DB arm had a second, independent defect - since the tester DB guard
(SignalDatabaseActive) it reads 0 in tester and optimizer but non-zero
live, so any backtest of CONF_DB/CONF_BLENDED could not reproduce live
trading. And what the DB produces is a filter-RANKING win rate, not a
per-trade win probability.
Both confidence numbers are still recorded per trade (aiConfidence /
dbConfidence) and still bucketed against outcome in TradeJournalReport.
Recording is what keeps the question answerable; acting on it was the
part with no evidence behind it. ConfidenceBridge.mqh now carries an
explicit telemetry-only rule at the top.
ENUM ORDINALS PINNED. Removing a member vacated a value in four enums at
once and MT5 does not validate an enum input replayed from a saved .set
or a stored optimization pass. TRAILING_STRATEGY and
MONEY_MANAGEMENT_STRATEGY now carry explicit values so the survivors keep
the numbers they were saved as, and ValidateBarrierInputs is widened into
ValidateTradeManagementInputs covering SL_Mode, TP_Mode,
Entry_Multiplier, TrailingStrategy and MM_STRATEGY. Without that gate a
chart saved with the Intelligent stop would feed SL_Mode = -1 into a
multiplier now used verbatim, placing the stop on the wrong side of entry.
RETRAIN-NEUTRAL: neither SL_Mode nor TP_Mode appears in
BuildModelFingerprint() or ComputeDbConfigFingerprint() since the
swing-pivot target replaced the barrier labels. No .nnw, .cfg or .db
re-keys. Also drops the now-dead g_TradeRewardRiskRatio bridge, the
CMoneyRiskBase::AdjustRiskAmount hook and the unsigned AIConfidence().
Compile-verified in _claude_stage: 0 errors, 0 warnings.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-25 10:10:20 -04:00
|
|
|
//--- publish this signal's current signed confidence - see g_LiveAISignedConfidence in
|
|
|
|
|
//--- Variables\ConfidenceBridge.mqh. Cheap: SignedAIConfidence() just reads the already-computed
|
|
|
|
|
//--- dPrevSignal.
|
|
|
|
|
//--- TELEMETRY SINCE 2026-08-25. The last consumer that could act on this - the confidence-adaptive
|
|
|
|
|
//--- trailing stop - was removed with the rest of the confidence-scaled trade management, so what
|
|
|
|
|
//--- this feeds now is the per-trade journal columns and the confidence-vs-outcome buckets in
|
|
|
|
|
//--- Database\TradeJournalReport.mqh. It is kept running rather than deleted precisely because
|
|
|
|
|
//--- those buckets are the only way the question "is this number worth anything?" ever gets an
|
|
|
|
|
//--- answer, and a trade cannot be scored against a conviction nobody recorded.
|
feat(gates): derive the recall floor instead of configuring it, publish what the configuration can PROVE, and stop an arbitrary member driving live exits
Three changes, all from the same principle: measure what is there before aiming
at it, and never certify a number you do not trade.
1. THE RECALL FLOOR IS DERIVED, AND IT MOVES BELOW CHANCE.
MinRecall=40 was a constant doing a statistical job. Its reference point is the
33.3% recall a zero-skill 3-class model gets on EVERY class, and against that the
constant was accidentally calibrated for exactly one sample size: on USDJPY CONV
(n_eff 195) 40% is chance + 2.0 SE; on SP500 PAI (n_eff 42) the same 40% is chance
+ 0.9 SE. One chart was being held to a bar twice as strict as the other, for no
reason anyone chose.
CollapseRecallFloorPct() computes it per class from that class's own effective
sample - EffectiveSampleSize(), so the overlap deflation the rest of the gates use
applies here too - as chance - EDGE_MIN_SIGMAS x SE. 26.5% at n_eff 195, 18.8% at
n_eff 42.
BELOW chance, deliberately, and this is the substantive change rather than the
arithmetic. This gate's only job is refusing to call a COLLAPSED model converged.
It is not a quality bar; the deploy gate is the quality bar and it is already
rigorous (chance + 2 SE on the deflated sample, Sidak over candidate eras, then the
cross-instrument pooled certificate). A convergence gate that ALSO demands
provably-above-chance recall on all three classes double-counts that job, and it
has failed that way twice here: MinRecall=60 blocked every SP500 H1 run in 2026-07,
and the 40 that replaced it made Neutral structurally unreachable once first-touch
resolution cut Neutral to a 0.65% residue. A floor nothing can reach does not make
a funded account safer, it stops the run converging at all.
Testing significantly BELOW chance instead catches what a fixed 40 was actually
catching - a model that has stopped emitting a class - and cannot become
unreachable by construction. It also fixes the direction the old constant scaled:
it now widens on a thin OOS window, where low recall genuinely cannot be told from
noise, and tightens on a rich one. Today's SP500 PAI (Buy 51 / Sell 18 / Neutral 30)
is still correctly blocked on Sell.
This also resolves a standing contradiction the code half-admitted at the
isBetterEra comment: selection ranks on coverage-weighted PRECISION while
convergence gated on RECALL, so a sparse high-precision abstainer - precisely the
model that could clear the deploy bar - was blocked by the floor.
The era line now PRINTS the derived floor. Anyone comparing these recalls against a
remembered "40" is reading the wrong bar.
2. DETECTABILITY: WHAT THIS CONFIGURATION COULD PROVE, BEFORE IT TRAINS.
The DEPLOY BAR line states the bar. It never said what reaching it would take, and
that is the actionable direction. ReportDetectability() inverts the same identity -
the gate passes when edge >= z x sqrt(p(1-p)/n_eff), so certifying an edge d needs
n_eff >= z^2 p(1-p)/d^2 independent calls, hence L times as many raw ones - and
prints a +2 / +5 / +10pp ladder as required independent calls, raw calls, and share
of the OOS window, marking any rung that needs more than the window holds IMPOSSIBLE.
Every term is a property of the CONFIGURATION - geometry via break-even, horizon via
mean label lifespan, window via oosCutoff - so no amount of training moves any of
them. It fires once, at the first healthy sweep, beside ReportFeatureHealth, for the
same reason: that is the first moment the bar grid, the measured geometry and the
lifespan are real numbers rather than defaults. It gates nothing.
This is the EdgeFinder discipline applied to our own gate: establish what the market
and the measurement design have to offer, then point the net at it - rather than
spending a thousand eras chasing something this OOS window could never certify.
3. AN ARBITRARY MEMBER WAS DRIVING LIVE EXITS AND TRAILING (user-identified).
Every ensemble member ran
g_LiveAISignedConfidence = SignedAIConfidence();
unconditionally, every tick. Last writer wins. Its consumers are the AI early-exit
route (CExpertSignalCustom::LiveSignedConfidence) and TrailingIntelligent - so on a
four-model chart an LSTM entry could be closed, and its stop moved, on the
Perceptron's opinion alone, decided by scheduling order. Not the vote, not a
weighted blend.
Now the mean across registered members, matching how the ensemble actually trades:
the open decision is the weighted-average vote, and an abstaining member contributes
0 and dilutes exactly as it does there. Members still training read 0, so a
half-trained ensemble reads WEAKER rather than louder - the safe direction for an
exit trigger. Deployed and paused members are included, which is the opposite of the
era barrier's exemption rule and correct for the opposite reason: that one asks who
must be waited for, this asks who has an opinion.
Latent today and staying that way for now by choice - Min_Vote_Close ships Disabled
(101, unreachable on both scales it drives) and TrailingStrategy is off, so live
exits are SL/TP only and the certified hold-to-barrier win rate is what actually
gets traded. Fixed now precisely because the plan is to enable vote exits once the
models are accurate, at which point a scheduling-order exit would be both harmful
and very hard to see.
STILL OPEN, and needs a decision before vote exits go on: the member gate and the
ensemble vote gate both grade hold-to-barrier, so enabling vote exits makes the
certified number stop describing the traded one. Warrior_EA.mq5 currently argues
barrier models may keep vote exits because "their label IS the vote's own horizon" -
that does not hold, since a vote flip at bar 5 of a 64-bar horizon is not the
target-before-stop outcome the gate measured. Either grade the OOS call on the real
exit rule (first of SL / TP / vote-flip / horizon) through the fill engine, or set
HoldToBarrier for ensemble members so the policy cannot drift from the certificate.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-17 16:15:08 -04:00
|
|
|
//--- ENSEMBLE: the AVERAGE across members, not this member's own reading. Every member ran this line
|
|
|
|
|
//--- unconditionally, every tick, so the global was simply whichever member's OnTick happened to run
|
refactor(kiss): drop the AI sub-vote early-exit route; certified == traded
First of the AI vote layers to go. CheckClosePosition had two exit routes:
the stock blended vote, and an AI-only one reading the AI members' sub-vote
undiluted. The second existed because an AI reversal averaged in with the
classic filters could be diluted below the threshold before it could close
a position.
It is gone, and with it m_lastAiVote and the aiResult/aiWeightSum pair
Direction() carried to feed it.
This CLOSES the certified-vs-traded gap rather than widening it. The
deploy gate certifies a win rate measured on hold-to-resolution outcomes,
and CheckClosePosition already gated the blended route off whenever an AI
model's derived geometry was on the order - so the AI route was the only
vote exit an AI-certified trade could take, and the exit replay existed to
reproduce it. With it removed, an AI-certified position holds to its
barrier by construction instead of by reconstruction, so Warrior_EA.mq5
now pushes ExitPolicy(0.0, true) unconditionally. Previously it forwarded
Min_Vote_Close and relied on Disabled arriving as 1.01 to switch the
simulated exit off by arithmetic - correct at the shipped default, and one
input change away from the simulation and the live path describing
different games.
Min_Vote_Close keeps its meaning for the classic route and is now
documented as inert wherever an AI certificate governs, rather than
appearing to drive an exit it can no longer reach.
Comment debt cleared while here: a tombstone block for m_ai_exit_threshold
(a member deleted 2026-08-18) still sat in the header, and four sites still
named LiveSignedConfidence's "two consumers" - it had one, the intelligent
trailing stop, since that same date.
NOT touched, and deliberately: NMS declustering is NOT a quality layer. It
gates the live signal at Inference.mqh:226 (NmsLiveAccept), and the
undeclustered population is ~8x what the EA trades. Removing it would
multiply live position count, not simplify a scoring path.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-19 20:27:28 -04:00
|
|
|
//--- last. So on a four-model chart an LSTM entry could have its stop moved on the Perceptron's
|
refactor(trade-mgmt): remove all confidence-scaled trade management
Five modes went, all of them staking real risk on the model's confidence:
Intelligent entry (ENTRY_INTELLIGENT), stop (SL_INTELLIGENT), target
(TP_INTELLIGENT), trailing (CTrailingIntelligent) and lot size
(CMoneyIntelligent's quarter-Kelly). With them, the Confidence_Source
input and the CONFIDENCE_SOURCE enum, whose only job was choosing which
number those five read.
The reason is calibration, not correctness: the confidence magnitude is
known to be miscalibrated against the label prior, so every one of these
modes multiplied money by a quantity whose units were never established.
The DB arm had a second, independent defect - since the tester DB guard
(SignalDatabaseActive) it reads 0 in tester and optimizer but non-zero
live, so any backtest of CONF_DB/CONF_BLENDED could not reproduce live
trading. And what the DB produces is a filter-RANKING win rate, not a
per-trade win probability.
Both confidence numbers are still recorded per trade (aiConfidence /
dbConfidence) and still bucketed against outcome in TradeJournalReport.
Recording is what keeps the question answerable; acting on it was the
part with no evidence behind it. ConfidenceBridge.mqh now carries an
explicit telemetry-only rule at the top.
ENUM ORDINALS PINNED. Removing a member vacated a value in four enums at
once and MT5 does not validate an enum input replayed from a saved .set
or a stored optimization pass. TRAILING_STRATEGY and
MONEY_MANAGEMENT_STRATEGY now carry explicit values so the survivors keep
the numbers they were saved as, and ValidateBarrierInputs is widened into
ValidateTradeManagementInputs covering SL_Mode, TP_Mode,
Entry_Multiplier, TrailingStrategy and MM_STRATEGY. Without that gate a
chart saved with the Intelligent stop would feed SL_Mode = -1 into a
multiplier now used verbatim, placing the stop on the wrong side of entry.
RETRAIN-NEUTRAL: neither SL_Mode nor TP_Mode appears in
BuildModelFingerprint() or ComputeDbConfigFingerprint() since the
swing-pivot target replaced the barrier labels. No .nnw, .cfg or .db
re-keys. Also drops the now-dead g_TradeRewardRiskRatio bridge, the
CMoneyRiskBase::AdjustRiskAmount hook and the unsigned AIConfidence().
Compile-verified in _claude_stage: 0 errors, 0 warnings.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-25 10:10:20 -04:00
|
|
|
//--- opinion alone, purely by scheduling order. (User-identified 2026-08-17.) That bug is now
|
|
|
|
|
//--- unreachable twice over - the aggregate is a mean, AND nothing acts on it - but the shape stays
|
|
|
|
|
//--- correct so that a future consumer inherits a defined number rather than a race.
|
feat(gates): derive the recall floor instead of configuring it, publish what the configuration can PROVE, and stop an arbitrary member driving live exits
Three changes, all from the same principle: measure what is there before aiming
at it, and never certify a number you do not trade.
1. THE RECALL FLOOR IS DERIVED, AND IT MOVES BELOW CHANCE.
MinRecall=40 was a constant doing a statistical job. Its reference point is the
33.3% recall a zero-skill 3-class model gets on EVERY class, and against that the
constant was accidentally calibrated for exactly one sample size: on USDJPY CONV
(n_eff 195) 40% is chance + 2.0 SE; on SP500 PAI (n_eff 42) the same 40% is chance
+ 0.9 SE. One chart was being held to a bar twice as strict as the other, for no
reason anyone chose.
CollapseRecallFloorPct() computes it per class from that class's own effective
sample - EffectiveSampleSize(), so the overlap deflation the rest of the gates use
applies here too - as chance - EDGE_MIN_SIGMAS x SE. 26.5% at n_eff 195, 18.8% at
n_eff 42.
BELOW chance, deliberately, and this is the substantive change rather than the
arithmetic. This gate's only job is refusing to call a COLLAPSED model converged.
It is not a quality bar; the deploy gate is the quality bar and it is already
rigorous (chance + 2 SE on the deflated sample, Sidak over candidate eras, then the
cross-instrument pooled certificate). A convergence gate that ALSO demands
provably-above-chance recall on all three classes double-counts that job, and it
has failed that way twice here: MinRecall=60 blocked every SP500 H1 run in 2026-07,
and the 40 that replaced it made Neutral structurally unreachable once first-touch
resolution cut Neutral to a 0.65% residue. A floor nothing can reach does not make
a funded account safer, it stops the run converging at all.
Testing significantly BELOW chance instead catches what a fixed 40 was actually
catching - a model that has stopped emitting a class - and cannot become
unreachable by construction. It also fixes the direction the old constant scaled:
it now widens on a thin OOS window, where low recall genuinely cannot be told from
noise, and tightens on a rich one. Today's SP500 PAI (Buy 51 / Sell 18 / Neutral 30)
is still correctly blocked on Sell.
This also resolves a standing contradiction the code half-admitted at the
isBetterEra comment: selection ranks on coverage-weighted PRECISION while
convergence gated on RECALL, so a sparse high-precision abstainer - precisely the
model that could clear the deploy bar - was blocked by the floor.
The era line now PRINTS the derived floor. Anyone comparing these recalls against a
remembered "40" is reading the wrong bar.
2. DETECTABILITY: WHAT THIS CONFIGURATION COULD PROVE, BEFORE IT TRAINS.
The DEPLOY BAR line states the bar. It never said what reaching it would take, and
that is the actionable direction. ReportDetectability() inverts the same identity -
the gate passes when edge >= z x sqrt(p(1-p)/n_eff), so certifying an edge d needs
n_eff >= z^2 p(1-p)/d^2 independent calls, hence L times as many raw ones - and
prints a +2 / +5 / +10pp ladder as required independent calls, raw calls, and share
of the OOS window, marking any rung that needs more than the window holds IMPOSSIBLE.
Every term is a property of the CONFIGURATION - geometry via break-even, horizon via
mean label lifespan, window via oosCutoff - so no amount of training moves any of
them. It fires once, at the first healthy sweep, beside ReportFeatureHealth, for the
same reason: that is the first moment the bar grid, the measured geometry and the
lifespan are real numbers rather than defaults. It gates nothing.
This is the EdgeFinder discipline applied to our own gate: establish what the market
and the measurement design have to offer, then point the net at it - rather than
spending a thousand eras chasing something this OOS window could never certify.
3. AN ARBITRARY MEMBER WAS DRIVING LIVE EXITS AND TRAILING (user-identified).
Every ensemble member ran
g_LiveAISignedConfidence = SignedAIConfidence();
unconditionally, every tick. Last writer wins. Its consumers are the AI early-exit
route (CExpertSignalCustom::LiveSignedConfidence) and TrailingIntelligent - so on a
four-model chart an LSTM entry could be closed, and its stop moved, on the
Perceptron's opinion alone, decided by scheduling order. Not the vote, not a
weighted blend.
Now the mean across registered members, matching how the ensemble actually trades:
the open decision is the weighted-average vote, and an abstaining member contributes
0 and dilutes exactly as it does there. Members still training read 0, so a
half-trained ensemble reads WEAKER rather than louder - the safe direction for an
exit trigger. Deployed and paused members are included, which is the opposite of the
era barrier's exemption rule and correct for the opposite reason: that one asks who
must be waited for, this asks who has an opinion.
Latent today and staying that way for now by choice - Min_Vote_Close ships Disabled
(101, unreachable on both scales it drives) and TrailingStrategy is off, so live
exits are SL/TP only and the certified hold-to-barrier win rate is what actually
gets traded. Fixed now precisely because the plan is to enable vote exits once the
models are accurate, at which point a scheduling-order exit would be both harmful
and very hard to see.
STILL OPEN, and needs a decision before vote exits go on: the member gate and the
ensemble vote gate both grade hold-to-barrier, so enabling vote exits makes the
certified number stop describing the traded one. Warrior_EA.mq5 currently argues
barrier models may keep vote exits because "their label IS the vote's own horizon" -
that does not hold, since a vote flip at bar 5 of a 64-bar horizon is not the
target-before-stop outcome the gate measured. Either grade the OOS call on the real
exit rule (first of SL / TP / vote-flip / horizon) through the fill engine, or set
HoldToBarrier for ensemble members so the policy cannot drift from the certificate.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-17 16:15:08 -04:00
|
|
|
//--- Averaging matches how the ensemble actually trades: the open decision is the weighted-average
|
|
|
|
|
//--- vote, and a member that abstains contributes 0 and dilutes, exactly as it does there. Members
|
|
|
|
|
//--- still training read 0 from SignedAIConfidence(), so a half-trained ensemble reads WEAKER rather
|
refactor(trade-mgmt): remove all confidence-scaled trade management
Five modes went, all of them staking real risk on the model's confidence:
Intelligent entry (ENTRY_INTELLIGENT), stop (SL_INTELLIGENT), target
(TP_INTELLIGENT), trailing (CTrailingIntelligent) and lot size
(CMoneyIntelligent's quarter-Kelly). With them, the Confidence_Source
input and the CONFIDENCE_SOURCE enum, whose only job was choosing which
number those five read.
The reason is calibration, not correctness: the confidence magnitude is
known to be miscalibrated against the label prior, so every one of these
modes multiplied money by a quantity whose units were never established.
The DB arm had a second, independent defect - since the tester DB guard
(SignalDatabaseActive) it reads 0 in tester and optimizer but non-zero
live, so any backtest of CONF_DB/CONF_BLENDED could not reproduce live
trading. And what the DB produces is a filter-RANKING win rate, not a
per-trade win probability.
Both confidence numbers are still recorded per trade (aiConfidence /
dbConfidence) and still bucketed against outcome in TradeJournalReport.
Recording is what keeps the question answerable; acting on it was the
part with no evidence behind it. ConfidenceBridge.mqh now carries an
explicit telemetry-only rule at the top.
ENUM ORDINALS PINNED. Removing a member vacated a value in four enums at
once and MT5 does not validate an enum input replayed from a saved .set
or a stored optimization pass. TRAILING_STRATEGY and
MONEY_MANAGEMENT_STRATEGY now carry explicit values so the survivors keep
the numbers they were saved as, and ValidateBarrierInputs is widened into
ValidateTradeManagementInputs covering SL_Mode, TP_Mode,
Entry_Multiplier, TrailingStrategy and MM_STRATEGY. Without that gate a
chart saved with the Intelligent stop would feed SL_Mode = -1 into a
multiplier now used verbatim, placing the stop on the wrong side of entry.
RETRAIN-NEUTRAL: neither SL_Mode nor TP_Mode appears in
BuildModelFingerprint() or ComputeDbConfigFingerprint() since the
swing-pivot target replaced the barrier labels. No .nnw, .cfg or .db
re-keys. Also drops the now-dead g_TradeRewardRiskRatio bridge, the
CMoneyRiskBase::AdjustRiskAmount hook and the unsigned AIConfidence().
Compile-verified in _claude_stage: 0 errors, 0 warnings.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-25 10:10:20 -04:00
|
|
|
//--- than louder, which is the safe direction for anything that could ever close a position.
|
feat(gate): grade OOS calls on the exit policy actually in force, and move vote combining out of the members and into the orchestrator
Option (a) from the exit-policy question: the certified number must be the traded
number. Plus the modularity correction the user called for on 778b6c0.
1. VOTE COMBINING BELONGS TO THE ORCHESTRATOR, NOT TO A MEMBER.
778b6c0 fixed the last-writer-wins bug on g_LiveAISignedConfidence by having a
member average its siblings through g_warriorEnsemble. That trades a scheduling bug
for a coupling bug, and it is the wrong shape for this EA: every signal runs in its
own instance, minds its own state, and VOTES to the orchestrator, which is the only
thing allowed to combine opinions.
Replaced with a publish/aggregate pair in Variables\ConfidenceBridge.mqh whose split
is enforced by shape rather than by convention:
- PublishAIVote(slot, conf) - a member writes ONLY its own slot, reads nobody's;
- AggregateAIVotes() - called by CExpertSignalCustom::LiveSignedConfidence.
CExpertSignalAIBase::EnsembleLiveSignedConfidence is gone. The orchestrator also
republishes the aggregate into g_LiveAISignedConfidence, because the intelligent
trailing reads that global directly and must act on the same number the exit route
does rather than on a leftover from whichever member ticked last. A solo AI signal
owns slot 0, so the non-ensemble path is unchanged.
2. THE GATE NOW REPLAYS THE REAL EXIT RULE.
SimulateTradeOutcome() walks the same price series with the same fill/barrier/spread
convention as ComputeLabelForBar - deliberately by copy, so a disagreement between
the two can only be a policy effect and never a discrepancy between two pieces of our
own arithmetic - and terminates at the FIRST of stop / target / vote reversal /
horizon. Barriers are tested before the vote on the same bar: intrabar we cannot know
which came first, and the barrier is what the broker executes automatically, so
checking the vote first would credit the exit policy with escapes a real stop would
have taken out of its hands.
It runs AFTER pass 3, not inside it. A vote-flip exit for a trade entered at bar r is
decided by the model's output at bars r-1, r-2, ... - NEWER bars - and pass 3 walks
oldest-to-newest, so at the moment r is graded its own exit does not exist yet. Only
once m_oosDecisionSeries is complete over the whole OOS window can the replay run.
In ensemble mode that series carries the member's adjusted decision and the live exit
reads the ensemble aggregate, which is the coupling the user identified: an LSTM entry
really can be closed by the ensemble turning against it.
3. THE STATISTIC HAS TO CHANGE WITH THE POLICY, AND THAT IS THE REAL FINDING.
A barrier exit pays a fixed R. A vote exit pays whatever the close happens to be. So
the moment vote exits are enabled the payoff is CONTINUOUS, and "win rate vs
break-even" stops being a meaningful test - there is no fixed break-even for a
variable payoff. SimulateTradeOutcome therefore returns R rather than a bool, and the
replay reports expectancy in R with its SE taken from the R distribution (overlap-
deflated on the same EffectiveSampleSize doctrine as every other SE here), not from a
binomial.
This is the same class of error as win-based scoring in 2026-08-09: measuring a
variable-payoff process with a fixed-payoff statistic. Naming it now, while vote exits
are still off, is much cheaper than discovering it after they go on.
4. WHY THIS IS SAFE TO SHIP TODAY.
Min_Vote_Close ships Disabled (101, unreachable on both scales it drives) and reaches
the AI signal through the new ExitPolicy() setter as 1.01, which the setter turns into
0.0 = no vote exit. Under that policy every replayed trade resolves at a barrier and
the simulation is arithmetically the same trade the deploy gate already certifies -
they cannot drift. The report says so explicitly, and prints ONCE per run in that
state; when vote exits are on it prints every era, because then the divergence is the
thing to watch. Nothing about today's numbers moves.
The gate switchover is wired but dormant by construction: it becomes exit-aware the
moment the input is enabled, which is exactly what "the certified number is the traded
number" has to mean.
KNOWN LIMIT, stated rather than hidden: only the AI early-exit route is replayed. The
rule-based averaged-vote close (m_threshold_close) depends on every other filter's
live vote, which pass 3 does not reproduce, so a position the classic filters would
have closed is held to its barrier here. The replay therefore holds LONGER than live
and overstates barrier-reached outcomes. Faithful only while the AI is the dominant
vote - which is the configuration this is being built for.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-17 16:25:57 -04:00
|
|
|
//--- PUBLISH ONLY THIS SIGNAL'S OWN VOTE. Combining is the orchestrator's job, never a member's - see
|
|
|
|
|
//--- the vote board in Variables\ConfidenceBridge.mqh for the scheduling bug this replaces and for why
|
|
|
|
|
//--- "have a member average its siblings" was the wrong shape of fix in a codebase whose whole point is
|
|
|
|
|
//--- that signals do not reach into each other. A solo AI signal owns slot 0 and the aggregate is then
|
|
|
|
|
//--- just its own value, so the non-ensemble path is unchanged.
|
|
|
|
|
PublishAIVote(m_ensembleMember ? m_ensembleIndex : 0, SignedAIConfidence());
|
fix(panel,arrows): one deploy predicate, a deployed-only readout, and persist the vote arrows
Four reported symptoms, three of them one root cause: the ensemble's
certified record was session-scoped and written ONLY at pass-3
completion. A deployed ensemble runs no further eras, so every restart
lost the aggregate win rate, the aggregate panel line and the overlay
snapshots - and could never regenerate them, because regeneration only
happens at an era end that will never come.
THE SELF-CONTRADICTION. Member rows read "Live - learning from new bars"
(from m_trainingComplete) while the line under them read "training, not
tradable yet" (from `prospective`, which means "this number came from
ProspectiveVote() rather than a real Direction() call" - what happens on
any bar where every member abstains, and which says nothing whatever
about training state). Both now resolve through one predicate:
WarriorChartModelsDeployed(), fed by members publishing their own state
on the same slot and cadence as their vote. Adds a third verdict word,
"armed (bar still open)", for a deployed model on a prospective
recompute - the case that used to claim it was training.
DEPLOYED PANEL. Once every published model is converged the per-member
rows are dropped: what ships is the aggregate vote win rate, the live
vote, and the verdict. While training the rows stay - they are the only
way a collapsed or lagging member is visible, since a collapsed member
abstains and so is invisible in the aggregate by construction.
ACCURACY NOW RESPECTS THE ENTRY THRESHOLD. The panel's "precision 65%"
came from m_cumOosCorrect/m_cumOosTotal, which counts every bar a model
called Buy or Sell - threshold-blind, and per-model rather than
per-vote. The correct number already existed (votePrecPct: bars where
|vote| >= threshold and the direction policy allows) and is now what the
panel shows, with the threshold named in the text because the number is
meaningless without it.
VOTE ARROWS PERSIST. With DrawUnfilteredSignals off - the default - the
chart shows SIG_VOTE_PREFIX arrows, and nothing saved them:
CChartUI's .arrows sidecar is member-scoped and never saw that layer.
New CVoteArrowStore mirrors them to a chart-keyed sidecar and restores
them progressively at init, on the same budgeted non-blocking path.
The header stores the open/close thresholds; a mismatch on load DISCARDS
the arrows rather than redrawing a picture of a strategy no longer
configured - stale arrows are worse than none, because none is visibly
empty and stale is confidently wrong.
Also: .stats bumped to WST7 carrying the ensemble record (guarded on
threshold match, most-complete-copy-wins), and the loader's version
tests collapsed from an or-chain to ">=" - the magics are ASCII 'WST1'..
'WST7' so they are already ordered, and a missed arm in that chain reads
the NEXT field's bytes into this one, which fails as plausible numbers
rather than as an error.
Compile-verified in _claude_stage: 0 errors, 0 warnings.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-25 13:00:15 -04:00
|
|
|
//--- ...and this member's own training state, on the same slot and the same cadence. Published
|
|
|
|
|
//--- HERE, beside the vote, so the number and the word describing it can never come from different
|
|
|
|
|
//--- moments - see WarriorChartModelsDeployed() in ExpertSignalCustom.mqh for the contradiction
|
|
|
|
|
//--- that produced.
|
|
|
|
|
PublishModelConverged(m_ensembleMember ? m_ensembleIndex : 0, m_trainingComplete);
|
2026-08-01 11:27:28 -04:00
|
|
|
datetime lastBarDate = (datetime)SeriesInfoInteger(m_symbol.Name(), m_period, SERIES_LASTBAR_DATE);
|
2026-08-22 00:25:52 -04:00
|
|
|
//--- A failed lookup (0) must not silently read as "dtStudied is already caught up, nothing
|
|
|
|
|
//--- pending" - that would freeze this function into never re-triggering training/signal refresh
|
|
|
|
|
//--- again until some other path happens to bump dtStudied.
|
2026-08-01 11:27:28 -04:00
|
|
|
bool newBarPending = (dPrevSignal == -2 || lastBarDate <= 0 || ((m_inferenceOnly ? m_lastBarTime : dtStudied) < lastBarDate));
|
2026-08-22 00:25:52 -04:00
|
|
|
//--- A MODEL THAT HAS NOT FINISHED TRAINING IS ALWAYS PENDING. Gating them on the watermark
|
|
|
|
|
//--- meant training could only advance when a new BAR closed. On H1 that is one 120ms chunk per
|
|
|
|
|
//--- hour. A POST-TRAINING WALK IS ALSO PENDING.
|
refactor(online-learning): OnlineLearning is a real collaborator, not a raw-include partial (S5)
Expert\AIBase\OnlineLearning.mqh (595 lines) -> Expert\OnlineLearning\:
IOnlineLearningView.mqh (abstract, ~50 accessors) + AIBaseOnlineLearningView.mqh/
AIBaseOnlineLearningViewImpl.mqh (the adapter) + OnlineLearning.mqh (COnlineLearning).
STATEFUL, unlike CModelPersistence: grep-verified the shadow net, the OOS
continual-learning simulation state and the pattern-database backfill state are
genuinely exclusive to this file's own methods - Training.mqh/Topology.mqh/
Lifecycle.mqh/the signal's own header only ever CHECKED or RESET this state at
era/lifecycle boundaries, never owned it, so it moved onto the collaborator as
real members (same doctrine as Excursion). Those external touch points became
consolidated view/forward calls instead of raw field pokes - AbortSimIfActive()
replaces THREE separate copies of the same delete/null/false triple (Training.mqh's
stop path, FlushTrainRun, ResetWeights), matching the geometry-scan duplicate-reset
precedent in project memory. ResetForFreshTopology() replaces Topology.mqh's five-
field reset block, DeployNet() replaces the shadow-preferred net selection duplicated
in Inference.mqh and ChartScoreBarForRescan, and BlendTowardNet() replaces the era-end
blend Training.mqh used to poke m_shadowNet for directly.
Reused the signal's existing Data*()/Chart*()/Persist*() getters wherever one already
answered the question (labels/outcome/history/horizon/priors/servable-bars/etc.);
added ~30 new Online*() wrappers only for what nothing else exposed yet. The three
PersistOnline*() get/set pairs (WST3 .stats fields) now forward through the owning
member instead of touching the field directly - CModelPersistence is unaffected.
Every method body is a pure relocation of the original's statements in original
order; verified against `git show HEAD~1:Expert/AIBase/Excursion.mqh`-style diff
against the pre-extraction file kept in the working tree until this commit.
Compiled 0 errors, 0 warnings (stage mirror + MetaEditor64 /compile, ~91s).
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-08-23 22:59:02 -04:00
|
|
|
bool postTrainWalkPending = (m_onlineLearning.SimRunActive() || m_onlineLearning.BackfillActive());
|
fix: the DB backfill could never run, and HEAD did not compile
Four defects in 64c5dd5/1a05e63, found by review + a baseline compile.
Goals 1-8 of that session are unchanged; this makes 6 and 8 actually reachable.
1. HEAD DID NOT COMPILE - 6 errors. CControlPanel::Minimize/Maximize were
declared `virtual bool ... override`, but CAppDialog declares both as
`virtual void` (Controls\Dialog.mqh). errors 265 + 404 on each, plus 151
on `bool ok = CAppDialog::Minimize()`. Return type is void now; there was
never a success flag to forward. Verified: 0 errors, 0 warnings.
2. THE BACKFILL COULD NEVER ADVANCE, and neither could the OOS continual
simulation (that one has been dead since it was written). Both are armed
at the instant convergence is declared, and both advance only from inside
Train(), one chunk per call. But ScheduleTrainingIfNeeded's only per-tick
ArmStudyEvent site sits in the `else` of a branch taken whenever
m_trainingComplete is set and m_trainRunActive is clear - which is exactly
the state FinalizeTrainRun() leaves behind one line before they are armed.
Train() was never called again, so the walks sat at their start index
forever: no "simulation complete" line, and not one row written to the DB
this feature exists to fill. Only a manual Resume/Retrain unstuck them.
Both flags now keep the model schedulable.
3. IN AI_HYBRID - the mode this ships in - the backfill was never even armed.
Ensemble members deploy at Train() ENTRY and return immediately (so no era
is wasted), which skips the era-end block the backfill was started from.
All four members were a no-op for a second, independent reason. Armed on
the ensemble deploy path too, from m_resumeBars/m_resumeOosCutoff.
4. RE-RUNS DUPLICATED ROWS. RegisterSignal inserts unconditionally - no key,
no duplicate check - and m_dbBackfillDone is in-memory, so every later
attach that retrained to convergence wrote a second full set of rows for
the same bars. The ranking would count one bar once per model that ever
deployed, weighting superseded opinions as heavily as the live one. A
.dbfill marker stamps the deployed era; written only on completion (an
interrupted walk redoes itself rather than ranking a partial window) and
deleted with the other sidecars on reset-weights.
Also: WarmBlocking's timeout was silent, which restored the exact silent
pin failure it was added to prevent - it now says so in the journal, and
returns true for "no reference pairs to wait for" so the warning stays rare
enough to be read.
Not addressed, needs a decision: the backfill scores the OOS window with the
checkpoint that was SELECTED as best on that same window, then writes those
win rates into the table filter weights rank on - the selection set consumed
twice, undiscounted, while the deploy gate right next to it applies a
family-wise correction for exactly that effect. The rows are also simulated
triple-barrier outcomes at today's spread sharing a table with realised
fills. The completion log line now states both plainly.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-16 21:25:51 -04:00
|
|
|
bool trainingPending = !(m_trainingComplete || m_inferenceOnly) || postTrainWalkPending;
|
2026-08-01 11:27:28 -04:00
|
|
|
//--- m_inferenceOnly (single backtest) takes the converged/inference branch even if the seeded model
|
|
|
|
|
//--- wasn't flagged complete, so a backtest never drops into Train()'s era loop - see m_inferenceOnly.
|
fix: the DB backfill could never run, and HEAD did not compile
Four defects in 64c5dd5/1a05e63, found by review + a baseline compile.
Goals 1-8 of that session are unchanged; this makes 6 and 8 actually reachable.
1. HEAD DID NOT COMPILE - 6 errors. CControlPanel::Minimize/Maximize were
declared `virtual bool ... override`, but CAppDialog declares both as
`virtual void` (Controls\Dialog.mqh). errors 265 + 404 on each, plus 151
on `bool ok = CAppDialog::Minimize()`. Return type is void now; there was
never a success flag to forward. Verified: 0 errors, 0 warnings.
2. THE BACKFILL COULD NEVER ADVANCE, and neither could the OOS continual
simulation (that one has been dead since it was written). Both are armed
at the instant convergence is declared, and both advance only from inside
Train(), one chunk per call. But ScheduleTrainingIfNeeded's only per-tick
ArmStudyEvent site sits in the `else` of a branch taken whenever
m_trainingComplete is set and m_trainRunActive is clear - which is exactly
the state FinalizeTrainRun() leaves behind one line before they are armed.
Train() was never called again, so the walks sat at their start index
forever: no "simulation complete" line, and not one row written to the DB
this feature exists to fill. Only a manual Resume/Retrain unstuck them.
Both flags now keep the model schedulable.
3. IN AI_HYBRID - the mode this ships in - the backfill was never even armed.
Ensemble members deploy at Train() ENTRY and return immediately (so no era
is wasted), which skips the era-end block the backfill was started from.
All four members were a no-op for a second, independent reason. Armed on
the ensemble deploy path too, from m_resumeBars/m_resumeOosCutoff.
4. RE-RUNS DUPLICATED ROWS. RegisterSignal inserts unconditionally - no key,
no duplicate check - and m_dbBackfillDone is in-memory, so every later
attach that retrained to convergence wrote a second full set of rows for
the same bars. The ranking would count one bar once per model that ever
deployed, weighting superseded opinions as heavily as the live one. A
.dbfill marker stamps the deployed era; written only on completion (an
interrupted walk redoes itself rather than ranking a partial window) and
deleted with the other sidecars on reset-weights.
Also: WarmBlocking's timeout was silent, which restored the exact silent
pin failure it was added to prevent - it now says so in the journal, and
returns true for "no reference pairs to wait for" so the warning stays rare
enough to be read.
Not addressed, needs a decision: the backfill scores the OOS window with the
checkpoint that was SELECTED as best on that same window, then writes those
win rates into the table filter weights rank on - the selection set consumed
twice, undiscounted, while the deploy gate right next to it applies a
family-wise correction for exactly that effect. The rows are also simulated
triple-barrier outcomes at today's spread sharing a table with realised
fills. The completion log line now states both plainly.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-16 21:25:51 -04:00
|
|
|
if((m_trainingComplete || m_inferenceOnly) && !m_trainingStopRequested && !m_trainRunActive &&
|
|
|
|
|
!postTrainWalkPending)
|
2026-08-01 11:27:28 -04:00
|
|
|
{
|
|
|
|
|
if(newBarPending)
|
|
|
|
|
RefreshConvergedSignal();
|
|
|
|
|
}
|
|
|
|
|
else
|
fix(ensemble): responsive panel + synchronized eras + combined-vote accuracy
Four user-reported/requested items, one root cause chain:
1) DEAD CONTROL PANEL in AI_HYBRID mode. All members posted custom event
id 1 and handled id 1001, and CExpertCustom broadcasts every chart
event to every filter - so each posted event ran a train chunk in ALL
N members (N*N chunks per round) and the chart thread never idled
long enough to deliver clicks/drags. profiling.csv: 99.45% of time in
OnChartEventHandler. Fix: per-instance study-event ids
(STUDY_EVENT_ID_BASE + construction order, offset above the Controls
library's ON_* codes - id 1 was also ON_DBL_CLICK, so panel
double-clicks fired training chunks). ArmStudyEvent() is the single
post site; lost-event watchdog replaces the accidental
sibling-clears-my-flag rescue.
2) WARM-UP DUPLICATION. The auto-tune sweep is deterministic over
identical features/labels, and it ends in the full MI diagnostic
suite, which the MI-share gate never intercepted on the sweep path -
four members ran four identical ~36s sweep+report blocks. First
member publishes outcome (g_ensembleChartTuneDone/Installed/Settings);
the rest apply it and skip both.
3) DEINIT STRANDED PANEL+ARROWS (user repro 18:52). Root cause from the
log: the 4,500ms budget runs from MetaTrader's stop REQUEST - a heavy
autosave in flight ate it, OnDeinit got ~430ms and died in the first
member's arrow persist ("Abnormal termination" 432ms in). Fix: early
visible-UI sweep (native prefix deletes for status/panel/dialog)
right after ClearStatusLabel, and a fast path for still-training
models - their arrows are re-rendered every era, so they get one bulk
purge instead of scan+atomic-write in the death window.
4) ENSEMBLE FEATURES (user requests): era BARRIER - members advance era
by era together; a member ahead of the slowest still-training member
declines Train() calls and its chunk budget is donated
(TRAIN_TIME_BUDGET_MS = 120/activeTrainers, UI headroom constant).
COMBINED-VOTE OOS SCORE - each member's pass-3 scan contributes its
adjusted per-bar decision (0.0 on abstain) to a shared row buffer;
the last member to finish the era scores the averaged vote vs the
mirrored Min_Vote_Open against the same target-before-stop outcomes
members grade themselves on, publishing an "Ensemble vote" line on
the aggregated panel. Member headlines now carry their lifetime win
rate with break-even.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-16 19:06:04 -04:00
|
|
|
{
|
|
|
|
|
//--- Lost-event watchdog: an armed event that never arrived (chart-event queue overflow) would
|
|
|
|
|
//--- otherwise leave bEventStudy true forever and silently stall training - the accidental rescue
|
|
|
|
|
//--- (any sibling's event clearing this flag) went away with the per-instance ids. See
|
|
|
|
|
//--- STUDY_EVENT_LOST_MS for why a false trip is not a realistic concern.
|
|
|
|
|
if(bEventStudy && GetTickCount() - m_studyArmedTick > STUDY_EVENT_LOST_MS)
|
|
|
|
|
{
|
|
|
|
|
Print(ID + ": study event armed " + IntegerToString(STUDY_EVENT_LOST_MS / 1000) +
|
|
|
|
|
"s ago never arrived (chart-event queue overflow?) - re-arming.");
|
|
|
|
|
bEventStudy = false;
|
|
|
|
|
}
|
fix: training could only advance one 120ms chunk per bar
ScheduleTrainingIfNeeded() armed the next Train() call only when
dtStudied < lastBarDate. That watermark test is right for a CONVERGED
model - one inference refresh per new bar - and wrong for a training
run, because Train() is chunked: it does ~120ms of work and yields,
needing thousands of calls to finish one era, and every one of those
calls has to be armed from there.
dtStudied is two incompatible things. Train() sets it to the training
WINDOW START (~2008); FinalizeTrainRun() sets it to the last bar
SCANNED (~now). So the moment any run finalized, the scheduler went
silent until the next candle closed. On H1 that is one chunk per hour.
The symptom was indistinguishable from a hang: no era lines, no
heartbeats, not one of the six instrumented stall branches - because
Train() was not being CALLED. The TRAIN STALL line that caught it
reported runActive=Y only because m_trainRunActive had been set
microseconds earlier in that same call, and eraResume=N proved no era
was in flight. Two log bursts, 28 minutes apart, exactly one H1 bar.
Before 0c85c54 this was survivable rather than correct: the saved
watermark left almost no bars eligible per era, so eras were nearly
free and one call per bar still looked like progress.
An unconverged model is now always pending. Pause/stop are handled by
m_trainingPaused/m_trainingStopRequested, which Train() checks itself.
Also: the one Train() exit that tears down the whole run on a buffer
failure was completely silent - it now says so. And the build tag moves
to train-dispatch-v2; it had not moved since ce52654, which is why the
running binary could not be identified from its own log.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-10 10:06:49 -04:00
|
|
|
if(!m_trainingStopRequested && !bEventStudy && (newBarPending || trainingPending))
|
fix(ensemble): responsive panel + synchronized eras + combined-vote accuracy
Four user-reported/requested items, one root cause chain:
1) DEAD CONTROL PANEL in AI_HYBRID mode. All members posted custom event
id 1 and handled id 1001, and CExpertCustom broadcasts every chart
event to every filter - so each posted event ran a train chunk in ALL
N members (N*N chunks per round) and the chart thread never idled
long enough to deliver clicks/drags. profiling.csv: 99.45% of time in
OnChartEventHandler. Fix: per-instance study-event ids
(STUDY_EVENT_ID_BASE + construction order, offset above the Controls
library's ON_* codes - id 1 was also ON_DBL_CLICK, so panel
double-clicks fired training chunks). ArmStudyEvent() is the single
post site; lost-event watchdog replaces the accidental
sibling-clears-my-flag rescue.
2) WARM-UP DUPLICATION. The auto-tune sweep is deterministic over
identical features/labels, and it ends in the full MI diagnostic
suite, which the MI-share gate never intercepted on the sweep path -
four members ran four identical ~36s sweep+report blocks. First
member publishes outcome (g_ensembleChartTuneDone/Installed/Settings);
the rest apply it and skip both.
3) DEINIT STRANDED PANEL+ARROWS (user repro 18:52). Root cause from the
log: the 4,500ms budget runs from MetaTrader's stop REQUEST - a heavy
autosave in flight ate it, OnDeinit got ~430ms and died in the first
member's arrow persist ("Abnormal termination" 432ms in). Fix: early
visible-UI sweep (native prefix deletes for status/panel/dialog)
right after ClearStatusLabel, and a fast path for still-training
models - their arrows are re-rendered every era, so they get one bulk
purge instead of scan+atomic-write in the death window.
4) ENSEMBLE FEATURES (user requests): era BARRIER - members advance era
by era together; a member ahead of the slowest still-training member
declines Train() calls and its chunk budget is donated
(TRAIN_TIME_BUDGET_MS = 120/activeTrainers, UI headroom constant).
COMBINED-VOTE OOS SCORE - each member's pass-3 scan contributes its
adjusted per-bar decision (0.0 on abstain) to a shared row buffer;
the last member to finish the era scores the averaged vote vs the
mirrored Min_Vote_Open against the same target-before-stop outcomes
members grade themselves on, publishing an "Ensemble vote" line on
the aggregated panel. Member headlines now carry their lifetime win
rate with break-even.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-16 19:06:04 -04:00
|
|
|
ArmStudyEvent((long)MathMax(0, MathMin(iTime(m_symbol.Name(), PERIOD_CURRENT, (int)(100 * Net.recentAverageSmoothingFactor * (m_trainingComplete ? 1 : 10))), dtStudied)), "New Bar");
|
|
|
|
|
}
|
2026-08-01 11:27:28 -04:00
|
|
|
//--- Train() (see its declaration comment) now yields every ~TRAIN_TIME_BUDGET_MS instead of
|
|
|
|
|
//--- blocking for a whole era, so while a run is active this per-tick line would otherwise
|
|
|
|
|
//--- overwrite Train()'s own full-detail status label on every single tick between chunks -
|
2026-08-22 00:25:52 -04:00
|
|
|
//--- flickering between the two instead of showing one steady picture.
|
fix(ensemble+depth): the barrier had no liveness escape, and the depth gate could not report the one state the evidence pointed at
Two charts (USDJPY 50,179 bars / XAUUSD 33,982) sat at era 0 for 38 minutes with
four of their eight members completely silent. Nothing in this commit guesses at
why the sweep fails - the last five guesses were all wrong. It makes the failure
say what it is, and stops one broken member taking its whole chart down with it.
WHAT THE LOG ACTUALLY SAYS, before any of this.
- The running build IS d9f834d (pulled 14:18, compiled 14:19:01, 0 errors), so
every depth instrument from 1dda479/7e63a8b/45c9e21 was live.
- It printed NOTHING. Zero "PRIMING", zero "CAPPED", zero "Per-indicator depth"
in 27 MB of journal. The instrument built to find the depth shortfall returned
"not this".
- On USDJPY at 14:24, CONV-cad8 completed eras 0 AND 1 across all 50,179 bars -
same chart, same 832-value window, same indicators, byte-identical fingerprint -
while LSTM-cad8 and HYB-cad8 reported ok=0 failed=50163. So it is not the
symbol, the history, the bar count or the indicator depth. It is per-member.
- ok=0 means the NEWEST anchors failed too, and a short indicator cannot do that.
The depth reading in project_silent_block_failures is therefore retired by its own
instrumentation. THE ROOT CAUSE IS STILL UNKNOWN and this commit does not claim one.
1. THE DEPTH GATE'S SILENT PATH WAS THE STATE IT WAS HUNTING.
ServableBars() read `if(servable <= 0 || servable >= want) return want;` - one
branch over three unrelated states, silent in all of them:
enabled == 0 -> nothing tunable is on. No cap. Healthy.
enabled > 0, servable == -1 -> a handle answered INVALID.
enabled > 0, servable == 0 -> created, never calculated.
BarsCalculated() returns -1 for a dead handle, so a dead MA is indistinguishable
from "no tunable indicators enabled" - and both returned `want` without printing a
character. That is exactly the state a per-member, every-index, depth-independent
failure produces, and it is the single reason a build carrying full depth
instrumentation logged nothing through the whole outage.
TunableBarsCalculated() now also reports HOW MANY indicators it consulted, and the
dead-handle case is reported (latched, with per-handle depths). The RETURN is
deliberately unchanged - what to do about a dead handle is not yet known, and
changing control flow on an unproven cause is how the last four fixes here went
wrong. SettledBars() routes its three pass-through states via ServableBars() so the
report is reachable from the training sweep, which is the only caller that hits it.
2. THE STALL REPORT NAMED A SLOT, NEVER A BLOCK.
"lookback slot 0 REJECTED (window had 24 of 832 values)" plus a guess ("an
indicator warm-up or a history-edge read"). Which guard fired was INFERRED by
counting 4+5+4+4+6+1 = 24 and concluding feature 25 must be the MA. The arithmetic
was right; every conclusion drawn from it was wrong, because a value count names a
POSITION and a position cannot tell cold from capped from invalid from off-the-end.
Every guard that can reject a bar now records itself - m_featureFailBlock - and the
report carries it, the series index, IndicatorDepthReport()'s per-handle depths,
and for each indicator whether the NEWEST bar reads. That last field is the whole
diagnosis in one word: newest-also-EMPTY means the buffer is unreadable everywhere
(cold or dead handle), newest-reads means a genuine history edge. Instrumented:
open, ATR, MA, RSI, MACD, Ichimoku, and all five AD blocks via ADIndicatorCold().
3. THE TOTAL-FAILURE BACKOFF WAS GATED ON THE WRONG QUESTION.
It armed only when m_featureFailTransient was set. Keeping that flag correct across
every guard is a list that has to stay right forever - the same shape of fix the
feature cache abandoned for the same reason - and the gate is pointless anyway: a
sweep where ZERO of 50,163 bars produced a window will produce zero again if it
restarts a millisecond later, transient or not. Doing that at full speed is what
starved six indicator threads on a six-core box. The backoff is now unconditional
on a total failure. The flag keeps its real job, deciding whether a MISS may be
cached, which is a per-bar question and not a scheduling one.
4. THE ERA BARRIER DEADLOCKED, AND SILENCED THE MEMBERS IT FROZE.
EnsembleMinTrainingEra() exempted deployed, stopped and paused members and its
comment concluded "so nothing deadlocks". Those three are all VOLUNTARY. A member
that simply CANNOT finish an era is none of them, so it pinned the minimum at its
own era with no time limit - and the hold branch's only action was
`m_lastEraCompleteTick = GetTickCount()`, which silences the stall watchdog. So on
USDJPY the two members that could not train reported, and the two healthy members
frozen behind them wrote nothing anywhere. The outage was visible only through the
members that were not suffering it.
- BarrierEraHeartbeat() stamps a clock on real era CHANGE, kept separate from
m_lastEraCompleteTick precisely because the barrier resets that one. Only a
member AT the minimum can be a blocker; a member ahead is idle by design and is
never counted as stuck.
- After ENSEMBLE_BARRIER_STUCK_MS (12 min) a non-advancing member is dropped from
the barrier minimum. It keeps training and rejoins the instant it completes an
era - at which point, being behind, it legitimately becomes the minimum again,
which is the documented resumed-laggard behaviour.
- Both transitions say so loudly, and the release states plainly that the
combined-vote score cannot be computed while the ensemble is desynchronised.
- A held member now writes a rate-limited journal line naming WHICH members it is
waiting on, so the blocker is read off one line.
5. THE PANEL FLICKER.
OnTickHandler gates its terse writer on !m_trainRunActive, and a barrier-held member
returns from Train() before ever setting it - so both writers thought they were the
only one updating the label and fought every tick. That is the reported "Getting
ready..." <-> "Waiting at era N for slower ensemble members" oscillation, and it hit
Perceptron but not Convolutional purely because Convolutional had a run active from
a completed era and Perceptron, resumed from disk, never did. Train()'s message is
the specific one, so it wins.
NEXT STEP once this is running: the stall line now ends in "REJECTED BY: ..." and
the per-handle depths. Read it. Do not reason around it.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-17 15:21:15 -04:00
|
|
|
if(m_ensembleMember && EnsembleEraBarrierHolds())
|
|
|
|
|
return;
|
2026-08-01 11:27:28 -04:00
|
|
|
if(!m_trainRunActive)
|
|
|
|
|
{
|
2026-08-22 00:25:52 -04:00
|
|
|
//--- Compact, accurate end-state text.
|
refactor(online-learning): OnlineLearning is a real collaborator, not a raw-include partial (S5)
Expert\AIBase\OnlineLearning.mqh (595 lines) -> Expert\OnlineLearning\:
IOnlineLearningView.mqh (abstract, ~50 accessors) + AIBaseOnlineLearningView.mqh/
AIBaseOnlineLearningViewImpl.mqh (the adapter) + OnlineLearning.mqh (COnlineLearning).
STATEFUL, unlike CModelPersistence: grep-verified the shadow net, the OOS
continual-learning simulation state and the pattern-database backfill state are
genuinely exclusive to this file's own methods - Training.mqh/Topology.mqh/
Lifecycle.mqh/the signal's own header only ever CHECKED or RESET this state at
era/lifecycle boundaries, never owned it, so it moved onto the collaborator as
real members (same doctrine as Excursion). Those external touch points became
consolidated view/forward calls instead of raw field pokes - AbortSimIfActive()
replaces THREE separate copies of the same delete/null/false triple (Training.mqh's
stop path, FlushTrainRun, ResetWeights), matching the geometry-scan duplicate-reset
precedent in project memory. ResetForFreshTopology() replaces Topology.mqh's five-
field reset block, DeployNet() replaces the shadow-preferred net selection duplicated
in Inference.mqh and ChartScoreBarForRescan, and BlendTowardNet() replaces the era-end
blend Training.mqh used to poke m_shadowNet for directly.
Reused the signal's existing Data*()/Chart*()/Persist*() getters wherever one already
answered the question (labels/outcome/history/horizon/priors/servable-bars/etc.);
added ~30 new Online*() wrappers only for what nothing else exposed yet. The three
PersistOnline*() get/set pairs (WST3 .stats fields) now forward through the owning
member instead of touching the field directly - CModelPersistence is unaffected.
Every method body is a pure relocation of the original's statements in original
order; verified against `git show HEAD~1:Expert/AIBase/Excursion.mqh`-style diff
against the pre-extraction file kept in the working tree until this commit.
Compiled 0 errors, 0 warnings (stage mirror + MetaEditor64 /compile, ~91s).
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-08-23 22:59:02 -04:00
|
|
|
bool onlineActive = m_onlineLearning.Enabled() && !m_inferenceOnly
|
2026-08-01 11:27:28 -04:00
|
|
|
&& !MQLInfoInteger(MQL_TESTER) && !MQLInfoInteger(MQL_OPTIMIZATION) && !MQLInfoInteger(MQL_FORWARD)
|
|
|
|
|
&& CheckPointer(Net) != POINTER_INVALID && !Net.CpuInference();
|
|
|
|
|
//--- Simple end-state panel (default, VerboseMode off): plain-language status + the model's
|
|
|
|
|
//--- compounded/persistent Buy/Sell win-rate (directional accuracy, Neutral excluded - persisted in
|
|
|
|
|
//--- .stats WST5, so it survives a fresh chart reload and is meaningful the moment a drop-and-go user
|
|
|
|
|
//--- attaches the EA) + the current call. The verbose era/forecast dump below stays for power users.
|
|
|
|
|
if(!VerboseMode)
|
|
|
|
|
{
|
|
|
|
|
string statusPlain;
|
|
|
|
|
if(m_trainingComplete)
|
|
|
|
|
statusPlain = onlineActive ? "Live - learning from new bars" : "Ready for live trading";
|
|
|
|
|
else if(m_trainingStopRequested)
|
|
|
|
|
statusPlain = "Paused - progress saved";
|
|
|
|
|
else if(m_trainingPaused)
|
|
|
|
|
statusPlain = "Paused";
|
|
|
|
|
else
|
|
|
|
|
statusPlain = "Getting ready...";
|
|
|
|
|
ENUM_SIGNAL liveSig = DoubleToSignal(dPrevSignal);
|
|
|
|
|
string liveSigPlain = (liveSig == Buy) ? "Buy" : (liveSig == Sell) ? "Sell" : "Neutral (no trade)";
|
2026-08-25 22:51:50 -04:00
|
|
|
string simpleLive;
|
2026-08-15 16:54:43 -04:00
|
|
|
//--- The ensemble headline is the FIRST line, so lead with the signal - on the combined
|
2026-08-25 22:51:50 -04:00
|
|
|
//--- panel each member's one line must answer "what is this model saying right now". Branched
|
|
|
|
|
//--- FIRST, not built-then-overwritten: ComputeCompoundedAccuracyLine() is a real string build
|
|
|
|
|
//--- (OosTally lookups, formatting) whose result the ensemble branch used to discard outright.
|
2026-08-15 16:54:43 -04:00
|
|
|
if(m_ensembleMember)
|
fix(ensemble): responsive panel + synchronized eras + combined-vote accuracy
Four user-reported/requested items, one root cause chain:
1) DEAD CONTROL PANEL in AI_HYBRID mode. All members posted custom event
id 1 and handled id 1001, and CExpertCustom broadcasts every chart
event to every filter - so each posted event ran a train chunk in ALL
N members (N*N chunks per round) and the chart thread never idled
long enough to deliver clicks/drags. profiling.csv: 99.45% of time in
OnChartEventHandler. Fix: per-instance study-event ids
(STUDY_EVENT_ID_BASE + construction order, offset above the Controls
library's ON_* codes - id 1 was also ON_DBL_CLICK, so panel
double-clicks fired training chunks). ArmStudyEvent() is the single
post site; lost-event watchdog replaces the accidental
sibling-clears-my-flag rescue.
2) WARM-UP DUPLICATION. The auto-tune sweep is deterministic over
identical features/labels, and it ends in the full MI diagnostic
suite, which the MI-share gate never intercepted on the sweep path -
four members ran four identical ~36s sweep+report blocks. First
member publishes outcome (g_ensembleChartTuneDone/Installed/Settings);
the rest apply it and skip both.
3) DEINIT STRANDED PANEL+ARROWS (user repro 18:52). Root cause from the
log: the 4,500ms budget runs from MetaTrader's stop REQUEST - a heavy
autosave in flight ate it, OnDeinit got ~430ms and died in the first
member's arrow persist ("Abnormal termination" 432ms in). Fix: early
visible-UI sweep (native prefix deletes for status/panel/dialog)
right after ClearStatusLabel, and a fast path for still-training
models - their arrows are re-rendered every era, so they get one bulk
purge instead of scan+atomic-write in the death window.
4) ENSEMBLE FEATURES (user requests): era BARRIER - members advance era
by era together; a member ahead of the slowest still-training member
declines Train() calls and its chunk budget is donated
(TRAIN_TIME_BUDGET_MS = 120/activeTrainers, UI headroom constant).
COMBINED-VOTE OOS SCORE - each member's pass-3 scan contributes its
adjusted per-bar decision (0.0 on abstain) to a shared row buffer;
the last member to finish the era scores the averaged vote vs the
mirrored Min_Vote_Open against the same target-before-stop outcomes
members grade themselves on, publishing an "Ensemble vote" line on
the aggregated panel. Member headlines now carry their lifetime win
rate with break-even.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-16 19:06:04 -04:00
|
|
|
{
|
2026-08-15 16:54:43 -04:00
|
|
|
simpleLive = statusPlain + " -> " + liveSigPlain;
|
fix(ensemble): responsive panel + synchronized eras + combined-vote accuracy
Four user-reported/requested items, one root cause chain:
1) DEAD CONTROL PANEL in AI_HYBRID mode. All members posted custom event
id 1 and handled id 1001, and CExpertCustom broadcasts every chart
event to every filter - so each posted event ran a train chunk in ALL
N members (N*N chunks per round) and the chart thread never idled
long enough to deliver clicks/drags. profiling.csv: 99.45% of time in
OnChartEventHandler. Fix: per-instance study-event ids
(STUDY_EVENT_ID_BASE + construction order, offset above the Controls
library's ON_* codes - id 1 was also ON_DBL_CLICK, so panel
double-clicks fired training chunks). ArmStudyEvent() is the single
post site; lost-event watchdog replaces the accidental
sibling-clears-my-flag rescue.
2) WARM-UP DUPLICATION. The auto-tune sweep is deterministic over
identical features/labels, and it ends in the full MI diagnostic
suite, which the MI-share gate never intercepted on the sweep path -
four members ran four identical ~36s sweep+report blocks. First
member publishes outcome (g_ensembleChartTuneDone/Installed/Settings);
the rest apply it and skip both.
3) DEINIT STRANDED PANEL+ARROWS (user repro 18:52). Root cause from the
log: the 4,500ms budget runs from MetaTrader's stop REQUEST - a heavy
autosave in flight ate it, OnDeinit got ~430ms and died in the first
member's arrow persist ("Abnormal termination" 432ms in). Fix: early
visible-UI sweep (native prefix deletes for status/panel/dialog)
right after ClearStatusLabel, and a fast path for still-training
models - their arrows are re-rendered every era, so they get one bulk
purge instead of scan+atomic-write in the death window.
4) ENSEMBLE FEATURES (user requests): era BARRIER - members advance era
by era together; a member ahead of the slowest still-training member
declines Train() calls and its chunk budget is donated
(TRAIN_TIME_BUDGET_MS = 120/activeTrainers, UI headroom constant).
COMBINED-VOTE OOS SCORE - each member's pass-3 scan contributes its
adjusted per-bar decision (0.0 on abstain) to a shared row buffer;
the last member to finish the era scores the averaged vote vs the
mirrored Min_Vote_Open against the same target-before-stop outcomes
members grade themselves on, publishing an "Ensemble vote" line on
the aggregated panel. Member headlines now carry their lifetime win
rate with break-even.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-16 19:06:04 -04:00
|
|
|
//--- one-line accuracy on the member headline (user request 2026-08-16: the solo panel
|
2026-08-22 00:25:52 -04:00
|
|
|
//--- shows accuracy, the ensemble panel did not).
|
fix(ensemble): responsive panel + synchronized eras + combined-vote accuracy
Four user-reported/requested items, one root cause chain:
1) DEAD CONTROL PANEL in AI_HYBRID mode. All members posted custom event
id 1 and handled id 1001, and CExpertCustom broadcasts every chart
event to every filter - so each posted event ran a train chunk in ALL
N members (N*N chunks per round) and the chart thread never idled
long enough to deliver clicks/drags. profiling.csv: 99.45% of time in
OnChartEventHandler. Fix: per-instance study-event ids
(STUDY_EVENT_ID_BASE + construction order, offset above the Controls
library's ON_* codes - id 1 was also ON_DBL_CLICK, so panel
double-clicks fired training chunks). ArmStudyEvent() is the single
post site; lost-event watchdog replaces the accidental
sibling-clears-my-flag rescue.
2) WARM-UP DUPLICATION. The auto-tune sweep is deterministic over
identical features/labels, and it ends in the full MI diagnostic
suite, which the MI-share gate never intercepted on the sweep path -
four members ran four identical ~36s sweep+report blocks. First
member publishes outcome (g_ensembleChartTuneDone/Installed/Settings);
the rest apply it and skip both.
3) DEINIT STRANDED PANEL+ARROWS (user repro 18:52). Root cause from the
log: the 4,500ms budget runs from MetaTrader's stop REQUEST - a heavy
autosave in flight ate it, OnDeinit got ~430ms and died in the first
member's arrow persist ("Abnormal termination" 432ms in). Fix: early
visible-UI sweep (native prefix deletes for status/panel/dialog)
right after ClearStatusLabel, and a fast path for still-training
models - their arrows are re-rendered every era, so they get one bulk
purge instead of scan+atomic-write in the death window.
4) ENSEMBLE FEATURES (user requests): era BARRIER - members advance era
by era together; a member ahead of the slowest still-training member
declines Train() calls and its chunk budget is donated
(TRAIN_TIME_BUDGET_MS = 120/activeTrainers, UI headroom constant).
COMBINED-VOTE OOS SCORE - each member's pass-3 scan contributes its
adjusted per-bar decision (0.0 on abstain) to a shared row buffer;
the last member to finish the era scores the averaged vote vs the
mirrored Min_Vote_Open against the same target-before-stop outcomes
members grade themselves on, publishing an "Ensemble vote" line on
the aggregated panel. Member headlines now carry their lifetime win
rate with break-even.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-16 19:06:04 -04:00
|
|
|
if(m_cumOosTotal > 0)
|
feat(target): delete the barrier/geometry stack - the label is the verdict
Step 3 of the swing-pivot plan, whole-hog. The swing label is now the ONE
target and the era verdict is precision + recall per class against the
label's own base rate - no win rate, no break-even, no expectancy, no
geometry anywhere in training.
DELETED
- Expert/Excursion/ (4), Expert/BarrierHorizon/ (4), GeometrySweep,
FirstPassageLadder, Labeling/TripleBarrier.mqh (CLabelOverlap survives
in Labeling/LabelOverlap.mqh), 3 test EAs.
- TripleBarrierLabel + walk, fractal label, geometry derivation/scan/
adoption, exit-policy replay, excursion MI targets, the drift verdict
(DIRECTION_INTELLIGENT), the recall floor, balanced-accuracy telemetry,
the barrier defines, the .cfg geometry adopt (slots kept as zeros for
the positional layout), the derived-geometry live-order override.
- TRAINING_TARGET input/enum: direction models are always swing; META2
re-keys the meta head onto label agreement (descriptor loses its two
geometry slots).
REWORKED
- Labels.mqh (1795 -> ~370 lines): AdvanceSwingLabelState with
FINALITY-GATED CACHING - an unresolved bar (pivot pair uncommitted) is
never cached, so it can never freeze as a false Neutral; training,
calibration, OOS scoring and online learning all skip unresolved bars.
- SDeployVerdict: significance-only; SOosTally chance = larger
directional class share; pooled gate poolability = timeframe (record v2).
- Purge/embargo/declustering gaps: the measured mean label resolution
lag (LabelResolutionBars), not a barrier horizon.
- Pool purge key + backfill DB rows: marked at the bar the label
resolved on (m_labelResolveAge), not a fabricated barrier touch.
- Online learning frontier: finality, not a horizon delay.
- m_bestBalancedOos -> m_bestSelectionScore, m_erasSinceBestBalanced ->
m_erasSinceBest, ensemble vote outcome arrays -> label arrays.
STEP 4 folded in: Entry_Multiplier / SL_Mode / TP_Mode / tradingdirection
are inputs again - trade management is the tester GA's search space.
Fingerprints: every direction model re-keys (TGT:SWG1 now unconditional,
CUT token gone); META1 -> META2. Full retrain, as planned.
Compile-verified in _claude_stage: Warrior_EA + both surviving test EAs,
0 errors, 0 warnings each.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-24 20:42:31 -04:00
|
|
|
simpleLive += StringFormat(" | precision %d%%", (int)MathRound(m_cumOosCorrect * 100.0 / m_cumOosTotal));
|
fix(ensemble): responsive panel + synchronized eras + combined-vote accuracy
Four user-reported/requested items, one root cause chain:
1) DEAD CONTROL PANEL in AI_HYBRID mode. All members posted custom event
id 1 and handled id 1001, and CExpertCustom broadcasts every chart
event to every filter - so each posted event ran a train chunk in ALL
N members (N*N chunks per round) and the chart thread never idled
long enough to deliver clicks/drags. profiling.csv: 99.45% of time in
OnChartEventHandler. Fix: per-instance study-event ids
(STUDY_EVENT_ID_BASE + construction order, offset above the Controls
library's ON_* codes - id 1 was also ON_DBL_CLICK, so panel
double-clicks fired training chunks). ArmStudyEvent() is the single
post site; lost-event watchdog replaces the accidental
sibling-clears-my-flag rescue.
2) WARM-UP DUPLICATION. The auto-tune sweep is deterministic over
identical features/labels, and it ends in the full MI diagnostic
suite, which the MI-share gate never intercepted on the sweep path -
four members ran four identical ~36s sweep+report blocks. First
member publishes outcome (g_ensembleChartTuneDone/Installed/Settings);
the rest apply it and skip both.
3) DEINIT STRANDED PANEL+ARROWS (user repro 18:52). Root cause from the
log: the 4,500ms budget runs from MetaTrader's stop REQUEST - a heavy
autosave in flight ate it, OnDeinit got ~430ms and died in the first
member's arrow persist ("Abnormal termination" 432ms in). Fix: early
visible-UI sweep (native prefix deletes for status/panel/dialog)
right after ClearStatusLabel, and a fast path for still-training
models - their arrows are re-rendered every era, so they get one bulk
purge instead of scan+atomic-write in the death window.
4) ENSEMBLE FEATURES (user requests): era BARRIER - members advance era
by era together; a member ahead of the slowest still-training member
declines Train() calls and its chunk budget is donated
(TRAIN_TIME_BUDGET_MS = 120/activeTrainers, UI headroom constant).
COMBINED-VOTE OOS SCORE - each member's pass-3 scan contributes its
adjusted per-bar decision (0.0 on abstain) to a shared row buffer;
the last member to finish the era scores the averaged vote vs the
mirrored Min_Vote_Open against the same target-before-stop outcomes
members grade themselves on, publishing an "Ensemble vote" line on
the aggregated panel. Member headlines now carry their lifetime win
rate with break-even.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-16 19:06:04 -04:00
|
|
|
}
|
2026-08-25 22:51:50 -04:00
|
|
|
else
|
|
|
|
|
{
|
|
|
|
|
simpleLive = DisplayName() + " - " + statusPlain + "\n";
|
|
|
|
|
//--- Only show the accuracy line once at least one signal has been validated (compounded
|
|
|
|
|
//--- counts persist across restarts, so a deployed model shows real numbers immediately,
|
|
|
|
|
//--- not "measuring").
|
|
|
|
|
if(m_cumIsTotal > 0 || m_cumOosTotal > 0)
|
|
|
|
|
simpleLive += ComputeCompoundedAccuracyLine() + "\n";
|
|
|
|
|
}
|
2026-08-15 16:54:43 -04:00
|
|
|
PublishStatus(simpleLive);
|
2026-08-01 11:27:28 -04:00
|
|
|
return;
|
|
|
|
|
}
|
|
|
|
|
string completeText = onlineActive ? "Complete - live (adapting to new bars)" : "Complete - ready for live (inference)";
|
|
|
|
|
string trainingState = m_trainingStopRequested
|
|
|
|
|
? (m_trainingComplete ? completeText : "Stopped - resumable (weights kept)")
|
|
|
|
|
: (m_trainingPaused ? "Paused" : (m_trainingComplete ? completeText : "In progress"));
|
|
|
|
|
//--- same "Forecast: <signal> -> <value>" line the active training loop's status label ends on
|
|
|
|
|
//--- (see the classLine-terminated StringFormat below), instead of a raw bEventStudy/dPrevSignal/
|
|
|
|
|
//--- dtStudied debug dump - this is what stays on screen once training stops/pauses/completes.
|
2026-08-15 16:54:43 -04:00
|
|
|
PublishStatus(StringFormat(
|
|
|
|
|
ID + " : Era %d -> Training %s\n" +
|
|
|
|
|
"Forecast: %s -> %.2f",
|
|
|
|
|
m_eraCount, trainingState,
|
|
|
|
|
EnumToString(DoubleToSignal(dPrevSignal)), dPrevSignal));
|
2026-08-01 11:27:28 -04:00
|
|
|
}
|
|
|
|
|
}
|
|
|
|
|
//+------------------------------------------------------------------+
|
|
|
|
|
//| Timer-driven equivalent of OnTickHandler()'s scheduling, called |
|
|
|
|
|
//| from Warrior_EA.mq5's always-on OnTimer() so training keeps |
|
|
|
|
|
//| progressing purely on wall-clock time - no dependency on ticks, |
|
|
|
|
|
//| which simply don't arrive while the market is closed. |
|
|
|
|
|
//+------------------------------------------------------------------+
|
|
|
|
|
void CExpertSignalAIBase::PollTraining(void)
|
|
|
|
|
{
|
2026-08-22 00:25:52 -04:00
|
|
|
//--- Drain a slice of the queued chart-arrow restore FIRST, and skip training work on any tick
|
|
|
|
|
//--- where restoring is still in flight.
|
|
|
|
|
//--- STOP BEFORE ANY OF IT. Drawing arrows for a chart that is unloading is the worst case of the
|
|
|
|
|
//--- three: it lengthens the very sweep that has to finish.
|
fix(shutdown): make ExitPolicy public, and stop every long loop the moment MT5 asks
Two things, one of which was a compile error.
1. ExitPolicy() was declared in the protected block but is pushed in from
Warrior_EA.mq5:770. Moved to public beside the other EA-facing setters.
2. Chart objects surviving OnDeinit. The 4,500 ms teardown budget is measured
from the STOP REQUEST, not from OnDeinit's first line, and OnDeinit cannot
begin until whatever is in flight returns - so a scan still running after
_StopFlag is raised does not delay the cleanup, it SPENDS it, and the purge
never gets its turn.
New CExpertSignalAIBase::ShutdownRequested() = IsStopped() || m_shutdownInProgress.
Deliberately NOT m_trainingStopRequested: that latches, and a latched flag
would permanently disable scans that must run again on the next Start.
Guarded, longest first:
- TuneIndicatorsByFilter - per candidate, restoring the OPERATOR's settings
on the way out (best[] is mutated in place; the tuner otherwise keeps the
last trial's parameters, which nothing chose).
- ReportBarrierGeometryScan - per pairing, breaking to ONE restore point so
m_barrierScanLiveLabels can never be left true (that makes ComputeLabelForBar
read the last candidate's multiples as the configured geometry).
- ReportFeatureLabelInformation / ReportExcursionInformation / lag profile -
nulls ABANDON rather than truncate: fewer draws is not a smaller null, it
is a wrong one, and p shifts toward significance. m_dirEvidence staying
false is the safe direction.
- SimulateExitPolicyOutcomes - zeroes its accumulators so the divergence line
is dropped instead of latching a partial expectancy as the run's only report.
- ReportGeometryExpectancyScan - per ladder rung.
- HttpGet - one choke point for up to a dozen blocking WebRequests per
first-pass Update(). An in-flight request cannot be cancelled; refusing to
start another is the whole remedy.
- PollTraining, OnChartEventHandler's study event, TuneIndicatorsAndTrain -
entry points, so a queued event cannot open an era during teardown.
TuneIndicatorsAndTrain's guard is the first statement, ahead of the
m_tuneFilterDone / g_ensembleChartTuneDone latches.
- OnTick / OnTimer / OnChartEvent.
Training's own bar loops already honoured this (pass 1 per bar, passes 2/2.5/3
yield on a 120 ms budget); the warm-up scans did not, and they are the longest
uninterruptible stretches the EA has.
StopTraining() is unchanged: the operator's Stop still finalises synchronously.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-17 17:03:11 -04:00
|
|
|
if(ShutdownRequested())
|
|
|
|
|
return;
|
refactor(chart): ChartUI is a real collaborator, not a raw-include partial (S2)
Expert/AIBase/ChartUI.mqh was 869 lines of method bodies of
CExpertSignalAIBase, #included after the class declaration - free to touch
any of its ~500 members. First of the eleven AIBase/*.mqh partials to come
out (fewest inbound edges - see the SOLID campaign session order), using
the same view+adapter shape already proven for CTrainingDataView.
CChartView (Expert/Chart/IChartView.mqh) is the abstract read/behaviour
surface a chart-rendering collaborator needs - identity, bar/model access,
the prediction cache, and the training/vote/meta scalars the panel and HUD
line summarise. CAIBaseChartView is the adapter the signal owns and binds
to itself (MQL5 gives a class exactly one base, so CExpertSignalAIBase
cannot implement the view directly). CChartUI is the real collaborator: it
owns the arrow-restore queue, the rescan queue/tally, the last-arrows-saved
count and the purge-mismatch latch as its own fields (verified via grep to
be touched nowhere else in Expert/), and reaches everything else - including
StartChartSignalRescan, moved in from its old inline home in the header
since it drives the exact same rescan state machine AdvanceChartSignalRescan
drains - through the view.
m_arrowSignalCache and m_signalClusterWindow stay on the signal: Training.mqh
writes the cache directly every era and the training-data view already reads
it, so moving it would mean rewriting Training.mqh's write sites too - out of
scope here. CChartUI reaches it through four bounds-checked accessors instead
of a raw member poke. All 10 public methods keep their exact signatures and
become one-line forwards on the signal, so no other file's call sites change
except Training.mqh's one era-end status refresh, which now reads
RefreshStatusLabel() rather than reaching into CChartUI's now-private
last-displayed-neuron cache directly.
Verified structurally, not compiled (never compile - the operator does, in
MetaEditor): brace balance checked on every touched/new file against HEAD,
and the view/adapter/impl method lists cross-diffed to confirm all 59
accessors match 1:1 across the interface, the adapter declaration and the
adapter body.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-08-23 20:03:52 -04:00
|
|
|
if(m_chartUI.ArrowRestorePending())
|
2026-08-01 11:27:28 -04:00
|
|
|
{
|
|
|
|
|
AdvanceChartSignalRestore();
|
|
|
|
|
return;
|
|
|
|
|
}
|
|
|
|
|
//--- Same one-thread reasoning as the arrow restore above: a manual rescan (Show Signals) also competes
|
|
|
|
|
//--- for the single MQL5 thread, and its per-bar feedForward is real compute rather than a cheap object
|
|
|
|
|
//--- write, so it must finish its own slices before training resumes rather than interleaving with it.
|
refactor(chart): ChartUI is a real collaborator, not a raw-include partial (S2)
Expert/AIBase/ChartUI.mqh was 869 lines of method bodies of
CExpertSignalAIBase, #included after the class declaration - free to touch
any of its ~500 members. First of the eleven AIBase/*.mqh partials to come
out (fewest inbound edges - see the SOLID campaign session order), using
the same view+adapter shape already proven for CTrainingDataView.
CChartView (Expert/Chart/IChartView.mqh) is the abstract read/behaviour
surface a chart-rendering collaborator needs - identity, bar/model access,
the prediction cache, and the training/vote/meta scalars the panel and HUD
line summarise. CAIBaseChartView is the adapter the signal owns and binds
to itself (MQL5 gives a class exactly one base, so CExpertSignalAIBase
cannot implement the view directly). CChartUI is the real collaborator: it
owns the arrow-restore queue, the rescan queue/tally, the last-arrows-saved
count and the purge-mismatch latch as its own fields (verified via grep to
be touched nowhere else in Expert/), and reaches everything else - including
StartChartSignalRescan, moved in from its old inline home in the header
since it drives the exact same rescan state machine AdvanceChartSignalRescan
drains - through the view.
m_arrowSignalCache and m_signalClusterWindow stay on the signal: Training.mqh
writes the cache directly every era and the training-data view already reads
it, so moving it would mean rewriting Training.mqh's write sites too - out of
scope here. CChartUI reaches it through four bounds-checked accessors instead
of a raw member poke. All 10 public methods keep their exact signatures and
become one-line forwards on the signal, so no other file's call sites change
except Training.mqh's one era-end status refresh, which now reads
RefreshStatusLabel() rather than reaching into CChartUI's now-private
last-displayed-neuron cache directly.
Verified structurally, not compiled (never compile - the operator does, in
MetaEditor): brace balance checked on every touched/new file against HEAD,
and the view/adapter/impl method lists cross-diffed to confirm all 59
accessors match 1:1 across the interface, the adapter declaration and the
adapter body.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-08-23 20:03:52 -04:00
|
|
|
if(m_chartUI.RescanPending())
|
2026-08-01 11:27:28 -04:00
|
|
|
{
|
|
|
|
|
AdvanceChartSignalRescan();
|
|
|
|
|
return;
|
|
|
|
|
}
|
feat(vote): replay pass rebuilds a deployed model's ladder without retraining
The previous commit persisted the tier ladder, which fixes this going
forward but did nothing for models whose .stats predates WST7 - they
still had to retrain to mint one. They never did. Every number a
converged model needs in order to vote is a pure function of weights
already on disk plus labels derivable from the chart, so replay them:
stage 1 build the label cache (existing chunked prebuild)
stage 2 rescan history (existing chunked rescan, deployed net)
stage 3 score + rank + persist (one walk over two arrays)
ScoreReplayFromCache() walks m_arrowSignalCache against
m_labelCacheBuy/Sell, fills the same m_oosTierFired/Hits and per-class
totals pass 3 fills, and hands them to RankTiersFromOos() - deliberately
feeding the existing ranker rather than reimplementing it. The shrinkage,
the chance reference and the module trust weight are subtle enough that a
second copy would drift, and a ladder measured by a slightly different
rule would be silently incomparable with every ladder training produced.
AdvanceDeployedRebuild() sequences the three stages off the timer. It has
to be a sequence: stages 1 and 2 are each minutes of work draining in
time-boxed slices, and stage 2's output is meaningless until stage 1 has
labels to score against. The previous version ran the rescan with no
labels at all, which is why it could only ever rebuild arrows and never
the ladder - the thing actually blocking the vote.
The result is written to .stats immediately. The failure being repaired
is state that lived in memory and was never written down; recomputing it
and not saving it would repeat that exactly.
Also routes every rescan completion through one hook, so there is a
single place that knows what a finished rescan means - republish for a
manual one, score and rank for a rebuild.
Compile-verified in _claude_stage: 0 errors, 0 warnings.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-25 15:01:24 -04:00
|
|
|
if(m_isInitialized && AdvanceDeployedRebuild())
|
|
|
|
|
return;
|
2026-08-01 11:27:28 -04:00
|
|
|
if(m_isInitialized)
|
|
|
|
|
ScheduleTrainingIfNeeded();
|
|
|
|
|
}
|
|
|
|
|
//+------------------------------------------------------------------+
|
feat(vote): replay pass rebuilds a deployed model's ladder without retraining
The previous commit persisted the tier ladder, which fixes this going
forward but did nothing for models whose .stats predates WST7 - they
still had to retrain to mint one. They never did. Every number a
converged model needs in order to vote is a pure function of weights
already on disk plus labels derivable from the chart, so replay them:
stage 1 build the label cache (existing chunked prebuild)
stage 2 rescan history (existing chunked rescan, deployed net)
stage 3 score + rank + persist (one walk over two arrays)
ScoreReplayFromCache() walks m_arrowSignalCache against
m_labelCacheBuy/Sell, fills the same m_oosTierFired/Hits and per-class
totals pass 3 fills, and hands them to RankTiersFromOos() - deliberately
feeding the existing ranker rather than reimplementing it. The shrinkage,
the chance reference and the module trust weight are subtle enough that a
second copy would drift, and a ladder measured by a slightly different
rule would be silently incomparable with every ladder training produced.
AdvanceDeployedRebuild() sequences the three stages off the timer. It has
to be a sequence: stages 1 and 2 are each minutes of work draining in
time-boxed slices, and stage 2's output is meaningless until stage 1 has
labels to score against. The previous version ran the rescan with no
labels at all, which is why it could only ever rebuild arrows and never
the ladder - the thing actually blocking the vote.
The result is written to .stats immediately. The failure being repaired
is state that lived in memory and was never written down; recomputing it
and not saving it would repeat that exactly.
Also routes every rescan completion through one hook, so there is a
single place that knows what a finished rescan means - republish for a
manual one, score and rank for a rebuild.
Compile-verified in _claude_stage: 0 errors, 0 warnings.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-25 15:01:24 -04:00
|
|
|
//| A DEPLOYED MODEL REBUILDS ITS OWN VOTE, WITHOUT RETRAINING. |
|
|
|
|
|
//| |
|
|
|
|
|
//| Everything a converged model needs in order to vote - the tier |
|
|
|
|
|
//| ladder, the module trust weight, the overlay snapshot the arrows |
|
|
|
|
|
//| are drawn from - is produced by a completed pass 3 and by nothing |
|
|
|
|
|
//| else. A converged model runs no passes. So a model that lost those |
|
|
|
|
|
//| (a .stats predating WST7, a wipe, a fresh deploy from a build that |
|
|
|
|
|
//| never stored them) was permanently mute: no vote, no trades, no |
|
|
|
|
|
//| arrows, and a win rate stuck on "measuring...". |
|
|
|
|
|
//| |
|
|
|
|
|
//| It never needed a retrain. Every one of those numbers is a pure |
|
fix(replay): resolve labels inline - the prebuilt cache's window never overlapped the rescan
The 15:13 session proved the replay pass ran end-to-end on all 24 models
and scored ZERO labelled bars on every one of them, while each rescan sat
on ~5000 scored predictions (~2755 Buy / ~2232 Sell). The two windows
never overlapped:
StartLabelCachePrebuild deliberately keeps a CONVERGED model's
dtStudied watermark (it gates inference recency and must not move), so
the prebuild's window was the handful of bars since the last studied
bar - all with uncommitted pivots, hence "label cache pre-built -
Buy: 0 | Sell: 0 | Neutral: 0" on every member.
The label never needed a cache. SwingPivotDirectionLabel(idx) is a pure
function of the ZigZag/Close/ATR buffers the rescan itself refreshes over
exactly the scoring window, and m_lastLabelLifespan == 0 is its own
unresolved flag - the same finality gate the cache applies, applied
directly. ScoreReplayFromCache now resolves each bar's label inline and
the label-prebuild stage is deleted from the rebuild state machine
outright; going through a cache built for a different window was
indirection that changed the answer.
Also splits the empty-result diagnostics: "no resolved labels" (a
windowing/data fault) is now distinguished from "labels present, every
call Neutral" (a calibration verdict). The first version reported the
second message for both, which mislabelled this very bug as a calibration
outcome in the same breath as reporting scored=0.
Honest limitation, stated in the code too: the replay window includes
bars the model trained on, so a replay-minted ladder is measured partly
in-sample and will read stronger than a holdout-measured one. It is
replaced by the genuine article at the next completed scoring pass; until
then it is what makes a restarted deployed model able to vote at all.
Compile-verified in _claude_stage: 0 errors, 0 warnings.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-25 15:22:01 -04:00
|
|
|
//| function of weights already on disk plus labels derivable from the |
|
|
|
|
|
//| chart, so this replays them: run the deployed net over history |
|
|
|
|
|
//| (the existing chunked rescan), then score each prediction against |
|
|
|
|
|
//| the swing label RESOLVED INLINE for the same bar and rank the |
|
|
|
|
|
//| ladder from the result (ScoreReplayFromCache, via the rescan's |
|
|
|
|
|
//| completion hook). |
|
feat(vote): replay pass rebuilds a deployed model's ladder without retraining
The previous commit persisted the tier ladder, which fixes this going
forward but did nothing for models whose .stats predates WST7 - they
still had to retrain to mint one. They never did. Every number a
converged model needs in order to vote is a pure function of weights
already on disk plus labels derivable from the chart, so replay them:
stage 1 build the label cache (existing chunked prebuild)
stage 2 rescan history (existing chunked rescan, deployed net)
stage 3 score + rank + persist (one walk over two arrays)
ScoreReplayFromCache() walks m_arrowSignalCache against
m_labelCacheBuy/Sell, fills the same m_oosTierFired/Hits and per-class
totals pass 3 fills, and hands them to RankTiersFromOos() - deliberately
feeding the existing ranker rather than reimplementing it. The shrinkage,
the chance reference and the module trust weight are subtle enough that a
second copy would drift, and a ladder measured by a slightly different
rule would be silently incomparable with every ladder training produced.
AdvanceDeployedRebuild() sequences the three stages off the timer. It has
to be a sequence: stages 1 and 2 are each minutes of work draining in
time-boxed slices, and stage 2's output is meaningless until stage 1 has
labels to score against. The previous version ran the rescan with no
labels at all, which is why it could only ever rebuild arrows and never
the ladder - the thing actually blocking the vote.
The result is written to .stats immediately. The failure being repaired
is state that lived in memory and was never written down; recomputing it
and not saving it would repeat that exactly.
Also routes every rescan completion through one hook, so there is a
single place that knows what a finished rescan means - republish for a
manual one, score and rank for a rebuild.
Compile-verified in _claude_stage: 0 errors, 0 warnings.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-25 15:01:24 -04:00
|
|
|
//| |
|
fix(replay): resolve labels inline - the prebuilt cache's window never overlapped the rescan
The 15:13 session proved the replay pass ran end-to-end on all 24 models
and scored ZERO labelled bars on every one of them, while each rescan sat
on ~5000 scored predictions (~2755 Buy / ~2232 Sell). The two windows
never overlapped:
StartLabelCachePrebuild deliberately keeps a CONVERGED model's
dtStudied watermark (it gates inference recency and must not move), so
the prebuild's window was the handful of bars since the last studied
bar - all with uncommitted pivots, hence "label cache pre-built -
Buy: 0 | Sell: 0 | Neutral: 0" on every member.
The label never needed a cache. SwingPivotDirectionLabel(idx) is a pure
function of the ZigZag/Close/ATR buffers the rescan itself refreshes over
exactly the scoring window, and m_lastLabelLifespan == 0 is its own
unresolved flag - the same finality gate the cache applies, applied
directly. ScoreReplayFromCache now resolves each bar's label inline and
the label-prebuild stage is deleted from the rebuild state machine
outright; going through a cache built for a different window was
indirection that changed the answer.
Also splits the empty-result diagnostics: "no resolved labels" (a
windowing/data fault) is now distinguished from "labels present, every
call Neutral" (a calibration verdict). The first version reported the
second message for both, which mislabelled this very bug as a calibration
outcome in the same breath as reporting scored=0.
Honest limitation, stated in the code too: the replay window includes
bars the model trained on, so a replay-minted ladder is measured partly
in-sample and will read stronger than a holdout-measured one. It is
replaced by the genuine article at the next completed scoring pass; until
then it is what makes a restarted deployed model able to vote at all.
Compile-verified in _claude_stage: 0 errors, 0 warnings.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-25 15:22:01 -04:00
|
|
|
//| There is deliberately NO label-prebuild stage any more. The first |
|
|
|
|
|
//| version had one, and it is exactly why that version scored zero |
|
|
|
|
|
//| bars on 24/24 models: the prebuild's window is anchored at |
|
|
|
|
|
//| dtStudied, which a converged model keeps at its recency watermark |
|
|
|
|
|
//| - so the "label cache" covered a handful of just-closed bars whose |
|
|
|
|
|
//| pivots cannot have committed yet, while the rescan sat on five |
|
|
|
|
|
//| thousand scored predictions it could never be matched against. |
|
|
|
|
|
//| The label is a pure function of buffers the rescan itself |
|
|
|
|
|
//| refreshes; going through a cache built for a different window was |
|
|
|
|
|
//| indirection that changed the answer. |
|
feat(vote): replay pass rebuilds a deployed model's ladder without retraining
The previous commit persisted the tier ladder, which fixes this going
forward but did nothing for models whose .stats predates WST7 - they
still had to retrain to mint one. They never did. Every number a
converged model needs in order to vote is a pure function of weights
already on disk plus labels derivable from the chart, so replay them:
stage 1 build the label cache (existing chunked prebuild)
stage 2 rescan history (existing chunked rescan, deployed net)
stage 3 score + rank + persist (one walk over two arrays)
ScoreReplayFromCache() walks m_arrowSignalCache against
m_labelCacheBuy/Sell, fills the same m_oosTierFired/Hits and per-class
totals pass 3 fills, and hands them to RankTiersFromOos() - deliberately
feeding the existing ranker rather than reimplementing it. The shrinkage,
the chance reference and the module trust weight are subtle enough that a
second copy would drift, and a ladder measured by a slightly different
rule would be silently incomparable with every ladder training produced.
AdvanceDeployedRebuild() sequences the three stages off the timer. It has
to be a sequence: stages 1 and 2 are each minutes of work draining in
time-boxed slices, and stage 2's output is meaningless until stage 1 has
labels to score against. The previous version ran the rescan with no
labels at all, which is why it could only ever rebuild arrows and never
the ladder - the thing actually blocking the vote.
The result is written to .stats immediately. The failure being repaired
is state that lived in memory and was never written down; recomputing it
and not saving it would repeat that exactly.
Also routes every rescan completion through one hook, so there is a
single place that knows what a finished rescan means - republish for a
manual one, score and rank for a rebuild.
Compile-verified in _claude_stage: 0 errors, 0 warnings.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-25 15:01:24 -04:00
|
|
|
//+------------------------------------------------------------------+
|
|
|
|
|
bool CExpertSignalAIBase::AdvanceDeployedRebuild(void)
|
|
|
|
|
{
|
fix(replay): resolve labels inline - the prebuilt cache's window never overlapped the rescan
The 15:13 session proved the replay pass ran end-to-end on all 24 models
and scored ZERO labelled bars on every one of them, while each rescan sat
on ~5000 scored predictions (~2755 Buy / ~2232 Sell). The two windows
never overlapped:
StartLabelCachePrebuild deliberately keeps a CONVERGED model's
dtStudied watermark (it gates inference recency and must not move), so
the prebuild's window was the handful of bars since the last studied
bar - all with uncommitted pivots, hence "label cache pre-built -
Buy: 0 | Sell: 0 | Neutral: 0" on every member.
The label never needed a cache. SwingPivotDirectionLabel(idx) is a pure
function of the ZigZag/Close/ATR buffers the rescan itself refreshes over
exactly the scoring window, and m_lastLabelLifespan == 0 is its own
unresolved flag - the same finality gate the cache applies, applied
directly. ScoreReplayFromCache now resolves each bar's label inline and
the label-prebuild stage is deleted from the rebuild state machine
outright; going through a cache built for a different window was
indirection that changed the answer.
Also splits the empty-result diagnostics: "no resolved labels" (a
windowing/data fault) is now distinguished from "labels present, every
call Neutral" (a calibration verdict). The first version reported the
second message for both, which mislabelled this very bug as a calibration
outcome in the same breath as reporting scored=0.
Honest limitation, stated in the code too: the replay window includes
bars the model trained on, so a replay-minted ladder is measured partly
in-sample and will read stronger than a holdout-measured one. It is
replaced by the genuine article at the next completed scoring pass; until
then it is what makes a restarted deployed model able to vote at all.
Compile-verified in _claude_stage: 0 errors, 0 warnings.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-25 15:22:01 -04:00
|
|
|
if(m_deployedRebuildStage >= 2)
|
feat(vote): replay pass rebuilds a deployed model's ladder without retraining
The previous commit persisted the tier ladder, which fixes this going
forward but did nothing for models whose .stats predates WST7 - they
still had to retrain to mint one. They never did. Every number a
converged model needs in order to vote is a pure function of weights
already on disk plus labels derivable from the chart, so replay them:
stage 1 build the label cache (existing chunked prebuild)
stage 2 rescan history (existing chunked rescan, deployed net)
stage 3 score + rank + persist (one walk over two arrays)
ScoreReplayFromCache() walks m_arrowSignalCache against
m_labelCacheBuy/Sell, fills the same m_oosTierFired/Hits and per-class
totals pass 3 fills, and hands them to RankTiersFromOos() - deliberately
feeding the existing ranker rather than reimplementing it. The shrinkage,
the chance reference and the module trust weight are subtle enough that a
second copy would drift, and a ladder measured by a slightly different
rule would be silently incomparable with every ladder training produced.
AdvanceDeployedRebuild() sequences the three stages off the timer. It has
to be a sequence: stages 1 and 2 are each minutes of work draining in
time-boxed slices, and stage 2's output is meaningless until stage 1 has
labels to score against. The previous version ran the rescan with no
labels at all, which is why it could only ever rebuild arrows and never
the ladder - the thing actually blocking the vote.
The result is written to .stats immediately. The failure being repaired
is state that lived in memory and was never written down; recomputing it
and not saving it would repeat that exactly.
Also routes every rescan completion through one hook, so there is a
single place that knows what a finished rescan means - republish for a
manual one, score and rank for a rebuild.
Compile-verified in _claude_stage: 0 errors, 0 warnings.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-25 15:01:24 -04:00
|
|
|
return false;
|
|
|
|
|
//--- WHO NEEDS THIS: a converged model that cannot currently vote (no ladder) or cannot currently
|
|
|
|
|
//--- be drawn (no snapshot). Never in the tester, never for an inference-only run, and never while
|
|
|
|
|
//--- a training run is live - a training pass produces all of this itself, correctly, and racing it
|
|
|
|
|
//--- would have two writers on the same counters.
|
|
|
|
|
if(m_deployedRebuildStage == 0)
|
|
|
|
|
{
|
|
|
|
|
if(!m_trainingComplete || m_inferenceOnly || m_trainRunActive || m_trainingPaused
|
|
|
|
|
|| MQLInfoInteger(MQL_TESTER) || MQLInfoInteger(MQL_OPTIMIZATION) || MQLInfoInteger(MQL_FORWARD))
|
|
|
|
|
return false;
|
|
|
|
|
if(m_tiersSelfRanked && m_overlaySnapBars > 0)
|
|
|
|
|
{
|
fix(replay): resolve labels inline - the prebuilt cache's window never overlapped the rescan
The 15:13 session proved the replay pass ran end-to-end on all 24 models
and scored ZERO labelled bars on every one of them, while each rescan sat
on ~5000 scored predictions (~2755 Buy / ~2232 Sell). The two windows
never overlapped:
StartLabelCachePrebuild deliberately keeps a CONVERGED model's
dtStudied watermark (it gates inference recency and must not move), so
the prebuild's window was the handful of bars since the last studied
bar - all with uncommitted pivots, hence "label cache pre-built -
Buy: 0 | Sell: 0 | Neutral: 0" on every member.
The label never needed a cache. SwingPivotDirectionLabel(idx) is a pure
function of the ZigZag/Close/ATR buffers the rescan itself refreshes over
exactly the scoring window, and m_lastLabelLifespan == 0 is its own
unresolved flag - the same finality gate the cache applies, applied
directly. ScoreReplayFromCache now resolves each bar's label inline and
the label-prebuild stage is deleted from the rebuild state machine
outright; going through a cache built for a different window was
indirection that changed the answer.
Also splits the empty-result diagnostics: "no resolved labels" (a
windowing/data fault) is now distinguished from "labels present, every
call Neutral" (a calibration verdict). The first version reported the
second message for both, which mislabelled this very bug as a calibration
outcome in the same breath as reporting scored=0.
Honest limitation, stated in the code too: the replay window includes
bars the model trained on, so a replay-minted ladder is measured partly
in-sample and will read stronger than a holdout-measured one. It is
replaced by the genuine article at the next completed scoring pass; until
then it is what makes a restarted deployed model able to vote at all.
Compile-verified in _claude_stage: 0 errors, 0 warnings.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-25 15:22:01 -04:00
|
|
|
m_deployedRebuildStage = 2; // nothing missing - never look again
|
feat(vote): replay pass rebuilds a deployed model's ladder without retraining
The previous commit persisted the tier ladder, which fixes this going
forward but did nothing for models whose .stats predates WST7 - they
still had to retrain to mint one. They never did. Every number a
converged model needs in order to vote is a pure function of weights
already on disk plus labels derivable from the chart, so replay them:
stage 1 build the label cache (existing chunked prebuild)
stage 2 rescan history (existing chunked rescan, deployed net)
stage 3 score + rank + persist (one walk over two arrays)
ScoreReplayFromCache() walks m_arrowSignalCache against
m_labelCacheBuy/Sell, fills the same m_oosTierFired/Hits and per-class
totals pass 3 fills, and hands them to RankTiersFromOos() - deliberately
feeding the existing ranker rather than reimplementing it. The shrinkage,
the chance reference and the module trust weight are subtle enough that a
second copy would drift, and a ladder measured by a slightly different
rule would be silently incomparable with every ladder training produced.
AdvanceDeployedRebuild() sequences the three stages off the timer. It has
to be a sequence: stages 1 and 2 are each minutes of work draining in
time-boxed slices, and stage 2's output is meaningless until stage 1 has
labels to score against. The previous version ran the rescan with no
labels at all, which is why it could only ever rebuild arrows and never
the ladder - the thing actually blocking the vote.
The result is written to .stats immediately. The failure being repaired
is state that lived in memory and was never written down; recomputing it
and not saving it would repeat that exactly.
Also routes every rescan completion through one hook, so there is a
single place that knows what a finished rescan means - republish for a
manual one, score and rank for a rebuild.
Compile-verified in _claude_stage: 0 errors, 0 warnings.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-25 15:01:24 -04:00
|
|
|
return false;
|
|
|
|
|
}
|
|
|
|
|
Print(ID + ": deployed but " + (!m_tiersSelfRanked ? "WITHOUT A TIER LADDER (so it cannot vote)"
|
|
|
|
|
: "without a historical vote snapshot (so it cannot draw arrows)") +
|
|
|
|
|
" - replaying history against the deployed weights to rebuild it. No training will run;"
|
|
|
|
|
" these numbers are a function of the weights already on disk.");
|
|
|
|
|
m_deployedRebuildStage = 1;
|
|
|
|
|
}
|
fix(replay): resolve labels inline - the prebuilt cache's window never overlapped the rescan
The 15:13 session proved the replay pass ran end-to-end on all 24 models
and scored ZERO labelled bars on every one of them, while each rescan sat
on ~5000 scored predictions (~2755 Buy / ~2232 Sell). The two windows
never overlapped:
StartLabelCachePrebuild deliberately keeps a CONVERGED model's
dtStudied watermark (it gates inference recency and must not move), so
the prebuild's window was the handful of bars since the last studied
bar - all with uncommitted pivots, hence "label cache pre-built -
Buy: 0 | Sell: 0 | Neutral: 0" on every member.
The label never needed a cache. SwingPivotDirectionLabel(idx) is a pure
function of the ZigZag/Close/ATR buffers the rescan itself refreshes over
exactly the scoring window, and m_lastLabelLifespan == 0 is its own
unresolved flag - the same finality gate the cache applies, applied
directly. ScoreReplayFromCache now resolves each bar's label inline and
the label-prebuild stage is deleted from the rebuild state machine
outright; going through a cache built for a different window was
indirection that changed the answer.
Also splits the empty-result diagnostics: "no resolved labels" (a
windowing/data fault) is now distinguished from "labels present, every
call Neutral" (a calibration verdict). The first version reported the
second message for both, which mislabelled this very bug as a calibration
outcome in the same breath as reporting scored=0.
Honest limitation, stated in the code too: the replay window includes
bars the model trained on, so a replay-minted ladder is measured partly
in-sample and will read stronger than a holdout-measured one. It is
replaced by the genuine article at the next completed scoring pass; until
then it is what makes a restarted deployed model able to vote at all.
Compile-verified in _claude_stage: 0 errors, 0 warnings.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-25 15:22:01 -04:00
|
|
|
//--- Run the deployed net over the window. StartChartSignalRescan queues; the drain is handled by
|
|
|
|
|
//--- the RescanPending() branch above this in PollTraining, which is why returning true here is
|
|
|
|
|
//--- correct - the next slices go there, and completion arrives via OnChartRescanComplete.
|
|
|
|
|
if(m_deployedRebuildStage == 1 && !m_chartUI.RescanPending())
|
feat(vote): replay pass rebuilds a deployed model's ladder without retraining
The previous commit persisted the tier ladder, which fixes this going
forward but did nothing for models whose .stats predates WST7 - they
still had to retrain to mint one. They never did. Every number a
converged model needs in order to vote is a pure function of weights
already on disk plus labels derivable from the chart, so replay them:
stage 1 build the label cache (existing chunked prebuild)
stage 2 rescan history (existing chunked rescan, deployed net)
stage 3 score + rank + persist (one walk over two arrays)
ScoreReplayFromCache() walks m_arrowSignalCache against
m_labelCacheBuy/Sell, fills the same m_oosTierFired/Hits and per-class
totals pass 3 fills, and hands them to RankTiersFromOos() - deliberately
feeding the existing ranker rather than reimplementing it. The shrinkage,
the chance reference and the module trust weight are subtle enough that a
second copy would drift, and a ladder measured by a slightly different
rule would be silently incomparable with every ladder training produced.
AdvanceDeployedRebuild() sequences the three stages off the timer. It has
to be a sequence: stages 1 and 2 are each minutes of work draining in
time-boxed slices, and stage 2's output is meaningless until stage 1 has
labels to score against. The previous version ran the rescan with no
labels at all, which is why it could only ever rebuild arrows and never
the ladder - the thing actually blocking the vote.
The result is written to .stats immediately. The failure being repaired
is state that lived in memory and was never written down; recomputing it
and not saving it would repeat that exactly.
Also routes every rescan completion through one hook, so there is a
single place that knows what a finished rescan means - republish for a
manual one, score and rank for a rebuild.
Compile-verified in _claude_stage: 0 errors, 0 warnings.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-25 15:01:24 -04:00
|
|
|
{
|
fix(replay): resolve labels inline - the prebuilt cache's window never overlapped the rescan
The 15:13 session proved the replay pass ran end-to-end on all 24 models
and scored ZERO labelled bars on every one of them, while each rescan sat
on ~5000 scored predictions (~2755 Buy / ~2232 Sell). The two windows
never overlapped:
StartLabelCachePrebuild deliberately keeps a CONVERGED model's
dtStudied watermark (it gates inference recency and must not move), so
the prebuild's window was the handful of bars since the last studied
bar - all with uncommitted pivots, hence "label cache pre-built -
Buy: 0 | Sell: 0 | Neutral: 0" on every member.
The label never needed a cache. SwingPivotDirectionLabel(idx) is a pure
function of the ZigZag/Close/ATR buffers the rescan itself refreshes over
exactly the scoring window, and m_lastLabelLifespan == 0 is its own
unresolved flag - the same finality gate the cache applies, applied
directly. ScoreReplayFromCache now resolves each bar's label inline and
the label-prebuild stage is deleted from the rebuild state machine
outright; going through a cache built for a different window was
indirection that changed the answer.
Also splits the empty-result diagnostics: "no resolved labels" (a
windowing/data fault) is now distinguished from "labels present, every
call Neutral" (a calibration verdict). The first version reported the
second message for both, which mislabelled this very bug as a calibration
outcome in the same breath as reporting scored=0.
Honest limitation, stated in the code too: the replay window includes
bars the model trained on, so a replay-minted ladder is measured partly
in-sample and will read stronger than a holdout-measured one. It is
replaced by the genuine article at the next completed scoring pass; until
then it is what makes a restarted deployed model able to vote at all.
Compile-verified in _claude_stage: 0 errors, 0 warnings.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-25 15:22:01 -04:00
|
|
|
if(!StartChartSignalRescan())
|
feat(vote): replay pass rebuilds a deployed model's ladder without retraining
The previous commit persisted the tier ladder, which fixes this going
forward but did nothing for models whose .stats predates WST7 - they
still had to retrain to mint one. They never did. Every number a
converged model needs in order to vote is a pure function of weights
already on disk plus labels derivable from the chart, so replay them:
stage 1 build the label cache (existing chunked prebuild)
stage 2 rescan history (existing chunked rescan, deployed net)
stage 3 score + rank + persist (one walk over two arrays)
ScoreReplayFromCache() walks m_arrowSignalCache against
m_labelCacheBuy/Sell, fills the same m_oosTierFired/Hits and per-class
totals pass 3 fills, and hands them to RankTiersFromOos() - deliberately
feeding the existing ranker rather than reimplementing it. The shrinkage,
the chance reference and the module trust weight are subtle enough that a
second copy would drift, and a ladder measured by a slightly different
rule would be silently incomparable with every ladder training produced.
AdvanceDeployedRebuild() sequences the three stages off the timer. It has
to be a sequence: stages 1 and 2 are each minutes of work draining in
time-boxed slices, and stage 2's output is meaningless until stage 1 has
labels to score against. The previous version ran the rescan with no
labels at all, which is why it could only ever rebuild arrows and never
the ladder - the thing actually blocking the vote.
The result is written to .stats immediately. The failure being repaired
is state that lived in memory and was never written down; recomputing it
and not saving it would repeat that exactly.
Also routes every rescan completion through one hook, so there is a
single place that knows what a finished rescan means - republish for a
manual one, score and rank for a rebuild.
Compile-verified in _claude_stage: 0 errors, 0 warnings.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-25 15:01:24 -04:00
|
|
|
{
|
fix(replay): resolve labels inline - the prebuilt cache's window never overlapped the rescan
The 15:13 session proved the replay pass ran end-to-end on all 24 models
and scored ZERO labelled bars on every one of them, while each rescan sat
on ~5000 scored predictions (~2755 Buy / ~2232 Sell). The two windows
never overlapped:
StartLabelCachePrebuild deliberately keeps a CONVERGED model's
dtStudied watermark (it gates inference recency and must not move), so
the prebuild's window was the handful of bars since the last studied
bar - all with uncommitted pivots, hence "label cache pre-built -
Buy: 0 | Sell: 0 | Neutral: 0" on every member.
The label never needed a cache. SwingPivotDirectionLabel(idx) is a pure
function of the ZigZag/Close/ATR buffers the rescan itself refreshes over
exactly the scoring window, and m_lastLabelLifespan == 0 is its own
unresolved flag - the same finality gate the cache applies, applied
directly. ScoreReplayFromCache now resolves each bar's label inline and
the label-prebuild stage is deleted from the rebuild state machine
outright; going through a cache built for a different window was
indirection that changed the answer.
Also splits the empty-result diagnostics: "no resolved labels" (a
windowing/data fault) is now distinguished from "labels present, every
call Neutral" (a calibration verdict). The first version reported the
second message for both, which mislabelled this very bug as a calibration
outcome in the same breath as reporting scored=0.
Honest limitation, stated in the code too: the replay window includes
bars the model trained on, so a replay-minted ladder is measured partly
in-sample and will read stronger than a holdout-measured one. It is
replaced by the genuine article at the next completed scoring pass; until
then it is what makes a restarted deployed model able to vote at all.
Compile-verified in _claude_stage: 0 errors, 0 warnings.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-25 15:22:01 -04:00
|
|
|
Print(ID + ": WARNING - could not start the replay rescan (no servable history, or the"
|
|
|
|
|
" model is not ready to infer). This member stays silent; it will rebuild on the"
|
|
|
|
|
" next attach or the next training pass.");
|
|
|
|
|
m_deployedRebuildStage = 2;
|
|
|
|
|
return false;
|
feat(vote): replay pass rebuilds a deployed model's ladder without retraining
The previous commit persisted the tier ladder, which fixes this going
forward but did nothing for models whose .stats predates WST7 - they
still had to retrain to mint one. They never did. Every number a
converged model needs in order to vote is a pure function of weights
already on disk plus labels derivable from the chart, so replay them:
stage 1 build the label cache (existing chunked prebuild)
stage 2 rescan history (existing chunked rescan, deployed net)
stage 3 score + rank + persist (one walk over two arrays)
ScoreReplayFromCache() walks m_arrowSignalCache against
m_labelCacheBuy/Sell, fills the same m_oosTierFired/Hits and per-class
totals pass 3 fills, and hands them to RankTiersFromOos() - deliberately
feeding the existing ranker rather than reimplementing it. The shrinkage,
the chance reference and the module trust weight are subtle enough that a
second copy would drift, and a ladder measured by a slightly different
rule would be silently incomparable with every ladder training produced.
AdvanceDeployedRebuild() sequences the three stages off the timer. It has
to be a sequence: stages 1 and 2 are each minutes of work draining in
time-boxed slices, and stage 2's output is meaningless until stage 1 has
labels to score against. The previous version ran the rescan with no
labels at all, which is why it could only ever rebuild arrows and never
the ladder - the thing actually blocking the vote.
The result is written to .stats immediately. The failure being repaired
is state that lived in memory and was never written down; recomputing it
and not saving it would repeat that exactly.
Also routes every rescan completion through one hook, so there is a
single place that knows what a finished rescan means - republish for a
manual one, score and rank for a rebuild.
Compile-verified in _claude_stage: 0 errors, 0 warnings.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-25 15:01:24 -04:00
|
|
|
}
|
|
|
|
|
}
|
fix(replay): resolve labels inline - the prebuilt cache's window never overlapped the rescan
The 15:13 session proved the replay pass ran end-to-end on all 24 models
and scored ZERO labelled bars on every one of them, while each rescan sat
on ~5000 scored predictions (~2755 Buy / ~2232 Sell). The two windows
never overlapped:
StartLabelCachePrebuild deliberately keeps a CONVERGED model's
dtStudied watermark (it gates inference recency and must not move), so
the prebuild's window was the handful of bars since the last studied
bar - all with uncommitted pivots, hence "label cache pre-built -
Buy: 0 | Sell: 0 | Neutral: 0" on every member.
The label never needed a cache. SwingPivotDirectionLabel(idx) is a pure
function of the ZigZag/Close/ATR buffers the rescan itself refreshes over
exactly the scoring window, and m_lastLabelLifespan == 0 is its own
unresolved flag - the same finality gate the cache applies, applied
directly. ScoreReplayFromCache now resolves each bar's label inline and
the label-prebuild stage is deleted from the rebuild state machine
outright; going through a cache built for a different window was
indirection that changed the answer.
Also splits the empty-result diagnostics: "no resolved labels" (a
windowing/data fault) is now distinguished from "labels present, every
call Neutral" (a calibration verdict). The first version reported the
second message for both, which mislabelled this very bug as a calibration
outcome in the same breath as reporting scored=0.
Honest limitation, stated in the code too: the replay window includes
bars the model trained on, so a replay-minted ladder is measured partly
in-sample and will read stronger than a holdout-measured one. It is
replaced by the genuine article at the next completed scoring pass; until
then it is what makes a restarted deployed model able to vote at all.
Compile-verified in _claude_stage: 0 errors, 0 warnings.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-25 15:22:01 -04:00
|
|
|
return true;
|
feat(vote): replay pass rebuilds a deployed model's ladder without retraining
The previous commit persisted the tier ladder, which fixes this going
forward but did nothing for models whose .stats predates WST7 - they
still had to retrain to mint one. They never did. Every number a
converged model needs in order to vote is a pure function of weights
already on disk plus labels derivable from the chart, so replay them:
stage 1 build the label cache (existing chunked prebuild)
stage 2 rescan history (existing chunked rescan, deployed net)
stage 3 score + rank + persist (one walk over two arrays)
ScoreReplayFromCache() walks m_arrowSignalCache against
m_labelCacheBuy/Sell, fills the same m_oosTierFired/Hits and per-class
totals pass 3 fills, and hands them to RankTiersFromOos() - deliberately
feeding the existing ranker rather than reimplementing it. The shrinkage,
the chance reference and the module trust weight are subtle enough that a
second copy would drift, and a ladder measured by a slightly different
rule would be silently incomparable with every ladder training produced.
AdvanceDeployedRebuild() sequences the three stages off the timer. It has
to be a sequence: stages 1 and 2 are each minutes of work draining in
time-boxed slices, and stage 2's output is meaningless until stage 1 has
labels to score against. The previous version ran the rescan with no
labels at all, which is why it could only ever rebuild arrows and never
the ladder - the thing actually blocking the vote.
The result is written to .stats immediately. The failure being repaired
is state that lived in memory and was never written down; recomputing it
and not saving it would repeat that exactly.
Also routes every rescan completion through one hook, so there is a
single place that knows what a finished rescan means - republish for a
manual one, score and rank for a rebuild.
Compile-verified in _claude_stage: 0 errors, 0 warnings.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-25 15:01:24 -04:00
|
|
|
}
|
|
|
|
|
//+------------------------------------------------------------------+
|
2026-08-01 11:27:28 -04:00
|
|
|
//| |
|
|
|
|
|
//+------------------------------------------------------------------+
|
|
|
|
|
void CExpertSignalAIBase::OnChartEventHandler(const int id,
|
|
|
|
|
const long &lparam,
|
|
|
|
|
const double &dparam,
|
|
|
|
|
const string &sparam)
|
|
|
|
|
{
|
2026-08-22 00:25:52 -04:00
|
|
|
//--- Match THIS instance's id only. See STUDY_EVENT_ID_BASE (ExpertSignalAIBase.mqh) for the
|
|
|
|
|
//--- full failure shape.
|
fix(ensemble): responsive panel + synchronized eras + combined-vote accuracy
Four user-reported/requested items, one root cause chain:
1) DEAD CONTROL PANEL in AI_HYBRID mode. All members posted custom event
id 1 and handled id 1001, and CExpertCustom broadcasts every chart
event to every filter - so each posted event ran a train chunk in ALL
N members (N*N chunks per round) and the chart thread never idled
long enough to deliver clicks/drags. profiling.csv: 99.45% of time in
OnChartEventHandler. Fix: per-instance study-event ids
(STUDY_EVENT_ID_BASE + construction order, offset above the Controls
library's ON_* codes - id 1 was also ON_DBL_CLICK, so panel
double-clicks fired training chunks). ArmStudyEvent() is the single
post site; lost-event watchdog replaces the accidental
sibling-clears-my-flag rescue.
2) WARM-UP DUPLICATION. The auto-tune sweep is deterministic over
identical features/labels, and it ends in the full MI diagnostic
suite, which the MI-share gate never intercepted on the sweep path -
four members ran four identical ~36s sweep+report blocks. First
member publishes outcome (g_ensembleChartTuneDone/Installed/Settings);
the rest apply it and skip both.
3) DEINIT STRANDED PANEL+ARROWS (user repro 18:52). Root cause from the
log: the 4,500ms budget runs from MetaTrader's stop REQUEST - a heavy
autosave in flight ate it, OnDeinit got ~430ms and died in the first
member's arrow persist ("Abnormal termination" 432ms in). Fix: early
visible-UI sweep (native prefix deletes for status/panel/dialog)
right after ClearStatusLabel, and a fast path for still-training
models - their arrows are re-rendered every era, so they get one bulk
purge instead of scan+atomic-write in the death window.
4) ENSEMBLE FEATURES (user requests): era BARRIER - members advance era
by era together; a member ahead of the slowest still-training member
declines Train() calls and its chunk budget is donated
(TRAIN_TIME_BUDGET_MS = 120/activeTrainers, UI headroom constant).
COMBINED-VOTE OOS SCORE - each member's pass-3 scan contributes its
adjusted per-bar decision (0.0 on abstain) to a shared row buffer;
the last member to finish the era scores the averaged vote vs the
mirrored Min_Vote_Open against the same target-before-stop outcomes
members grade themselves on, publishing an "Ensemble vote" line on
the aggregated panel. Member headlines now carry their lifetime win
rate with break-even.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-16 19:06:04 -04:00
|
|
|
if(id == CHARTEVENT_CUSTOM + m_studyEventId)
|
2026-08-01 11:27:28 -04:00
|
|
|
{
|
fix(shutdown): make ExitPolicy public, and stop every long loop the moment MT5 asks
Two things, one of which was a compile error.
1. ExitPolicy() was declared in the protected block but is pushed in from
Warrior_EA.mq5:770. Moved to public beside the other EA-facing setters.
2. Chart objects surviving OnDeinit. The 4,500 ms teardown budget is measured
from the STOP REQUEST, not from OnDeinit's first line, and OnDeinit cannot
begin until whatever is in flight returns - so a scan still running after
_StopFlag is raised does not delay the cleanup, it SPENDS it, and the purge
never gets its turn.
New CExpertSignalAIBase::ShutdownRequested() = IsStopped() || m_shutdownInProgress.
Deliberately NOT m_trainingStopRequested: that latches, and a latched flag
would permanently disable scans that must run again on the next Start.
Guarded, longest first:
- TuneIndicatorsByFilter - per candidate, restoring the OPERATOR's settings
on the way out (best[] is mutated in place; the tuner otherwise keeps the
last trial's parameters, which nothing chose).
- ReportBarrierGeometryScan - per pairing, breaking to ONE restore point so
m_barrierScanLiveLabels can never be left true (that makes ComputeLabelForBar
read the last candidate's multiples as the configured geometry).
- ReportFeatureLabelInformation / ReportExcursionInformation / lag profile -
nulls ABANDON rather than truncate: fewer draws is not a smaller null, it
is a wrong one, and p shifts toward significance. m_dirEvidence staying
false is the safe direction.
- SimulateExitPolicyOutcomes - zeroes its accumulators so the divergence line
is dropped instead of latching a partial expectancy as the run's only report.
- ReportGeometryExpectancyScan - per ladder rung.
- HttpGet - one choke point for up to a dozen blocking WebRequests per
first-pass Update(). An in-flight request cannot be cancelled; refusing to
start another is the whole remedy.
- PollTraining, OnChartEventHandler's study event, TuneIndicatorsAndTrain -
entry points, so a queued event cannot open an era during teardown.
TuneIndicatorsAndTrain's guard is the first statement, ahead of the
m_tuneFilterDone / g_ensembleChartTuneDone latches.
- OnTick / OnTimer / OnChartEvent.
Training's own bar loops already honoured this (pass 1 per bar, passes 2/2.5/3
yield on a 120 ms budget); the warm-up scans did not, and they are the longest
uninterruptible stretches the EA has.
StopTraining() is unchanged: the operator's Stop still finalises synchronously.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-17 17:03:11 -04:00
|
|
|
//--- The study event IS the training driver, so a queued one landing after the stop request would
|
|
|
|
|
//--- open a whole era inside the teardown window. Checked here as well as in Warrior_EA.mq5's
|
|
|
|
|
//--- OnChartEvent because CExpertCustom re-posts these between members.
|
|
|
|
|
if(ShutdownRequested())
|
|
|
|
|
return;
|
2026-08-01 11:27:28 -04:00
|
|
|
TuneIndicatorsAndTrain(lparam);
|
|
|
|
|
bEventStudy = false;
|
|
|
|
|
OnTickHandler();
|
|
|
|
|
}
|
|
|
|
|
}
|
2026-08-24 04:39:17 -04:00
|
|
|
//--- AcquireConfigLock/ReleaseConfigLock bodies now live on CConfigLock (m_configLock) - see
|
|
|
|
|
//--- Expert\ConfigLock\ConfigLock.mqh's class comment, including g_initFatalReason's declaration.
|
2026-08-01 11:27:28 -04:00
|
|
|
#endif
|