FaisalPratama/Warrior_EA
Watch
1
1
Fork
You've already forked Warrior_EA
 0
forked from animatedread/Warrior_EA
Code Pull requests Activity
Warrior_EA/Expert/AIBase/Lifecycle.mqh

759 lines
34 KiB
MQL5
Raw Permalink Normal View History

refactor(ai): extract Layer.mqh and deduplicate AI config - Moves CLayer neuron construction to AI/Impl/Layer.mqh to keep Network.mqh clean - Unifies four previously duplicated architecture initialisation blocks (MLP/CONV/LSTM/HYBRID) into a single shared function - Eliminates risk of behavioural drift where one architecture missed a setter, causing mismatched feature sets or targets
2026-08-01 11:27:28 -04:00
//+------------------------------------------------------------------+
//| Lifecycle.mqh |
//| |
//| Construction/destruction, the CExpertSignal vote API |
//| (LongCondition/ShortCondition/ConfidenceTier/pattern weights), |
//| tick + chart-event dispatch, and the per-config chart lock. |
//| |
//| PARTIAL IMPLEMENTATION FILE - not standalone. |
//| CExpertSignalAIBase method BODIES only. The class declaration |
//| lives in Expert\ExpertSignalAIBase.mqh, which includes this file |
//| at the bottom, after the declaration. Do not include it |
//| anywhere else and do not compile it on its own. |
//+------------------------------------------------------------------+
#ifndef WARRIOR_AIBASE_LIFECYCLE_MQH
#define WARRIOR_AIBASE_LIFECYCLE_MQH
//+------------------------------------------------------------------+
//| Constructor |
//+------------------------------------------------------------------+
//--- These are only fallback defaults for a fresh object before the EA's OnInit() applies the
//--- active input values via the public setters in Warrior_EA.mq5. The input-driven values are the
//--- source of truth for the actual run configuration.
CExpertSignalAIBase::CExpertSignalAIBase(void) :
ID("NULL"),
m_neuronsCount(0),
m_minTrainYear(1970),
m_optimizationAlgo(TrainingOptimizer), // see the member declaration comment
//--- Placeholder only; InitNeuralNetwork() replaces it with ComputeFirstLayerWidth() before anything
//--- reads it. Deliberately the floor rather than 0, so a hypothetical path that built a topology
//--- without going through init would produce a small usable net instead of a zero-width layer.
m_initialNeuronsCount(FIRST_LAYER_MIN_WIDTH),
m_outputNeuronsCount(OUTPUT_CLASSIFICATION),
//--- Frozen. Nothing reads these to build a topology any more - the taper derives its own endpoints
//--- (BuildFreshTopology) - but they still occupy positional slots in the .cfg sidecar and the weights
//--- fingerprint. Held at their historical defaults so both stay byte-stable; changing either value
//--- would re-key every model on disk for no behavioural reason whatsoever.
m_minNeuronsCount(MIN_NEURONS_20),
m_neuronsReduction(RF_70),
m_hiddenLayersCount(3),
m_lstmHiddenSize(32),
m_convFilterCount(16),
m_historyBars(14),
m_fractalPeriods(5),
m_pattern_0(25),
m_pattern_1(50),
m_pattern_2(75),
m_pattern_3(100),
feat: S2 meta-labeling head - binary trade-quality model over the classic-candidate corpus The NN now has a target that is not per-bar direction (closed, best-of-999 p=1.0000): P(win | this journaled candidate, at the EA's own SL/TP, net of cost). One net for all 52 pattern-sides, AIType=AI_META. - NetForward.mqh: the host-side softmax+CE gradient generalized total==3 -> 2||3 on both backprop paths; a 2-class softmax IS a logistic head, and no compute backend changes. - SignalMETA.mqh (new): corpus loaded read-only from the LARGEST signal DB on disk (decoupled from the config fingerprint that burned four S1 runs); the GMT->server offset is measured PER ROW against entryPrice vs bar open (DST-immune, histogram logged); a window-span regime filter drops the pre-2017 daily-backfill rows; 31-feature setup descriptor appended at the input (26 one-hot + side + tanh netVote + SL/TP ATR + spread/ATR). - Training.mqh: candidate-queued pass 1, binary-target pass 2, per-candidate calibration (2.5) and OOS (3) walks. Counter mapping win->Buy / loss->Sell lets checkpoint selection, the edge floor, the plateau ladder and the family-wise deploy gate run UNCHANGED: precision reads as win rate among traded candidates, chance as the base win rate, recalls as sensitivity/ specificity. Era-end META line: coverage x (p - break-even) vs the null. - Labels are the side-conditional triple-barrier win caches - never the DB's stop-and-reverse outcome. Logit adjustment deliberately skipped (~40% base rate). Live inference + online learning guarded off until S3. - Fingerprint: conditional |TGT:META1; State\META\ folder + 2-output filename slot keep meta models fully separate from direction models. Compiles clean (0 errors, 0 warnings). S2 run = attach a chart with AIType=AI_META; S3 wires the votes via the per-side hooks. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-13 06:52:31 -04:00
m_trainTarget(0),
m_metaCandCount(0),
refactor(ai): extract Layer.mqh and deduplicate AI config - Moves CLayer neuron construction to AI/Impl/Layer.mqh to keep Network.mqh clean - Unifies four previously duplicated architecture initialisation blocks (MLP/CONV/LSTM/HYBRID) into a single shared function - Eliminates risk of behavioural drift where one architecture missed a setter, causing mismatched feature sets or targets
2026-08-01 11:27:28 -04:00
m_useVolumes(true),
m_useTime(true),
m_useATR(true),
m_useMA(false),
m_useRSI(false),
m_useMACD(false),
m_useIchimoku(false),
m_useSwingContext(false),
m_useNews(false),
fix(signals): revive a dead MA model, and demote Sanyaku from state to event Two defects surfaced by research/test_classic.py, both verified fixed by re-running the transcription against 178k bars of EURUSD H1. CSignalMA model 1 could never fire. For any recursive average - and MA_TYPE_EMA is the shipped default - MA(i) = a*Close(i) + (1-a)*MA(i+1), so DiffMA(i) = a * (Close(i) - MA(i+1)) DiffCloseMA(i) = (1-a) * (Close(i) - MA(i+1)) are positive multiples of one quantity and always share a sign. Model 1 asks for a close BELOW a RISING average, which is precisely the combination that identity forbids: 0.000% of bars, either direction, any symbol. The MQL5 standard library this was ported from defaults to MODE_SMA, where the two are merely correlated - the bug arrived with the EMA default, not with the port. Reading the slope one bar back (DiffMAPrev) breaks the tie for every MA type while keeping the model's stated meaning. Now fires on 7.92% of bars. CSignalIchimoku model 11 fired on 27% of bars at weight 100. Sanyaku is three standing STATES conjoined with no transition term, so it held across long stretches - and being last in the if-chain at the top weight, the module's highest-conviction reading was also its most common one, overwriting all eight event models below it on a quarter of all bars. The old comment rejected an event form because "demanding all three flip on the same bar would fire almost never" - true, but that is not the alternative. Kouten is the TURN: the ALIGNMENT transitions, and only one role need change for it to. Testing !Sanyaku(idx+1) fires once per aligned stretch. Now 2.17%, in line with Kumo breakout (2.4%) and the strong TK cross (1.1%). DataReady() extended one bar deeper to cover the lookback. Neither pattern showed edge before or after; this is about the models meaning what they say and the vote not being dominated by a constant. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-01 17:14:34 -04:00
m_useCrossAsset(false),
feat(ai): spread as a volatility-regime feature, and fix a stale-index cache in both new blocks Adds spread/ATR and the spread change ratio as network inputs (EnableSpreadFeature, default on). Spread is the one microstructure channel that is both FX-available and genuinely historical in the Strategy Tester - "during testing, the spread is not modeled but is taken from historical data" - so unlike swap, signed tick flow or depth of market it is something a backtest can honestly validate. What it encodes, stated precisely because the raw measurement overstates it. research/test_spread.py found spr/atr the strongest single feature in this codebase, on 5 of 8 instrument/geometry cells at 2-4x any volume feature. But the barrier LABEL charges the spread inside its own barriers, so a wide-spread bar is mechanically likelier to resolve as a loss and the feature would partly be predicting its own cost model. Relabelling at zero cost and re-measuring the identical feature showed 20-40% of it WAS that tautology and the majority was not (XAUUSD retained 97%). What survives is a volatility-regime reading: spread is near-fixed while ATR is not, so the ratio runs high exactly when realised volatility is below its own ATR estimate, which genuinely predicts whether ATR-scaled barriers get reached. It is UNSIGNED - Neutral-vs-directional only, never a side. Also fixes a stale-index bug I introduced with the cross-asset panel and had just repeated in the spread series. Both cached on length alone: if(m_crossAsset.Bars() >= bars) return true; MQL5 series indices are relative to NOW, so one new closed candle shifts every index by one. Keyed only on length, the panel keeps serving its index 0 as a bar that is no longer the newest, and every cross-asset value is read one bar out of step with the price features sitting beside it in the same vector - silently, with no error and no shape change. This is the same class of defect as the dtStudied watermark behind the zero-direction backtests. Both now carry a datetime anchor on m_Time.GetData(0), the same invalidation key the label/feature bar caches already use. And a performance fix that fell out of it: with correct invalidation the panel rebuilds on every new bar, and RefreshConvergedSignal runs per bar - which in the tester would mean one full multi-symbol resample per simulated bar at training depth. Inference only reads bars 0..m_historyBars-1 plus the panel's own slow window, so it now requests exactly that. The cache check is >=, so a deeper panel left from training still satisfies it. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-01 17:42:40 -04:00
m_useSpreadFeature(false),
m_spreadSeriesBars(0),
m_spreadSeriesAnchor(0),
m_crossAssetAnchor(0),
feat: pin the cross-asset pair set train->serve + warm the sync at init The reference-pair set was re-discovered from Market Watch on every build, so adding or removing a terminal symbol silently changed what a trained model's six cross-asset features meant - the last open train/serve parity gap from the 2026-08-11 audit. The set a model's FIRST successful build actually used is now stamped into its .cfg (append-and-length-guard, adopt-don't-compare - the derived-barrier pattern) and every later build constructs the panel from exactly that list; a pinned pair that is temporarily unavailable is skipped, never substituted. Also warms SymbolSelect/SeriesInfo for every reference symbol at InitNeuralNetwork, so the terminal's ~minute of async cross-symbol download starts at init instead of when the first Build() trips over an unselected symbol - the source of the startup 'only 0 usable reference pairs' console failures. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-11 21:29:14 -04:00
m_crossAssetPairsPinned(""),
m_crossAssetCfgSaved(false),
refactor(ai): extract Layer.mqh and deduplicate AI config - Moves CLayer neuron construction to AI/Impl/Layer.mqh to keep Network.mqh clean - Unifies four previously duplicated architecture initialisation blocks (MLP/CONV/LSTM/HYBRID) into a single shared function - Eliminates risk of behavioural drift where one architecture missed a setter, causing mismatched feature sets or targets
2026-08-01 11:27:28 -04:00
m_newsFeatureWindowMinutes(60),
m_useADCumulativeDelta(false),
m_useADShorteningOfThrust(false),
m_useADWyckoffEventStream(false),
m_useADWyckoffFailedStructure(false),
m_useADWyckoffSignificantBarInversion(false),
m_autoTuneIndicators(false),
m_indicatorsPtr(NULL),
Net(NULL),
m_shadowNet(NULL),
TempData(NULL),
dError(-1),
dUndefine(0),
dForecast(0),
dPrevSignal(0),
m_refreshOk(0),
m_refreshFailFeatures(0),
m_refreshFailShort(0),
m_refreshBuy(0),
m_refreshSell(0),
m_refreshNeutral(0),
m_voteGateBlocked(0),
m_voteGatePassed(0),
m_voteGateCompleteAtFirst(-1),
m_voteGateLoadedAtFirst(-1),
m_signalClusterWindow(6),
m_nmsLiveBuyTime(0),
m_nmsLiveSellTime(0),
m_nmsLiveBuyAccept(false),
m_nmsLiveSellAccept(false),
m_nmsLiveKeptTime(0),
m_nmsLiveKeptDir(Neutral),
m_nmsLiveKeptConf(0),
dtStudied(0),
m_eraCount(0),
m_trainingComplete(false),
m_inferenceOnly(false),
m_modelLoadedFromDisk(false),
m_topologySuperseded(false),
m_mqlInferenceValidated(false),
m_shadowBootstrapAttempted(false),
m_enableOnlineLearning(true),
m_freezePriorCalibration(false),
m_onlineLearnedUpToTime(0),
m_onlineRollingAcc(-1.0),
m_onlineSamples(0),
m_onlineBarsSincePersist(0),
m_onlineBlendFrozen(false),
bEventStudy(false),
m_oosSplitPct(30),
dOosError(-1),
dOosForecast(0),
m_oosSamples(0),
m_cumIsCorrect(0),
m_cumIsTotal(0),
m_cumOosCorrect(0),
m_cumOosTotal(0),
m_oosOutSpreadSum(0),
m_oosOutCount(0),
m_countBuySignals(0),
m_countSellSignals(0),
m_countNeutralSignals(0),
m_trueBuyCount(0),
m_trueSellCount(0),
m_trueNeutralCount(0),
m_logitAdjustTau(1.0),
m_logitAdjustLogged(false),
m_logitAdjustSkipWarned(false),
m_prevEraTrueBuyCount(0),
m_prevEraTrueSellCount(0),
m_prevEraTrueNeutralCount(0),
m_oosBuyHits(0),
m_oosBuyTotal(0),
m_oosSellHits(0),
m_oosSellTotal(0),
m_oosNeutralHits(0),
m_oosNeutralTotal(0),
m_oosBuyPredicted(0),
m_oosBuyPredictedHits(0),
m_oosSellPredicted(0),
m_oosSellPredictedHits(0),
m_oosNeutralPredicted(0),
m_oosNeutralPredictedHits(0),
fix: the deploy gate was benchmarking a win rate against a label frequency The gate rests on an invariant stated at ExpertSignalAIBase.mqh:199 - under a driftless walk P(touch +k before -m) is m/(m+k), and break-even for a k:m trade is ALSO m/(m+k), so "beats chance" and "is profitable" are the same test. That invariant needs reward >= risk, and the measured geometry no longer satisfies it. With target 1.62*ATR and stop 3.33*ATR, break-even is 67.3%, but both-won bars were stripped out of Buy and Sell so the label base rate read 37.5%. chancePrecPct is max(BuyTotal,SellTotal)/bars, so the gate was clearing models nearly 30pp short of break-even: 42% "directional precision" is +4 sigma against 37.5% and loses money on every single trade. Live since 217b9bc. Root cause is that label agreement stopped being the same question as trade profitability. Buy implies winLong, but the converse fails on every both-won bar, and the label can only name one of two directions that both pay. So stop asking the model whether it matched a label and start asking whether its trade paid: - cache winLong/winShort per bar beside the label, under the same validity flag; published from the barrier walk before the collapse to 3 classes - dirPrecPct now counts wins on the side actually called - chancePrecPct is max(P(winLong), P(winShort)), MEASURED - the textbook m/(m+k) would credit SP500's drift to the model - the NMS "what would I have made" pair, the live-fired precision, and the IS/OOS cumulative win rates all move to the same test. IS and OOS are read side by side as the overfitting signal, so measuring one in wins and the other in agreement would put a fixed gap between them that has nothing to do with generalization - the confidence threshold is FITTED on wins too, so the operating point maximises what the gate grades - per-class label-agreement precision is still computed and logged; it is the right diagnostic for class separation, just not for a deploy decision - era line renamed dir-precision -> win-rate, chance -> chance=break-even Both build variants compile 0 errors, 0 warnings. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-09 16:00:51 -04:00
m_oosBuyPredictedWins(0),
m_oosSellPredictedWins(0),
m_oosWinLongTotal(0),
m_oosWinShortTotal(0),
refactor(ai): extract Layer.mqh and deduplicate AI config - Moves CLayer neuron construction to AI/Impl/Layer.mqh to keep Network.mqh clean - Unifies four previously duplicated architecture initialisation blocks (MLP/CONV/LSTM/HYBRID) into a single shared function - Eliminates risk of behavioural drift where one architecture missed a setter, causing mismatched feature sets or targets
2026-08-01 11:27:28 -04:00
m_oosConfidenceSum(0),
m_confidenceCalScale(1.0),
m_minDirectionalRecallPct(40),
m_priorBuy(0.0),
m_priorSell(0.0),
m_priorNeutral(0.0),
m_oosBuyFired(0),
m_oosBuyFiredHits(0),
m_oosSellFired(0),
m_oosSellFiredHits(0),
fix: NMS gates the TRADE, not just the arrow - one arrow is now one trade NmsLiveAccept() appeared in exactly one place: wrapped around DrawObject(). It never touched dPrevSignal, and dPrevSignal is what LongCondition() / ShortCondition() / SignedAIConfidence() read. So a declustered bar lost its arrow and still opened a position. Measured on SP500 H1 2026-08-09: CONV called a direction on 64% of bars, so the ~500 bars visible on screen held ~320 decisions - and ~40 arrows were drawn. Roughly one arrow per eight positions the EA would take. And the survivors are not a random eighth. Rule 2 of the declustering keeps the HIGHER-CONFIDENCE side of a cluster, so the visible set is systematically the best member of each run. A chart showing the best of every eight decisions and hiding the rest reads far better than the model is - the same best-of-N selection error already corrected in the geometry scan, the indicator tuner, the lag profile and the deploy gate, this time on the display layer, where it is most likely to mislead the person deciding whether to trade. Fixed by neutralising dPrevSignal when NMS rejects, rather than adding a "may trade" flag consulted at each read site: that leaves exactly ONE definition of what the model decided this bar, so the arrow, the panel's "Current signal", the confidence feeding sizing/SL/TP/trailing, the refresh tally and the order itself cannot drift apart again. Also reports the consequence instead of hiding it. Every OOS counter on the era line still scores every directional call - a population ~8x larger than what now trades - so the line carries a second figure: | TRADED (declustered) NN% on N calls (edge +Npp) replaying the identical rule over pass 3 (which walks OOS bars oldest to newest, the same order the live sweep sees). Its cursors are separate members from the live ones so a training pass can never disturb the live chart's declustering. Deliberately NOT switched into selectionScore yet. Declustering cuts coverage from ~64% of bars to ~8%, well under MIN_COVERAGE_FRACTION_OF_BASE_RATE, which would make every checkpoint undeployable overnight - the minRR collision and the recall-floor catch-22 twice over. The floor gets re-derived from these measurements first. Compiles clean: 0 errors, 0 warnings. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-09 10:22:31 -04:00
m_oosNmsFired(0),
m_oosNmsHits(0),
m_oosNmsLastBuyIdx(-1),
m_oosNmsLastSellIdx(-1),
m_oosNmsKeptIdx(-1),
m_oosNmsKeptConf(0.0),
m_oosNmsKeptDir(Neutral),
refactor(ai): extract Layer.mqh and deduplicate AI config - Moves CLayer neuron construction to AI/Impl/Layer.mqh to keep Network.mqh clean - Unifies four previously duplicated architecture initialisation blocks (MLP/CONV/LSTM/HYBRID) into a single shared function - Eliminates risk of behavioural drift where one architecture missed a setter, causing mismatched feature sets or targets
2026-08-01 11:27:28 -04:00
m_lastBuyFiredPrecPct(-1),
m_lastSellFiredPrecPct(-1),
m_lastBuyFired(0),
m_lastSellFired(0),
m_maxClassSampleWeight(1.5),
m_swingConfirmationBars(100),
m_barrierHorizonBars(BARRIER_HORIZON_FALLBACK),
m_barrierHorizonResolved(false),
m_barrierFallbackWarned(false),
m_lastBarrierTimedOut(false),
fix: a restart no longer loses the measured geometry or the training window Terminal restart, 22:25: all four resumed models sat on empty windows with enum 2:6 barriers. Three interlocking causes, all visible in one log excerpt: 1) THE PRE-SCAN WINDOW WAS SIZED BY THE SAVED WATERMARK. A resumed model's dtStudied sits at its last studied bar, so Bars(dtStudied, now) ~ 0 and the resumed-model MI pre-scan built a zero-bar "complete" label cache - logged as "Buy: 0 | Sell: 0 | Neutral: 0". Train()'s own era start RESETS dtStudied to the training-window rule before computing its window; the pre-scan did not. The rule is now factored into TrainWindowStart() and both use it. The scan also refuses to arm before SERIES_SYNCHRONIZED (it ran in the same second as OnInit), and deployed models keep their watermark - for them it gates inference recency, not a training window. 2) THE HORIZON LATCHED ON AN INDICATOR WARM-UP. ComputeBarrierHorizonBars ran against a ZigZag with 0 calculated legs, fell back, and EnsureBarrierHorizon latched fallback(32) x slMult x tpMult = 384 for the process lifetime. A leg-starved horizon is now PROVISIONAL: re-resolved on the next rebuild, the label cache wiped if it moved (labels from two horizons answer different questions), and the geometry deriver refuses to run from it - a pair derived over a warm-up window would get PINNED. 3) THE DERIVED GEOMETRY WAS NEVER PERSISTED. The .cfg is written at model creation and at weights-reset - both BEFORE era 0 derives - so the measured pair lived only in memory: every restart read back zeros, adopted nothing, fell back to the enum barriers, and the era-0-only gate meant a resumed model could NEVER re-derive. A full day of training on 3.33/1.62 resumed as 2:6. Now: the settled pair is pinned to the .cfg the moment derivation completes (one-shot, atomic write), and the derive gate accepts any model with no pinned pair, not just era 0 - mid-run stability is carried by m_geometryDerived itself, which never allows a second derivation. Both build variants compile 0 errors, 0 warnings. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-09 22:40:43 -04:00
m_barrierHorizonLegStarved(false),
m_horizonStarvedWarned(false),
m_geometryCfgSaved(false),
fix: both-won bars were labelled "do not trade" - resolve by first touch Removing the min-reward:risk raise let the MEASURED geometry come back with the target NEARER than the stop (SP500 H1: target 1.62*ATR at q50 of favourable, stop 3.33*ATR at q75 of adverse). That reopened a branch the code called unreachable: price can reach +target and -target inside one horizon, winning in BOTH directions, and those bars fell through to Neutral. Neutral has only three producers, both-lost is unreachable (you cannot touch -3.33 without crossing -1.62 first, which wins the short), and timeouts logged at 1.0% of Neutral - so ~27% of ALL bars were being handed to the model as the abstain class when a trade either way would have collected its target. The cleanest positives in the sample, labelled "do not trade", while the fitted confidence threshold was being asked to find selectivity in what was left. Resolved by FIRST TOUCH: the target reached earlier is the trade that would have closed first. Same forward window, no extra lookahead. Same-bar ties stay Neutral - OHLC cannot order two touches, and unlike an intrabar stop tie there is no pessimistic side to fall to, so a guess would inject a coin-flip direction into the target. Also: - count both-won and its same-bar tie subset in the prebuild line, so the share is measured rather than inferred from arithmetic on a log line - scope the timeout counter to IS, matching the tally it is reported as a percentage OF; it was incremented over the whole scan and divided by an in-sample denominator - clear m_lastBarrierTimedOut at the top of the walk with the excursions, not at the bottom - the two early returns published the previous bar's verdict - mark the pass-1 label line PROVISIONAL. It prints the enum fallback because geometry can only be derived from excursions that do not exist yet, and it reads exactly like a config change that failed to take effect FORCES RETRAIN. Both build variants compile 0 errors, 0 warnings. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-09 15:35:42 -04:00
m_lastBarrierBothWon(false),
m_lastBarrierBothWonTied(false),
refactor(ai): extract Layer.mqh and deduplicate AI config - Moves CLayer neuron construction to AI/Impl/Layer.mqh to keep Network.mqh clean - Unifies four previously duplicated architecture initialisation blocks (MLP/CONV/LSTM/HYBRID) into a single shared function - Eliminates risk of behavioural drift where one architecture missed a setter, causing mismatched feature sets or targets
2026-08-01 11:27:28 -04:00
m_labelPrebuildTimeoutCount(0),
fix: both-won bars were labelled "do not trade" - resolve by first touch Removing the min-reward:risk raise let the MEASURED geometry come back with the target NEARER than the stop (SP500 H1: target 1.62*ATR at q50 of favourable, stop 3.33*ATR at q75 of adverse). That reopened a branch the code called unreachable: price can reach +target and -target inside one horizon, winning in BOTH directions, and those bars fell through to Neutral. Neutral has only three producers, both-lost is unreachable (you cannot touch -3.33 without crossing -1.62 first, which wins the short), and timeouts logged at 1.0% of Neutral - so ~27% of ALL bars were being handed to the model as the abstain class when a trade either way would have collected its target. The cleanest positives in the sample, labelled "do not trade", while the fitted confidence threshold was being asked to find selectivity in what was left. Resolved by FIRST TOUCH: the target reached earlier is the trade that would have closed first. Same forward window, no extra lookahead. Same-bar ties stay Neutral - OHLC cannot order two touches, and unlike an intrabar stop tie there is no pessimistic side to fall to, so a guess would inject a coin-flip direction into the target. Also: - count both-won and its same-bar tie subset in the prebuild line, so the share is measured rather than inferred from arithmetic on a log line - scope the timeout counter to IS, matching the tally it is reported as a percentage OF; it was incremented over the whole scan and divided by an in-sample denominator - clear m_lastBarrierTimedOut at the top of the walk with the excursions, not at the bottom - the two early returns published the previous bar's verdict - mark the pass-1 label line PROVISIONAL. It prints the enum fallback because geometry can only be derived from excursions that do not exist yet, and it reads exactly like a config change that failed to take effect FORCES RETRAIN. Both build variants compile 0 errors, 0 warnings. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-09 15:35:42 -04:00
m_labelPrebuildBothWonCount(0),
m_labelPrebuildBothWonTieCount(0),
refactor(ai): extract Layer.mqh and deduplicate AI config - Moves CLayer neuron construction to AI/Impl/Layer.mqh to keep Network.mqh clean - Unifies four previously duplicated architecture initialisation blocks (MLP/CONV/LSTM/HYBRID) into a single shared function - Eliminates risk of behavioural drift where one architecture missed a setter, causing mismatched feature sets or targets
2026-08-01 11:27:28 -04:00
m_maxErasPerRun(300),
m_arrowRestoreIndex(0),
m_arrowRestorePending(false),
m_arrowRestoreStartMs(0),
m_rescanIndex(0),
m_rescanHi(0),
m_rescanBarsNow(0),
m_rescanPending(false),
m_rescanStartMs(0),
m_rescanRawBuy(0),
m_rescanRawSell(0),
m_rescanRawNeutral(0),
m_trainRunActive(false),
m_eraResumePending(false),
m_resumeBars(0),
m_resumeTotalIter(0),
m_resumeOosCutoff(0),
m_resumeBarIndex(0),
m_resumeAddLoop(false),
m_isTrainQueueCount(0),
m_isTrainCursor(0),
m_isPass2Active(false),
m_isPass2Done(false),
m_isPass3Active(false),
m_oosScoreIndex(0),
m_oosScoreStartIndex(0),
fix: the operating point was fitted on bars the net had memorized FitDirConfThreshold harvested its margin histogram from pass 2's own backprop samples. Pairing every fit against the same era's OOS result shows what that measured: PAI era 1 IS 25% cov @ 66.1% (-0.8pp) -> OOS 64% (-3pp) gap +2.1pp PAI era 76 IS 90% cov @ 79.6% (+12.7pp) -> OOS 65% (-2pp) gap +14.6pp LSTM era 9 IS 77% cov @ 81.6% (+14.6pp) -> OOS 63% (-4pp) gap +18.6pp The gap grows monotonically while OOS stays flat, so within a handful of eras the curve stops describing behaviour on unseen bars. That is fatal here specifically, because the objective branches on the SIGN of (p - break-even): the memorized curve reads +12pp at 95% coverage, so coverage x (p - p0) correctly maximises coverage and returns ~0.02 - fire on every bar. The "p < p0 -> get more selective" branch, which is the actual regime and the entire point of 983a6a3, could never fire because IS never showed p < p0. Carve a calibration slice out of the IS span - DIR_CONF_CALIB_PCT_OF_IS, purged from backprop by one label horizon on BOTH sides (the far-side purge is not optional: without it the newest training bars carry labels partly decided by price action inside the slice, putting the memorization straight back into the curve). Score it in a new chunked pass 2.5, after pass 2 has trained and before pass 3 grades - the only position where the histogram is simultaneously not-trained-on, not-graded, and current with the weights it will be applied to. Costs 15% of the training data. Worth it beyond honesty: the deploy gate needs dirPrecPct > chance + EDGE_MIN_SIGMAS*SE, and a threshold pinned near zero dilutes any edge concentrated in the confident bars across every bar the model calls, driving dirPrecPct toward chance by construction. A threshold that can be selective is the only mechanism by which a small, concentrated edge could ever clear that gate. Also: a sparse histogram now KEEPS the previous threshold instead of resetting to 0.0. A failed measurement must not decay to the most exposed setting in the range. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-10 15:58:18 -04:00
m_isCalibActive(false),
m_isCalibDone(false),
m_calibIndex(0),
m_calibStartIndex(0),
feat: excursion-size head (Stage 1, measurement only) Direction is closed - normalised asymmetry fails on three instruments with a working positive control, and the classifier's own best-of-999 era-cap test agrees (+0.9pp = 1.48 sigma, family-wise p=1.0000). SIZE is a different question and RANGE clears at ~4x its null. Checked the denomination before building on that, since the source memo warns to: m_excUpCache holds (maxHigh - fill)/ATR, so "RANGE is predictable" is a claim about travel RELATIVE to current ATR, not a restatement of "ATR is autocorrelated". It is exactly the part a fixed multiple (stop 3.31*ATR, target 1.64*ATR) discards. A second small CNet, 760 -> 24 -> 32 sigmoid outputs = P(price reaches ladder rung k) upward and downward. Survival parameterisation rather than regressing the multiple, because it needs nothing new from CNet: sigmoid outputs and the per-neuron delta the `total != 3` branch already applies (a quantile head would need a linear activation and a pinball gradient in Network.mqh, Network.cl and the DirectML path, on a class four topologies share). Targets are free - m_ladderUpAt already records first-touch age per rung with 0 meaning never reached. Separate net, not extra outputs on the classifier: more outputs would change m_outputNeuronsCount, the .nnw shape and the fingerprint, and push the count off 3 - the exact condition backProp uses to select the joint softmax gradient the 3-class head depends on. The classifier is bit-for-bit unaffected and this is removable without trace. STAGE 1 PLACES NO ORDERS. It reports a Brier skill score against the constant per-rung base rate - the baseline a fixed ATR multiple already assumes - with both predictors fitted IS and evaluated OOS, so neither gets a look at the test set. Positive skill justifies Stage 2 (drive SL/TP and sizing off ExcursionQuantile, which is defined and deliberately uncalled). Zero or negative means ATR already carries everything and Stage 2 must not be built. Trains only on primary occurrences: the replay queue oversamples for CLASS balance, and a direction-balanced sample is a biased SIZE sample. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-11 07:40:01 -04:00
m_excNet(NULL),
m_excTgt(NULL),
m_excOut(NULL),
m_excHeadFailed(false),
m_excBaseTotal(0),
m_excScored(0),
fix: the excursion gate would have passed Stage 2 on an artifact I made Second-opinion review killed the +4.2% far-rung result, correctly, and the mechanism is my own bug. A head trained toward {0.05,0.9} converges to 0.05+0.85p, so its bias is 0.05-0.15p: negative where p is near 1, POSITIVE where p < 1/3, growing monotonically as the rung gets farther. Against a baseline frozen at the IS rate, an upward-biased head scores positive Brier skill whenever the OOS rate merely sits above the IS rate. Predicted signature: huge negatives near, ~zero at p=1/3, growing positives far. Observed: -82% ... -0.6% ... +1.2/+2.7/+4.2. The far rungs were not the clean end of a distorted measurement, they were the other face of the same artifact. Everything before 25aca83 is void. The gate was a bare `skill >= 2%` point estimate over 8 rungs x 4 topologies x N eras, reported per era - a best-of-~300 with no interval and no multiplicity control, which is the shape of the four traps already documented here. It now needs FOUR things at once: DECISION RUNGS only the rungs ExcursionQuantile actually reads at the live geometry (target 1.62, stop 3.31 ATR), fixed before looking. Skill at 5 ATR is skill about a distance no order is placed at - and the TARGET side currently interpolates 1.5/2.0, which measured -2.2% and -1.3%. DISJOINT SAMPLE one bar per horizon. Adjacent bars share 63 of 64 horizon bars, so ~16k scored bars is ~250 independent ones and every SE over the full set is ~8x understated. VS ORACLE the best constant achievable ON THE SCORED BLOCK, closed form from H and n (Brier = H*(1-H/n)). A head that learned only a LEVEL nearer the OOS rate than the frozen IS constant scores positive against the old baseline and <= 0 here. This is the control that separates per-bar skill from base-rate drift. MONOTONE CURVE P(reach k) must be non-increasing in k. Nothing constrained 8 independent sigmoids to obey that, and ExcursionQuantile returns the FIRST crossing - so a tangled curve is misread exactly where the head is least sure. Counted and reported, not silently used. The pass message now also states what a pass would and would not buy: expectancy is -costs at zero directional edge whatever the stop distance, and under prop DD limits LOWER variance also lowers P(reach target before limit), so "better drawdown" is a choice of failure mode, not a win. Still owed before any Stage 2: a race against a trailing-quantile incumbent and a vol-feature logistic. Beating a frozen global constant is the weakest admissible bar for replacing a global constant. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-11 15:49:32 -04:00
m_excScoredD(0),
m_excMonoViol(0),
perf: the excursion head cost 3.6x era time - cut its dispatches ~250x Measured on exc-race-v3: LSTM era 300s -> 1087s (net 272->748s, "other" 30->337s). My estimate had been "single-digit percent". The cost is per-DISPATCH, not per-FLOP, and therefore hits EVERY backend: the head is 19k weights and ~2.4 GFLOP an era - seconds of arithmetic - but ~48k forward/backward calls x several layer submits each, and its 760-wide layer exceeds the CPU DLL's inline threshold so each one pays a real handoff. The classifier's own net time tripled too, from contention with a second pool on an already-full box. Three changes, all backend-neutral because they remove submits rather than tune threads: SCORE ONLY DISJOINT WINDOWS (~64x). Adjacent bars share all but one bar of their horizon, so 16k consecutive bars were always ~250 independent observations - the full-sample tally was never worth more than the disjoint one, it just quoted an n that was ~64x too large. Dropping it costs nothing statistically and removes 63 of every 64 forward passes. The two parallel tallies collapse into one, which is also less code. The trailing ring still advances on every bar: it needs the outcome SEQUENCE, and that is array lookups, not a forward pass. TRAIN ON EVERY 4th PRIMARY BAR (4x). The target is low-dimensional and strongly autocorrelated - neighbouring bars carry near-identical excursion information - so per-bar training buys resolution the target does not have. Strided on ATTEMPTS, not acceptances, so a stretch of unlabelled bars cannot silently change the spacing. OWN TIMING COLUMN. The head's passes were landing in the era line's "other" bucket, which is how a 3.6x regression read as an unexplained jump in the one column nobody attributes. A cost that cannot be seen in the timing line cannot be traded off against anything. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-11 16:38:29 -04:00
m_excTrainTick(0),
m_excUs(0),
feat: race the excursion head against a trailing-quantile incumbent Beating a frozen global constant is the weakest admissible bar for replacing a global constant. The honest incumbent is a rolling rung frequency: it adapts to the volatility regime - exactly what the head claims to predict - and needs no model, no 760 inputs and no training. Implemented as a ring of per-bar outcome bitmasks (32 rungs fit one ulong), sized horizon + EXCURSION_TRAIL_WINDOW. The newest `horizon` entries are held back UNRESOLVED: a bar's rung outcomes are only known one horizon later, so using them would be lookahead and would flatter the incumbent into an opponent the head could never fairly beat. Pass 3 walks oldest-to-newest, so "pushed more than horizon bars ago" is exactly "resolved by now". Each push is O(rungs), not O(window). The head's decision-rung Brier is pro-rated to the trailing estimate's coverage before the ratio, since the incumbent only scores bars where its window is warm. This line is worth reading on its own, independently of the head: if the trailing quantile beats the global constant, that is a cheap risk-control win available with no machine learning at all - and it is the same number either way, so the run answers both questions in one pass. The ring is deliberately NOT reset per era - it estimates the market, not the era, and re-warming 500 bars every era would leave the incumbent unusable over the first chunk of every scoring pass, handing the head a free win on exactly those bars. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-11 15:57:11 -04:00
m_excTrailHead(0),
m_excTrailCount(0),
m_excTrailN(0),
m_excTrailScored(0),
refactor(ai): extract Layer.mqh and deduplicate AI config - Moves CLayer neuron construction to AI/Impl/Layer.mqh to keep Network.mqh clean - Unifies four previously duplicated architecture initialisation blocks (MLP/CONV/LSTM/HYBRID) into a single shared function - Eliminates risk of behavioural drift where one architecture missed a setter, causing mismatched feature sets or targets
2026-08-01 11:27:28 -04:00
m_lastStatusLabelUpdateTick(0),
m_lastBuyRecallPct(-1),
m_lastSellRecallPct(-1),
m_lastDisplayNeuron0(0),
m_lastDisplayNeuron1(0),
m_lastDisplayNeuron2(0),
m_lastDisplaySignal(0),
m_lastBarTime(0),
m_modelEta(InitialEtaForOptimizer()),
m_etaCeiling(InitialEtaForOptimizer()),
m_erasSinceCooldown(0),
m_bestOosForecast(-1),
m_bestBalancedOos(-1),
m_bestPassedRecall(false),
fix: a one-sided era can no longer become the best checkpoint Measured on HYBRID, era 29 of the first win-scored run: the model collapsed to always-Buy and was crowned "new best selection score 67.1%". Under win-based scoring that is not a coincidence - the always-call-the-drift-side model IS the chance reference, so it scores exactly chance (P(winLong) ~ 67% on SP500), while every honest two-sided era scores 63-66% because shorts win less often against the drift. Raw score ranking therefore actively prefers the degenerate model, every regression restores back to it, and live NMS collapses its near-constant signal to ~25 trades per era - observed as "hybrid barely trades". bothSidesLive already blocked one-sided eras from DEPLOYING (tradeableOK, 371f8aa), but among not-yet-deployable eras the score alone ranked - the same early phase the coverage credit was added for, failing the same way through a different door. The ranking key is now three lexicographic tiers: deployable > two-sided > score. A one-sided era cannot displace a two-sided best regardless of score - by construction its score is a property of the data's drift, not the model - and a two-sided era displaces a one-sided best no matter how much lower it scores. m_bestBothSidesLive is snapshotted with the checkpoint and reset with the rest of the best-tracking state. Both build variants compile 0 errors, 0 warnings. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-09 21:19:44 -04:00
m_bestBothSidesLive(false),
diag: slow eras must explain themselves - heartbeat + era time split + pass-1 paint The 23:42 restart left all four charts grinding ~25x slower than the 18:01 baseline (era lines in 86 seconds there; 20+ minutes of nothing here), and NOTHING could say why from outside: pass 1 logs nothing, its status paint sat inside the !wouldQueue branch so the IS sweep - 80% of the pass, processed FIRST - painted nothing either, the VPS has no debugger for a thread stack, and the hourly new-bar cache invalidation cancels and restarts an unfinished era, so a slow era can stay invisible FOREVER. Externals gave: four chart threads at ~95% pure user-mode compute, DLL pool idle, no file writes. That narrows it to "MQL5-side per-item work in the era passes" and no further. So training now explains itself: - TrainHeartbeat: one line per 4096 processed items, only after an era has already run 60s, at most 6 lines per era - a healthy era stays exactly as quiet as before. Reports position and the cumulative split: feature-window builds vs net forward/backprop vs everything else. Hooked into all three passes. - The era summary line gains "| ERA TOOK Ns (feature windows X, net fwd/back Y, other Z)" whenever an era exceeded 120s. - Pass 1 paints its progress for QUEUED bars too, not just the OOS slice, so the panel shows "learning (era N)" instead of sitting on the idle writer's "Getting ready..." for the entire IS sweep. The label is throttled internally; painting per bar costs nothing. Both build variants compile 0 errors, 0 warnings. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-10 00:10:45 -04:00
m_eraStartTick(0),
m_passFeatUs(0),
m_passNetUs(0),
m_passHeartbeatPrints(0),
diag: an era that discards itself now says so instead of scanning forever add_loop is exactly "at least one bar produced a usable feature window". When it stays false, pass 2, pass 3, the era counter, the checkpoint and every log line in the era-end block are ALL skipped: Train() returns having done nothing, m_eraResumePending is still false, and the next call restarts the SAME era from bar 0. That is an infinite 0->100% "scan" loop that prints absolutely nothing - the only remaining silent restart path in Train(), and it matches the reported symptom exactly. Pass 1 now counts usable vs unusable windows and reports at the pass boundary, which demonstrably executes: - total failure routes through ReportTrainStall (already capped at one line a minute, and carries the run-state flags) naming the counts, the required window width and the bar count - success prints how long the scan took and how many samples it handed to pass 2, but only once the era has passed 10s - a fast era stays as quiet as before, a slow one distinguishes "advancing" from "sweeping the same bars forever" A PARTIAL failure is normal and deliberately does not shout: pass 1 walks oldest-to-newest and the deepest bars predate the indicators' warm-up, so those windows fail and are cached as misses. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-10 11:29:33 -04:00
m_passWindowOk(0),
m_passWindowFail(0),
fix: the recall gate was unsatisfiable and the LR decay was a spiral Both made the run structurally unable to succeed, independently of any signal in the data. Found by reading the 13:01 log. RECALL GATE. m_objectiveMet required Buy, Sell AND Neutral OOS recall each >= 40%. First-touch resolution (ce52654) collapsed Neutral from the ~94% majority it was under exact-pivot labels to a same-bar-tie residue - 250 of 38,261 bars, 0.65% - so the floor was asking the model to identify 40% of coin-flip ties before it could converge. Measured: CONV, LSTM and HYBRID all logged "Neutral:0% (need >=40% each)" on every era. No model could ever satisfy it; every run was destined for the plateau ladder or the era cap. Only the DIRECTIONAL floors are load-bearing for the anti-collapse job the gate exists to do: an all-Neutral model shows Buy and Sell recall at 0% and is blocked by them. Neutral's own floor guarded the mirror bias (over-calling Buy/Sell at Neutral's expense), which was real at 94% prevalence and is not at 0.65% - there, almost never calling Neutral is correct rather than biased. Prevalence-guarded rather than hardcoded off, so it returns by itself if a future label rule makes Neutral substantial again. Deliberately NOT extended to Buy/Sell: exempting a thin directional class reopens the era-44-46 hole, which directionalRecallMeasured only half-covers - it checks those classes were MEASURED, not that they passed. ETA DECAY. A regressing era restored the checkpoint, reset the optimizer and cut eta - all on the FIRST regression. The next era then started from an identical state with a smaller step, regressed again, and got the same treatment. The loop is self-sustaining and cannot discover anything, because rolling the weights back is exactly what removes the exploration that would end it. Measured on PAI: eras 2-11 every one a regression against era 1, eta 0.000594 -> 0.000024, dW/W 0.000%/0.000% from era 2 onward. Ten eras, ~45s each, reproducing era 1 exactly and unable to do anything else. Now requires ETA_DECAY_PATIENCE_ERAS consecutive regressions - the standard ReduceLROnPlateau formulation. A single bad era is noise, and an improving era clears the counter so alternating runs never accumulate into a decay. Build tag -> gate-patience-v3. It had not moved in six commits, which is why the running binary could not be identified from its own log. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-10 13:28:58 -04:00
m_consecutiveRegressions(0),
fix: a resumed model cached a cold ATR as permanent, so it never trained BufferTempData cached EVERY failure - m_featureCacheHasValue[idx]=true with m_featureCacheValid[idx]=false - and the cache never re-tries a miss. So a single feature read taken before the terminal had finished calculating the indicator buffers marked those bars unusable for the rest of the process, even though the data arrived milliseconds later. MT5 fills an indicator's buffers asynchronously after the handle is created, and a cold ATR returns 0 for EVERY index, not just its warm-up tail. BufferTempDataCompute rejects a bar with no ATR (correctly - the price features would be meaningless), so the whole window failed, and the whole cache was poisoned. Only resumed models were hit, because only they read features that early. Topology.mqh sets m_warmupPassesRemaining = netLoaded ? 0 : 3: a fresh start sits through three separately-scheduled Train() calls before anything touches a feature, which is exactly what those passes are for. A resumed one skips them and TuneIndicatorsAndTrain drives StartLabelCachePrebuild and the MI report from the first chart event. Its rationale - "a restart already has a proven-synced history" - holds for HISTORY and not for INDICATORS, which are recreated every process start. Downstream: BuildFeatureWindow failed on every bar of every era, so add_loop never went true, so pass 2, pass 3, the era counter and the checkpoint were all skipped and pass 1 swept 0->100% forever. The "0 samples" MI report line at startup was the same failure, four seconds earlier, already visible in the log. - a miss is now cached only when it is PERMANENT; the two "not ready yet" guards mark m_featureFailTransient and are recomputed on the next visit. Steady-state cost is ~ind_Periods bars per era, not 54k. - an era that discards itself now drops the feature cache before restarting, so any remaining cause of this state self-heals instead of looping. Deleting the .nnw "fixed" this only by turning the model back into a fresh one. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-10 11:41:31 -04:00
m_featureFailTransient(false),
diag: name the cause when every feature window fails, and enforce the width contract Era 0 stalls with "NOT ONE of 54681 scanned bars produced a usable feature window, windows ok=0 failed=54681" and nothing else. That line reads identically for a cold ATR, a conditionally-missing optional feature block and an out-of-range index, so it cannot be diagnosed without one restart per hypothesis. Two changes: 1. WIDTH CONTRACT in BufferTempData. Every enabled block must emit exactly m_neuronsCount values on EVERY bar. A block that emits its values on some bars and skips them on others (indicator, panel or series unavailable for that bar) does not merely shorten the window - it SHIFTS every feature after it into the wrong slot, and the net then trains on silently misaligned inputs that still look like a valid window to everything downstream. Now rejected, rolled back and reported once, naming the optional blocks (XA / SPR / swing context) as the ones carrying an availability test. Worth having independently of the current stall. 2. BuildFeatureWindow records WHICH lookback slot rejected and how much of the window was assembled, and the pass-1 stall report renders it: "slot 0 of 20 REJECTED (window had 0 of 760)" is an indicator warm-up or history-edge read; "every lookback bar ACCEPTED and the window was still short: 640 of 760" is a missing 6-value block. No behaviour change on a healthy run: the width check is an equality that already holds, and the diagnostics render only inside the total-failure branch. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-11 10:31:56 -04:00
m_windowFailSlot(-2),
m_windowFailTotal(0),
m_featureWidthWarned(false),
diag: the heartbeat could be outrun by the condition it watched for It fired only on 4096-item boundaries once an era had already run 60s. Those boundaries are all crossed in the first few chunks of pass 1, so an era that became slow AFTER them printed nothing at all - which is precisely what happened: 20 minutes, four pegged cores, zero heartbeats. I read that silence as "the era loop is never reached" and went looking for a wedge above it. The silence may simply have meant "past the last boundary". A diagnostic whose trigger can be outrun by the condition it watches for is worse than no diagnostic, because it produces confident wrong conclusions. Now time-gated: checked every 256 items (the mask only keeps GetTickCount off the hot path), prints when the era has run >60s and >30s since the last line, up to 12 per era. Progress/phase for the panel is still published on every call, before any gate. Both build variants compile 0 errors, 0 warnings. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-10 07:44:03 -04:00
m_lastHeartbeatTick(0),
fix: the panel showed "100%" for the whole of pass 1 The simple panel derived its percentage from pass 2's counters: (m_isTrainCursor+1) / max(m_isTrainQueueCount,1). During pass 1 those are 0 and 0, so the expression is (0+1)/max(0,1) = 100%. An era spends its first pass scanning ~38k bars - minutes of work - and the panel reported that phase as finished the entire time. Observed by the user as "started learning at 100% of their era and are stuck there", and it actively misled the diagnosis: the one number on screen said the opposite of what was happening. The UI cannot fix this on its own - it can see pass 2's counters but has no way to know which pass owns them. So each pass now PUBLISHES its own progress and a short phase name through TrainHeartbeat (which every pass already calls per item), and the panel just displays them: "learning (era 45, scan 34%)". Published before the heartbeat's 4096-item journal gate, so the panel updates continuously while the journal stays quiet. Both build variants compile 0 errors, 0 warnings. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-10 07:22:29 -04:00
m_passProgressPct(0),
m_passLabel("starting"),
fix: prebuild and era sized different windows; diag: Train() names its branch TWO things, one incident. 1) THE BUG I SHIPPED IN 0c85c54. m_tuneStartTrainBar is declared, initialised to 0, and NEVER ASSIGNED - the assignment existed before the God-class split and the split dropped it, leaving a dead member. Harmless while nothing read it; a real defect the moment 0c85c54 made StartLabelCachePrebuild() reset dtStudied from it. Train() then computed the window as max(StartTrainBar, floor) while the prebuild computed max(0, floor), where StartTrainBar is the non-zero datetime OnChartEventHandler passes through from the "New Bar" event. The two therefore disagreed about `bars`, so EnsureBarCachesCapacity() saw a changed size at era start, wiped the caches, and re-armed a full 38k-bar prebuild - instead of training. Restored the assignment so both sides evaluate the identical expression. 2) THE REASON IT TOOK ALL NIGHT TO FIND. Train() is a state machine with six early-return branches above the era loop and every one of them is silent. Four charts burned a core each for 15 minutes with an empty journal: the pass heartbeats (694b756) proved the era loop was never reached, no prebuild completion line appeared either, and nothing external can see inside a single MQL5 thread - per-thread CPU says "busy", file writes say nothing, and the VPS has no debugger. That is an undiagnosable state, and it is the thing to fix, not just the bug of the day. ReportTrainStall() now names the branch Train() is taking whenever no era has completed for 3 minutes, at most once a minute per signal, with the state that decides the branch: run/prebuild/simOos/resume flags, era, dtStudied, and - for the cache-invalidation branch specifically - BOTH bar counts, since two sizings disagreeing is exactly what re-arms the prebuild forever. Silent on a healthy run: an era completing resets the clock. Both build variants compile 0 errors, 0 warnings. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-10 07:32:08 -04:00
m_lastEraCompleteTick(0),
m_lastStallReportTick(0),
refactor(ai): extract Layer.mqh and deduplicate AI config - Moves CLayer neuron construction to AI/Impl/Layer.mqh to keep Network.mqh clean - Unifies four previously duplicated architecture initialisation blocks (MLP/CONV/LSTM/HYBRID) into a single shared function - Eliminates risk of behavioural drift where one architecture missed a setter, causing mismatched feature sets or targets
2026-08-01 11:27:28 -04:00
m_haveOosCheckpoint(false),
feat: gate deployment on the null of the MAXIMUM, not the per-era null EDGE_MIN_SIGMAS is a PER-ERA test and the deployed model is the MAXIMUM over every era a run ranks. A 2-sigma one-sided test passes on noise with probability 0.0228 per era, so over N eras the chance at least one clears it is 1-(1-0.0228)^N: 34% by era 18, 80% by era 70, 93% by era 112. The gate was near-certain to open on a long run whatever the data held. It did. HYBRID deployed 2026-08-08 at dir-precision 35.5% vs 34% chance - +1.5pp, best of 112 eras whose per-era values wandered 30%..35.5%. At the call counts these runs produce that is p_family 0.92..0.9999. Every OTHER best-of-N decision here already carries this correction, and every one REJECTS on this data: the barrier-geometry winner (null of the maximum over 6, p=0.3902), the indicator tuner (Sidak, p=1.0000), the MI lag profile (null of the maximum over 21 lags). The one decision that ships a model to a live account had none. BestCheckpointSurvivesSelection() re-tests the checkpoint that is about to deploy: z = (precision - chance)/SE, SE = sqrt(p0(1-p0)/n) p_single = P(Z >= z) p_family = 1 - (1-p_single)^N against DEPLOY_FAMILY_WISE_ALPHA. It uses the checkpoint's OWN snapshotted precision/chance/call-count, not the latest era's, because the model that ships is the one that has to clear the bar. N counts CANDIDATE eras (coverage measurable, at least one directional call) - an era that called nothing directional could never have become the best, so counting it would make the gate stricter than the search that actually happened. Conservative on purpose: consecutive eras share OOS bars and differ by one gradient step, so they are nowhere near N independent draws and the true family-wise error is below this bound. This gate decides what trades real money and the house posture is reject-unless-demonstrated. Effect at 2900 directional calls / N=112: required edge goes 1.76pp -> 2.92pp. A real edge clears it; +1.5pp does not. Applied to BOTH automatic paths - the plateau ladder's stage-3 deploy and the m_trainingComplete assignment - which must stay identical or the flag persisted into the .nnw disagrees with the decision to stop, and a reload runs inference on a model the ladder refused. NOT applied to the two operator paths (era-cap deploy, panel Deploy button). Those stay the operator's call; ReportSelectionGateVerdict() logs the verdict beside them so an authorised deploy can never later be misread as a validated one. NormalUpperTail() is A&S 26.2.17 (|err| < 7.5e-8), self-contained rather than pulling in Math\Stat. Verified against reference values to 6dp: Q(1.645)=0.049985, Q(1.96)=0.024998, Q(3.0)=0.001350. Its locals are ntB1..ntB5 because AI\Network.mqh line 79 does "#define b1 AdamBeta1". Compiles clean: 0 errors, 0 warnings. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-08 18:01:04 -04:00
m_bestDirPrecPct(-1.0),
m_bestChancePrecPct(-1.0),
m_bestDirCalls(0),
m_deployCandidateEras(0),
refactor(ai): extract Layer.mqh and deduplicate AI config - Moves CLayer neuron construction to AI/Impl/Layer.mqh to keep Network.mqh clean - Unifies four previously duplicated architecture initialisation blocks (MLP/CONV/LSTM/HYBRID) into a single shared function - Eliminates risk of behavioural drift where one architecture missed a setter, causing mismatched feature sets or targets
2026-08-01 11:27:28 -04:00
m_oosStable(false),
m_objectiveMet(false),
m_erasSinceBestBalanced(0),
m_plateauStage(0),
fix: training-stability audit fixes F1/F2/F3/F5 - unbiased shuffle, real plateau escapes, fresh optimizer state on restore, pure OOS metric Four of the six findings from research/training_pipeline_audit_2026-08-09.md (F4 mini-batching and F6 feature re-encode deliberately deferred - see the report's implementation-status section for why): - F1: pass-2 Fisher-Yates (and AutoTune's MI block shuffle) used MathRand()%, which is 15-bit - provably non-uniform on every full-history era over 32,768 queued samples. New 30-bit ShuffleRandomIndex(). - F2: plateau warm restarts were a no-op whenever eta already sat at its ceiling (the normal state of a non-regressing plateau) - the ladder was just a 24-era countdown. Restarts now overshoot to 5x the ceiling (PLATEAU_RESTART_BOOST) and anneal geometrically back over the patience window, SGDR-style; ETA_MIN widened 1e-4 -> 1e-5 so the decay schedule has real range. - F3: checkpoint restores put weights back but kept the rejected trajectory's Adam moments, so the optimizer immediately pushed back toward the rolled-back state (the restore->regress->restore oscillation). CNet::ResetOptimizerState() zeroes moments/momentum/step counters (weights, BN statistics, gamma/beta untouched) on every mid-run restore, every boosted restart, and the deploy-time restore that online learning continues from. - F5: batch-norm running statistics now freeze for the pass-3 OOS scoring walk, so the selection metric the checkpoint ranking and deploy gate read is a pure function of the checkpoint instead of partly measuring BN drift. Defensive unfreeze in FinalizeTrainRun covers stop-mid-pass; live/online adaptation and the OOS continual-learning simulation stay adaptive by design. Compiled clean (0 errors, 0 warnings) via the staged-tree recipe. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-09 10:54:09 -04:00
m_restartBoostErasLeft(0),
refactor(ai): extract Layer.mqh and deduplicate AI config - Moves CLayer neuron construction to AI/Impl/Layer.mqh to keep Network.mqh clean - Unifies four previously duplicated architecture initialisation blocks (MLP/CONV/LSTM/HYBRID) into a single shared function - Eliminates risk of behavioural drift where one architecture missed a setter, causing mismatched feature sets or targets
2026-08-01 11:27:28 -04:00
m_syncWaitStartTick(0),
m_warmupPassesRemaining(0),
fix(resume): model reload stalled training - three hardenings on the resume path A resumed META model hot-looped pass 1 (0->100% scan oscillation, silent for 3 minutes until the stall reporter fired) because EVERY window failed at the first AD/Wyckoff feature: the init-time param adoption called ReInitADIndicators unconditionally, destroying five freshly-calculating indicator instances to recreate them with BYTE-IDENTICAL params (verified by parsing the .nnw header - the MI tuner had kept the configured settings), at process start, on a box with 1 GB free of 31. The replacements sat cold for 6+ minutes while full-history resweeps starved the indicator threads harder. - AdoptIndicatorParams: installs a loaded param set into the tuner and rebuilds handles ONLY when the set actually differs from what the live indicators run. Both call sites (resume init + panel reload) use it. - Resumed models get the same 3 warm-up passes as fresh ones. The skip was the shared root cause of the cold-ATR (ba13eef), cold-AD (2026-08-11) and this incident - custom indicators recompute from scratch every process start regardless of what the .nnw proves. - Cold-sweep backoff: a pass-1 sweep in which every window failed on a TRANSIENT cause arms a 5s era-start pause instead of an immediate full-history resweep, so the retry loop stops consuming the CPU/memory the warming indicators need. The stall reporter names the backoff branch. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-13 10:23:11 -04:00
m_coldSweepTick(0),
refactor(ai): extract Layer.mqh and deduplicate AI config - Moves CLayer neuron construction to AI/Impl/Layer.mqh to keep Network.mqh clean - Unifies four previously duplicated architecture initialisation blocks (MLP/CONV/LSTM/HYBRID) into a single shared function - Eliminates risk of behavioural drift where one architecture missed a setter, causing mismatched feature sets or targets
2026-08-01 11:27:28 -04:00
m_labelCacheBars(0),
m_labelCacheAnchorTime(0),
m_labelCachePrebuilt(false),
feat: derive the ATR multiples from measured excursions - no hardcoded geometry The barrier was still two constants. SL_Mode/TP_Mode left the Inputs tab in 3482b6c, but the fallback was a hardcoded 2:6 and the geometry scan only ever chose from a hardcoded grid {2,3} x {2,3,4,6,8,10}. Picking the least-bad of eleven guesses is not deriving anything. WHY THE SCAN WAS THE WRONG INSTRUMENT, now measurable rather than argued. It ranks pairings by how predictable their OUTCOME is - a question about direction. The excursion test (2c78f3b) ran on SP500 H1 and direction is the one thing absent: ASYMMETRY p=0.0846, against RANGE/UP/DOWN all at p=0.0050, with RANGE scoring 0.01345 vs a 0.00343 null - 4x, where the barrier label sits at 1.01x. Hence the scan failing its own gate on every run, and its "winner" wandering 2:8 -> 3:8 -> 2:8 -> 2:4 across four runs of the same data. Excursion SIZE is strongly measurable, so derive the geometry from that instead. stop = q25 of measured ADVERSE travel (ordinary noise does not reach it) target = q50 of measured FAVOURABLE travel (reached ~half the time, by construction, inside the horizon) Continuous, in ATR units, superseding the enum multiples. Reachability ("target on X% of bars, stop on Y%") and the implied break-even are printed so the choice is auditable rather than trusted. FIXED-POINT ITERATION, not one-shot. ComputeBarrierHorizonBars scales the horizon with the target (first-passage time grows with the band) and the excursions are measured OVER the horizon, so target -> horizon -> excursions -> target is a real loop - deriving once sizes the target from travel measured under the PREVIOUS horizon. Re-measures until the multiples move <5%, capped at 3 passes, and says so if it does not settle. Does NOT create expectancy, and the log says as much: chance precision equals break-even at every geometry (m/(m+k) on both sides). It buys a target the market reaches and a stop that survives noise. Where Min_Risk_Reward_Ratio forces a target the market rarely reaches, it WARNS rather than overriding - the ratio is the user's risk policy, so the honest move is to state its cost. That is the collision that once rejected 100% of setups. Pinned in the .cfg as doubles appended AFTER this morning's two ints, so .cfg files written earlier today still load (their length guard finds no doubles) and a model that carries them was trained on them and never re-derives. Also fixes a message from e5ceed6 that claimed "this model resumed from disk" unconditionally - it printed above a "seeding era 0" line on a brand-new model, because the branch fires whenever the cache is not built, which is equally true before a fresh model's first prebuild. A diagnostic that misreports its own trigger is worse than one that says nothing: it gets quoted back as evidence. FORCES A FULL RETRAIN (labels change). Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-07 12:06:25 -04:00
m_lastExcUp(0.0),
m_lastExcDown(0.0),
m_derivedSlMult(0.0),
m_derivedTpMult(0.0),
feat: fitted directional confidence threshold - selectivity gets a mechanism The training loss and the selection metric wanted different things and only the second one knew it. Logit-adjusted cross-entropy has no term for "how often should I trade", so the head calls a direction on 87-91% of bars. The selection metric is precision x coverage credit, saturating at the coverage floor - above the floor extra calls earn NOTHING and only precision counts. So selection wanted few good calls, the loss produced many mediocre ones, and all selection could do was pick the least-bad era out of what it was handed. Nothing pushed the model toward selectivity. This gives the decision RULE the policy instead of distorting the loss (which is estimating class probabilities correctly, and a probability estimate should not be bent to encode a trading policy - Elkan 2001: estimate, then choose the operating point separately). AdjustedSignalFromSoftmax now abstains unless the winning direction's softmax margin over its best rival clears a fitted threshold. Margin, not the winning probability: the latter moves with overall calibration rather than with how close the decision actually was. Fitted on IS, applied to OOS and live. Pass 2 already forward-passes every IS sample, so the margin histogram is harvested there for free (primary occurrences only, so the oversampled replay queue cannot skew the operating point); the fit runs at the end of pass 2, BEFORE pass 3, so the deploy gate grades the thresholded model on bars the threshold never saw. Fitting on pass 3's own predictions would be choosing the operating point on the data being graded - the best-of-N error corrected in five other places here. Objective: maximise IS directional precision subject to still clearing the SAME coverage floor the deploy gate uses (base rate x 0.25, re-derived locally so the two cannot drift apart). Swept top-down in one pass; ties go to the LOWER threshold, since equal precision for less coverage is strictly worse. Under DIR_CONF_MIN_FIT_CALLS (200) it runs unthresholded rather than on a guess. The threshold is part of the MODEL, not the run: captured with Net.CaptureWeights(), restored with the weights at both restore sites, and appended to the .cfg under the same length-guard convention so a deployed model reloads at the operating point its gate actually cleared. A pre-2026-08-09 .cfg reads 0.0, which is exactly the behaviour it was trained under. Per-era line now prints "@margin>=X.XX" next to coverage, so a coverage drop can be attributed to the operating point rather than guessed at. Both build variants compile 0 errors / 0 warnings. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-09 15:04:37 -04:00
//--- 0.0 = unthresholded until the first pass 2 fits it (see DIR_CONF_THRESHOLD_BINS). Deliberately
//--- the permissive value: a model that has not measured its own operating point must not silently
//--- abstain on everything.
m_dirConfThreshold(0.0),
m_bestDirConfThreshold(0.0),
m_dirConfPrimaryBars(0),
m_dirConfSparseWarned(false),
feat: derive the ATR multiples from measured excursions - no hardcoded geometry The barrier was still two constants. SL_Mode/TP_Mode left the Inputs tab in 3482b6c, but the fallback was a hardcoded 2:6 and the geometry scan only ever chose from a hardcoded grid {2,3} x {2,3,4,6,8,10}. Picking the least-bad of eleven guesses is not deriving anything. WHY THE SCAN WAS THE WRONG INSTRUMENT, now measurable rather than argued. It ranks pairings by how predictable their OUTCOME is - a question about direction. The excursion test (2c78f3b) ran on SP500 H1 and direction is the one thing absent: ASYMMETRY p=0.0846, against RANGE/UP/DOWN all at p=0.0050, with RANGE scoring 0.01345 vs a 0.00343 null - 4x, where the barrier label sits at 1.01x. Hence the scan failing its own gate on every run, and its "winner" wandering 2:8 -> 3:8 -> 2:8 -> 2:4 across four runs of the same data. Excursion SIZE is strongly measurable, so derive the geometry from that instead. stop = q25 of measured ADVERSE travel (ordinary noise does not reach it) target = q50 of measured FAVOURABLE travel (reached ~half the time, by construction, inside the horizon) Continuous, in ATR units, superseding the enum multiples. Reachability ("target on X% of bars, stop on Y%") and the implied break-even are printed so the choice is auditable rather than trusted. FIXED-POINT ITERATION, not one-shot. ComputeBarrierHorizonBars scales the horizon with the target (first-passage time grows with the band) and the excursions are measured OVER the horizon, so target -> horizon -> excursions -> target is a real loop - deriving once sizes the target from travel measured under the PREVIOUS horizon. Re-measures until the multiples move <5%, capped at 3 passes, and says so if it does not settle. Does NOT create expectancy, and the log says as much: chance precision equals break-even at every geometry (m/(m+k) on both sides). It buys a target the market reaches and a stop that survives noise. Where Min_Risk_Reward_Ratio forces a target the market rarely reaches, it WARNS rather than overriding - the ratio is the user's risk policy, so the honest move is to state its cost. That is the collision that once rejected 100% of setups. Pinned in the .cfg as doubles appended AFTER this morning's two ints, so .cfg files written earlier today still load (their length guard finds no doubles) and a model that carries them was trained on them and never re-derives. Also fixes a message from e5ceed6 that claimed "this model resumed from disk" unconditionally - it printed above a "seeding era 0" line on a brand-new model, because the branch fires whenever the cache is not built, which is equally true before a fresh model's first prebuild. A diagnostic that misreports its own trigger is worse than one that says nothing: it gets quoted back as evidence. FORCES A FULL RETRAIN (labels change). Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-07 12:06:25 -04:00
m_geometryDerived(false),
m_geometryDerivePasses(0),
fix: excursion window must not depend on the barrier it sizes DIRECTION IS NOT THERE, and this run is what establishes it. Three symbols: raw ASYMMETRY clears on all three (p=0.0199 / 0.0050 / 0.0050) norm ASYMMETRY collapses on all three (p=0.3433 / 0.5075 / 0.2736), USDCAD landing BELOW its own null RANGE control strengthens to 3-5x its null everywhere Divide sigma out and the apparent directional signal vanishes entirely. What cleared was volatility leaking through an unnormalised difference. Note this would have passed any replication test: three instruments at p=0.005 is exactly the evidence one would accept before committing to a rebuild, and the confound reproduces perfectly. Replication was never going to catch it - only the normalisation could. Two defects of mine, both surfaced by the same run. 1. THE GEOMETRY DERIVATION WAS DIVERGING, NOT CONVERGING. It produced a 14.57*ATR stop and a 29.14*ATR target that only 5.7% of bars ever reach. Excursions were measured over the barrier horizon; the horizon scales with the target; the target is a quantile of the excursions - so target -> horizon -> excursions -> target ran away, and "settled" only because the horizon ladder caps at 384 bars. A saturated runaway, which the iteration guard could not catch because it watches for OSCILLATION. Fixed at the root: excursions now accumulate only over m_swingMedianBars - the UNSCALED median ZigZag leg, a property of the instrument that owes nothing to the barrier. The barrier walk still runs the full horizon, because that is how long the trade is held; only the MEASUREMENT used to size the barrier is confined to a geometry-independent window. (The Min_Risk_Reward_Ratio warning fired correctly and is what flagged it - the diagnostic worked while the derivation behind it did not.) 2. THE CONFOUND VERDICT WAS UNREACHABLE. `sizeCleared && !asymCleared` was tested first and is true whenever size clears - i.e. always - so the branch that NAMES the volatility confound never printed; all three symbols showed the generic size-not-direction message instead. Verdict chain rewritten with the specific case first, and the dangling elses my first patch introduced removed. FORCES A FULL RETRAIN (the excursion window changes every derived barrier). Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-07 13:57:23 -04:00
m_swingMedianBars(0),
refactor(ai): extract Layer.mqh and deduplicate AI config - Moves CLayer neuron construction to AI/Impl/Layer.mqh to keep Network.mqh clean - Unifies four previously duplicated architecture initialisation blocks (MLP/CONV/LSTM/HYBRID) into a single shared function - Eliminates risk of behavioural drift where one architecture missed a setter, causing mismatched feature sets or targets
2026-08-01 11:27:28 -04:00
m_labelPrebuildActive(false),
m_prebuildSeedPending(false),
m_labelPrebuildBars(0),
m_labelPrebuildOosCutoff(0),
m_labelPrebuildIndex(-1),
m_labelPrebuildBuyCount(0),
m_labelPrebuildSellCount(0),
m_labelPrebuildNeutralCount(0),
m_simOosNet(NULL),
m_simOosRunActive(false),
m_simOosCutoff(0),
m_simOosBarIndex(-1),
m_simOosForecast(0),
m_simOosSamples(0),
m_tuneTrialIndex(-1),
m_tuneBestOosForecast(-1),
m_tuneLastTrialWasWin(true),
m_tuneHaveBestCheckpoint(false),
m_tuneStartTrainBar(0),
m_tuneFilterDone(false),
m_trainingPaused(false),
m_trainingStopRequested(false),
m_activeFileCommon(true),
m_configLockName(""),
m_isInitialized(false),
fix(deinit): a full model write was running ahead of the cheap cleanup "Abnormal termination" is back, and this time it is not the arrows. The timing names the culprit exactly: 16:02:31.547 OnDeinit: shutting down 16:02:36.003 Abnormal termination <- 4.46 s, MetaTrader gave up 16:02:36.226 chart signals - persisted <- cleanup finished 0.2 s LATE OnDeinit called StopTraining() BEFORE the chart cleanup. StopTraining() finalises an in-flight run, and FinalizeTrainRun() restores the best checkpoint and then persists it - a full ~1MB model write per signal. So the expensive step ran ahead of the cheap bounded one, which is precisely the inversion the shutdown ordering exists to prevent. The previous fix put PersistWeightsOnShutdown last and missed that StopTraining smuggles a second save in at the front. Two changes: Cleanup now runs FIRST, then StopTraining, then the weight save. The visible teardown is cheap and bounded, so it always completes even when everything after it is killed. And the deploy-persist inside FinalizeTrainRun is suppressed during shutdown. RestoreWeights() is an in-MEMORY swap, so the best checkpoint is already the live net by that line, and PersistWeightsOnShutdown writes exactly those weights moments later. The old path wrote the same model twice per signal - eight full writes across four charts - for no benefit. A user-pressed Stop still persists immediately, because nothing else would. Compiles 0 errors / 0 warnings. Build tag deinit-order-v2. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-01 16:06:40 -04:00
m_shutdownInProgress(false),
diag(autotune): one label shuffle cannot settle the no-edge question The permutation baseline added in 018afb1 came back on all four charts as 0.00401 nats against floors of 0.00267 / 0.00298 / 0.00318 - three draws whose spread is as wide as the excess being judged, because one shuffle is one sample from the null, not the null. That is not enough to retire a topology on. Now MI_NOISE_PERMUTATIONS draws, reported as mean +/- sd with a z-score, plus two numbers the mean over 26 columns cannot express: - the STRONGEST single feature's MI, against its own shuffled value. One informative column among 25 useless ones is precisely the case the mean hides, and precisely the case worth finding. - the excess as a percentage of H(Y). At these sample sizes a z-score can be comfortably significant while the effect is worthless, so "is it real" and "is it big enough to matter" are asked separately and answered separately. The verdict line also now states the measure's limit every time rather than only when the news is bad: this is a MARGINAL, PER-BAR statistic and the network reads m_historyBars bars jointly, so it can prove signal exists but never that it does not. It rules out a per-feature edge - and therefore any indicator retuning - not an edge that lives in a combination or across time. Compiles 0 errors / 0 warnings, standard and Market. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-01 13:32:12 -04:00
m_lastArrowsSaved(0),
fix: the Adam second moment was never Adam - all four tiers Root cause of the B=32 regression, and it predates F4 entirely. Every Adam kernel stored v already square-rooted and then fed that stored value back in as if it were the variance: v_new = sqrt(b2 * v_old + (1 - b2) * g^2) That recursion has a fixed point at v ~= b2 = 0.999 for ANY gradient below unit scale, so the denominator stops tracking the gradient and Adam degrades into plain SGD with lr = lt. Measured against the shipped WarriorCPU.dll (batch_accum_check.cpp, TestOptimizerScaleInvariance), 4000 steps of a constant gradient: 3285x less displacement at |g|=1e-5 than at |g|=1, where a scale-invariant optimizer gives the same distance for both. After the fix all six magnitudes read 1.199 and v tracks |g| exactly. It hit conv/LSTM specifically because they sit behind a batch-norm with running variance ~2.6e+05, so their gradients arrive divided by ~500 - deep in the degraded regime - while the dense stack near the loss stayed in the working one. In situ on SP500 H1: lstm1 dW/W 2.62/10.0/7.14% -> 0.024/0.022/ 0.003%, conv1 decaying to 0.000% by era 30. NeuronBatchNorm.mqh already squared v back for gamma/beta and its comment named the kernels as wrong, which is exactly why gamma/beta kept training while the stages behind froze. Persisted .nnw needs no migration - v keeps its std-dev meaning. Also, the two ways F4 exposed it, both mine: - No LR compensation for B fewer steps per era. sqrt(B) for adaptive methods (Krizhevsky 2014; Granziol et al. 2022), applied once in InitialEtaForOptimizer(). Linear scaling (Goyal et al. 2017) is for SGD. - Plateau patience denominated in eras, so raising B made the ladder 32x more impatient in its only unit. PAI converged at era 41 on ~49k updates where the same config had been finding new bests at era 1028. TrainPlateauPatienceEras() stretches it by the same sqrt(B). TRAIN_BATCH_SIZE 32 -> 8 so the patience stretch stays affordable (8 -> 23 eras per stage, not 8 -> 45). Both helpers are identities at B=1. Deploy gate: DEPLOY_MIN_SIDE_RECALL_PCT (10%) folded into tradeableOK. The perceptron reported Sell:0% recall in all 41 eras, cleared the floor on Buy alone at 36.6% vs 34% chance, deployed, and sprayed buy arrows. Folded into the ranking key rather than checked at deploy time so a one-sided era cannot become best-so-far in the first place. Deinit: the arrow purge now runs BEFORE ExtPanel.Destroy(), an unbounded CAppDialog teardown that sat ahead of it - the same ordering inversion the rule there exists to prevent. CONV was force-terminated 4.8 s into OnDeinit (vs ~1.1 s for the three that finished) having reached none of its cleanup, so its arrows stayed on the chart. Steps are now timed in the log. PurgeChart's verification rescan filtered on OBJ_ARROW, the same blind spot as the bulk delete, so "persisted 10 ... cleared 0" passed silently. It now walks every object type and reports the object counts when both are zero. Both build variants compile 0 errors / 0 warnings; both DLLs rebuilt. FORCES A RETRAIN (already forced by N1) and both DLLs must ship with the .ex5. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-09 14:02:35 -04:00
m_purgeMismatchWarned(false),
diag(autotune): one label shuffle cannot settle the no-edge question The permutation baseline added in 018afb1 came back on all four charts as 0.00401 nats against floors of 0.00267 / 0.00298 / 0.00318 - three draws whose spread is as wide as the excess being judged, because one shuffle is one sample from the null, not the null. That is not enough to retire a topology on. Now MI_NOISE_PERMUTATIONS draws, reported as mean +/- sd with a z-score, plus two numbers the mean over 26 columns cannot express: - the STRONGEST single feature's MI, against its own shuffled value. One informative column among 25 useless ones is precisely the case the mean hides, and precisely the case worth finding. - the excess as a percentage of H(Y). At these sample sizes a z-score can be comfortably significant while the effect is worthless, so "is it real" and "is it big enough to matter" are asked separately and answered separately. The verdict line also now states the measure's limit every time rather than only when the news is bad: this is a MARGINAL, PER-BAR statistic and the network reads m_historyBars bars jointly, so it can prove signal exists but never that it does not. It rules out a per-feature edge - and therefore any indicator retuning - not an edge that lives in a combination or across time. Compiles 0 errors / 0 warnings, standard and Market. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-01 13:32:12 -04:00
m_miBestColumn(0.0),
diag(autotune): stop making the feature test cost a trained model The permutation test lived inside TuneIndicatorsByFilter, which is gated on era 0 - correctly, because re-running the SWEEP would change the input vector out from under weights already fitted to the old one. But the test itself reads cached features and writes nothing, so none of that applies to it, and the gate meant the only way to see the answer on a running model was to delete the model. Today that price was PAI's 45 trained eras and CONV's 31, spent to re-ask a read-only question. Split into ReportFeatureLabelInformation(), called from the sweep when it runs and directly when it does not - a resumed model, a disabled tuner, nothing tunable. Once per attach either way. Compiles 0 errors / 0 warnings. Build tag permtest-v2. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-01 14:01:32 -04:00
m_miLabelEntropy(0.0),
diag(autotune): a positive control, and a scan that separates "no signal" from "signal knocked out of step" Four architecturally different networks landed on the same precision - Buy 23-25% against a 25.4% base rate, Sell 19-22% against 22.0% - while making completely different calls (HYBRID votes Sell on 69% of bars, PAI on 41%). Precision equal to the base rate is what INDEPENDENCE looks like, and precision under independence is fixed by the label distribution, not by the architecture, so all four converging on it is arithmetic rather than coincidence. Accuracy meanwhile tracks coverage exactly as independence predicts (31.1/30.3/25.0 predicted vs 31.8/28.9/24.6 observed for PAI/CONV/HYB). But "no information in the data" and "information destroyed upstream of every topology" produce that identical picture, and the MI test alone cannot tell them apart either. Two additions: POSITIVE CONTROL. Three "measurements" in this codebase have turned out to be silent no-ops that produced plausible numbers - the MI scorer reading an array nobody filled, the eval-mode guard that switched off the imbalance correction, the alternation gate whose premise was never true. So the estimator now has to prove it responds to a signal known to be present before any floor reading is believed: the label of a neighbouring sample row, ~19 bars away and far inside the 128-bar barrier horizon, so the two outcome windows overlap heavily and MUST be associated. Same binning, same estimator. Near the floor => every MI figure is void. ALIGNMENT SCAN. Re-scores against the label taken from bar i+k for k in -5..+5. A peak at k != 0 is a feature/label misalignment - an off-by-one in the label index, a horizon applied to the wrong bar, a feature window that lags what it claims - which would destroy the information before any topology saw it and would look identical in every accuracy number this EA prints. A flat profile says the features simply do not carry this target. The sampled range is trimmed by |k| at both ends so a shift is measured rather than an edge effect, and both bars must carry a real label. Also: BuildMiSample publishes its stride instead of the report recomputing that arithmetic (it would drift), and the control sizes its buffers from its own sample count rather than the caller's. Compiles 0 errors / 0 warnings, standard and Market. Build tag mi-control-align-v1. Redeploy only - no retrain, no model deletion; the diagnostic runs on resumed models. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-01 14:12:10 -04:00
m_miStrideBars(0),
fix(diag): the symbol sweep was measuring its own sampling, not the market Twelve cells came back with higher-timeframe "signal" 5-9x anything on H1, at p=0.005. It was an artifact, and the sweep's own columns gave it away: excess tracked the sampling STRIDE almost monotonically, and the three D1 cells - stride collapsed to 1-5 bars against a 128-bar horizon, i.e. ~99% window overlap - were the three highest. Three flaws, all the same family: comparing numbers without the spread that belongs to them. 1. THE NULL ASSUMED INDEPENDENCE THE LABELS DO NOT HAVE. Triple-barrier labels overlap; two rows less than one horizon apart share most of their outcome window. A free Fisher-Yates shuffle destroys that dependence along with the association, making the null far narrower than the truth and handing out significance that isn't there - Lopez de Prado ch. 4 arriving through the back door of the significance test. Now permutes contiguous BLOCKS of at least one horizon, so the null keeps the autocorrelation and the p-value means what it says. It degrades honestly: severe overlap leaves few blocks, the null widens, nothing reaches significance. The block count is now printed, because THAT - not the row count - is the sample size a p-value rests on, and a warning fires under 30 blocks so "not significant" is not misread as "no signal" when it means "not enough independent history to tell". 2. THE POSITIVE CONTROL'S STRENGTH DEPENDED ON THE DATASET. It paired each row's label with the NEXT SAMPLE ROW's, whose distance is the stride - so on M5, where stride ran 160-717 bars against a 128-bar horizon, it was pairing two windows that never overlap. All three M5 cells duly reported a FAILED estimator and voided their own results with nothing wrong. A control whose strength varies with the cell cannot certify the cell. Now pinned to a quarter of the horizon, where ~75% overlap is guaranteed by construction. 3. THE LOOKAHEAD VERDICT HAD NO MARGIN. It flagged 7 of 12 cells on gaps of 0.00008-0.00040 nats against a measured null sd of ~0.00030 - noise, every one. Now requires 3 sd, the same discipline the deploy floor applies to precision. Compiles 0 errors / 0 warnings, standard and Market. Build tag blockperm-v1. Supersedes every number from the sweep. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-01 15:11:40 -04:00
m_miNullBlocks(0),
feat(labels): measure which barrier is predictable at entry, don't guess The alignment scan settled the shape of the problem: 4.7x more is knowable 5 bars into a 128-bar window than at the entry the model actually trades. A 6xATR target reached over 128 bars is decided overwhelmingly by what happens DURING the window, so whatever the entry state knows is buried under 128 bars of later noise. That is a property of the TARGET, and it is why four different architectures all landed on precision exactly equal to the base rate - no topology can undo it. So measure the target. For each SL/TP pairing a user can actually select, relabel the same sampled bars and score how much the SAME features say about THAT outcome at entry. Seconds, no training, no topology, and it runs on the diagnostic path that already exists. Ranked on excess over its OWN null as a share of its OWN H(Y), never on raw nats: each geometry has a different class balance, hence a different finite-sample bias and a different amount of information there to find, so raw MI would rank the most BALANCED label rather than the most PREDICTABLE one. The break-even win rate m/(m+k) is printed beside each so the ranking is read next to the bar the model must clear. Stated in the output because it is the easy thing to get wrong: chance precision EQUALS break-even at every geometry, so a tighter target does not hand you expectancy. It buys predictability - less noise piled on top of what the entry state knows - which is the one thing changing topology cannot do. Read-only by construction: it relabels a sampled copy via TripleBarrierLabel(), never writes the label cache (which belongs to the configured geometry), and restores the horizon and overrides it borrowed. The overrides apply only when BOTH are positive, so a half-set pair can never silently relabel a live run. Compiles 0 errors / 0 warnings, standard and Market. Build tag geometry-scan-v1. Redeploy only - no retrain to READ the ranking. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-01 14:32:04 -04:00
m_miReportDone(false),
Add new research scripts for trading strategy analysis - Implemented sqx_audit.py to audit StrategyQuant X trade lists, focusing on performance metrics and cost analysis. - Created sqx_portfolio.py to evaluate portfolio performance based on uncorrelated components and their impact on risk and return. - Developed swing.py to analyze cost ratios across different holding periods and assess swing trading structures. - Introduced test_management.py to investigate the effectiveness of exit rules on random entries and their impact on expectancy.
2026-08-02 12:25:20 -04:00
m_miReportDeferrals(0),
feat(labels): measure which barrier is predictable at entry, don't guess The alignment scan settled the shape of the problem: 4.7x more is knowable 5 bars into a 128-bar window than at the entry the model actually trades. A 6xATR target reached over 128 bars is decided overwhelmingly by what happens DURING the window, so whatever the entry state knows is buried under 128 bars of later noise. That is a property of the TARGET, and it is why four different architectures all landed on precision exactly equal to the base rate - no topology can undo it. So measure the target. For each SL/TP pairing a user can actually select, relabel the same sampled bars and score how much the SAME features say about THAT outcome at entry. Seconds, no training, no topology, and it runs on the diagnostic path that already exists. Ranked on excess over its OWN null as a share of its OWN H(Y), never on raw nats: each geometry has a different class balance, hence a different finite-sample bias and a different amount of information there to find, so raw MI would rank the most BALANCED label rather than the most PREDICTABLE one. The break-even win rate m/(m+k) is printed beside each so the ranking is read next to the bar the model must clear. Stated in the output because it is the easy thing to get wrong: chance precision EQUALS break-even at every geometry, so a tighter target does not hand you expectancy. It buys predictability - less noise piled on top of what the entry state knows - which is the one thing changing topology cannot do. Read-only by construction: it relabels a sampled copy via TripleBarrierLabel(), never writes the label cache (which belongs to the configured geometry), and restores the horizon and overrides it borrowed. The overrides apply only when BOTH are positive, so a half-set pair can never silently relabel a live run. Compiles 0 errors / 0 warnings, standard and Market. Build tag geometry-scan-v1. Redeploy only - no retrain to READ the ranking. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-01 14:32:04 -04:00
m_barrierScanSlMult(0.0),
m_barrierScanTpMult(0.0),
fix(labels): the geometry scan rewarded the labels it should reject First run named 3:10 on all four charts, at 2.3x the configured 2:6. That answer was wrong and the fault was the ranking statistic. 3:10 wants a horizon of ~swingMedian*30 (~320 bars) and gets BARRIER_HORIZON_MAX. Clamped, most trades never resolve, the unresolved remainder all lands in Neutral, and H(Y) collapses. The old statistic divided the excess BY H(Y) - so a collapsing denominator made the most degenerate label look like the most predictable one. Every geometry from 2:6 upward was already showing the clamped h128, and the two widest scored highest, which is the fingerprint of the artefact rather than of signal. Two fixes: Rank on the raw excess in nats. Subtracting each geometry's OWN measured null already removes the class-balance bias, which is the only thing the normalisation was ever needed for. Disqualify clamped geometries outright rather than ranking them down. The deployed EA holds until SL or TP with no bar limit, so a truncated label trains the model on a question the strategy never asks. They are still printed, marked '!', so the disqualification is visible instead of a silent omission - and the scan now says so explicitly when nothing eligible is left, because "the limit is the feature set, not the target" is itself the finding in that case. The scan also reports each geometry's directional share and timeout share now. A label nobody can trade is not a candidate however well it scores, and that has to be visible in the same line as the score. Compiles 0 errors / 0 warnings. Build tag geometry-scan-v2. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-01 14:46:25 -04:00
m_barrierScanLiveLabels(false),
m_barrierScanTimeouts(0),
m_barrierHorizonClamped(false)
refactor(ai): extract Layer.mqh and deduplicate AI config - Moves CLayer neuron construction to AI/Impl/Layer.mqh to keep Network.mqh clean - Unifies four previously duplicated architecture initialisation blocks (MLP/CONV/LSTM/HYBRID) into a single shared function - Eliminates risk of behavioural drift where one architecture missed a setter, causing mismatched feature sets or targets
2026-08-01 11:27:28 -04:00
{
//--- indicator tuning defaults live in CADIndicatorTuner's own constructor (Expert\ADIndicatorTuner.mqh),
//--- which runs automatically for the m_indicatorTuner member above.
}
//+------------------------------------------------------------------+
//| Destructor |
//+------------------------------------------------------------------+
CExpertSignalAIBase::~CExpertSignalAIBase(void)
{
//--- deliberately NOT calling PersistOnShutdown() here: OnDeinit() (Warrior_EA.mq5) already calls
//--- it explicitly for every signal, one call stack frame shallower, BEFORE Expert.Deinit() tears
//--- these objects down. Doing it again here nested inside that same teardown cascade doubled the
//--- stack depth of an already-deep recursive save (layers -> neurons -> connections) right at the
//--- point in MT5's lifecycle (EA recompile while attached) that has the least stack headroom, and
//--- reliably crashed the terminal with a stack overflow. Keep this destructor cheap.
if(CheckPointer(Net) != POINTER_INVALID)
delete Net;
if(CheckPointer(m_shadowNet) != POINTER_INVALID)
delete m_shadowNet;
if(CheckPointer(TempData) != POINTER_INVALID)
delete TempData;
if(CheckPointer(m_simOosNet) != POINTER_INVALID)
delete m_simOosNet;
feat: excursion-size head (Stage 1, measurement only) Direction is closed - normalised asymmetry fails on three instruments with a working positive control, and the classifier's own best-of-999 era-cap test agrees (+0.9pp = 1.48 sigma, family-wise p=1.0000). SIZE is a different question and RANGE clears at ~4x its null. Checked the denomination before building on that, since the source memo warns to: m_excUpCache holds (maxHigh - fill)/ATR, so "RANGE is predictable" is a claim about travel RELATIVE to current ATR, not a restatement of "ATR is autocorrelated". It is exactly the part a fixed multiple (stop 3.31*ATR, target 1.64*ATR) discards. A second small CNet, 760 -> 24 -> 32 sigmoid outputs = P(price reaches ladder rung k) upward and downward. Survival parameterisation rather than regressing the multiple, because it needs nothing new from CNet: sigmoid outputs and the per-neuron delta the `total != 3` branch already applies (a quantile head would need a linear activation and a pinball gradient in Network.mqh, Network.cl and the DirectML path, on a class four topologies share). Targets are free - m_ladderUpAt already records first-touch age per rung with 0 meaning never reached. Separate net, not extra outputs on the classifier: more outputs would change m_outputNeuronsCount, the .nnw shape and the fingerprint, and push the count off 3 - the exact condition backProp uses to select the joint softmax gradient the 3-class head depends on. The classifier is bit-for-bit unaffected and this is removable without trace. STAGE 1 PLACES NO ORDERS. It reports a Brier skill score against the constant per-rung base rate - the baseline a fixed ATR multiple already assumes - with both predictors fitted IS and evaluated OOS, so neither gets a look at the test set. Positive skill justifies Stage 2 (drive SL/TP and sizing off ExcursionQuantile, which is defined and deliberately uncalled). Zero or negative means ATR already carries everything and Stage 2 must not be built. Trains only on primary occurrences: the replay queue oversamples for CLASS balance, and a direction-balanced sample is a biased SIZE sample. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-11 07:40:01 -04:00
//--- Excursion head: never persisted (Stage 1 is a measurement), so teardown is the whole lifecycle.
if(CheckPointer(m_excNet) != POINTER_INVALID)
delete m_excNet;
if(CheckPointer(m_excTgt) != POINTER_INVALID)
delete m_excTgt;
if(CheckPointer(m_excOut) != POINTER_INVALID)
delete m_excOut;
fix(chart): arrows survived the EA that drew them - persist, then clear Reported: on deinit the panel and status label go, the signal arrows stay. Two independent causes, both fixed here. 1. It was partly deliberate. ShutdownChartCleanup carried a second behaviour selected by a `preserveChartArrows` flag derived from the deinit reason: on RECOMPILE / PARAMETERS / CHARTCHANGE / TEMPLATE the arrows were left on the chart on purpose, to avoid a reload flicker. That branch IS the reported symptom, an operator cannot tell it apart from a cleanup that failed, and it was outright wrong whenever the reload changed the config - REASON_PARAMETERS means exactly that, and the preserved arrows then belonged to a model the chart no longer runs, with nothing marking them stale. It is gone, along with the flag and m_purgeChartOnDestruct. One path now: persist, clear, restore on the next attach. 2. Whatever remains was unfalsifiable. PurgeChart was a single ObjectsDeleteAll(prefix) whose return value was discarded, with no caller ever looking at the chart again - so "the arrows are still there" and "the arrows were never there" produced identical evidence, which is why the report survived three sessions. It now verifies: after the bulk delete it walks the OBJ_ARROW-typed list (a handful of objects, not the whole chart), deletes any surviving WarSig_ by name, and says so. Costs one typed scan when the bulk delete works, which is the normal case; names the root cause when it does not. Every failure mode of SaveChartSignals was also silent - it returned void and had three bare early returns. It returns bool now, logs the open error with the filename, and the shutdown purge is CONDITIONAL on it: for a converged model the chart objects are the only copy of its signal history (nothing redraws them - the renderer runs per training era and a deployed model has none left), so a chart left littered because the disk write failed beats a clean chart bought by destroying the history. Either way the log now says which happened. Also states the user's rule once, where arrows come back rather than across InitNeuralNetwork's several exits: no weights loaded for this config => clear the sidecar and start visually clean. A fresh run must not inherit calls it never made, and the first save would otherwise adopt them (the sidecar is rebuilt by scanning the chart). Compiles 0 errors / 0 warnings, standard and Market. Needs redeploy. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-01 13:28:34 -04:00
//--- Unconditional now that the warm-reload "leave the arrows up" branch is gone (see
//--- ShutdownChartCleanup). Cheap and idempotent: OnDeinit already purged, so this normally deletes
//--- nothing - it exists for the teardown paths that never reach OnDeinit (a failed OnInit).
PurgeChart();
refactor(ai): extract Layer.mqh and deduplicate AI config - Moves CLayer neuron construction to AI/Impl/Layer.mqh to keep Network.mqh clean - Unifies four previously duplicated architecture initialisation blocks (MLP/CONV/LSTM/HYBRID) into a single shared function - Eliminates risk of behavioural drift where one architecture missed a setter, causing mismatched feature sets or targets
2026-08-01 11:27:28 -04:00
//--- Last, and cheap by design (one global-variable delete): the claim must outlive every save above
//--- it, or a chart re-attaching during this teardown could start writing the same files mid-save.
ReleaseConfigLock();
}
//+------------------------------------------------------------------+
//| Sets the file/id identity a subclass constructor would otherwise |
//| repeat verbatim (ID, m_id, m_folderPath, m_fileName, pattern count)|
//+------------------------------------------------------------------+
void CExpertSignalAIBase::SetIdentity(string id, string shortId, int patternCount = 4)
{
ID = id;
m_id = shortId;
m_folderPath = eaName + "\\" + "Neural Networks" + "\\" + "State" + "\\" + m_id + "\\";
m_fileName = m_folderPath + _Symbol + "_" + IntegerToString(_Period);
m_pattern_count = patternCount;
}
//+------------------------------------------------------------------+
//| "Voting" that price will grow. |
//+------------------------------------------------------------------+
int CExpertSignalAIBase::LongCondition(void)
{
int result = 0;
//--- Readiness gate: live trading still requires a converged model, but an inference-only tester run
//--- may replay a model that was ACTUALLY loaded from disk even if its persisted trainingComplete flag
//--- is false. Without that exception the tester could seed/refresh dPrevSignal from the deployed model
//--- and draw chart arrows from those weights, yet this gate would still hard-zero the trading vote.
//--- A fresh random topology still cannot trade in the tester because m_modelLoadedFromDisk stays false.
//--- Census: this gate is invisible to the refresh-path counters and is a live candidate for the
//--- all-bars-zero-direction backtest - see m_voteGateBlocked.
NoteVoteGate(DoubleToSignal(dPrevSignal) == Buy);
if(!m_trainingComplete && !(m_inferenceOnly && m_modelLoadedFromDisk))
return 0;
//--- No alternation gate any more - see the removal note at m_voteGateBlocked's declaration. Under
//--- triple-barrier labels consecutive same-direction setups are ordinary and correct.
//--- "not yet studied" sentinel - dPrevSignal == -2 is not a real Sell. Its MAGNITUDE is 2, so it
//--- passed straight through the confidence floor that used to sit here (|-2| exceeds any 0..1
//--- threshold); only the m_trainingComplete gate above was keeping it out. Checked explicitly now
//--- that the floor is gone, rather than left resting on that.
if(dPrevSignal == -2)
return 0;
//--- NO confidence floor here, by design - see m_minSignalConfidence's former declaration site. A
//--- weak call is not blocked at the AI's own boundary; it votes at its tier weight (as low as
//--- m_pattern_0) and is then filtered by Min vote to open, exactly like a weak classic vote.
if(DoubleToSignal(dPrevSignal) == Buy)
{
int tier = ConfidenceTier();
result = PatternWeightForTier(tier);
m_active_pattern = "Pattern_" + IntegerToString(tier);
m_active_direction = "Buy";
}
return(result);
}
//+------------------------------------------------------------------+
//| "Voting" that price will fall. |
//+------------------------------------------------------------------+
int CExpertSignalAIBase::ShortCondition(void)
{
int result = 0;
//--- Readiness gate - see LongCondition's matching comment.
NoteVoteGate(DoubleToSignal(dPrevSignal) == Sell);
if(!m_trainingComplete && !(m_inferenceOnly && m_modelLoadedFromDisk))
return 0;
//--- "not yet studied" sentinel, and no confidence floor - see LongCondition's matching comments.
if(dPrevSignal == -2)
return 0;
if(DoubleToSignal(dPrevSignal) == Sell)
{
int tier = ConfidenceTier();
result = PatternWeightForTier(tier);
m_active_pattern = "Pattern_" + IntegerToString(tier);
m_active_direction = "Sell";
}
return result;
}
//+------------------------------------------------------------------+
//| Buckets the live confidence magnitude into one of 4 equal bands |
//| between the head's own structural floor and 1.0 - see m_pattern_0's|
//| declaration comment for the resulting tier/weight table. Neither |
//| head has a configurable floor: the boundary is 1/3 for the 3-class|
//| softmax and 0.5 for regression (DoubleToSignal's own threshold), |
//| both arithmetic properties of the head rather than settings. |
//+------------------------------------------------------------------+
int CExpertSignalAIBase::ConfidenceTier(void)
{
//--- The head's own STRUCTURAL decision boundary - the lowest confidence magnitude that head can
//--- possibly report for a directional call - not a user setting:
//--- - 3-class classification: the winning class of a 3-way softmax is arithmetically >= 1/3, since
//--- three probabilities summing to 1 cannot all be below it. Nothing can ever be read below this.
//--- - single-neuron regression: 0.5, DoubleToSignal()'s own decision boundary.
//--- Quartiling from HERE (rather than from an input, as the classification branch used to) is what
//--- makes the tier boundaries a fixed property of the model instead of something that silently moves
//--- whenever the trader adjusts an unrelated vote threshold - and it is what lets the same tier
//--- weights mean the same thing on both heads. See m_pattern_0's declaration comment for the weights.
double floorConf = (m_outputNeuronsCount == 3) ? (1.0 / 3.0) : 0.5;
double span = MathMax(1.0 - floorConf, 0.0001);
double t = (CalibratedConfidenceMagnitude() - floorConf) / span;
int tier = (int)MathFloor(t * 4.0);
return MathMax(0, MathMin(tier, 3));
}
//+------------------------------------------------------------------+
//| Returns the given tier's current pattern weight (0-100) |
//+------------------------------------------------------------------+
int CExpertSignalAIBase::PatternWeightForTier(int tier)
{
switch(tier)
{
case 0:
return m_pattern_0;
case 1:
return m_pattern_1;
case 2:
return m_pattern_2;
default:
return m_pattern_3;
}
}
//+------------------------------------------------------------------+
//| Set the specified pattern's weight to the specified value |
//+------------------------------------------------------------------+
void CExpertSignalAIBase::ApplyPatternWeight(int patternNumber, int weight)
{
switch(patternNumber)
{
case 0:
Pattern_0(weight);
break;
case 1:
Pattern_1(weight);
break;
case 2:
Pattern_2(weight);
break;
case 3:
Pattern_3(weight);
break;
default:
break;
}
}
//+------------------------------------------------------------------+
//| OnTick function |
//+------------------------------------------------------------------+
void CExpertSignalAIBase::OnTickHandler(void)
{
ScheduleTrainingIfNeeded();
}
//+------------------------------------------------------------------+
//| Schedules the next training pass (if one is due) and refreshes |
//| the per-tick status label. Factored out of OnTickHandler() so |
//| Warrior_EA.mq5's always-on timer (see PollTraining()) can drive |
//| this on a fixed wall-clock schedule too - training must not stall |
//| just because the market is closed and no ticks are arriving. |
//+------------------------------------------------------------------+
void CExpertSignalAIBase::ScheduleTrainingIfNeeded(void)
{
//--- stopped: no new training passes get scheduled at all (StartTraining() re-arms this).
//--- paused: still schedule so bEventStudy/dtStudied bookkeeping stays current, but Train() itself
//--- blocks at the next era boundary until resumed - keeps in-memory state coherent either way.
//--- complete: training already converged - a plain new bar must NOT re-enter Train()'s full era
//--- loop, which would otherwise reset the best-checkpoint/eta-decay tracking and run real
//--- Net.backProp() passes again, forever, once per bar, on an already-converged model (see
//--- RefreshConvergedSignal()'s declaration comment). Just keep the live signal current instead.
//--- publish this signal's current signed confidence for the intelligent trailing (and any other
//--- live-confidence consumer) - see g_LiveAISignedConfidence in Variables\ConfidenceBridge.mqh.
//--- Cheap: SignedAIConfidence() just reads the already-computed dPrevSignal.
g_LiveAISignedConfidence = SignedAIConfidence();
datetime lastBarDate = (datetime)SeriesInfoInteger(m_symbol.Name(), m_period, SERIES_LASTBAR_DATE);
// A failed lookup (0) must not silently read as "dtStudied is already caught up, nothing pending" -
// that would freeze this function into never re-triggering training/signal refresh again until some
// other path happens to bump dtStudied. Treat a failed lookup as pending instead (same >0-guard
// philosophy as the SERIES_FIRSTDATE lookup elsewhere in this class) so a transient history-sync
// hiccup costs one extra harmless check, not a silent stall.
bool newBarPending = (dPrevSignal == -2 || lastBarDate <= 0 || ((m_inferenceOnly ? m_lastBarTime : dtStudied) < lastBarDate));
fix: training could only advance one 120ms chunk per bar ScheduleTrainingIfNeeded() armed the next Train() call only when dtStudied < lastBarDate. That watermark test is right for a CONVERGED model - one inference refresh per new bar - and wrong for a training run, because Train() is chunked: it does ~120ms of work and yields, needing thousands of calls to finish one era, and every one of those calls has to be armed from there. dtStudied is two incompatible things. Train() sets it to the training WINDOW START (~2008); FinalizeTrainRun() sets it to the last bar SCANNED (~now). So the moment any run finalized, the scheduler went silent until the next candle closed. On H1 that is one chunk per hour. The symptom was indistinguishable from a hang: no era lines, no heartbeats, not one of the six instrumented stall branches - because Train() was not being CALLED. The TRAIN STALL line that caught it reported runActive=Y only because m_trainRunActive had been set microseconds earlier in that same call, and eraResume=N proved no era was in flight. Two log bursts, 28 minutes apart, exactly one H1 bar. Before 0c85c54 this was survivable rather than correct: the saved watermark left almost no bars eligible per era, so eras were nearly free and one call per bar still looked like progress. An unconverged model is now always pending. Pause/stop are handled by m_trainingPaused/m_trainingStopRequested, which Train() checks itself. Also: the one Train() exit that tears down the whole run on a buffer failure was completely silent - it now says so. And the build tag moves to train-dispatch-v2; it had not moved since ce52654, which is why the running binary could not be identified from its own log. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-10 10:06:49 -04:00
//--- A MODEL THAT HAS NOT FINISHED TRAINING IS ALWAYS PENDING. The watermark test above answers
//--- "has a new bar closed since the last one we processed", which is the right question for a
//--- CONVERGED model (one inference refresh per bar, see the branch below) and the wrong one for a
//--- training run: Train() is chunked - it does ~TRAIN_TIME_BUDGET_MS of work and yields, needing
//--- thousands of calls to finish a single era - yet every one of those calls has to be armed from
//--- here. Gating them on the watermark meant training could only advance when a new BAR closed.
//--- On H1 that is one 120ms chunk per hour.
//--- The trap is that dtStudied is two different things: Train() sets it to the training WINDOW
//--- START (TrainWindowStart, ~2008) while FinalizeTrainRun() sets it to the last bar SCANNED
//--- (~now). So the moment any run finalized, dtStudied >= lastBarDate and this function went
//--- silent until the next candle - no era lines, no heartbeats, no stall branch, nothing, because
//--- Train() was not being CALLED at all. Observed 2026-08-10: four charts, 28 minutes of silence
//--- between two bursts exactly one H1 bar apart, and the TRAIN STALL line that finally caught it
//--- reported runActive=Y only because m_trainRunActive had been set microseconds earlier in that
//--- same call. Before 0c85c54 this was survivable rather than correct: eras were nearly free (the
//--- saved watermark left almost no bars eligible), so one call per bar still looked like progress.
//--- There is never a reason to withhold a Train() call from an unconverged model - pause/stop are
//--- handled by m_trainingPaused/m_trainingStopRequested, which Train() checks for itself.
bool trainingPending = !(m_trainingComplete || m_inferenceOnly);
refactor(ai): extract Layer.mqh and deduplicate AI config - Moves CLayer neuron construction to AI/Impl/Layer.mqh to keep Network.mqh clean - Unifies four previously duplicated architecture initialisation blocks (MLP/CONV/LSTM/HYBRID) into a single shared function - Eliminates risk of behavioural drift where one architecture missed a setter, causing mismatched feature sets or targets
2026-08-01 11:27:28 -04:00
//--- m_inferenceOnly (single backtest) takes the converged/inference branch even if the seeded model
//--- wasn't flagged complete, so a backtest never drops into Train()'s era loop - see m_inferenceOnly.
if((m_trainingComplete || m_inferenceOnly) && !m_trainingStopRequested && !m_trainRunActive)
{
if(newBarPending)
RefreshConvergedSignal();
}
else
fix: training could only advance one 120ms chunk per bar ScheduleTrainingIfNeeded() armed the next Train() call only when dtStudied < lastBarDate. That watermark test is right for a CONVERGED model - one inference refresh per new bar - and wrong for a training run, because Train() is chunked: it does ~120ms of work and yields, needing thousands of calls to finish one era, and every one of those calls has to be armed from there. dtStudied is two incompatible things. Train() sets it to the training WINDOW START (~2008); FinalizeTrainRun() sets it to the last bar SCANNED (~now). So the moment any run finalized, the scheduler went silent until the next candle closed. On H1 that is one chunk per hour. The symptom was indistinguishable from a hang: no era lines, no heartbeats, not one of the six instrumented stall branches - because Train() was not being CALLED. The TRAIN STALL line that caught it reported runActive=Y only because m_trainRunActive had been set microseconds earlier in that same call, and eraResume=N proved no era was in flight. Two log bursts, 28 minutes apart, exactly one H1 bar. Before 0c85c54 this was survivable rather than correct: the saved watermark left almost no bars eligible per era, so eras were nearly free and one call per bar still looked like progress. An unconverged model is now always pending. Pause/stop are handled by m_trainingPaused/m_trainingStopRequested, which Train() checks itself. Also: the one Train() exit that tears down the whole run on a buffer failure was completely silent - it now says so. And the build tag moves to train-dispatch-v2; it had not moved since ce52654, which is why the running binary could not be identified from its own log. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-10 10:06:49 -04:00
if(!m_trainingStopRequested && !bEventStudy && (newBarPending || trainingPending))
refactor(ai): extract Layer.mqh and deduplicate AI config - Moves CLayer neuron construction to AI/Impl/Layer.mqh to keep Network.mqh clean - Unifies four previously duplicated architecture initialisation blocks (MLP/CONV/LSTM/HYBRID) into a single shared function - Eliminates risk of behavioural drift where one architecture missed a setter, causing mismatched feature sets or targets
2026-08-01 11:27:28 -04:00
bEventStudy = EventChartCustom(ChartID(), 1, (long)MathMax(0, MathMin(iTime(m_symbol.Name(), PERIOD_CURRENT, (int)(100 * Net.recentAverageSmoothingFactor * (m_trainingComplete ? 1 : 10))), dtStudied)), 0, "New Bar");
//--- Train() (see its declaration comment) now yields every ~TRAIN_TIME_BUDGET_MS instead of
//--- blocking for a whole era, so while a run is active this per-tick line would otherwise
//--- overwrite Train()'s own full-detail status label on every single tick between chunks -
//--- flickering between the two instead of showing one steady picture. Only write this terse
//--- summary when nothing else is actively updating the status label (idle/stopped/paused/cooldown).
if(!m_trainRunActive)
{
//--- Compact, accurate end-state text. The completed state distinguishes a model that is genuinely
//--- adapting live (online learning active - a live chart with EnableOnlineLearning, not the tester)
//--- from one running pure inference (the Strategy Tester, or online learning off), so the label is
//--- literally true either way and never over-promises "keeps learning" when it doesn't - see
//--- OnlineLearnStep()'s gate for exactly when adaptation runs.
bool onlineActive = m_enableOnlineLearning && !m_inferenceOnly
&& !MQLInfoInteger(MQL_TESTER) && !MQLInfoInteger(MQL_OPTIMIZATION) && !MQLInfoInteger(MQL_FORWARD)
&& CheckPointer(Net) != POINTER_INVALID && !Net.CpuInference();
//--- Simple end-state panel (default, VerboseMode off): plain-language status + the model's
//--- compounded/persistent Buy/Sell win-rate (directional accuracy, Neutral excluded - persisted in
//--- .stats WST5, so it survives a fresh chart reload and is meaningful the moment a drop-and-go user
//--- attaches the EA) + the current call. The verbose era/forecast dump below stays for power users.
if(!VerboseMode)
{
string statusPlain;
if(m_trainingComplete)
statusPlain = onlineActive ? "Live - learning from new bars" : "Ready for live trading";
else if(m_trainingStopRequested)
statusPlain = "Paused - progress saved";
else if(m_trainingPaused)
statusPlain = "Paused";
else
statusPlain = "Getting ready...";
ENUM_SIGNAL liveSig = DoubleToSignal(dPrevSignal);
string liveSigPlain = (liveSig == Buy) ? "Buy" : (liveSig == Sell) ? "Sell" : "Neutral (no trade)";
string simpleLive = DisplayName() + " - " + statusPlain + "\n";
//--- Only show the accuracy line once at least one signal has been validated (compounded counts
//--- persist across restarts, so a deployed model shows real numbers immediately, not "measuring").
if(m_cumIsTotal > 0 || m_cumOosTotal > 0)
simpleLive += ComputeCompoundedAccuracyLine() + "\n";
simpleLive += "Current signal: " + liveSigPlain;
SetStatusLabel(simpleLive);
return;
}
string completeText = onlineActive ? "Complete - live (adapting to new bars)" : "Complete - ready for live (inference)";
string trainingState = m_trainingStopRequested
? (m_trainingComplete ? completeText : "Stopped - resumable (weights kept)")
: (m_trainingPaused ? "Paused" : (m_trainingComplete ? completeText : "In progress"));
//--- same "Forecast: <signal> -> <value>" line the active training loop's status label ends on
//--- (see the classLine-terminated StringFormat below), instead of a raw bEventStudy/dPrevSignal/
//--- dtStudied debug dump - this is what stays on screen once training stops/pauses/completes.
SetStatusLabel(StringFormat(
ID + " : Era %d -> Training %s\n" +
"Forecast: %s -> %.2f",
m_eraCount, trainingState,
EnumToString(DoubleToSignal(dPrevSignal)), dPrevSignal));
}
}
//+------------------------------------------------------------------+
//| Timer-driven equivalent of OnTickHandler()'s scheduling, called |
//| from Warrior_EA.mq5's always-on OnTimer() so training keeps |
//| progressing purely on wall-clock time - no dependency on ticks, |
//| which simply don't arrive while the market is closed. |
//+------------------------------------------------------------------+
void CExpertSignalAIBase::PollTraining(void)
{
//--- Drain a slice of the queued chart-arrow restore FIRST, and skip training work on any tick where
//--- restoring is still in flight. Both compete for the one MQL5 thread; letting the arrows finish
//--- quickly (a few hundred ms of slices) means the user sees a complete chart almost immediately,
//--- whereas interleaving them with 80ms training chunks would stretch the restore over minutes.
if(m_arrowRestorePending)
{
AdvanceChartSignalRestore();
return;
}
//--- Same one-thread reasoning as the arrow restore above: a manual rescan (Show Signals) also competes
//--- for the single MQL5 thread, and its per-bar feedForward is real compute rather than a cheap object
//--- write, so it must finish its own slices before training resumes rather than interleaving with it.
if(m_rescanPending)
{
AdvanceChartSignalRescan();
return;
}
if(m_isInitialized)
ScheduleTrainingIfNeeded();
}
//+------------------------------------------------------------------+
//| |
//+------------------------------------------------------------------+
void CExpertSignalAIBase::OnChartEventHandler(const int id,
const long &lparam,
const double &dparam,
const string &sparam)
{
if(id == 1001)
{
TuneIndicatorsAndTrain(lparam);
bEventStudy = false;
OnTickHandler();
}
}
//+------------------------------------------------------------------+
//| Claim m_activeFileName for this chart, terminal-wide. |
//| |
//| Two charts running the same AIType with the same retrain-affecting|
//| inputs resolve to the SAME .nnw/.cfg/.stats/checkpoint set. Both |
//| then train independently and save over each other, so whichever |
//| writes last wins and the other's eras are discarded - silently, |
//| because every individual file operation succeeds. A five-chart |
//| comparison run on 2026-07-29 lost both its HYBRID models this way |
//| (one chart left at the AIType default), and the only evidence |
//| anywhere was that model path appearing twice as often in the log. |
//| |
//| A terminal-wide global variable is the right lock rather than a |
//| lock FILE: GlobalVariableTemp() is an atomic create-if-absent, |
//| and a TEMPORARY variable dies with the terminal, so a crash can |
//| never leave a stale lock that blocks the next start. Within one |
//| session a stale entry is still possible (an EA removed without a |
//| clean deinit), so the owner's chart id is stored and revalidated. |
//+------------------------------------------------------------------+
bool CExpertSignalAIBase::AcquireConfigLock(void)
{
//--- FNV-1a over the resolved filename: every retrain-affecting input is already folded into that
//--- name, so equal names mean genuinely equal configs and nothing else has to be compared. Hashed
//--- because MQL5 caps global-variable names at 63 characters and the path alone exceeds that.
uint h = 2166136261;
int len = StringLen(m_activeFileName);
for(int i = 0; i < len; i++)
{
h ^= (uint)StringGetCharacter(m_activeFileName, i);
h *= 16777619;
}
string name = "WarriorAI_" + m_id + "_" + StringFormat("%08x", h);
long self = ChartID();
//--- Atomic: true means it did not exist and is now ours.
if(GlobalVariableTemp(name))
{
GlobalVariableSet(name, (double)self);
m_configLockName = name;
return true;
}
long owner = (long)GlobalVariableGet(name);
//--- Our own entry: this chart is re-initializing after a parameter change or a recompile whose
//--- OnDeinit never reached ReleaseConfigLock(). Reclaim it instead of refusing to start.
if(owner == self)
{
m_configLockName = name;
return true;
}
//--- Owner recorded but its chart no longer runs an expert - take the claim over. owner == 0 is
//--- deliberately NOT treated as stale: it means another instance created the variable microseconds
//--- ago and has not stamped its id yet, which is a live claim, not a dead one.
bool ownerAlive = false;
if(owner != 0)
{
long id = ChartFirst();
while(id >= 0)
{
if(id == owner)
{
ownerAlive = (StringLen(ChartGetString(id, CHART_EXPERT_NAME)) > 0);
break;
}
id = ChartNext(id);
}
}
if(owner != 0 && !ownerAlive)
{
GlobalVariableSet(name, (double)self);
m_configLockName = name;
return true;
}
Print(ID + ": REFUSED to start - another chart is already training this exact configuration. Both" +
" would save into the same files (" + m_activeFileName + ".nnw plus its .cfg/.stats/checkpoints)" +
" and overwrite each other's progress with no error reported anywhere. Owner: " +
(owner != 0 ? "chart " + IntegerToString(owner) + " (" + ChartSymbol(owner) + " " +
EnumToString((ENUM_TIMEFRAMES)ChartPeriod(owner)) + ")" : "another chart, still initializing") +
". Change AIType or any retrain-affecting input on THIS chart so it trains its own model, or" +
" remove one of the two charts. Note AIType defaults to " + EnumToString(AI_HYBRID) +
" - a chart whose AIType was never actually changed lands here.");
return false;
}
//+------------------------------------------------------------------+
//| Drop this instance's claim (see AcquireConfigLock). |
//+------------------------------------------------------------------+
void CExpertSignalAIBase::ReleaseConfigLock(void)
{
if(StringLen(m_configLockName) == 0)
return;
//--- Only delete a claim we still hold: if a later instance took this entry over via the stale-owner
//--- path above, deleting it here would silently hand the config to a third chart.
if((long)GlobalVariableGet(m_configLockName) == ChartID())
GlobalVariableDel(m_configLockName);
m_configLockName = "";
}
#endif
Copy permalink