4088850/Warrior_EA
Watch
1
0
Fork
You've already forked Warrior_EA
 0
forked from animatedread/Warrior_EA
Code Pull requests Activity
Warrior_EA/Expert/ExpertSignalAIBase.mqh
AnimateDread 000c45fdbb feat: print a self-verifying config line per chart at startup 
A multi-chart comparison is only valid if every chart is identical except
the axis under test, and a drifted setting was previously invisible: the
model filename carries a HASH, so two charts that should match and do not
look merely "different" with no indication of which field moved.

Each signal now logs its effective config plus the raw fingerprint string,
unconditionally (not gated on VerboseMode). The six lines diff directly, so
an accidental divergence in study period, feature set, focal gamma or
anything else feeding training shows up at startup rather than as an
unexplained result hours later.

Both builds compile 0 errors, 0 warnings.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-29 20:22:22 -04:00

3520 lines
243 KiB
MQL5
Raw Permalink Blame History

//+------------------------------------------------------------------+
//| Warrior_EA |
//| AnimateDread |
//| |
//+------------------------------------------------------------------+
#include "ExpertSignalCustom.mqh"
#include "..\AI\Network.mqh"
#include "..\Variables\IndicatorResources.mqh"
#include "..\Variables\IndicatorTuneRanges.mqh"
#include "..\System\StatusLabel.mqh"
#include "..\System\NewsRelevance.mqh"
#include "ADIndicatorTuner.mqh"
//--- Hard 0/1 one-hot targets for the 3-neuron classification head - matches the book's
//--- (nnbook.txt sec. 1.4 "Cross-entropy") cross-entropy formulation, which defines the reference
//--- distribution's occurring-event probability as exactly 1.0 (missing event exactly 0.0), no
//--- smoothing. This used to be softened to 0.9/0.1 (see git history) as a workaround for the OLD
//--- per-neuron independent-sigmoid backward gradient: since that gradient was each neuron's own
//--- raw (target-sigmoid_output) delta with nothing else driving it to zero, a literal 1.0/0.0
//--- target was only ever approached asymptotically, so weights kept growing (up to the MAX_WEIGHT
//--- clamp) chasing it forever. Now that CNet::backProp()/backPropOCL() (AI\Network.mqh) compute a
//--- joint softmax+categorical-cross-entropy gradient (softmax_i - target_i) across all 3 neurons
//--- instead, the winning class's SOFTMAX probability - not any one neuron's raw sigmoid value -
//--- is what needs to approach the target, and softmax can reach very close to 1.0 for the winning
//--- class from ordinary (non-saturated) logit separation, so the old runaway-weight failure mode
//--- is no longer expected to require smoothed targets to avoid. If it resurfaces in practice (the
//--- per-neuron SIGMOID forward pass can still individually saturate before softmax normalizes),
//--- that's the first thing to check before reintroducing smoothing.
// Soft labels keep the classification head from overfitting to hard one-hot targets while still
// preserving a clear target for the true class. The true class gets 0.9 and the others 0.05 each.
#define LABEL_SMOOTH_HIGH 0.9
#define LABEL_SMOOTH_LOW 0.05
//--- Namespace prefix for the directional signal arrows this class draws (DrawObject/DeleteObject). Two
//--- reasons it exists: (1) so PurgeChart() can delete ONLY our arrows and never the user's own manual
//--- chart drawings (a full ObjectsDeleteAll(0) on a client's chart is not acceptable for a commercial
//--- product), and (2) so arrows can survive an EA re-init instead of being wiped on every InitIndicators
//--- - the whole point of keeping them on screen (the panel has a hide toggle if the user wants them gone).
#define SIG_ARROW_PREFIX "WarSig_"
//--- Upper bound on arrows restored from a .arrows file (see LoadChartSignals). Purely a guard against a
//--- corrupt header declaring a garbage count - a long converged run legitimately accumulates a few
//--- thousand, so this sits well above that. Restoring is chunked across timer calls regardless, so a
//--- large-but-valid count costs progressive fill-in, never a frozen OnInit.
#define MAX_RESTORED_ARROWS 50000
//--- How many of the MOST RECENT arrows are kept on the chart and in the .arrows sidecar. Arrows would
//--- otherwise accumulate for the life of the model (a converged SP500 H1 run had reached 2896), which
//--- clutters the chart, slows every save/restore, and preserves history nobody scrolls back to. Both
//--- SaveChartSignals (which also DELETES the pruned objects from the chart) and LoadChartSignals select
//--- by TIME, not by scan order - ObjectsTotal() order is arbitrary, so "the last N scanned" would keep a
//--- random subset rather than the newest.
#define MAX_PERSISTED_ARROWS 1000
//--- Manual "rescan" window (see RescanChartSignals): how many of the MOST RECENT bars get re-inferred
//--- from the currently deployed weights when the operator asks for a fresh signal set. Bounded well
//--- below a full StudyPeriods re-render (which can be years of bars and would freeze the one MQL5 chart
//--- thread, same doctrine as MAX_RESTORED_ARROWS/ARROW_RESTORE_BUDGET_MS) - a rescan is meant to replace
//--- stale old arrows with what the model calls on RECENT history, not reproduce the whole training run.
#define SIGNAL_RESCAN_LOOKBACK_BARS 5000
//--- Wall-clock budget per chunk of the deferred arrow restore. Same doctrine as TRAIN_TIME_BUDGET_MS:
//--- MQL5 gives a chart ONE thread, so "async" here means small time-boxed slices between which the
//--- terminal can service the panel, the chart and the journal - never a single long blocking pass.
//--- 50ms sits between the training chunk (120ms, already tolerated) and the 500ms timer period, so the
//--- restore completes in a handful of slices while leaving ~90% of each timer window free for the UI.
#define ARROW_RESTORE_BUDGET_MS 50
//--- Max allowed |Δ| between the compute backend's and the pure-MQL5 path's outputs for a model to be
//--- marked MQL5-inference-safe (see ValidateCpuInference). The outputs are bounded sigmoid values;
//--- backend-vs-double summation-order noise is ~1e-6 on the CPU-DLL path (double) and stays well under
//--- this even on a float32 OpenCL backend, while a genuine math/port bug shows up as >0.01. Fails safe.
#define CPU_INFERENCE_MAX_DIFF 1.0e-3
extern bool g_signalsVisible;
//--- Genetic + successive-halving auto-tuner tuning knobs (see TuneIndicatorsAndTrain). GA_SEEDS is the
//--- number of fixed-seed repeat runs averaged per candidate (the "average of 3 runs" stability measure);
//--- GA_RUNGS successive-halving rungs prune bad candidates cheaply before spending a realistic budget on
//--- the survivors, with per-rung era budgets in GaRungEras(). GA_SEED_BASE offsets MathSrand() per seed.
#define GA_SEEDS 3
#define GA_RUNGS 3
#define GA_SEED_BASE 90001
#define GA_MAX_GENERATIONS 4
//--- Outer safety cap on how many times a single minority-class (Buy/Sell) bar can be duplicated into
//--- m_isTrainQueue - see that queueing block's comment for the full rationale (data-level oversampling
//--- to work around Adam's near-invariance to constant loss reweighting, per Kingma & Ba 2015). The
//--- queueing block derives the REAL per-bar repCount from the RAW previous-era class ratio (uncapped),
//--- bounded by this constant, and applies NO additional per-occurrence loss weight on top of it -
//--- duplication alone carries the whole correction. Two earlier versions of this both stacked a second
//--- multiplicative correction (m_maxClassSampleWeight) on top of repCount and both collapsed in
//--- opposite directions - see the queueing block's comment for the full history. This constant is now
//--- the ONLY thing bounding the correction's strength, so it directly trades off correction speed
//--- against the momentum-correlation risk of a single bar recurring too often in one era's shuffled
//--- replay order. Was 3, which left a real observed ~5.2-5.3x Buy/Sell imbalance only partially
//--- corrected (repCount=3 -> Neutral still ~1.75-1.8x more frequent than each minority class in the
//--- queue) - the network stayed inside the same Neutral-collapse basin the old loss-weight-only
//--- mechanism plateaued in for eras 20-115 (~73.6% OOS, 0% Buy/Sell recall), just reaching that
//--- plateau faster. Raised to 5 to let repCount reach much closer to full class parity for that
//--- observed ratio, per Buda, Maki & Mazurowski 2018 (oversampling to parity is competitive-or-better
//--- and doesn't meaningfully overfit) - the Fisher-Yates shuffle already scatters duplicates across
//--- the whole era's replay order, so the momentum-correlation risk this cap guards against scales with
//--- how clustered repeats land, not with the raw repCount number itself.
//--- 2026-07-19: repurposed from a hand-tuned working cap into an ABSOLUTE SANITY CEILING only. The
//--- working oversampling rate is now fully measurement-driven so it adapts to any market/timeframe
//--- with no retuning: era 0's rate comes from the eager prebuild's upfront label tally (see
//--- AdvanceLabelCachePrebuild's seeding of m_prevEraTrue*), every later era's from its own
//--- just-measured tally, with repCount = round(measured majority/minority ratio) - parity per Buda
//--- et al., whatever the ratio happens to be (~5x with widened labels, ~35x with exact-pivot labels,
//--- anything else on other symbols/timeframes). This constant only guards the degenerate-tally case
//--- (a class with a handful of true bars in some window producing a 1000x+ ratio - replicating 3
//--- bars 1000x each is noise amplification, not class balance) and must simply sit far ABOVE any
//--- ratio a healthy market/timeframe produces; the history above (3 undershot, 5 fit only the
//--- widened-label regime) is exactly why no hand-tuned value should ever be the binding term.
#define MAX_OVERSAMPLE_REPLICAS 100
//--- Fraction of full class parity the minority (Buy/Sell) oversampling aims for. 1.0 = replicate each
//--- minority bar up to the measured majority/minority ratio, i.e. an equal 1:1:1 queue - which
//--- maximally pushes the softmax boundary toward directional calls. On the 160+-era exact-pivot run
//--- (2026-07-20, SP500 H4) that produced a healthy, stable model that nonetheless OVER-called
//--- direction: Buy fired on ~7% of OOS bars at ~15% precision, Sell ~11% at ~11%, against a ~3%
//--- true base rate each - lots of low-quality directional calls. Oversampling to only a fraction of
//--- parity leaves Neutral proportionally more represented in the training queue, shifting the boundary
//--- back toward calling Neutral unless the directional evidence is stronger - fewer, higher-precision
//--- Buy/Sell calls. 0.7 is a deliberately gentle first step off full parity (the model was already
//--- good); lower it further toward more conservative directional calling, raise toward 1.0 for more.
//--- Measurement-driven like the ratio itself: this scales whatever each era actually measured, so it
//--- stays adaptive across markets/timeframes. Applied to rawRatio BEFORE the round/cap below.
#define OVERSAMPLE_PARITY_FRACTION 0.6
//--- Buy/Sell ground truth is the single exact bar where AdvanceZigZagLabelState() confirms a ZigZag
//--- pivot - on real data that's ~2-3% of bars per class (everything else, including the bar directly
//--- next to a real reversal, is Neutral). Observed in practice as OOS Buy/Sell recall stuck at 0-3%
//--- across many real training eras despite class-balanced loss weighting, label smoothing, and a
//--- bounded output activation all correctly doing their jobs - reweighting a rare example's gradient
//--- makes the optimizer try harder to fit that one bar, it can't manufacture separability between the
//--- pivot bar and its near-identical neighbor if the feature window genuinely can't tell them apart.
//--- AdvanceLabelCachePrebuild() widens each confirmed pivot into a +-LABEL_WINDOW_BARS neighborhood
//--- (never overwriting a DIFFERENT genuine pivot's own exact label) after the sequential ZigZag scan
//--- resolves, so the network is trained/scored against "was this bar near a real reversal" instead of
//--- "was this bar the exact reversal tick." Set to 0 for exact-bar-only labels.
//--- 2026-07-19: back to 0 (exact reversal candle only) by explicit user decision - the model should
//--- learn to pinpoint the reversal bar itself, not its neighborhood. The 0-3%-recall failure described
//--- above that originally motivated the widening was observed on the OLD training stack, BEFORE the
//--- sign-agreement-gate removal, CLASS_LOGIT_SCALE, the WEIGHT_DECAY reduction, the mid-run
//--- label-cache-wipe fix, and the era-diluted "OOS calls %" diagnostic fix (which understated how
//--- directional those runs actually were) - so it is tainted evidence, and exact labels get a clean
//--- re-test on the fixed stack. If Buy/Sell recall pins near 0% again DESPITE a healthy raw-out
//--- spread in the era log, that's the genuine pivot-vs-neighbor separability wall described above -
//--- revert to 2, or give the features swing structure (EnableSwingContext) before giving up on
//--- exact labels. NOTE: exact labels shrink Buy/Sell counts ~5x (~270 each vs ~9100 Neutral, a ~35x
//--- imbalance) - the oversampling rate is measured per era (era 0's from the prebuild tally), so
//--- parity adapts to this automatically; MAX_OVERSAMPLE_REPLICAS above is only a sanity ceiling.
#define LABEL_WINDOW_BARS 0
//--- Reduce-on-regression learning-rate decay. `eta` (AI\Network.mqh) is a plain global read fresh
//--- by every weight-update call on every backend (native/OpenCL/CPU-DLL/DirectML-DLL alike), so
//--- shrinking it here takes effect on the very next backProp() everywhere at once. A fixed step
//--- size that is large enough to escape a bad random init quickly (see the sharp era 5->14 OOS
//--- accuracy climb this was tuned against) is, by the same token, large enough to overshoot once
//--- training gets close to a good solution - the era 14->16 regression right after that climb is
//--- the classic signature of that overshoot, not a structural bug. The existing best-checkpoint
//--- mechanism just below (m_bestOosForecast + CNet::CaptureWeights) already guarantees the FINAL deployed
//--- model can't regress; this constant lets the live training process itself settle down instead
//--- of continuing to oscillate around a good solution once it finds one.
#define ETA_DECAY_REGRESSION_PCT 5.0 // only decay after a real regression, not per-era noise
#define ETA_DECAY_FACTOR 0.7
#define ETA_MIN 0.0001
//--- Balanced accuracy (macro-recall) of a model that puts every bar in ONE class: (0+0+100)/3. It is
//--- the FLOOR of the balanced metric, not a midpoint - any genuinely multi-class model scores above it.
//--- ComputeFirstLayerWidth() constants. SECONDS_PER_YEAR is the mean Julian year (365.25 days), the
//--- same convention MQL5's own date arithmetic uses. MARKET_OPEN_FRACTION allows for closed hours and
//--- weekends - ~0.72 is right for both a 24/5 FX week and an index with extended sessions, and the
//--- ladder in that function makes anything in the 0.6-0.85 range land on the same rung anyway.
//--- FIRST_LAYER_MIN_WIDTH is a floor for degenerate configs (a very short study period, or a symbol
//--- whose history cannot support the requested window) - below this the taper has nothing to work with.
#define SECONDS_PER_YEAR 31557600.0
#define MARKET_OPEN_FRACTION 0.72
#define FIRST_LAYER_MIN_WIDTH 16
//--- Where the dense taper ENDS. The last hidden layer wants to be small enough to force the network to
//--- commit to a compressed representation, but comfortably wider than the decision itself so it is not
//--- the bottleneck - a few units per class is the usual heuristic. The absolute floor covers the
//--- single-output regression head, where 4x1 would be absurdly narrow.
#define HIDDEN_TAPER_OUTPUT_MULTIPLE 4
#define HIDDEN_TAPER_MIN_WIDTH 8
#define BALANCED_COLLAPSE_PCT (100.0 / 3.0)
//--- How far above that floor a best-so-far checkpoint must sit before the regression handler is
//--- allowed to defend it during the pre-recall-pass phase. See the guard's comment in Train(): the
//--- point is to distinguish "the best we have is still basically a collapse, keep exploring freely"
//--- from "we found a real multi-class state and are now sliding off it", which is the case that ran
//--- unchecked for 228 eras on SP500 H1 (2026-07-29).
#define BALANCED_WORTH_DEFENDING_MARGIN_PCT 5.0
//--- PLATEAU LADDER ------------------------------------------------------------------------------
//--- The two branches above only fire on a NEW BEST (isBetterEra) or on a real REGRESSION
//--- (isWorseEra, a drop of more than ETA_DECAY_REGRESSION_PCT below the best). Between them sits a
//--- dead zone - "not better, not meaningfully worse" - where NOTHING happened: no checkpoint, no
//--- restore, no eta change. A run that settles into that band is stuck, and before this ladder
//--- existed it stayed stuck until the era cap (observed 2026-07-25: balanced accuracy pinned in a
//--- 64-69% band for 15 consecutive eras while eta sat frozen and the softmax outputs slowly
//--- compressed toward uniform, with ~970 eras still to burn before the cap would end it).
//---
//--- So: count eras since the last NEW BEST, and escalate when that count says the run has stopped
//--- improving on its own. Two ideas from the literature, in this order:
//--- 1. WARM RESTART (Loshchilov & Hutter, SGDR, ICLR 2017). On a plateau the correct move is a
//--- BIGGER step, not a smaller one - decaying eta into a plateau just entrenches whatever basin
//--- the model is sitting in. Note this is the opposite of the isWorseEra branch's decay, and
//--- deliberately so: decay answers overshoot, restart answers stagnation.
//--- 2. GAMMA ANNEALING (Mukhoti et al., "Calibrating Deep Neural Networks using Focal Loss",
//--- NeurIPS 2020, which schedules gamma DOWN over training rather than fixing it). Focal loss's
//--- (1-pt)^gamma modulator goes to ~0 on everything the model already classifies well, so late
//--- in a run the surviving gradient comes almost entirely from genuinely ambiguous bars - and
//--- near a pivot, ZigZag labels ARE ambiguous. Meanwhile WEIGHT_DECAY keeps pulling every
//--- weight toward zero on every step regardless (see AI\Network.mqh's note that a weight's
//--- sustainable magnitude is ~ its gradient SNR / WEIGHT_DECAY). Annealing gamma hands the
//--- easy-but-correct bars their gradient back, restoring the signal side of that ratio.
//--- Annealing is MONOTONE (gamma only ever decreases within a run), matching the scheduled-gamma
//--- literature; eta may still be bumped back up by the existing recovery bump.
//---
//--- Escalation is per-stage, every PLATEAU_PATIENCE_ERAS eras without a new best. ANY new best
//--- resets the counter and the stage to 0 (the ladder is a response to stagnation, so evidence the
//--- run is moving again retires it) - except the annealed gamma, which stays where it got to.
#define PLATEAU_PATIENCE_ERAS 8 // eras with no new best balanced accuracy before escalating a stage
#define PLATEAU_STAGE_RESTART 1 // warm-restart eta to its ceiling + first gamma step down
#define PLATEAU_STAGE_ANNEAL 2 // gamma to 0 (plain class-balanced CE) + one more warm restart
#define PLATEAU_STAGE_DEPLOY 3 // exhausted: deploy the best checkpoint and finish the run
#define PLATEAU_GAMMA_STEP 1.0 // gamma value stage 1 drops to (from the configured starting value)
//--- FILE-COMPATIBILITY SHIM for the removed "Min OOS accuracy % (converge)" input (was MinWR, default
//--- PCT_80). Convergence is decided by the plateau ladder now, so the value has no behavioural effect
//--- anywhere - but it still occupies a slot in TWO persisted identities that must not shift:
//--- 1. the topology .cfg layout (SaveTopologyConfiguration/LoadAndCompareTopologyConfiguration), and
//--- 2. the weights-FILENAME fingerprint (BuildConfigFingerprint) - change the hash and every existing
//--- model silently becomes unreachable and retrains from era 0.
//--- So keep writing/hashing the old default, and stop COMPARING the .cfg field (see that function) so a
//--- model saved under ANY previous MinWR still loads. Models trained with the shipped default 80 keep
//--- their exact filename and resume normally; one trained under a non-default MinWR gets a new filename
//--- and retrains once, which is unavoidable when a fingerprint field stops being a variable.
#define LEGACY_CONVERGE_WR_SLOT 80
//--- Minimum true OOS samples a class needs this era before its recall is trusted as a real pass -
//--- see directionalRecallOK's declaration comment for the era-44-46 false-convergence this prevents.
//--- 10 is a low bar (still lets a genuinely thin early-run OOS window fall back to "not blocking"
//--- via recallPct==-1), just enough to rule out the zero/near-zero-sample degenerate case.
#define MIN_OOS_CLASS_SAMPLES_FOR_GATE 10
//--- Bound on FindConfirmedZigZagPivot()'s backward scan (m_useSwingContext feature block) - ADZigZag's
//--- own stock defaults (Depth=12) produce pivots frequently enough in normal conditions that this cap
//--- should rarely bind, but a long, unusually strong single-direction run could genuinely go this long
//--- without a confirmed opposite-type pivot. Generous rather than tight: the scan is cheap (plain array
//--- reads, no indicator recompute) and its result is cached per-bar by BufferTempData()'s feature cache,
//--- so the one-time cost of a long scan is paid at most once per unique bar, not per training pass.
#define SWING_SCAN_CAP_BARS 750
//--- EMA shadow-weight deployment blend rate - see m_shadowNet's declaration comment for the full
//--- rationale. 0.01 matches the Tau range (0.001-0.01) used for target-network soft updates in
//--- Dmitriy Gizlyk's reference NeuroNet_DNG-based RL algorithms (references\MQL5\Experts\*\Study.mq5) -
//--- small enough that no single era's raw weights can move the deployed model far, large enough that
//--- the shadow still tracks real, sustained learning within a few dozen eras rather than lagging
//--- forever.
#define SHADOW_WEIGHT_TAU 0.01
//--- Online continual-learning tunables (see OnlineLearnStep()). Live-chart-only: once a model is
//--- deployed (m_trainingComplete) it keeps adapting to newly-CONFIRMED price structure as bars mature,
//--- exactly the supervised ZigZag-pivot task it was trained on - never trade P&L (that would be RL).
//--- ONLINE_LEARN_MAX_CATCHUP caps how many newly-confirmed bars one step may backprop, so a weekend
//--- gap or a restart can't stall a tick with an unbounded catch-up loop (excess is picked up over the
//--- next few bars). ONLINE_ACC_SMOOTH is the guardrail EMA horizon (bars) for the rolling predict-
//--- before-learn accuracy - far shorter than CNet::recentAverageSmoothingFactor (10000) so the
//--- guardrail actually reacts within a realistic live sample count. The deployed shadow is only blended
//--- toward Net while that rolling accuracy holds at/above max(ONLINE_LEARN_MIN_ACC, deploy_baseline -
//--- ONLINE_LEARN_ACC_MARGIN); if it drops, Net keeps adapting (so it can recover) but the blend is
//--- FROZEN so drift can never reach live - the conservative "reject the deployment of a bad update"
//--- guardrail (no weight-snapshot/revert needed). Persist every ONLINE_LEARN_PERSIST_EVERY updates so
//--- a crash loses at most that many bars of adaptation.
//---
//--- CLASS IMBALANCE (2026-07-28). This path streams the RAW live class distribution - on SP500 H1 that
//--- is ~94% Neutral - one bar at a time. It used to call Net.backProp() with the default sampleWeight
//--- of 1.0, i.e. no correction at all, which made it a single-class drift vector by construction: it
//--- actively pulled a balanced, converged model back toward Neutral and undid exactly what
//--- balanced-accuracy checkpoint selection exists to protect. Train() corrects the same imbalance with
//--- DATA-level oversampling (queueing a minority bar repCount times), which is not available here -
//--- there is no queue to duplicate into, and replaying one bar back-to-back is precisely the
//--- correlated-momentum failure the shuffle in pass 2 was added to avoid. The streaming-safe
//--- equivalent is a COST-level correction: alpha-balanced focal loss (Lin et al. 2017, "Focal Loss for
//--- Dense Object Detection", eq. 5) - weight = alpha_c * (1 - p_t)^gamma - which is a SINGLE loss, not
//--- two stacked corrections, and was designed for exactly this extreme foreground/background ratio.
//--- alpha_c: inverse class frequency from the measured, persisted priors (m_priorBuy/Sell/Neutral),
//--- majority class normalised to 1.0, dialled by the same OversampleParity input Train()
//--- uses so one knob governs both engines, then hard-capped (see below).
//--- gamma: m_focalGamma, the CONFIGURED value - not m_focalGammaRuntime, which is a training-run
//--- schedule artifact the plateau ladder anneals to 0 and which has no meaning post-deploy.
//--- ONLINE_LEARN_MAX_CLASS_WEIGHT caps alpha_c so one rare bar can never deliver an outsized kick to an
//--- already-validated deployed model; ConstrainReplay tightens it to 3.0 to match the effective
//--- correction Train() applies under the same input (repCount capped at 3).
//--- ONLINE_LEARN_ETA_SCALE: this path previously inherited whatever value the GLOBAL `eta` happened to
//--- be left at by the last Train() chunk of ANY model instance (PAI/CONV/LSTM/HYBRID share it), which
//--- is arbitrary. eta is now pinned to this model's own converged rate scaled down for the duration of
//--- the loop, then restored, so a live adaptation step is deliberately gentler than a training step.
#define ONLINE_LEARN_MAX_CATCHUP 64
#define ONLINE_ACC_SMOOTH 50.0
#define ONLINE_LEARN_WARMUP 20
#define ONLINE_LEARN_MIN_ACC 40.0
#define ONLINE_LEARN_ACC_MARGIN 10.0
#define ONLINE_LEARN_PERSIST_EVERY 32
#define ONLINE_LEARN_MAX_CLASS_WEIGHT 5.0
#define ONLINE_LEARN_ETA_SCALE 0.25
//--- Free function, not a class method: needed by CExpertSignalAIBase's constructor init list for
//--- both m_modelEta and m_etaCeiling (see their declaration comments), which runs before member-init
//--- order could safely make one depend on another - this only touches the TrainingOptimizer input
//--- (Variables\Inputs.mqh) and the SgdLearningRate/AdamLearningRate inputs (AI\Network.mqh's `lr`
//--- macro resolves to AdamLearningRate), never class members.
double InitialEtaForOptimizer(void)
{
return (TrainingOptimizer == SGD) ? SgdLearningRate : lr;
}
class CExpertSignalAIBase : public CExpertSignalCustom
{
protected:
string ID;
CiOpen m_Open;
CiClose m_Close;
CiHigh m_High;
CiLow m_Low;
CiVolumes m_Volumes;
CiTime m_Time;
//--- optional classic-indicator input features (m_useMA/m_useRSI/m_useMACD/m_useIchimoku below) -
//--- independent instances from the ones Signals\SignalMA.mqh/SignalRSI.mqh/SignalMACD.mqh/
//--- SignalIchimoku.mqh use for trade voting, since this class and those CSignal* classes are
//--- unrelated hierarchies (feature engineering vs. signal vote). The MA
//--- feature now uses the SAME unified indicator as the classic vote (CustomIndicators\ADMovingAverage.mq5)
//--- via CiCustom, so its type/period are tunable (see ADIndicatorTuner maType/maPeriod).
CiCustom m_MA;
CiRSI m_RSI;
//--- built-in Ci* wrappers, not CiCustom: MACD is defined on plain EMAs and Ichimoku on plain
//--- highest/lowest midpoints, so there is no unified AD* custom indicator to route them through the
//--- way m_MA goes through ADMovingAverage. Periods come from the tuner (macdFast/macdSlow/
//--- macdSignal, ichiTenkan/ichiKijun/ichiSenkou).
CiMACD m_MACDFeature;
CiIchimoku m_Ichimoku;
//--- custom price-action/volume indicators (CustomIndicators\*.mq5), loaded via iCustom/CiCustom
CiCustom m_ADCumulativeDelta;
CiCustom m_ADShorteningOfThrust;
CiCustom m_ADWyckoffEventStream;
CiCustom m_ADWyckoffFailedStructure;
CiCustom m_ADWyckoffSignificantBarInversion;
//--- ground truth for the training labels (see m_swingConfirmationBars' declaration comment) -
//--- CustomIndicators\ADZigZag.mq5, a renamed/rebranded copy of the stock MQL5 ZigZag indicator,
//--- always created (not gated behind an Enable* input like the AD* feature indicators above,
//--- since it isn't an optional input feature - it IS the label) and always run at its own stock
//--- defaults (Depth=12, Deviation=5, Backstep=3) - never touched by AutoTuneIndicators, since
//--- tuning the ground truth itself alongside the model being scored against it would let a trial
//--- "improve" its OOS score by cherry-picking an easier target rather than a better model.
CiCustom m_ADZigZag;
//--- currently-active tunable input values for each AD indicator (AutoTuneIndicators search space)
//--- plus their own flatten/unflatten/perturb/best-tracking logic - see CADIndicatorTuner's
//--- declaration comment (Expert\ADIndicatorTuner.mqh) for why this is a separate collaborator
//--- rather than loose fields/methods here: it's entirely self-contained (never touches Net,
//--- Train()'s state machine, or anything else in this class), unlike TuneIndicatorsAndTrain()
//--- below, which orchestrates Train()/Net/checkpointing around it and stays here for exactly that
//--- reason. InpContextMode/InpSessionType/InpSessionCount are session choices, not accuracy
//--- knobs, and are hardcoded to the indicators' own defaults in InitAD*() rather than stored/
//--- tuned in the collaborator.
CADIndicatorTuner m_indicatorTuner;
bool m_autoTuneIndicators;
int m_indicatorTuneTrials;
//--- rebuilds only the AD* indicator handles (in place) so ReInit picks up updated param structs
bool ReInitADIndicators(CIndicators *indicators);
//--- builds a fresh, untrained topology into Net (assumes Net is currently NULL); factored out of
//--- InitNeuralNetwork() so TuneIndicatorsAndTrain() can rebuild weights per trial without
//--- re-running indicator init (which would re-Add() the base indicators into indicators a 2nd time)
bool BuildFreshTopology();
//--- CIndicators collection passed into InitNeuralNetwork(), retained so
//--- TuneIndicatorsAndTrain() can call ReInitADIndicators() between trials
CIndicators *m_indicatorsPtr;
CNet *Net;
//--- EMA "shadow" copy of Net, blended a small step (SHADOW_WEIGHT_TAU) toward Net's weights at
//--- the end of every era (see Train()'s era-end block) rather than replaced outright. Live
//--- trading/inference (RefreshLatestSignal()) reads from this, not from Net directly, so any
//--- single era's raw weights - including an Adam overshoot event - can only ever nudge what's
//--- actually deployed by SHADOW_WEIGHT_TAU, not overwrite it wholesale. Bootstrapped as a clone of
//--- Net (via the same Save()/Load() pattern m_simOosNet uses) the first time InitNeuralNetwork()
//--- runs with no prior shadow file, and persisted alongside Net.Save() thereafter. NULL only
//--- during the brief window before that first bootstrap completes - every read site falls back to
//--- Net in that case, never blocks on it.
CNet *m_shadowNet;
//--- One-shot latch for EnsureShadowNet()'s clone bootstrap. Cloning a second full net can fail on the
//--- tester's CPU-DLL compute fallback (the chart's GPU/DirectML path succeeds); without this the
//--- bootstrap would retry on EVERY bar, re-initialising the compute backend each time - log spam and a
//--- multi-second-per-bar crawl through an inference-only backtest. A failed/skipped bootstrap is
//--- harmless: every read site falls back to Net, and a bootstrapped shadow is identical to Net until
//--- era-end blending diverges it (which never happens in a no-training run). Reset with m_shadowNet.
bool m_shadowBootstrapAttempted;
//--- Online continual-learning state (see OnlineLearnStep(); tunables at ONLINE_LEARN_* above). Live-
//--- chart-only - a deployed (m_trainingComplete) model keeps learning from newly-confirmed ZigZag
//--- structure as bars mature, waiting the full m_swingConfirmationBars confirmation delay just like
//--- training so a still-provisional (repainting) recent bar is NEVER backpropped. m_enableOnlineLearning
//--- is the input gate. m_onlineLearnedUpToTime is the bar-TIME watermark of the newest bar already
//--- learned from (indexed by time, not now-relative index, so it survives the per-bar index-frame
//--- shift); persisted in the .stats sidecar (WST3). m_onlineRollingAcc is the guardrail EMA (0-100)
//--- of predict-before-learn hits (seeded from the deploy OOS baseline), m_onlineSamples the cumulative
//--- update count (both persisted). m_onlineBarsSincePersist drives periodic saves (in-memory only).
bool m_enableOnlineLearning;
datetime m_onlineLearnedUpToTime;
double m_onlineRollingAcc;
long m_onlineSamples;
int m_onlineBarsSincePersist;
//--- Latched log state so the guardrail freeze/resume transition prints once per flip, not per bar.
bool m_onlineBlendFrozen;
CArrayDouble *TempData;
double dError;
double dUndefine;
double dForecast;
double dPrevSignal;
//--- Live-inference alternation gate for LongCondition()/ShortCondition(). Ground truth comes
//--- from AdvanceZigZagLabelState(): a confirmed ZigZag pivot is Sell at a top, Buy at a bottom,
//--- and a real ZigZag can never confirm two same-type pivots back to back - it tracks one pending
//--- high and one pending low, and only emits a new pivot by flipping to the opposite type (see
//--- ADZigZag.mq5). LABEL_WINDOW_BARS then widens each confirmed pivot into a same-direction
//--- neighborhood, but never bleeds into or overwrites an adjacent DIFFERENT pivot's own exact bar
//--- (see AdvanceLabelCachePrebuild()'s widening pass), so this holds at the "streak" level too:
//--- every contiguous run of non-Neutral ground truth is followed - after however many Neutral bars
//--- - by a run of the OPPOSITE direction, never the same one twice in a row. A live Buy directly
//--- after another live Buy, with no Sell in between, is therefore a pattern the model was never
//--- trained to produce - provably a false fire, never a true one - so LongCondition()/
//--- ShortCondition() suppress it to 0 (neutral/abstain) instead of opening a second same-direction
//--- trade before the awaited reversal. Live-only, in-memory, never persisted, and never read by
//--- Train()'s historical scoring/backprop loop (which computes/consumes its own dPrevSignal
//--- separately per historical bar) - same category as SignalMarketDepth.mqh's spread history.
ENUM_SIGNAL m_lastNonNeutralSignal;
//--- The gate's value as of the START of the current tick's vote, captured by BeginVote() before
//--- LongCondition()/ShortCondition() get a chance to consume it. RevokeVote() restores it when the
//--- parent throws this filter's vote away (Hybrid quorum shortfall / discarded tick) - see the
//--- BeginVote()/RevokeVote() declaration comment in Expert\ExpertSignalCustom.mqh. Snapshotting has
//--- to happen in BeginVote() and not at the top of each condition: Direction() evaluates
//--- LongCondition() FIRST, so a snapshot taken inside ShortCondition() would capture the value
//--- LongCondition() had just overwritten and "restore" the gate to the wrong state.
ENUM_SIGNAL m_gateSnapshot;
//--- Non-max suppression (NMS) of directional signals. Consecutive H4 bars near a real turn share
//--- almost their entire feature vector, so a single reversal fires the same class on a whole run of
//--- adjacent bars - a visual/emission cluster around one event, not several distinct turns (and the
//--- ZigZag ground truth never labels two same-direction pivots without an opposite between them, so
//--- a same-direction repeat is provably redundant - see m_lastNonNeutralSignal's comment). NMS keeps
//--- only the FIRST (earliest) bar of each same-direction run and drops same-direction neighbors
//--- within m_signalClusterWindow bars. Per-direction and independent, so a missed opposite signal
//--- never blocks a genuine later reversal (unlike the strict alternation gate). Causal - keeps the
//--- earliest/best-entry bar, never peeks ahead - so the live signal and the drawn history declutter
//--- to the same stream. NMS is post-processing ON TOP of the model: the raw per-bar recall/precision/
//--- accuracy stats deliberately stay un-NMS'd so they keep measuring the network itself. 0 disables.
//--- The cluster extends from the last SEEN same-direction bar, not the last KEPT one, so an entire
//--- contiguous run collapses to a single arrow no matter how long (tracking distance-from-kept
//--- instead re-emitted one arrow every window+1 bars inside a long run - the residual clustering the
//--- user still saw). A same-direction bar more than m_signalClusterWindow bars past the previous
//--- same-direction bar starts a fresh cluster; a real opposite turn is a multi-bar run of its own, so
//--- two genuine reversals separate naturally without needing the opposite class to reset state.
int m_signalClusterWindow;
//--- Per-bar predicted signed signal for THIS era (index = now-relative bar index; -2 = not scored
//--- this era). When NMS is on, the scan passes (1/2/3) ONLY record into this cache and draw nothing;
//--- PruneDirectionalClusters() then renders the whole declustered set once at era end. That is what
//--- keeps the chart from ever showing the raw mid-era clusters (the passes would otherwise draw every
//--- above-threshold bar as it scanned, and the sweep only cleaned up at era end). Recording (not
//--- drawing) also decouples NMS from pass 2's SHUFFLED order, which no inline cursor could dedup.
//--- Sized to bar count each fresh era.
double m_arrowSignalCache[];
//--- Live-side NMS state: bar TIME of the last SEEN signal per direction (advances on every same-
//--- direction bar, kept or suppressed, so a contiguous live run collapses to one) plus the cached
//--- accept/suppress decision for that exact bar (keeps repeated same-bar RefreshLatestSignal calls
//--- idempotent - re-evaluating the same bar returns its first decision, not a flipped one). 0 = none.
datetime m_nmsLiveBuyTime;
datetime m_nmsLiveSellTime;
bool m_nmsLiveBuyAccept;
bool m_nmsLiveSellAccept;
//--- Last KEPT live signal of either direction, for cross-direction resolution: a Buy and a Sell
//--- within m_signalClusterWindow bars are flicker at one turn zone (real opposite pivots are a whole
//--- leg apart), so only the higher-confidence side is kept. Confidence = |signed signal| = winning
//--- softmax probability. Same rule runs in PruneDirectionalClusters() for the historical chart.
datetime m_nmsLiveKeptTime;
ENUM_SIGNAL m_nmsLiveKeptDir;
double m_nmsLiveKeptConf;
datetime dtStudied;
long m_eraCount; // cumulative era counter, persisted in the .nnw so restarts don't look like they reset progress
bool m_trainingComplete; // persisted: true only once Train() converged (objective+stability), not just interrupted
bool bEventStudy;
//--- out-of-sample holdout: share (%) of the study period never trained on, used only to
//--- measure genuine forward accuracy so overfitting shows up in the stats, not just live/OOS trading
int m_oosSplitPct;
double dOosError; // smoothed OOS mismatch rate (0..100), lower is better
double dOosForecast; // smoothed OOS accuracy (0..100)
int m_oosSamples; // count of OOS predictions evaluated this Train() call
//--- per-era raw (pre-softmax) output-neuron stats over pass 3's OOS scan, reset at pass 3
//--- start; surfaced in the era-end log line. The per-bar logit spread (max-min across the 3
//--- outputs) is the collapse fingerprint: a healthy net differentiates bars (avg spread well
//--- above 0), a saturated all-one-class net pins all three outputs to the same value on every
//--- bar (avg spread ~0, mins/maxes flat) - see CLASS_LOGIT_SCALE's comment (AI\Network.mqh).
double m_oosOutMin[3];
double m_oosOutMax[3];
double m_oosOutSpreadSum;
int m_oosOutCount;
//--- per-era counts of the network's own classification of each bar it fed forward (IS+OOS),
//--- reset at the start of every era; surfaced in the status label text so class imbalance
//--- (e.g. the network collapsing to all-Neutral) is visible while training runs
int m_countBuySignals;
int m_countSellSignals;
int m_countNeutralSignals;
//--- per-era counts of the *true* label of every bar fed forward (IS+OOS), reset alongside the
//--- predicted counts above. Used both to display the actual class distribution in the status
//--- label text and, more importantly, to drive IS-sample oversampling in Train() (see backProp calls
//--- below) - rarer classes get replayed more times so gradient descent doesn't just learn to
//--- always call the majority (usually Neutral) class.
int m_trueBuyCount;
int m_trueSellCount;
int m_trueNeutralCount;
//--- snapshot of the class totals above, taken at the end of the PREVIOUS era (see Train()'s
//--- era-reset block) and held fixed for the whole of the current era. The oversampling ratio
//--- below is computed from these frozen counts rather than the live, still-accumulating
//--- m_true*Count values - using the live counts made the ratio order-dependent within a single
//--- era (chronological, oldest-to-newest bar processing means whichever class happens to be
//--- numerically behind at any given moment gets amplified up to 5x, even if that's just a
//--- transient artifact of which regime the era's early bars came from, not the class's true
//--- overall rarity) - a real source of run-to-run oscillation in the predicted class mix. All
//--- zero on era 0, when oversampling simply falls back to 1x (see the reps calc in Train()).
int m_prevEraTrueBuyCount;
int m_prevEraTrueSellCount;
int m_prevEraTrueNeutralCount;
//--- per-era OOS confusion counts, reset each era; used to compute per-class OOS recall (hits/total)
//--- for the status label text and, more importantly, as an additional convergence gate alongside the
//--- blended dOosForecast accuracy - a model that "wins" only by calling everything Neutral will
//--- have high dOosForecast but near-zero Buy/Sell recall, and should NOT be allowed to converge.
int m_oosBuyHits, m_oosBuyTotal;
int m_oosSellHits, m_oosSellTotal;
int m_oosNeutralHits, m_oosNeutralTotal;
//--- same per-era OOS confusion counts as above but keyed by PREDICTED class instead of true class,
//--- i.e. per-class precision (of the bars this era where the model called Sell, how many actually
//--- were Sell?) rather than recall (of the bars that actually were Sell, how many did it catch?).
//--- Recall alone can't distinguish "the model over-fires Sell and happens to also catch enough real
//--- Buys/Neutrals to clear their recall floors" from "the model is genuinely well-calibrated" - a
//--- skewed Predicted-this-era count (see m_countSellSignals) with recall still passing the gate is
//--- exactly that failure mode, and precision is what would expose it.
int m_oosBuyPredicted, m_oosBuyPredictedHits;
int m_oosSellPredicted, m_oosSellPredictedHits;
int m_oosNeutralPredicted, m_oosNeutralPredictedHits;
//--- Confidence calibration: the classification head's forward pass still uses SIGMOID per-neuron
//--- (see BuildFreshTopology()'s SIGMOID comment - bounded activation, avoids logit runaway), but
//--- the BACKWARD pass (CNet::backProp/backPropOCL in AI\Network.mqh) now trains all 3 neurons
//--- jointly against a true softmax+categorical-cross-entropy gradient (softmax_i - target_i), not
//--- 3 independent binary-cross-entropy targets. ApplyClassificationSoftmax() at READ time
//--- reproduces the same normalization the loss was actually trained against, so the value it
//--- returns is now a genuinely loss-consistent probability (still not literature-perfect
//--- calibration - no temperature scaling/Platt scaling has been applied - but no longer
//--- structurally decoupled from what was optimized). SignedAIConfidence()/g_AISignedConfidence
//--- exposes it, and Money\MoneyIntelligent.mqh:AdjustRiskAmount() scales position-sizing risk%
//--- directly off it.
//--- m_oosConfidenceSum accumulates MathAbs(dPrevSignal) (the claimed confidence) over every
//--- classified OOS bar this era; compared against this era's actual OOS accuracy
//--- ((m_oosBuyHits+m_oosSellHits+m_oosNeutralHits)/m_oosSamples) at era-end to derive
//--- m_confidenceCalScale - a single empirical multiplier ("the model claims 80% on average but is
//--- only right 60% of the time -> scale reported confidence by 0.75") applied to the MAGNITUDE
//--- only, never the sign/class decision, in SignedAIConfidence(). EMA-blended across eras (same
//--- smoothing factor as dOosForecast) so one noisy era can't swing it, and clamped to [0.3, 1.5]
//--- so a thin/degenerate OOS window can't drive it to something absurd. Starts at 1.0 (no
//--- correction) until the first era with OOS samples computes a real value.
double m_oosConfidenceSum;
double m_confidenceCalScale;
//--- minimum acceptable OOS recall (%) for the Buy and Sell classes individually before Train() is
//--- allowed to declare convergence; a class with zero OOS samples this era doesn't block (avoids a
//--- deadlock when a given era's OOS window happens to contain no examples of that class)
int m_minDirectionalRecallPct;
//--- There is deliberately NO minimum-confidence input here any more, and no member holding one.
//--- Confidence is expressed through the VOTE WEIGHT (ConfidenceTier() -> PatternWeightForTier()),
//--- not through a separate entry floor, so a weak call is not blocked at the AI's own boundary - it
//--- votes weakly and is then filtered by the same Min vote to open threshold that filters a weak
//--- classic vote. That is what makes ONE input genuinely govern both engines.
//--- The floor it replaced was a scale mismatch: Min_Vote_Open fed it directly as a 0..1 probability
//--- while ALSO being the 0..100 averaged-vote threshold, and a 3-class argmax winner is
//--- arithmetically >= 1/3, so every setting from 0 to 33 gated precisely nothing while every setting
//--- above that silently re-quartiled the confidence tiers as a side effect. See ConfidenceTier().
//--- Optional training-safety toggles for the replay and prior-correction paths. The replay path can
//--- be disabled entirely, or its aggression capped so it stays closer to the main pass's update scale.
bool m_enableMinorityReplay;
bool m_constrainReplay;
bool m_freezePriorCalibration;
bool m_useStaticPrior;
//--- Operator-tunable fraction of full class parity the minority oversampling aims for (see
//--- OVERSAMPLE_PARITY_FRACTION's declaration comment for the full rationale - this is the same dial,
//--- promoted from a compile-time constant to the `OversampleParity` input so it can be swept per run).
//--- Lower => Neutral stays proportionally heavier in the training queue => fewer, higher-precision
//--- Buy/Sell calls (but lower recall - watch the MinRecall convergence gate); higher (toward 1.0) =>
//--- more directional calls, higher recall, more over-calling. Defaults to OVERSAMPLE_PARITY_FRACTION.
double m_oversampleParity;
//--- Post-hoc logit-adjustment strength tau (0..1, from AILogitPriorStrength/100) - see
//--- AdjustedSignalFromSoftmax(). 0 disables it (raw argmax, pre-feature behaviour); 1 fully
//--- calibrates the decision to the true base rate. Applied to LIVE signal emission
//--- (RefreshLatestSignal) and the live-decision precision metric below, deliberately NOT to the raw
//--- argmax recall/convergence scoring (those must keep measuring the model's intrinsic class
//--- separation, or lowering directional recall via this correction could block convergence).
double m_logitPriorStrength;
//--- Training-time logit adjustment - see EnableLogitAdjustedLoss (Variables\Inputs.mqh).
bool m_useLogitAdjustedLoss;
double m_logitAdjustTau;
//--- True class base rates (natural, un-oversampled), measured from the label distribution each era
//--- (UpdateClassPriors, EMA-blended for stability) and PERSISTED alongside the weights (.stats
//--- sidecar) so live inference - including after a restart, when no training re-runs - calibrates
//--- exactly as training did. 0 = not yet measured => AdjustedSignalFromSoftmax falls back to raw.
double m_priorBuy, m_priorSell, m_priorNeutral;
//--- Per-era OOS "fired" confusion counts under the LIVE decision rule: a directional call is counted
//--- whenever the prior-corrected posterior is non-Neutral, i.e. exactly the bars on which the
//--- deployed EA would cast a directional vote. m_oosBuyFiredHits/m_oosBuyFired = live Buy precision;
//--- likewise Sell. This is the metric that predicts forward-trading performance (recall/argmax-
//--- precision above score the RAW argmax, before the base-rate correction the live decision applies).
//--- Reset each era. Whether a counted vote also clears Min_Vote_Open against the other filters'
//--- average is an aggregate question this per-bar scorer cannot see - see its use site.
int m_oosBuyFired, m_oosBuyFiredHits;
int m_oosSellFired, m_oosSellFiredHits;
//--- Cumulative (compounded, persistent) DIRECTIONAL accuracy = the win-rate of the model's Buy/Sell
//--- calls: of the bars it actually called Buy or Sell, how many matched the true label. Neutral ("no
//--- trade") calls are deliberately EXCLUDED - counting them inflates the rate to ~80%+ (Neutral is the
//--- ~94% majority the model gets right for free) and tells you nothing about trade quality. Summed
//--- across every era AND carried across restarts via the .stats sidecar (WST5); never reset era-to-era,
//--- so the panel shows a stable win-rate that only firms up as more pivots confirm. Incremented at the
//--- IS (pass 2) and OOS (pass 3) hit sites, only when the PREDICTION is directional, skipped for
//--- m_evalMode throwaway candidates. Reset only by ResetWeights (a fresh model).
long m_cumIsCorrect, m_cumIsTotal;
long m_cumOosCorrect, m_cumOosTotal;
//--- Latest live-fired precision (%) and fire count per direction (-1 = n/a), cached at era end for
//--- the status panel/log the same way m_lastBuyRecallPct is (see its comment).
int m_lastBuyFiredPrecPct, m_lastSellFiredPrecPct;
int m_lastBuyFired, m_lastSellFired;
//--- Ceiling on the class-balance sampleWeight multiplier (see its computation in Train(), keyed off
//--- m_prevEraTrueBuyCount/Sell/Neutral). MAX_WEIGHT_DELTA (AI\Network.mqh/.cl, DirectML\WarriorCPU.cpp)
//--- bounds how far a SINGLE weight can move in one step, but it does nothing to stop many small
//--- steps in the same direction from accumulating into a large net swing across an era - with real
//--- label counts around Buy 1340 / Sell 1312 / Neutral 7013 the uncapped ratio is ~5.3x on every
//--- single Buy/Sell example, every era. At the old hardcoded 3.0x ceiling that was STILL strong
//--- enough in practice to let one era's Buy gradients overwrite the separability the previous era
//--- had just built for Sell (and vice versa) - the observed anti-correlated Buy/Sell recall whipsaw
//--- (e.g. era 8 Buy:0%/Sell:26% -> era 9 Buy:10%/Sell:57% -> era 10 Buy:1%/Sell:37%), never both
//--- classes improving together. Was tunable (ClassSampleWeight input, CSW_15/1.5x default) as a
//--- loss-level multiplier, but that mechanism proved structurally too weak against Adam's near-
//--- invariance to constant gradient rescaling (Kingma & Ba 2015) - see the m_isTrainQueue queueing
//--- block's oversampling comment for the full history. Class-balance correction now happens
//--- entirely via data-level oversampling (repCount in that queueing block); this member is
//--- currently unread by that path. The ClassSampleWeight INPUT that used to set it was removed
//--- (it had no effect); the member is kept at its constructor default in case a smaller, additive
//--- loss-level nudge is ever reintroduced on top of oversampling.
double m_maxClassSampleWeight;
//--- Focal loss modulating exponent (Lin et al. 2017, "Focal Loss for Dense Object Detection") -
//--- multiplies on top of the class-balance sampleWeight above (3-neuron classification only; the
//--- 1-output regression head's tanh output has no clean "probability of the true class" reading,
//--- so it's left at factor 1.0 there). The class-balance weight above is a fixed per-CLASS
//--- multiplier - it keeps boosting every Buy/Sell example by the same ratio all era, whether the
//--- network is currently 10% or 95% confident on that particular bar, which is exactly what let a
//--- class it had already learned keep getting reinforced long past the point of diminishing
//--- returns (observed in practice: era 95-114 of one run staying Buy-recall 80-100%+ almost the
//--- entire stretch, weights carried over unreset the whole time). Focal loss instead scales EACH
//--- example individually by (1-pt)^gamma, where pt is the network's own current sigmoid output for
//--- that bar's true class neuron: near 1.0 (full weight) while the network is still wrong/unsure
//--- about this specific bar, decaying toward 0.0 once it's already confidently correct - so
//--- gradient keeps redirecting toward whatever the network hasn't learned yet instead of piling
//--- more identical pressure onto a class it already has, regardless of which class that happens to
//--- be. Multiplies the per-slot weight from m_isTrainQueueWeightScale (see that member's comment -
//--- currently always 1.0, class balance being handled by oversampling instead), and since
//--- (1-pt)^gamma <= 1.0 always, it can only ever shrink that weight further, never exceed it.
double m_focalGamma;
//--- Ground truth for the training labels below now comes directly from the real MQL5 ZigZag
//--- indicator (CustomIndicators\ADZigZag.mq5 - a renamed, logic-untouched copy of the stock
//--- ZigZag.mq5, run at its own stock defaults: Depth=12, Deviation=5 points, Backstep=3), not a
//--- hand-rolled fractal/deviation-%/ATR-trend-context approximation of one. Like any ZigZag, its
//--- most recent 1-3 legs are provisional and can still be revised (repainted) as new bars arrive
//--- (see ADZigZag.mq5's own ExtRecalc), so a candidate bar's ADZigZagBuffer value is only trusted
//--- once at least m_swingConfirmationBars MORE bars have closed after it - see
//--- AdvanceZigZagLabelState()'s declaration comment. Until then it stays Neutral.
int m_swingConfirmationBars;
//--- safety valve: Train()'s do-while loop has no other bound on how many eras it will run
//--- before giving up, so a config that can't reach the convergence objective (e.g. too few
//--- swing-confirmed examples for the min recall bar to be reachable) would otherwise loop
//--- forever, permanently keeping the era-progress status label up instead of the normal per-tick
//--- info line and burning CPU nonstop. When the cap is hit, the operator is prompted (see
//--- PromptContinuePastEraCap): CONTINUE resets the era counter and keeps training; STOP deploys
//--- the best checkpoint found so far (FinalizeTrainRun) and terminates training. Headless
//--- (tester/optimizer) runs can't prompt, so they take the STOP branch automatically.
int m_maxErasPerRun;
//--- Train() runs its per-bar loop synchronously, and MQL5 is single-threaded per chart - a
//--- multi-minute era would otherwise starve the terminal's chart-event queue for that whole
//--- stretch, including the control panel's own click/drag hit-testing (Panel\ControlPanel.mqh),
//--- which depends entirely on CHARTEVENT_MOUSE_MOVE being delivered promptly. Train() is
//--- therefore chunked: each call does at most ~TRAIN_TIME_BUDGET_MS of work, then returns and
//--- picks back up exactly where it left off on the next call (re-triggered via the same "New
//--- Bar" custom-event scheduling ScheduleTrainingIfNeeded() already used) - the members below
//--- are what makes that resumable across calls.
bool m_trainRunActive; // true: a run (schedule -> convergence/stop) is in progress, possibly spanning many Train() calls
bool m_eraResumePending; // true: yielded mid-bar-loop last call - resume the SAME era, don't start a new one
int m_resumeBars;
int m_resumeTotalIter;
int m_resumeOosCutoff;
int m_resumeBarIndex;
bool m_resumeAddLoop;
//--- Pass 2 of the era loop: bar indices pass 1 (sequential feedForward/scoring) queued as
//--- IS-eligible for backProp, trained on in a freshly shuffled order instead of pass 1's own
//--- fixed chronological (oldest-to-newest) visitation order. Root cause this targets: Adam's
//--- momentum (b1, AI\Network.mqh) has an effective memory window of ~1/(1-b1) steps, which lands
//--- close to LABEL_WINDOW_BARS's ~5-bar contiguous same-class label runs around each ZigZag
//--- pivot - replaying those runs in the SAME order every single era let momentum lock onto
//--- whichever run it was currently passing through, with the network's end-of-era state
//--- disproportionately reflecting whatever it last walked through (recency), not a globally
//--- balanced fit. Symptom observed in practice: IS error climbing era-over-era (0.16->0.55+ over
//--- 25 eras on the same fixed dataset) instead of settling, and OOS Buy/Sell recall whipsawing
//--- between near-0% and 70-100% despite near-equal Buy/Sell label counts. Shuffling is the
//--- standard SGD fix - see the m_isPass2Active declaration below for how it's sequenced against
//--- pass 1 within Train()'s existing resumable-chunk machinery.
int m_isTrainQueue[];
int m_isTrainQueueCount;
//--- Parallel to m_isTrainQueue (same index, kept in lockstep through the Fisher-Yates shuffle
//--- below) - the per-occurrence weight to apply when this queued slot is trained on in pass 2,
//--- decided once at queue time (see the queueing block's oversampling comment) rather than
//--- recomputed in pass 2. Currently always 1.0: class-balance correction is carried entirely by
//--- repCount (how many times a bar was duplicated into the queue), not by any per-occurrence
//--- weight scaling - see the queueing block's comment for why stacking a second correction here
//--- caused two separate training collapses. Kept as a real per-slot array rather than a literal
//--- 1.0 so a future, smaller/safer supplemental weight could be reintroduced without re-touching
//--- the queueing or shuffle code.
double m_isTrainQueueWeightScale[];
//--- Parallel to m_isTrainQueue too (same index, same lockstep swap in the shuffle): true on exactly
//--- ONE occurrence per duplicated bar - the rep-0 slot written at queue time. Its only job is to keep
//--- the reported IS accuracy (m_cumIsCorrect/m_cumIsTotal) measured on the NATURAL class distribution
//--- while backProp still trains on the oversampled one.
//--- Why: the queue duplicates each Buy/Sell bar up to repCount times (~21x at the observed 30.7:1
//--- imbalance), so counting every occurrence measured IS accuracy over a set that is ~58% directional,
//--- while the OOS counter measures the real ~6% directional distribution. Both excluded Neutral and
//--- both used the identical formula, so they LOOKED directly comparable - and reported things like
//--- "IS 77% / OOS 12%", which reads as catastrophic overfitting when the two numbers were simply
//--- scored against base rates an order of magnitude apart (77% vs a 58% baseline is a 1.33x lift;
//--- 12% vs a 6.1% baseline is 1.97x - the OOS side was actually the STRONGER one).
//--- Counting primaries only puts both metrics on the same footing, so the IS/OOS gap once again means
//--- what everyone reads it as meaning: generalisation. Training itself is completely unaffected -
//--- every occurrence still backprops exactly as before; this flag is read only by the counter.
bool m_isTrainQueuePrimary[];
//--- Pass 2's cursor into the (already shuffled) m_isTrainQueue - lets pass 2 itself yield/resume
//--- mid-queue under the same TRAIN_TIME_BUDGET_MS chunk budget pass 1 already yields under.
int m_isTrainCursor;
//--- true: pass 1 (sequential) has finished for this era and pass 2 (shuffled backProp) is either
//--- running or has yielded mid-queue - Train() skips straight past pass 1's loop on resume when
//--- this is set. Reset to false only at a fresh era's start (never mid-run).
bool m_isPass2Active;
//--- true: pass 2 has already run to natural completion for this era (m_isPass2Active's own
//--- false state is ambiguous between "not started yet" and "already finished" - both look
//--- identical to a plain `if(!m_isPass2Active)` check). Needed because pass 3 (OOS scoring) can
//--- itself yield/resume across multiple Train() calls same as passes 1/2 do; without this flag,
//--- every resume into an unfinished pass 3 fell through the `if(!m_isPass2Active)` guards on both
//--- pass 1 and pass 2 and re-ran the ENTIRE shuffled queue again (pass 1 itself was a no-op on
//--- resume since its own loop cursor `i` was already exhausted, but pass 2 re-shuffled and replayed
//--- from scratch every single time) - silently multiplying the predicted-class counts (and the
//--- extra, unintended backProp() calls) once per resume, for as long as pass 3 kept needing more
//--- than one chunk to finish. Reset to false only at a fresh era's start, alongside m_isPass2Active.
bool m_isPass2Done;
//--- Pass 3: chronological, OOS-region-only re-walk that happens AFTER pass 2 has actually trained
//--- on this era's IS data - see m_isTrainQueue's declaration comment for why OOS scoring can no
//--- longer just happen inline during pass 1 (that would score every era's OOS window against
//--- weights from BEFORE this era's training, one full era stale - and for era 0 specifically,
//--- against the still-untrained cold-start network, which is why era 0's OOS recall used to show
//--- a meaningless 100% Neutral / 0% Buy / 0% Sell every time). Cursor walks i downward from
//--- m_oosScoreStartIndex to 0, mirroring pass 1's own iteration bounds/order for whichever bars
//--- satisfy isOOS - order matters here (unlike pass 2) since dOosForecast/dOosError are recursive
//--- EMAs over the visitation sequence, not order-independent.
bool m_isPass3Active;
int m_oosScoreIndex;
int m_oosScoreStartIndex;
datetime m_lastBarTime;
//--- This model's own learning-rate trajectory. `eta` (AI\Network.mqh) is a single file-scope
//--- global shared by every CNet in the process - PAI/CONV/LSTM each train independently and
//--- asynchronously (their own Train() calls are interleaved via separate chart-event/timer
//--- scheduling, not synchronized), but all three previously read AND wrote that one global, so
//--- one model's era-end regression/recovery decay/recovery of `eta` (see Train()'s era-end
//--- block) silently changed the learning rate the OTHER two models' very next backProp() call
//--- used too - an unintended coupling between what's supposed to be three independent training
//--- trajectories. Train() now restores `eta` from this member right before touching it, and
//--- saves it back before returning (both at the mid-chunk yield and at the natural end of the
//--- call) - MQL5 gives one chart's EA a single execution thread and each Train() call runs to
//--- completion or to its own yield point before another event is dispatched, so this save/
//--- restore is enough to isolate each model's schedule with no change needed to the neuron-level
//--- or OpenCL/DirectML call signatures that read the global directly.
double m_modelEta;
//--- Ceiling the era-end recovery bump (Train()'s isBetterEra block) restores `eta` toward - used
//--- to be the raw `lr` constant unconditionally, which is only correct for ADAM. SGD's own starting
//--- rate is the separate SgdLearningRate input (AI\Network.mqh) - clamping SGD's recovery bump to
//--- plain `lr` (AdamLearningRate) would silently cap it back down to Adam's ceiling after the
//--- first regression+recovery cycle, undoing the whole point of giving SGD its own configured
//--- rate. Computed once at construction from this model's own m_optimizationAlgo, matching
//--- m_modelEta's per-instance isolation (see that member's comment).
double m_etaCeiling;
int m_erasSinceCooldown; // eras completed since the last cooldown reset - replaces the old per-call-only "erasThisCall"
CArrayDouble m_oosWindow; // run-scoped OOS stability window (used to be a Train()-local CArrayDouble)
double m_bestOosForecast;
//--- Balanced accuracy (macro-recall: mean of Buy/Sell/Neutral OOS recall) of the era the current
//--- checkpoint was taken from. This is the metric the checkpoint SELECTION ranks on now, in place
//--- of the blended dOosForecast - blended accuracy is dominated by Neutral (~96% of bars), so among
//--- otherwise-acceptable eras it silently preferred the MOST Neutral-leaning weights, deploying a
//--- model that scores well on paper but under-calls Buy/Sell. Balanced accuracy weights every class
//--- equally, so "best" now means "most accurate across all three classes" - exactly what a 3-class
//--- signal product wants. Kept SEPARATE from m_bestOosForecast (which still snapshots the blended
//--- value at the same checkpoint) because FinalizeTrainRun()/the restore-on-regression branch reset
//--- dOosForecast to m_bestOosForecast, and that must stay the blended EMA the rest of the machinery
//--- expects. The directional recall FLOOR (directionalRecallOK) still gates convergence unchanged;
//--- this only fixes which era gets deployed among the candidates. -1 until the first ranked era.
double m_bestBalancedOos;
//--- whether the era m_bestOosForecast/the checkpoint was taken from also cleared the per-class
//--- directional recall floor (see directionalRecallOK below) - part of the "best" ranking itself,
//--- not just a side note, so blended accuracy alone can never outrank a directionally-useful era
//--- (see the checkpoint/eta-decay comment in Train()'s era-end block for why that matters).
bool m_bestPassedRecall;
bool m_haveOosCheckpoint;
bool m_oosStable;
bool m_objectiveMet;
//--- Plateau ladder state (see the PLATEAU_* constants). m_erasSinceBestBalanced counts eras since
//--- the last NEW BEST balanced accuracy; m_plateauStage is how far up the escalation it has climbed.
//--- Both are run-scoped and deliberately NOT persisted, matching m_modelEta/m_bestBalancedOos, which
//--- also restart fresh - a resumed run re-earns its patience rather than resuming mid-escalation.
int m_erasSinceBestBalanced;
int m_plateauStage;
//--- RUNTIME focal gamma - the value the loss actually applies, which the ladder anneals downward.
//--- MUST stay separate from m_focalGamma: that member is the CONFIGURED value and is part of the
//--- weights-filename fingerprint (see BuildConfigFingerprint), so mutating it mid-run would change
//--- which .nnw this model saves to and orphan the run's own weights. Configured value = model
//--- identity; runtime value = training schedule. Reset to m_focalGamma at the start of each run.
double m_focalGammaRuntime;
uint m_syncWaitStartTick; // 0 = not waiting on history sync; else GetTickCount() when the wait began
//--- 3 no-op passes on a fresh start (see InitNeuralNetwork()/ResetWeights()), each its own separately-
//--- scheduled Train() call (not a tight in-process loop), so the broker/terminal's history sync gets
//--- several real, wall-clock-separated chances to finish before the era loop commits to a bar count.
int m_warmupPassesRemaining;
//--- fractal/swing-confirmation/trend-context Buy/Sell label cache: the label at a given now-relative
//--- bar index only depends on price/ATR history, never on model state, so recomputing it every era
//--- (as opposed to once per real bar close) is pure waste. Rebuilt fresh whenever `bars` changes
//--- (see the invalidation check in Train()) rather than incrementally appended, since MQL5 timeseries
//--- indices are relative to "now" and shift by one on every new candle - a full rebuild on change
//--- sidesteps needing datetime-keyed/incremental bookkeeping entirely.
bool m_labelCacheBuy[];
bool m_labelCacheSell[];
bool m_labelCacheHasValue[];
int m_labelCacheBars; // 0 = no cache built yet
datetime m_labelCacheAnchorTime; // m_Time.GetData(0) at last (re)build - 2nd invalidation key
void ComputeLabelForBar(int i, int bars, bool &buy, bool &sell);
void AdvanceZigZagLabelState(int i, int bars);
//--- full per-bar INPUT feature vector cache (everything BufferTempData() computes: ATR-normalized
//--- OHLC, time-of-day encoding, volume delta, AD indicator buffers, ...). Same rationale as the
//--- label cache above - a given now-relative bar index's feature vector only depends on price/
//--- indicator history, never on model state, so recomputing it is pure waste: BufferTempData()
//--- used to rebuild every bar from scratch once per historyBars-wide window it appears in, AND
//--- again on every subsequent era on top of that. Flat array (idx*m_neuronsCount + feature),
//--- not a true 2D array - MQL5 can't dynamically resize an inner dimension. Shares the label
//--- cache's invalidation trigger (see EnsureBarCachesCapacity()) since both are keyed on the exact
//--- same now-relative index frame.
double m_featureCache[];
bool m_featureCacheHasValue[]; // true once idx has been resolved (valid OR invalid)
bool m_featureCacheValid[]; // false: BufferTempDataCompute(idx) returned false (e.g.
// no ATR yet) - cached as a miss so it isn't retried forever
bool BufferTempDataCompute(int idx);
//--- Nearest confirmed (non-repainting) ZigZag pivot at or after fromIdx - see this method's
//--- definition comment and m_useSwingContext's declaration comment for the repainting-embargo
//--- rationale callers must apply to fromIdx before calling this.
bool FindConfirmedZigZagPivot(int fromIdx, int &pivotIdx, double &pivotPrice, bool &pivotIsLow);
bool EnsureBarCachesCapacity(int bars);
//--- Eager label-cache pre-build + true-label tally, run once per fresh start (see
//--- m_warmupPassesRemaining) BEFORE era 0's real training loop begins. Without this, era 0 trains
//--- with m_prevEraTrueBuyCount/Sell/Neutral all still 0 (see that member's declaration comment),
//--- so the class-balanced oversampler falls back to reps=1 for every class on era 0 specifically -
//--- i.e. era 0 gets ZERO correction for label imbalance, unlike every era after it. Pre-scanning the
//--- whole IS window's true labels upfront (reusing the same cache/ComputeLabelForBar() the era loop
//--- itself uses) lets era 0 start with a real oversampling ratio instead of an uncorrected one.
bool m_labelCachePrebuilt; // true once the one-time pre-scan has completed
bool m_labelPrebuildActive; // true while a chunked pre-scan is in progress
bool m_prebuildSeedPending; // true: era 0's era-start reset must NOT stomp the
// prebuild-seeded m_prevEraTrue* counts with the
// still-empty live tally (see Train()'s era-start block)
int m_labelPrebuildBars;
int m_labelPrebuildOosCutoff;
int m_labelPrebuildIndex;
int m_labelPrebuildBuyCount;
int m_labelPrebuildSellCount;
int m_labelPrebuildNeutralCount;
void StartLabelCachePrebuild(void);
void AdvanceLabelCachePrebuild(void);
//--- Evaluation-only continual-learning OOS simulation: once the core model converges, a CLONE of its
//--- weights (never the production Net itself) walks forward through the OOS window bar-by-bar,
//--- scoring each bar with its current weights THEN learning from it - simulating how the model would
//--- adapt in live/forward trading. This must never feed back into Net or the real OOS convergence
//--- metric (dOosForecast/m_oosSamples), so it's tracked in entirely separate members and the clone is
//--- discarded (never Save()'d) once each walk completes.
CNet *m_simOosNet; // NULL when no simulation is active
bool m_simOosRunActive;
int m_simOosCutoff; // oosCutoff snapshot from the run that converged
int m_simOosBarIndex; // resume point, m_simOosCutoff-1 down to 0
double m_simOosForecast; // smoothed accuracy - separate from dOosForecast
int m_simOosSamples;
void StartOosContinualSimulation(int bars, int oosCutoff);
void AdvanceOosSimulationChunk(void);
//--- same resumability problem one level up: TuneIndicatorsAndTrain()'s own trial loop calls
//--- Train() per trial and used to assume each call ran an entire trial to completion synchronously
int m_tuneTrialIndex; // -1 = no multi-trial tuning run in progress
double m_tuneBestOosForecast;
bool m_tuneLastTrialWasWin;
bool m_tuneHaveBestCheckpoint;
datetime m_tuneStartTrainBar;
//=== Genetic + successive-halving indicator auto-tuner (see TuneIndicatorsAndTrain) ==============
//--- Each candidate is a full ADIndicatorTuner Flatten() vector (all indicator params incl. MA type).
//--- A population evolves across generations (block crossover + mutation = interaction-aware search);
//--- within a generation, candidates are pruned by successive-halving rungs. Every candidate is scored
//--- in a THROWAWAY network (m_evalMode) with a FIXED per-seed weight init, averaged over GA_SEEDS runs,
//--- ranked on BALANCED accuracy (macro-recall). The deployed weights are produced ONLY by the single
//--- full retrain on the winning params at the very end - the search never touches the live model. All
//--- state below is persisted across the chunked, resumable Train() calls (one eval can span many ticks).
bool m_gaActive; // a genetic tuning search is in progress
bool m_gaFinalRetrain; // search done - running the single full deploy-retrain on the winner
bool m_evalMode; // Train() is scoring a throwaway candidate (no persistence, silent cap)
int m_evalEraBudget; // era cap for the current rung's candidate evaluation
int m_gaPopSize; // candidates per generation
int m_gaGen; // current generation (0-based)
int m_gaMaxGen; // total generations to run
int m_gaRung; // current successive-halving rung within this generation
int m_gaCandPos; // position within m_gaAlive currently being evaluated
int m_gaSeedIdx; // which of GA_SEEDS fixed-seed repeat runs for this candidate
double m_gaPop[]; // flat: m_gaPopSize * AD_TUNE_PARAM_COUNT candidate params
double m_gaScore[]; // per-candidate averaged balanced-accuracy score (size popSize)
double m_gaScoreSum[]; // per-candidate running score sum across seeds this rung
int m_gaAlive[]; // indices (into m_gaPop) of candidates still alive this rung
int m_gaAliveCount; // how many of m_gaAlive are live
double m_gaBestParams[]; // best candidate params seen across the whole search
double m_gaBestScore; // its balanced-accuracy score
bool m_gaHaveBest;
//--- GA helpers
int GaRungEras(int rung);
void GaExtract(int candIdx, double &out[]); // copy candidate candIdx out of m_gaPop
void GaStore(int candIdx, const double &in[]); // copy `in` into candidate candIdx of m_gaPop
void GaRandomCandidate(const double &base[], double &out[], int mutations);
void GaBlockCrossover(const double &pa[], const double &pb[], double &child[]);
void GaMutate(double &cand[], int mutations);
void GaSortAliveByScoreDesc(void);
void GaBreedNextGeneration(void);
//================================================================================================
void FinalizeTrainRun(void);
//--- The "this is now THE model" persistence sequence, shared by every deploy path so they can't
//--- drift apart: weights (carrying the current m_trainingComplete flag), the pure-MQL5 inference
//--- self-check, the calibration sidecar, and the EMA shadow. Callers set m_trainingComplete first -
//--- the flag is written INTO the .nnw here. Used by FinalizeTrainRun() (ladder/era-cap/stop deploys)
//--- and by DeployNow()/RetrainDeployed() (the panel buttons).
void PersistDeployedModel(void);
//--- On hitting the per-run era cap: asks the operator whether to keep training (true) or deploy
//--- the best checkpoint and stop (false). Headless (tester/optimizer) can't show a dialog, so it
//--- returns false. See m_maxErasPerRun's declaration comment.
bool PromptContinuePastEraCap(double bestOos);
//--- variables
//--- training control, driven by the control panel (Warrior_EA.mq5); Train()/OnTickHandler
//--- poll these rather than being torn down/rebuilt, so pausing/stopping never loses in-memory state
bool m_trainingPaused; // true: Train() blocks between eras until unpaused
bool m_trainingStopRequested; // true: OnTickHandler stops scheduling new training passes
//--- true for ANY Strategy Tester run - a single backtest AND every optimization pass (MQL_TESTER):
//--- the run must NEVER train. It seeds the agent-local cache from the deployed production model (see
//--- InitNeuralNetwork) and runs pure inference over the window - what a buyer expects from "backtest
//--- my deployed EA", and what makes optimizing TRADING parameters (SL/TP, filters, MM) fast and
//--- comparable (the AI is held fixed, not retrained per config). Training + online continual learning
//--- happen only on a live chart, where this is false.
bool m_inferenceOnly;
//--- true only when the current Net weights came from a saved .nnw on disk, not from a freshly-built
//--- random topology. Used to let an inference-only tester replay a seeded model even if that model's
//--- persisted trainingComplete flag is still false, while still blocking the "no model found, built a
//--- fresh topology" path from placing random-weight trades.
bool m_modelLoadedFromDisk;
//--- true once ValidateCpuInference() has confirmed this model's pure-MQL5 forward pass matches the
//--- compute backend's within tolerance (see CNet::SetCpuInference). Persisted in the .stats sidecar
//--- so an inference-only backtest can run DLL-free; if false (e.g. a conv/LSTM topology not yet
//--- ported, or a validation miss) the backtest falls back to the DLL. Measured at deploy on the
//--- chart (where a backend exists to compare against), never in the tester itself.
bool m_mqlInferenceValidated;
string m_fileName;
string m_folderPath;
//--- which file Train()'s Net.Save() calls (and this method's own Net.Load()) actually target:
//--- the shared FILE_COMMON production weights normally, or a LOCAL per-agent cache file when
//--- running inside the Strategy Tester/optimizer (see InitNeuralNetwork) so that repeated
//--- optimization passes with an unchanged topology can reuse an already-trained model instead of
//--- re-running every era from scratch, without ever touching the live production .nnw/.cfg.
string m_activeFileName;
bool m_activeFileCommon;
//--- Name of the terminal-wide global variable this instance holds as an exclusive claim on
//--- m_activeFileName, or "" when it holds none. See AcquireConfigLock().
string m_configLockName;
//--- user-settable via Inputs.mqh's TrainingOptimizer (SGD or ADAM), read into this member at
//--- construction. Honored by all three signal types - PAI/CONV via BuildFreshTopology(), and
//--- LSTM via CSignalLSTM::AddCustomLayers - since CNeuronLSTMOCL (AI\Network.mqh) now has an
//--- accelerated SGD+momentum path (LSTM_UpdateWeightsMomentum) alongside its original Adam-only
//--- one, and CNet::CNet's GPU/DirectML init gate no longer restricts LSTM topologies to ADAM.
int m_optimizationAlgo;
int m_historyBars;
int m_outputNeuronsCount;
int m_minNeuronsCount;
int m_initialNeuronsCount;
int m_neuronsCount;
double m_neuronsReduction;
int m_hiddenLayersCount;
//--- LSTM-only recurrent hidden-unit count - see LstmHiddenSize's declaration comment
//--- (Variables\Inputs.mqh). Harmless, unused constant contribution to m_fingerprint for MLP/CONV.
int m_lstmHiddenSize;
//--- CONV-only convolutional output-filter count - see ConvFilterCount's declaration comment
//--- (Variables\Inputs.mqh). Harmless, unused constant contribution to m_fingerprint for MLP/LSTM.
int m_convFilterCount;
int m_studyPeriod;
int m_minTrainYear;
bool m_isInitialized;
bool m_purgeChartOnDestruct;
int m_fractalPeriods;
//--- "weights" of the 4 CONFIDENCE TIERS a live AI fire can land in (0-100 each), the AI equivalent
//--- of classic indicators' several geometric m_pattern_N members. ConfidenceTier() buckets the
//--- live confidence magnitude (CalibratedConfidenceMagnitude()) into 4 equal bands between the
//--- HEAD'S OWN structural floor and 1.0 - 1/3 for the 3-class softmax, 0.5 for the regression head -
//--- tier 0 = weakest possible directional call, tier 3 = near-certain. UpdateSignalsWeights()
//--- (Expert\ExpertSignalCustom.mqh) then calibrates each tier's weight independently from its OWN
//--- realized win rate, same as any classic pattern.
//---
//--- The defaults span the SAME 0-100 conviction scale the classic patterns use, and that is the
//--- whole mechanism by which one Min vote to open governs both engines. Confidence is expressed as
//--- vote strength rather than as a separate entry floor, so Min_Vote_Open reads directly as "which
//--- tier is good enough", with no second scale to reason about:
//--- 25 tier 0 conf 0.333-0.500 (3-class) / 0.500-0.625 (regression) - a bare plurality
//--- 50 tier 1 conf 0.500-0.667 / 0.625-0.750
//--- 75 tier 2 conf 0.667-0.833 / 0.750-0.875
//--- 100 tier 3 conf 0.833-1.000 / 0.875-1.000 - near-certain
//--- So Min_Vote_Open = 50 lets tier 1 and up trade when the AI votes alone, 75 lets tier 2 and up,
//--- and so on. A near-coin-flip 0.34 argmax is NOT blocked at the AI boundary - it votes 25 and is
//--- filtered by the vote threshold, exactly as a weight-10 classic confirmation would be. Note these
//--- are only the DEFAULTS: with UseDatabaseRanking on, each tier's weight moves to its own measured
//--- win rate, which is precisely why the tiers must be separate patterns rather than one continuous
//--- confidence-to-weight formula - a formula would leave the ranking system nothing to calibrate.
//---
//--- This used to be a single m_pattern_0 covering every fire regardless of confidence. That was a
//--- real bug, not just a missed opportunity: with only one pattern, UpdateSignalsWeights()'
//--- averaging step collapses (average of one value is that value), so the SAME win rate got written
//--- into both the pattern weight AND the module weight (filter.Weight()) - and Direction() multiplies
//--- them together. A genuine 70%-win-rate model was therefore scored as (0.70*70)=49, not 70 - a
//--- quadratic, not linear, derating that got harsher the further from 100% the model's real win rate
//--- was. Classic indicators never hit this because their module weight blends across SEVERAL
//--- differently-performing patterns, diluting any one pattern's own score instead of squaring it.
//--- Splitting into 4 genuinely-different tiers restores that same blending for AI signals.
//---
//--- Defaults (constructor) are graduated 80/87/93/100 - all within the 80-100 band classic
//--- indicators reserve for their most rigorous patterns, since even tier 0 has already passed the
//--- training-completion gate, the confidence floor, and the alternation gate (LongCondition's
//--- comments) before ever reaching here. Hybrid mode (AIType==HYBRID) uses the same shared
//--- weighted-vote path as the other models; no quorum gate is needed anymore.
int m_pattern_0, m_pattern_1, m_pattern_2, m_pattern_3;
//--- functions
virtual bool InitIndicators(CIndicators *indicators);
//--- sets ID/m_id/m_folderPath/m_fileName/m_pattern_count from the subclass constructor - defaults
//--- to 4 (the confidence tiers - see m_pattern_0's declaration comment), not 1
void SetIdentity(string id, string shortId, int patternCount = 4);
//--- hook for neuron-type-specific layers (Conv+Pool, LSTM, ...); default is a plain perceptron (no-op)
virtual bool AddCustomLayers(CArrayObj *topology) { return true; }
//--- Reusable front-end stages, composed by the AddCustomLayers() overrides. Each subclass names the
//--- stages it wants instead of re-declaring the CLayerDescription fields, so the topologies cannot
//--- silently drift apart: HYBRID is *defined* as AddConvStage + AddLstmStage, which makes its
//--- "matches the standalone CONV front-end exactly, then adds LSTM" contract structural rather than
//--- a comment. (They had already drifted - HYBRID guarded the LSTM step with MathMax(1,...) and
//--- CSignalLSTM did not, so a historyBars of 1 gave the two a different step.)
bool AddConvStage(CArrayObj *topology);
bool AddLstmStage(CArrayObj *topology);
//--- Appends a batch-normalization layer, or does nothing (returning success) when EnableBatchNorm is
//--- off. `units` is advisory only - CNet sizes the layer from whatever sits below it, because a conv
//--- or pool stage's output width is derived inside the CNet constructor and is not knowable here.
//--- See AI\NeuronBatchNorm.mqh for what the layer does and why it exists.
bool AddBatchNormStage(CArrayObj *topology, int units);
//--- hardcoded activation for the common tapering Dense hidden-layer stack built by
//--- BuildFreshTopology() (below AddCustomLayers, above the output layer). PRELU (leaky ReLU,
//--- 0.01 slope) is the default: it doesn't saturate/vanish the way TANH does across a deep
//--- taper, and is what the Conv layer itself already uses (see CSignalCONV::AddCustomLayers).
//--- CSignalLSTM overrides this to TANH instead - these Dense layers sit directly on top of the
//--- LSTM layer's own TANH-bounded ([-1,1]) output, so keeping them bounded too avoids feeding an
//--- unbounded activation straight off a bounded recurrent output, which is untested territory
//--- here and not what the user asked for ("Tanh for LSTM gates").
virtual ENUM_ACTIVATION HiddenLayerActivation(void) { return PRELU; }
//--- Single source of truth for the output head's activation. BuildFreshTopology() stamps it into a
//--- NEW topology; EnforceTopologyContract() re-asserts it after every Load(), because a .nnw
//--- persists the activation and would otherwise pin a superseded architecture forever (see
//--- CNet::EnforceOutputActivation's declaration comment in AI\Network.mqh for the incident this
//--- comes from). Deliberately one expression called from both places - when these were two separate
//--- literals, changing the head in BuildFreshTopology() silently did nothing to any existing model.
//--- Regression (1 output): TANH - its native [-1,1] range maps directly onto the -1/0/1
//--- Sell/Neutral/Buy target convention. Classification (3 outputs): SIGMOID - see the long
//--- rationale at BuildFreshTopology()'s use of this method for why the head must stay BOUNDED.
ENUM_ACTIVATION OutputLayerActivation(void) const { return (m_outputNeuronsCount == 1) ? TANH : SIGMOID; }
//--- Width of the first dense layer, DERIVED rather than configured. It used to be an input
//--- (FIRST_LAYER_NEURONS, default 500) whose only sensible value depends entirely on two things the
//--- user cannot see: how wide the input vector ended up after feature selection, and how much
//--- in-sample data the study period actually yields. Left to a hand-picked constant it was badly
//--- wrong - 500 units against a 420-wide input is 210,500 weights, 72% of a 292,583-weight model,
//--- against ~36,500 training bars of which only ~2,236 are directional. That is ~8 parameters per
//--- sample, and it EXPANDS a set of highly correlated inputs instead of compressing them. The
//--- symptom is already in the logs: the shallowest topology consistently beat the deepest, which is
//--- what over-parameterization looks like from the outside.
//--- Computed from the STUDY PERIOD rather than from bars currently downloaded, so the answer is a
//--- deterministic function of the inputs and cannot drift as history fills in - and is then snapped
//--- to a coarse power-of-two ladder so even a large error in the estimate lands on the same rung.
//--- Every field it reads is already part of the weights-filename fingerprint, so the derived value
//--- needs no fingerprint entry of its own. MUST be called before the fingerprint is built and never
//--- again (see the note on fingerprint-feeding members at the top of this file).
int ComputeFirstLayerWidth(void) const;
//--- Re-assert everything about a just-loaded net that lives in the FILE but is owned by the CODE.
//--- Call after every successful Net.Load(); no-ops (and stays silent) when the file already agrees.
void EnforceTopologyContract(void);
//--- common network bootstrap: indicators, topology build/load, training-file bookkeeping
bool InitNeuralNetwork(CIndicators *indicators);
//--- Exclusive per-config claim, so two charts can never train into one set of model files. Every
//--- retrain-affecting input is already hashed into m_activeFileName, so "same file" IS "same
//--- config" - which makes the filename the only correct lock identity. Live charts only: each
//--- tester/optimizer agent is a separate process with its own sandboxed _optcache copy, and they
//--- are *meant* to run the same config in parallel.
bool AcquireConfigLock(void);
void ReleaseConfigLock(void);
void DrawObject(datetime time, double signal, double high, double low);
void DeleteObject(datetime time);
//--- Time-ordered NMS sweep over m_arrowSignalCache: prunes each same-direction run down to its
//--- earliest bar (deleting redundant neighbors within m_signalClusterWindow). Run once per era end.
void PruneDirectionalClusters(int bars);
//--- Live newest-bar NMS accept test (time-keyed, idempotent per bar time - see m_signalClusterWindow).
bool NmsLiveAccept(datetime barTime, ENUM_SIGNAL dir, double conf)
{
if(m_signalClusterWindow <= 0)
return true;
if(dir != Buy && dir != Sell)
return true;
// Idempotent re-eval of the same bar (RefreshLatestSignal can run more than once per bar).
if(dir == Buy && m_nmsLiveBuyTime == barTime)
return m_nmsLiveBuyAccept;
if(dir == Sell && m_nmsLiveSellTime == barTime)
return m_nmsLiveSellAccept;
long minGap = (long)m_signalClusterWindow * PeriodSeconds();
datetime lastSame = (dir == Buy) ? m_nmsLiveBuyTime : m_nmsLiveSellTime;
bool accept;
// 1) Same-direction contiguous collapse: suppress if within the window of the previous SEEN
// same-direction bar (advance last-seen below either way, so a whole run collapses to one).
if(lastSame != 0 && (long)(barTime - lastSame) <= minGap)
accept = false;
else
{
// 2) Cross-direction resolution vs the last KEPT opposite signal: keep the stronger side.
accept = true;
if(m_nmsLiveKeptTime != 0 && m_nmsLiveKeptDir != dir &&
(long)(barTime - m_nmsLiveKeptTime) <= minGap)
{
if(conf > m_nmsLiveKeptConf)
DeleteObject(m_nmsLiveKeptTime); // this bar is stronger: remove the weaker opposite arrow
else
accept = false; // the kept opposite is stronger: suppress this bar
}
}
if(dir == Buy)
{
m_nmsLiveBuyTime = barTime;
m_nmsLiveBuyAccept = accept;
}
else
{
m_nmsLiveSellTime = barTime;
m_nmsLiveSellAccept = accept;
}
if(accept)
{
m_nmsLiveKeptTime = barTime;
m_nmsLiveKeptDir = dir;
m_nmsLiveKeptConf = conf;
}
return accept;
}
void PurgeChart(void);
ENUM_SIGNAL DoubleToSignal(double value);
//--- Shared status-label formatting for all three of Train()'s era passes (pass 1 sequential scan/
//--- display, pass 2 shuffled backProp, pass 3 post-training OOS scoring) - see m_isTrainQueue's and
//--- m_isPass2Active's declaration comments for why the era loop is now three passes instead of one.
//--- Extracted so the panel keeps updating every bar throughout ALL three passes instead of freezing
//--- during pass 2/3 the way it did when this was pass 1-only inline code - a stalled-looking panel
//--- during a still-running era was reported as looking "stuck". Throttled internally (see
//--- m_lastStatusLabelUpdateTick) - SetStatusLabel() (System\StatusLabel.mqh) does real text-layout
//--- work (TextGetSize/WrapLineInto per line) AND an unconditional ChartRedraw(0) every call, which
//--- gets slower as more chart objects accumulate over a long backtest. Calling it unconditionally
//--- once per bar was already the existing (pass-1-only) behavior; doing that again independently in
//--- pass 2 AND pass 3 roughly TRIPLED total ChartRedraw() calls per era and was the actual cause of
//--- a training run observed taking 1.5+ hours without completing era 0 - not the shuffle itself.
//--- forceRefresh=true bypasses the throttle below - used exactly once per era, right after the
//--- era-end block (Train()) finalizes m_eraCount/dOosForecast, so the panel's "Era %d"/"OOS Acc"
//--- fields update in the SAME moment the console's "training in progress - era N" line does.
//--- Without it, the panel's last real redraw during a normal (throttled) bar-scan call happened
//--- DURING pass 3, before m_eraCount++ - so it kept showing era N-1's number (with what was, by
//--- then, already era N's near-final accuracy) for this era's entire duration, only catching up to
//--- the correct era number once the NEXT era's own bar-scan calls started - a full one-era-behind
//--- display lag relative to the console log, reported in practice.
void UpdateTrainingStatusLabel(const string &progressLine, double neuron0, double neuron1, double neuron2, double signalValue, bool forceRefresh = false);
//--- Per-instance (NOT a function-local static - see CExpertSignalCustom::Direction()'s declaration
//--- comment for why that distinction matters for a method shared across PAI/CONV/LSTM instances)
//--- wall-clock throttle gate for UpdateTrainingStatusLabel()'s ChartRedraw().
uint m_lastStatusLabelUpdateTick;
//--- Last values passed to UpdateTrainingStatusLabel() - cached (updated on EVERY call, throttled
//--- or not) so the forced era-end refresh above has something real to redraw with instead of a
//--- stale/zeroed placeholder, since no "current bar" exists once an era's own three passes are done.
double m_lastDisplayNeuron0, m_lastDisplayNeuron1, m_lastDisplayNeuron2, m_lastDisplaySignal;
//--- Latest OOS Buy/Sell recall (-1 = n/a, same convention as logBuyRecallPct etc.), cached the same
//--- way as m_lastDisplayNeuron0 above so UpdateTrainingStatusLabel() can show it on every call, not
//--- just the era-end one that actually just computed it. Surfaced on-chart (not just the Experts
//--- log) because a Buy/Sell-diluted-by-Neutral headline accuracy number is what a trader watching
//--- the panel sees by default, but Buy/Sell recall is what actually predicts trading performance -
//--- Neutral is "don't trade," so a model can look good on blended accuracy purely by calling Neutral
//--- often, while its actual Buy/Sell calls are unreliable.
int m_lastBuyRecallPct, m_lastSellRecallPct;
//--- turns the classification output layer's 3 values (TempData[0..2], SIGMOID activation - see
//--- BuildFreshTopology() - each already in [0,1]) into a softmax probability distribution in
//--- place, and returns the signed dPrevSignal convention (+P(buy), -P(sell), 0.0 exactly for
//--- neutral) used by both Train()'s live-forecast branch and RefreshLatestSignal(). Softmax is
//--- monotonic per-element so it can't change which of the 3 wins - it only exists to turn the 3
//--- independently-trained values into a normalized confidence. Max-subtracted before exp() for
//--- numerical stability regardless (safe since softmax(x) == softmax(x - max(x))).
double ApplyClassificationSoftmax(void);
//--- Post-hoc logit adjustment / prior correction: reads the raw softmax probabilities
//--- ApplyClassificationSoftmax() just left in TempData[0..2] and returns the PRIOR-CORRECTED signed
//--- decision (same +P'(buy)/-P'(sell)/0-neutral convention). This is the exact rule live trading
//--- fires on and the rule the live-fired precision metric scores. See the definition for the math.
double AdjustedSignalFromSoftmax(void);
//--- EMA-updates the persisted true class base rates (m_priorBuy/Sell/Neutral) from a just-finished
//--- era's true class counts. No-op on an empty/degenerate tally.
void UpdateClassPriors(long buyCnt, long sellCnt, long neutralCnt);
//--- Installs tau*log(prior_c) on Net from the freshly measured priors. Called once per era
//--- start, straight after UpdateClassPriors, so the offsets track the same distribution the
//--- era is scored against. No-op (and actively clears stale offsets) when the input is off.
void ApplyLogitAdjustment(void);
//--- Small binary sidecar (fileName + ".stats") persisting the calibration state that must survive a
//--- restart for live trading to behave like training: the true class priors and m_confidenceCalScale.
bool SaveModelStats(string fileName, bool common);
bool LoadModelStats(string fileName, bool common);
//--- Deploy-time (chart, backend present) self-check: runs the just-saved deployed model through both
//--- the backend and a temporary pure-MQL5 (CNet::SetCpuInference) clone on the same input window and
//--- returns true only if the outputs match within CPU_INFERENCE_MAX_DIFF. Gates whether an
//--- inference-only backtest may run DLL-free. Fails safe (returns false) on any error/mismatch or an
//--- architecture whose CPU path isn't ported yet - the caller then keeps the model on the DLL path.
bool ValidateCpuInference(void);
//--- Build the panel's "Buy/Sell accuracy: IS x% OOS y%" line (directional win-rate, Neutral excluded)
//--- from the cumulative counts (m_cumIsCorrect etc.); returns "...: measuring..." until at least one
//--- directional call has been validated. Shared by the training and live/complete simple panels.
string ComputeCompoundedAccuracyLine(void);
//--- Persist/restore the drawn directional arrows (the "WarSig_" objects) to a sidecar file so they
//--- survive an EA remove/re-add, recompile, or restart WITHOUT a retrain - the chart objects are
//--- destroyed on unload (destructor PurgeChart) and OnInit has no other way to bring them back.
//--- Stores each arrow's time/code/price and its hide state (OBJPROP_TIMEFRAMES), so the show/hide
//--- toggle is preserved too. Chart-only (a backtest has no persistent chart to restore to).
void SaveChartSignals(void);
void LoadChartSignals(void);
//--- Wipe this model's drawn arrows AND their .arrows sidecar, plus any deferred restore still in
//--- flight. Call from every path that discards or replaces the trained weights - see the definition
//--- for why leaving them behind resurrects a dead model's calls through SaveChartSignals.
void ClearPersistedChartSignals(const string reason);
//--- Deferred ("async") half of LoadChartSignals: LoadChartSignals only PARSES the sidecar into the
//--- m_arrowRestore* buffers (an ~80KB read - instant) and returns, so OnInit never blocks; this then
//--- creates the chart objects in ARROW_RESTORE_BUDGET_MS slices, driven by the same 500ms timer that
//--- already paces training. MQL5 has no threads - a chart runs one thread - so blocking OnInit is what
//--- made the terminal look frozen (no panel, no status label, no journal) while ~2900 arrows were
//--- rebuilt. Time-boxed slices give the terminal room to paint the UI between them instead.
void AdvanceChartSignalRestore(void);
//--- parsed-but-not-yet-drawn arrows, consumed by AdvanceChartSignalRestore (see above)
datetime m_arrowRestoreTime[];
int m_arrowRestoreCode[];
double m_arrowRestorePrice[];
long m_arrowRestoreTf[];
int m_arrowRestoreIndex;
bool m_arrowRestorePending;
uint m_arrowRestoreStartMs;
//--- Deferred ("async") half of StartChartSignalRescan (public, defined inline further down): drains
//--- the per-bar inference loop in ARROW_RESTORE_BUDGET_MS slices off PollTraining's timer instead of
//--- blocking the button-click handler for however long a full lookback scan takes. Internal-only -
//--- called from PollTraining(), never from outside the class - so this stays protected while
//--- StartChartSignalRescan()/RescanPending() (which the panel button needs) are public.
void AdvanceChartSignalRescan(void);
int m_rescanIndex;
int m_rescanHi;
int m_rescanBarsNow;
bool m_rescanPending;
uint m_rescanStartMs;
//--- Raw (PRE prior-correction) argmax tally, accumulated per-bar across AdvanceChartSignalRescan's
//--- slices - lets the completion log distinguish "the network itself calls Neutral almost everywhere"
//--- from "the network still discriminates, but AdjustedSignalFromSoftmax's logit-prior correction is
//--- suppressing it down to Neutral" - both produce an identical all-Neutral m_arrowSignalCache/empty
//--- chart otherwise.
int m_rescanRawBuy;
int m_rescanRawSell;
int m_rescanRawNeutral;
bool ResizeBuffers(int barIndex);
bool RefreshData();
bool BufferTempData(int idx);
//--- shared by OnTickHandler() and the timer-driven PollTraining() - see definition
void ScheduleTrainingIfNeeded(void);
void Train(datetime StartTrainBar = 0);
//--- outer loop around Train(): when AutoTuneIndicators is on, tries randomized AD indicator
//--- input variations across m_indicatorTuneTrials calls to Train(), keeping the best-OOS one
void TuneIndicatorsAndTrain(datetime StartTrainBar = 0);
//--- recomputes dPrevSignal/chart arrow for the most recent bar; used after restoring a
//--- checkpointed model at the end of Train() so the live signal matches the deployed weights
void RefreshLatestSignal();
//--- inference-only "new bar" handler used once m_trainingComplete is true - see
//--- ScheduleTrainingIfNeeded()'s declaration comment for why this must NOT call Net.backProp()
void RefreshConvergedSignal(void);
//--- Online continual-learning step (live chart only) - see its implementation comment and the
//--- ONLINE_LEARN_* tunables. Backprops the deployed Net on bars whose ZigZag label has just become
//--- CONFIRMED (m_swingConfirmationBars matured), then blends the shadow under a rolling-accuracy
//--- guardrail. No-op in the tester/optimizer (m_inferenceOnly) and while training is active.
void OnlineLearnStep(void);
//--- Alpha-balanced focal sample weight (Lin et al. 2017 eq. 5) for ONE streamed bar - see the
//--- ONLINE_LEARN_* block's CLASS IMBALANCE comment for the derivation. Shared deliberately by
//--- OnlineLearnStep() (the live path) and AdvanceOosSimulationChunk() (the simulation of that
//--- path): the simulation's reported accuracy is only a valid forecast of live continual-learning
//--- behaviour if it optimises the IDENTICAL objective, so the formula must exist in exactly one
//--- place. p* are this bar's pre-update softmax probabilities, already normalised in place by
//--- ApplyClassificationSoftmax(). Returns 1.0 for the regression head (no class structure).
double OnlineSampleWeight(ENUM_SIGNAL trueSignal, double pBuy, double pSell, double pNeutral);
//--- Confirmed ZigZag label for a now-relative bar index that is already old enough to be non-
//--- repainting (idx >= m_swingConfirmationBars) - the exact same verdict AdvanceZigZagLabelState()
//--- caches for training (LABEL_WINDOW_BARS is 0, so no widening to replicate). Returns Buy at a
//--- confirmed bottom, Sell at a confirmed top, else Neutral - all three are valid training targets.
ENUM_SIGNAL ConfirmedZigZagLabel(int idx);
//--- lazily bootstraps m_shadowNet if it's still NULL: tries loading a persisted shadow file
//--- first (continuity across EA restarts), falling back to cloning Net's current weights (via
//--- the same Save()/Load() pattern StartOosContinualSimulation() uses for m_simOosNet) if no
//--- compatible shadow file exists yet. No-op if m_shadowNet is already valid. See m_shadowNet's
//--- declaration comment for the full EMA shadow-weight deployment rationale.
void EnsureShadowNet(void);
//--- persists m_shadowNet alongside every Net.Save() call, using the same run metadata (error/
//--- undefine/forecast/era/trainingComplete/indicator params) the caller already computed for
//--- Net.Save() itself - see m_shadowNet's declaration comment. No-op if the shadow isn't
//--- bootstrapped yet.
void SaveShadowNet(const double &indicatorParams[]);
//--- method of initialization of the indicators
bool InitOpen(CIndicators *indicators);
bool InitClose(CIndicators *indicators);
bool InitHigh(CIndicators *indicators);
bool InitLow(CIndicators *indicators);
bool InitVolumes(CIndicators *indicators);
bool InitTime(CIndicators *indicators);
//--- addToCollection=false is used by ReInitADIndicators() to rebuild an already-collected
//--- handle's params (here: a re-tuned period) without re-adding the (same) pointer into
//--- indicators a second time
bool InitMA(CIndicators *indicators, bool addToCollection = true);
bool InitRSI(CIndicators *indicators, bool addToCollection = true);
bool InitMACDFeature(CIndicators *indicators, bool addToCollection = true);
bool InitIchimoku(CIndicators *indicators, bool addToCollection = true);
bool InitADCumulativeDelta(CIndicators *indicators, bool addToCollection = true);
bool InitADShorteningOfThrust(CIndicators *indicators, bool addToCollection = true);
bool InitADWyckoffEventStream(CIndicators *indicators, bool addToCollection = true);
bool InitADWyckoffFailedStructure(CIndicators *indicators, bool addToCollection = true);
bool InitADWyckoffSignificantBarInversion(CIndicators *indicators, bool addToCollection = true);
bool InitADZigZag(CIndicators *indicators, bool addToCollection = true);
//--- common=false targets a LOCAL (non-shared) file - used by the tester/optimizer per-agent
//--- weight cache so cross-pass reuse never touches the production FILE_COMMON config/weights.
bool SaveTopologyConfiguration(string fileName, int initialNeuronsCount, int hiddenLayersCount, double neuronsReduction, int minNeuronsCount, int optimizationAlgo, int historyBars, int outputNeuronsCount, int neuronsCount, int studyPeriod, int minTrainYear, bool isInitialized, int stopTrainWR, int fractalPeriods, bool common = true);
bool LoadAndCompareTopologyConfiguration(string fileName, int initialNeuronsCount, int hiddenLayersCount, double neuronsReduction, int minNeuronsCount, int optimizationAlgo, int historyBars, int outputNeuronsCount, int neuronsCount, int studyPeriod, int minTrainYear, bool isInitialized, int stopTrainWR, int fractalPeriods, bool common = true);
//--- Retry helpers for the tester/opt seed-copy race: a live chart's own atomic Save() (write .savetmp,
//--- then FileMove() over the real file) can hold the source or destination file for a moment, and a
//--- concurrent FileCopy/FileOpen from a Strategy Tester agent reading the SAME production file can hit
//--- a transient Windows sharing violation in that narrow window. Both retry a handful of times with a
//--- short pause rather than silently treating a transient lock as "no model"/"corrupt file" - see their
//--- call sites in InitNeuralNetwork.
bool CopyFileWithRetry(string srcFileName, string dstFileName);
bool CopySharedFile(string srcFileName, string dstFileName, bool quiet);
bool LoadNetWithRetry(double &indicatorParams[]);
//--- input data
bool m_useVolumes;
bool m_useTime;
bool m_useATR;
//--- Uses its own period (m_indicatorTuner.maPeriod), fed as ATR-normalized OHLC distance-from-MA
//--- (4 values, same convention as the base close-open/high-open/low-open features) plus the MA's
//--- own bar-over-bar change (1 value, ATR-normalized like every other price-domain feature here -
//--- not volume's previous-bar-ratio scheme, since a moving average lives in price units and
//--- already has ATR as its natural scale reference). See BufferTempDataCompute()'s m_useMA block
//--- for the exact 5 values. maPeriod starts equal to the Classic Signals PeriodMA input (see
//--- CADIndicatorTuner's constructor) but may diverge from it once AutoTuneIndicators searches a
//--- trial - the Classic Signals MA vote itself is untouched by that search, since it needs no
//--- training/warm-up and there is nothing for a tuning trial to validate it against.
bool m_useMA;
//--- RSI is already a 0-100 oscillator, so the only transform needed is /100 to match every other
//--- feature's roughly [-1,1]/[0,1] scale - no ATR or distance normalization applies. See
//--- BufferTempDataCompute()'s m_useRSI block for the exact value. Same maPeriod/rsiPeriod
//--- divergence-from-the-Classic-Signals-input note as m_useMA above applies to rsiPeriod.
bool m_useRSI;
//--- MACD as 3 ATR-normalized values (main line, signal line, histogram) - see
//--- BufferTempDataCompute()'s m_useMACD block. Deliberately kept to 3 despite MACD being cheap: the
//--- point of adding it next to m_useMA is the SECOND timescale (m_useMA supplies exactly one moving
//--- average) and the histogram, which is the only acceleration term anywhere in the feature vector -
//--- the level information itself is already covered by the MA distances. ATR-normalized rather than
//--- left raw because the MACD lines live in price units, exactly like the MA feature.
bool m_useMACD;
//--- Ichimoku as 8 values - see BufferTempDataCompute()'s m_useIchimoku block for each. This is the
//--- widest single classic-indicator feature here and it earns that width by carrying multi-timescale
//--- support/resistance GEOMETRY (three lookbacks plus a forward-projected cloud) that nothing else in
//--- the vector encodes: the swing-context block's confirmed pivots are >=m_swingConfirmationBars bars
//--- stale by construction, and its Donchian/SMA values are single-scale.
//--- LOOKAHEAD - MT5's iIchimoku stores raw per-bar values and shifts only the DRAWING, so the cloud
//--- sitting under bar idx is SenkouSpan*(idx + ichiKijun), and the Chikou value plotted at bar idx
//--- would be Close(idx - ichiKijun) - a FUTURE bar. The feature block applies the +Kijun offset and
//--- never calls ChinkouSpan(); Signals\SignalIchimoku.mqh's class comment documents the buffer
//--- convention in full, and the same reasoning governs both.
bool m_useIchimoku;
//--- Normalized ZigZag swing-context features: 5 confirmed-pivot values (direction/magnitude/age of
//--- the last CONFIRMED swing) plus 4 recent-price-action values (Donchian range position at 20/50
//--- bars, 20-bar return, 20-bar SMA extension) that give fresh, non-repainting trend/position
//--- context the >=100-bar-stale pivot anchor can't - see BufferTempDataCompute()'s m_useSwingContext
//--- block for the exact 9 values. Reads the same
//--- m_ADZigZag the training labels already come from (see that member's declaration comment) rather
//--- than a separate indicator instance - NOT gated behind AutoTuneIndicators/ReInitADIndicators like
//--- the AD* indicators below, since m_ADZigZag itself is deliberately never tuned (same reason as
//--- the label side: tuning the ground truth alongside the model scored against it would let a trial
//--- cherry-pick an easier target). Critical correctness constraint, not just a style choice: ZigZag's
//--- most recent 1-3 legs repaint (see m_swingConfirmationBars' declaration comment), so every read of
//--- m_ADZigZag for THIS feature - exactly like the label side - must only trust a pivot that is
//--- already at least m_swingConfirmationBars bars old relative to the bar the feature is being
//--- computed for. Skipping that embargo would leak future information a live bar couldn't actually
//--- have had yet - silent lookahead bias inflating backtest/training performance without being real.
bool m_useSwingContext;
//--- see System\NewsRelevance.mqh's declaration comment for what this feature actually encodes
//--- (event proximity + impact, not actual-vs-forecast deviation) and why the forward-looking half
//--- of it isn't lookahead bias.
bool m_useNews;
int m_newsFeatureWindowMinutes;
bool m_useADCumulativeDelta;
bool m_useADShorteningOfThrust;
bool m_useADWyckoffEventStream;
bool m_useADWyckoffFailedStructure;
bool m_useADWyckoffSignificantBarInversion;
public:
CExpertSignalAIBase(void);
~CExpertSignalAIBase(void);
//--- "voting" that price will grow/fall, common to every AI signal (single market model)
virtual int LongCondition(void);
virtual int ShortCondition(void);
//--- alternation-gate snapshot/rollback - see m_gateSnapshot and the base declarations
virtual void BeginVote(void) override { m_gateSnapshot = m_lastNonNeutralSignal; }
virtual void RevokeVote(void) override { m_lastNonNeutralSignal = m_gateSnapshot; }
// |dPrevSignal| is already a 0..1 confidence for classification output (softmax
// probability of the winning class) and typically bounded for regression output
// (tanh-activated network); OpenParams() clamps regardless. Scaled by m_confidenceCalScale
// (classification head only - 1.0/no-op for regression) so callers get an empirically
// calibrated magnitude instead of the raw, uncalibrated softmax value - see
// m_confidenceCalScale's declaration comment.
double CalibratedConfidenceMagnitude(void) const
{
double mag = MathAbs(dPrevSignal);
if(!MathIsValidNumber(mag))
return 0.0;
if(m_outputNeuronsCount == 3)
mag = MathMin(1.0, mag * m_confidenceCalScale);
if(!MathIsValidNumber(mag))
return 0.0;
return mag;
}
virtual double AIConfidence(void) override { return CalibratedConfidenceMagnitude(); }
// Signed for direction-aware use (AI-driven early exit): sign matches dPrevSignal's
// convention (+ buy, - sell, 0 neutral/no signal yet). dPrevSignal == -2 is the
// "not yet studied" sentinel, not a real sell signal - treat it as no confidence.
virtual double SignedAIConfidence(void) override
{
if(dPrevSignal == -2)
return 0.0;
double sign = (dPrevSignal > 0.0) ? 1.0 : (dPrevSignal < 0.0) ? -1.0 : 0.0;
if(sign == 0.0)
return 0.0;
return sign * CalibratedConfidenceMagnitude();
}
//--- event handlers, common to every AI signal
virtual void OnTickHandler(void);
//--- drives the same training-scheduling check as OnTickHandler(), but callable from a timer so
//--- it isn't dependent on ticks (which don't arrive while the market is closed)
void PollTraining(void);
virtual void OnChartEventHandler(const int id,
const long &lparam,
const double &dparam,
const string &sparam);
//--- methods of adjusting "weights" of the 4 confidence-tier market models - see m_pattern_0's
//--- declaration comment
void Pattern_0(int value) { m_pattern_0 = value; }
void Pattern_1(int value) { m_pattern_1 = value; }
void Pattern_2(int value) { m_pattern_2 = value; }
void Pattern_3(int value) { m_pattern_3 = value; }
virtual void ApplyPatternWeight(int patternNumber, int weight);
//--- buckets the live confidence magnitude into one of the 4 tiers above - see m_pattern_0's
//--- declaration comment. Public so PollTraining()/status-display code could surface which tier is
//--- currently active if ever useful, though LongCondition/ShortCondition are the only callers today.
int ConfidenceTier(void);
int PatternWeightForTier(int tier);
//--- methods of setting adjustable parameters
//--- No public setter for m_initialNeuronsCount. It feeds the weights-filename fingerprint, and it is
//--- now derived exactly once, inside InitNeuralNetwork(), before that fingerprint is built - see
//--- ComputeFirstLayerWidth(). An external setter could only ever be called after construction and
//--- would either be ignored (if before init) or silently re-key the model mid-run (if after).
void OutputNeuronsCount(int value) { m_outputNeuronsCount = value; }
//--- No setter: the taper's endpoints are derived, not configured. See BuildFreshTopology()'s taper
//--- block and the note in Variables\Inputs.mqh. Kept as members only because the .cfg topology
//--- sidecar's field layout is positional and rewriting it would invalidate every model on disk.
void HiddenLayersCount(int value) { m_hiddenLayersCount = value; }
void LstmHiddenSize(int value) { m_lstmHiddenSize = value; }
void ConvFilterCount(int value) { m_convFilterCount = value; }
//--- StopTrainWR(int) removed with the MinWR input - there is no absolute accuracy target any more;
//--- see LEGACY_CONVERGE_WR_SLOT and the plateau ladder.
void MinDirectionalRecall(int value) { m_minDirectionalRecallPct = value; }
//--- MinSignalConfidence(double) removed with the AI entry floor - confidence now reaches the trade
//--- decision as vote weight (ConfidenceTier), gated by the one Min vote to open threshold that the
//--- classic votes already answer to. See m_pattern_0's declaration comment.
//--- 0.1..1.0 fraction of full class parity for minority oversampling (see m_oversampleParity)
void OversampleParity(double value) { m_oversampleParity = MathMax(0.1, MathMin(1.0, value)); }
void LogitPriorStrength(double value) { m_logitPriorStrength = MathMax(0.0, value); }
void UseLogitAdjustedLoss(bool value) { m_useLogitAdjustedLoss = value; }
void LogitAdjustTau(double value) { m_logitAdjustTau = MathMax(0.0, value); }
void EnableMinorityReplay(bool value) { m_enableMinorityReplay = value; }
void ConstrainReplay(bool value) { m_constrainReplay = value; }
void FreezePriorCalibration(bool value) { m_freezePriorCalibration = value; }
void UseStaticPrior(bool value) { m_useStaticPrior = value; }
void SignalClusterWindow(int value) { m_signalClusterWindow = value; }
//--- Sets the CONFIGURED gamma (model identity, part of the weights-filename fingerprint) and seeds
//--- the runtime value the loss applies. Only the plateau ladder moves the runtime one from here.
void FocalLossGamma(double value) { m_focalGamma = value; m_focalGammaRuntime = value; }
void SwingConfirmationBars(int value) { m_swingConfirmationBars = value; }
void EnableOnlineLearning(bool value) { m_enableOnlineLearning = value; }
void MaxErasPerRun(int value) { m_maxErasPerRun = value; }
void OOSSplit(int value) { m_oosSplitPct = value; }
void HistoryBars(int value) { m_historyBars = value; }
void StudyPeriod(int value) { m_studyPeriod = value; }
void MinTrainYear(int value) { m_minTrainYear = value; }
void UseVolumes(bool value) { m_useVolumes = value; }
void UseTime(bool value) { m_useTime = value; }
void UseATR(bool value) { m_useATR = value; }
void UseMA(bool value) { m_useMA = value; }
void UseRSI(bool value) { m_useRSI = value; }
void UseMACD(bool value) { m_useMACD = value; }
void UseIchimoku(bool value) { m_useIchimoku = value; }
void UseSwingContext(bool value) { m_useSwingContext = value; }
void UseNews(bool value) { m_useNews = value; }
void NewsFeatureWindowMinutes(int value) { m_newsFeatureWindowMinutes = value; }
void UseADCumulativeDelta(bool value) { m_useADCumulativeDelta = value; }
void UseADShorteningOfThrust(bool value) { m_useADShorteningOfThrust = value; }
void UseADWyckoffEventStream(bool value) { m_useADWyckoffEventStream = value; }
void UseADWyckoffFailedStructure(bool value) { m_useADWyckoffFailedStructure = value; }
void UseADWyckoffSignificantBarInversion(bool value) { m_useADWyckoffSignificantBarInversion = value; }
void AutoTuneIndicators(bool value) { m_autoTuneIndicators = value; }
void IndicatorTuneTrials(int value) { m_indicatorTuneTrials = value; }
//--- control-panel API (Warrior_EA.mq5): current-config-only training/weights control.
//--- "current config" == this signal instance's own m_fileName (symbol+period+id+topology),
//--- never touches another signal type's or another symbol/timeframe's saved files.
void PauseTraining(void) { m_trainingPaused = true; PrintVerbose(ID + ": training paused by user (era " + IntegerToString(m_eraCount) + ")"); }
void ResumeTraining(void) { m_trainingPaused = false; PrintVerbose(ID + ": training resumed by user (era " + IntegerToString(m_eraCount) + ")"); }
bool IsTrainingPaused(void) const { return m_trainingPaused; }
bool IsTrainingStopped(void) const { return m_trainingStopRequested; }
long EraCount(void) const { return m_eraCount; }
bool TrainingComplete(void) const { return m_trainingComplete; }
void StopTraining(void)
{
m_trainingStopRequested = true;
m_trainingPaused = false;
//--- ScheduleTrainingIfNeeded() refuses to schedule another "New Bar" event while
//--- m_trainingStopRequested is set, so a run interrupted mid-chunk would otherwise never get
//--- called again to finalize (restore the best checkpoint, persist state) - do it synchronously
//--- here instead. Safe to block briefly: this is just a checkpoint file restore, not the
//--- multi-minute bar loop.
if(m_trainRunActive)
FinalizeTrainRun();
Print(ID + ": training stopped by user (era " + IntegerToString(m_eraCount) + ", weights as of last completed era retained)");
}
void StartTraining(void)
{
if(!m_trainingStopRequested && !m_trainingPaused)
return;
m_trainingStopRequested = false;
m_trainingPaused = false;
if(!bEventStudy)
bEventStudy = EventChartCustom(ChartID(), 1, (long)dtStudied, 0, "Resume");
Print(ID + ": training (re)started by user (era " + IntegerToString(m_eraCount) + ")");
}
//--- Has an era ever cleared the per-class recall floor and been checkpointed this run? This is the
//--- same quality bar the plateau ladder's auto-deploy requires (see PLATEAU_STAGE_DEPLOY), exposed so
//--- the panel can warn before a MANUAL deploy ships a model that ignores Buy or Sell.
bool HasRecallPassingCheckpoint(void) const { return m_bestPassedRecall; }
double BestBalancedAccuracy(void) const { return m_bestBalancedOos; }
//--- MANUAL deploy (panel "Deploy Model"): finalise whatever the run has found so far as THE model -
//--- exactly what the plateau ladder does on its own at stage 3, just triggered early by the operator.
//--- Restores the best checkpoint (not whatever era the run happened to be mid-way through), persists
//--- weights + calibration + shadow, and flips to live inference.
//--- Deliberately does NOT set m_trainingStopRequested: like every other deploy path, a deployed model
//--- still runs live inference AND online continual learning. A panel Stop is the thing that halts
//--- everything - see StopTraining()/OnlineLearnStep()'s gate. Reversible via RetrainDeployed().
bool DeployNow(void)
{
if(CheckPointer(Net) == POINTER_INVALID || !m_isInitialized)
return false;
if(m_trainingComplete)
return true; // already deployed - nothing to do
//--- Set BEFORE any save below: the flag is written INTO the .nnw, so persisting first would store
//--- "still training" and a restart would resume the era loop instead of running the deployed model.
m_trainingComplete = true;
m_trainingPaused = false;
m_trainingStopRequested = false;
if(m_trainRunActive || m_haveOosCheckpoint)
FinalizeTrainRun(); // restores the best checkpoint, persists, ends the run
else
{
//--- Nothing trained this session (e.g. deploying a model that was just loaded from disk), so
//--- there is no in-memory checkpoint to restore - persist exactly what is loaded right now.
PersistDeployedModel();
SaveChartSignals();
}
RefreshLatestSignal();
Print(ID + ": model DEPLOYED by user at era " + IntegerToString(m_eraCount) +
" (balanced accuracy " + (m_bestBalancedOos < 0 ? "n/a" : DoubleToString(m_bestBalancedOos, 1) + "%") +
", blended OOS " + DoubleToString(dOosForecast, 1) + "%) - training stopped, now running live inference" +
(m_enableOnlineLearning ? " with online continual learning" : "") +
". Use the panel's \"Retrain Model\" to resume training from here.");
return true;
}
//--- The inverse of DeployNow(), and the ONLY way back: while m_trainingComplete is set,
//--- ScheduleTrainingIfNeeded() routes every tick to the converged/inference branch, so StartTraining()
//--- alone can never revive a deployed model (it clears the stop flag, but the complete flag still wins
//--- that branch). Clearing it here re-arms the normal training path, continuing from the DEPLOYED
//--- weights rather than from scratch - "reset weights" is the separate, destructive button for that.
//--- The plateau ladder, best-checkpoint tracking and annealed gamma all reset themselves when Train()
//--- starts its next fresh run (see the !m_trainRunActive block), so a retrain does not inherit the
//--- exhausted stage that deployed the model and immediately re-deploy it.
void RetrainDeployed(void)
{
if(!m_trainingComplete)
return;
m_trainingComplete = false;
m_trainingStopRequested = false;
m_trainingPaused = false;
//--- Persist the cleared flag immediately. Otherwise a terminal restart before the first era
//--- completes would reload the .nnw still marked complete and silently go back to inference-only,
//--- looking like the button did nothing.
PersistDeployedModel();
if(!bEventStudy)
bEventStudy = EventChartCustom(ChartID(), 1, (long)dtStudied, 0, "Retrain");
Print(ID + ": RETRAINING the deployed model from era " + IntegerToString(m_eraCount) +
" - keeping its current weights as the starting point (use \"Delete & Reset Weights\" to start from scratch instead).");
}
//--- Manual "rescan" of the drawn signal arrows: purges every arrow currently on the chart (namespaced
//--- delete - user drawings untouched) and re-infers the last SIGNAL_RESCAN_LOOKBACK_BARS bars from the
//--- CURRENTLY deployed weights, then re-runs the same end-of-era NMS declutter (PruneDirectionalClusters)
//--- used during training so the fresh set matches what a live re-render would have produced. Wired to
//--- the panel's Hide->Show Signals sequence: without this, "restore" only ever replays whatever was
//--- last saved to the .arrows sidecar, which for a long-deployed model can be a stale historical render
//--- from whenever it was last actually trained - years-old arrows crowding out anything recent. Chart-only
//--- (no persistent chart in the tester/optimizer) and a no-op until a model has something to infer with.
//--- This only does the cheap setup (buffer resize, arrow purge, cache alloc) and QUEUES the per-bar
//--- inference loop for AdvanceChartSignalRescan() to drain in time-boxed slices off the timer - see
//--- that method's comment for why the loop itself must never run in one blocking pass. Returns true
//--- once a rescan has been queued (check RescanPending() for completion), false if there was nothing
//--- to rescan (no deployed model, tester/optimizer context, etc).
bool StartChartSignalRescan(void)
{
if(MQLInfoInteger(MQL_TESTER) || MQLInfoInteger(MQL_OPTIMIZATION) || MQLInfoInteger(MQL_FORWARD))
return false;
if(CheckPointer(Net) == POINTER_INVALID || !m_isInitialized || !m_trainingComplete)
return false;
if(m_outputNeuronsCount != 1 && m_outputNeuronsCount != 3)
return false;
int barsAvail = Bars(m_symbol.Name(), PERIOD_CURRENT);
int barsNow = MathMin(SIGNAL_RESCAN_LOOKBACK_BARS, barsAvail);
if(barsNow <= m_historyBars)
return false;
if(!ResizeBuffers(barsNow) || !RefreshData())
return false;
EnsureShadowNet();
//--- Drop only the arrows THIS rescan is about to re-judge - i.e. those within [now, oldest bar of
//--- the barsNow window] - not every namespaced arrow on the chart. PruneDirectionalClusters below
//--- only touches indices inside [0, barsNow), so anything older survives untouched either way; the
//--- previous version called ObjectsDeleteAll(0, SIG_ARROW_PREFIX) unconditionally, which wiped
//--- EVERY arrow ever drawn (years of history, going back to whenever this EA was first attached)
//--- and then only ever redrew the last SIGNAL_RESCAN_LOOKBACK_BARS (~7 months on H1) - anything
//--- older was gone permanently the moment Show Signals was clicked, with no way back short of the
//--- .arrows sidecar (itself only ever a snapshot from whatever the LAST save happened to catch).
//--- Reported 2026-07-26: a user's multi-year arrow history vanished down to whatever a stale
//--- .arrows file held, immediately after their first-ever successful Show Signals click. This scoped
//--- delete is the fix - older arrows are never in scope to be wiped in the first place.
datetime rescanCutoffTime = m_Time.GetData(barsNow - 1);
for(int oi = ObjectsTotal(0, 0, OBJ_ARROW) - 1; oi >= 0; oi--)
{
string onm = ObjectName(0, oi, 0, OBJ_ARROW);
if(StringFind(onm, SIG_ARROW_PREFIX) != 0)
continue;
if((datetime)ObjectGetInteger(0, onm, OBJPROP_TIME) >= rescanCutoffTime)
ObjectDelete(0, onm);
}
ArrayResize(m_arrowSignalCache, barsNow);
ArrayInitialize(m_arrowSignalCache, -2.0);
m_rescanBarsNow = barsNow;
m_rescanHi = barsNow - m_historyBars;
m_rescanIndex = 0;
m_rescanRawBuy = 0;
m_rescanRawSell = 0;
m_rescanRawNeutral = 0;
m_rescanPending = (m_rescanHi > 0);
m_rescanStartMs = GetTickCount();
if(m_rescanPending)
Print(ID + ": rescanning last " + IntegerToString(m_rescanHi) + " bars against the deployed model (progressive, non-blocking)...");
return m_rescanPending;
}
//--- true while a queued rescan (StartChartSignalRescan above) still has slices left for
//--- AdvanceChartSignalRescan to drain - polled by Warrior_EA.mq5's FinalizeSignalsRescanIfDone() to
//--- know when it's safe to (re)apply arrow visibility and report the Show Signals click as complete.
bool RescanPending(void) const { return m_rescanPending; }
//--- forces a save of the network's current in-memory weights/state regardless of era-completion
//--- state; called from OnDeinit() so shutdown/chart-removal never loses more than the current tick
//--- of learning, and a subsequent restart's Train() resumes from m_eraCount rather than the last
//--- fully-completed era only.
//--- Heavy weight/state persistence ONLY (net weights + calibration sidecar + shadow net). Split out
//--- from PersistOnShutdown() so OnDeinit() can run the cheap chart save+purge FIRST: on the CPU-DLL
//--- box this recursive save (two full nets) is the slow/fragile step, and if it ever stalls past MT5's
//--- deinit budget or faults, the arrows and status panel must already be gone, not stranded on the
//--- chart (the reported "cleanup not going well - signals/panel stay after Abnormal termination").
bool PersistWeightsOnShutdown(void)
{
if(CheckPointer(Net) == POINTER_INVALID || !m_isInitialized)
return false;
//--- An inference-only run (any Strategy Tester pass - see m_inferenceOnly) trains NOTHING, so there
//--- is no new state to persist and this save can only do harm. It is exactly what corrupted the
//--- tester cache on 2026-07-26: when the seeded load failed, the !netLoaded path reset the in-memory
//--- state to a fresh untrained net (era 0), and this unconditional shutdown save then wrote THAT
//--- over the good seeded copy - which, because seeding only re-runs when the cache file is absent,
//--- silently poisoned every subsequent backtest. Skipping it makes the tester cache strictly
//--- read-only for single backtests: it can only ever be (re)written by the seed copy from
//--- production, never by a run's own in-memory state.
if(m_inferenceOnly)
{
PrintVerbose(ID + ": inference-only run - skipping the shutdown weight save (nothing was trained; the cached model is left exactly as seeded).");
return true;
}
double currentIndicatorParams[];
m_indicatorTuner.Flatten(currentIndicatorParams);
bool ok = Net.Save(m_activeFileName + ".nnw", dError, dUndefine, dForecast, dtStudied, m_activeFileCommon, m_eraCount, m_trainingComplete, currentIndicatorParams);
//--- calibration state (class priors + confidence scale) must travel with the weights so live
//--- trading behaves like training after a restart - see SaveModelStats().
if(!SaveModelStats(m_activeFileName, m_activeFileCommon))
Print(ID + ": ERROR - shutdown SaveModelStats failed for " + m_activeFileName + ". Calibration state not persisted.");
//--- Deliberately do NOT save the shadow net here. On the CPU-DLL box a second full-net write
//--- (~18MB) roughly DOUBLES the shutdown cost, and OnDeinit has a limited budget before MT5 reports
//--- "Abnormal termination" and skips the rest of the teardown. (An earlier revision also blamed that
//--- overrun for the "failed to allocate layer 0" reloads; that was a misdiagnosis - the real cause
//--- was a lost virtual override in the read path, see AI\Network.mqh's CLayer::CreateElement note.
//--- Halving the shutdown cost is still worth it on its own.) The shadow is NOT lost: it is saved
//--- every era (Train()), at FinalizeTrainRun(), and every ONLINE_LEARN_PERSIST_EVERY bars during
//--- online learning, and it self-heals (EnsureShadowNet re-blends from the main Net when
//--- missing/stale). Worst case a
//--- shutdown loses only the shadow's in-progress-era drift, which re-converges - a far better
//--- trade than risking the whole model to an over-budget shutdown.
if(!ok)
Print(ID + ": ERROR - failed to persist weights on shutdown for " + m_activeFileName + ", error " + IntegerToString(GetLastError()));
else
PrintVerbose(ID + ": weights persisted on shutdown (era " + IntegerToString(m_eraCount) + ", trainingComplete=" + (string)m_trainingComplete + ")");
return ok;
}
//--- Persist the drawn arrows to disk, then remove THIS EA's chart visuals (arrows + status label).
//--- Called early in OnDeinit(), before the heavy weight save, so a later stall/fault in the save can
//--- never leave the chart littered. Idempotent - the destructor's PurgeChart() then simply no-ops.
//--- Deliberately NOT part of SaveWeightsNow(): a mid-session manual save must not wipe the chart.
void ShutdownChartCleanup(const bool preserveChartArrows = false)
{
bool isTesterRun = (MQLInfoInteger(MQL_TESTER) || MQLInfoInteger(MQL_OPTIMIZATION) || MQLInfoInteger(MQL_FORWARD));
if(isTesterRun)
{
// Tester runs do not need arrow sidecar persistence; keep shutdown cheap and deterministic.
if(preserveChartArrows)
{
m_purgeChartOnDestruct = false;
ClearStatusLabel();
return;
}
PurgeChart();
return;
}
//--- save arrows BEFORE purging them so a re-add/recompile restores them - see SaveChartSignals()/
//--- LoadChartSignals(). PurgeChart() then clears our arrows + the status label.
SaveChartSignals();
if(preserveChartArrows)
{
//--- Keep arrows visible across in-process reloads (recompile/inputs/template/chart-change).
//--- The object itself is still destroyed by Expert.Deinit(), so suppress the destructor purge.
m_purgeChartOnDestruct = false;
ClearStatusLabel();
return;
}
PurgeChart();
}
//--- Full shutdown persistence (weights + arrows), preserved for the panel's manual "save weights"
//--- button (SaveWeightsNow) - does NOT purge the chart. OnDeinit no longer calls this; it runs
//--- ShutdownChartCleanup() then PersistWeightsOnShutdown() so cleanup can't be starved by the save.
bool PersistOnShutdown(void)
{
bool ok = PersistWeightsOnShutdown();
//--- Persist the drawn arrows too so a re-add/recompile restores them without a retrain.
SaveChartSignals();
return ok;
}
//--- explicit manual save, identical persistence to PersistOnShutdown() but user-triggered from the panel
bool SaveWeightsNow(void) { return PersistOnShutdown(); }
//--- reloads this signal's current-config weights file from disk, discarding any unsaved in-memory
//--- training progress since the last successful save
bool LoadWeightsNow(void)
{
if(CheckPointer(Net) == POINTER_INVALID)
return false;
double loadedIndicatorParams[];
bool netLoaded = Net.Load(m_activeFileName + ".nnw", dError, dUndefine, dForecast, dtStudied, m_activeFileCommon, m_eraCount, m_trainingComplete, loadedIndicatorParams);
if(!netLoaded)
{
Print(ID + ": ERROR - failed to load weights from " + m_activeFileName + ".nnw, error " + IntegerToString(GetLastError()));
return false;
}
m_modelLoadedFromDisk = true;
//--- the file may carry a superseded architecture - correct it before anything reads the net
EnforceTopologyContract();
//--- restore the calibration state that pairs with these weights (priors + confidence scale) so a
//--- manual reload keeps live decisions calibrated exactly as the saved model was - see LoadModelStats().
LoadModelStats(m_activeFileName, m_activeFileCommon);
if(ArraySize(loadedIndicatorParams) == AD_TUNE_PARAM_COUNT)
{
m_indicatorTuner.Unflatten(loadedIndicatorParams);
if(m_indicatorsPtr != NULL)
ReInitADIndicators(m_indicatorsPtr);
}
//--- this just swapped dtStudied/m_eraCount/Net's weights out from under whatever a chunked
//--- Train() run (see its declaration comment) had cached for the era/trial it was mid-way
//--- through - discard that resumable state so the next Train() call starts a fresh era
//--- against the just-loaded dtStudied instead of resuming bar-loop bookkeeping computed
//--- against a now-stale one.
m_trainRunActive = false;
m_eraResumePending = false;
m_haveOosCheckpoint = false;
m_oosWindow.Clear();
m_tuneTrialIndex = -1;
RefreshLatestSignal();
Print(ID + ": weights reloaded from disk (era " + IntegerToString(m_eraCount) + ", trainingComplete=" + (string)m_trainingComplete + ")");
return true;
}
//--- deletes this signal's current-config saved files only (weights, topology config, in-progress
//--- checkpoint) and rebuilds a fresh untrained topology in memory so training restarts from era 0.
//--- Never touches another signal type's or another symbol/timeframe's files - m_fileName already
//--- embeds symbol+period+id+output-count+opt-algo.
bool ResetWeights(void)
{
bool stopped = m_trainingStopRequested;
m_trainingStopRequested = true; // hold off any in-flight Train() scheduling while we reset
//--- targets whichever file this run is actually training against (see InitNeuralNetwork): the
//--- shared production weights normally, or the local tester/optimizer cache during a backtest -
//--- so resetting from the panel during a visual-mode backtest can never wipe the live model.
int flags = m_activeFileCommon ? FILE_COMMON : 0;
string nnw = m_activeFileName + ".nnw";
string cfg = m_activeFileName + ".cfg";
string ckpt = m_activeFileName + "_ckpt.tmp";
//--- The calibration/online-learning sidecar (.stats: priors, confidence scale, CPU-inference marker,
//--- and the online watermark/guardrail - see SaveModelStats) and the deployed EMA shadow
//--- (_shadow.nnw - see SaveShadowNet) both pair with the weights being erased. Delete them too, or
//--- a fresh retrain would silently inherit the OLD model's calibration and blend into a stale shadow
//--- (EnsureShadowNet loads _shadow.nnw from disk before it ever clones the new Net).
string stats = m_activeFileName + ".stats";
string shadow = m_activeFileName + "_shadow.nnw";
ResetLastError();
if(FileIsExist(nnw, flags) && !FileDelete(nnw, flags))
Print(ID + ": ERROR - failed to delete " + nnw + ", error " + IntegerToString(GetLastError()));
ResetLastError();
if(FileIsExist(cfg, flags) && !FileDelete(cfg, flags))
Print(ID + ": ERROR - failed to delete " + cfg + ", error " + IntegerToString(GetLastError()));
ResetLastError();
if(FileIsExist(ckpt, flags) && !FileDelete(ckpt, flags))
Print(ID + ": ERROR - failed to delete " + ckpt + ", error " + IntegerToString(GetLastError()));
ResetLastError();
if(FileIsExist(stats, flags) && !FileDelete(stats, flags))
Print(ID + ": ERROR - failed to delete " + stats + ", error " + IntegerToString(GetLastError()));
ResetLastError();
if(FileIsExist(shadow, flags) && !FileDelete(shadow, flags))
Print(ID + ": ERROR - failed to delete " + shadow + ", error " + IntegerToString(GetLastError()));
//--- The drawn signal arrows and their .arrows sidecar belong to the model being erased, exactly
//--- like the .stats/_shadow sidecars above - see ClearPersistedChartSignals().
ClearPersistedChartSignals("weights reset from the panel");
m_eraCount = 0;
m_trainingComplete = false;
m_modelLoadedFromDisk = false;
dtStudied = 0;
dError = -1;
dUndefine = 0;
dForecast = 0;
dPrevSignal = 0;
m_lastNonNeutralSignal = Neutral;
m_gateSnapshot = Neutral;
m_nmsLiveBuyTime = 0;
m_nmsLiveSellTime = 0;
m_nmsLiveBuyAccept = false;
m_nmsLiveSellAccept = false;
m_nmsLiveKeptTime = 0;
m_nmsLiveKeptDir = Neutral;
m_nmsLiveKeptConf = 0;
dOosError = -1;
dOosForecast = 0;
m_oosSamples = 0;
//--- fresh model => wipe the compounded/persistent accuracy history too (it is only reset here; a
//--- normal restart restores it from .stats, and it survives era-to-era). See m_cumIsCorrect.
m_cumIsCorrect = 0;
m_cumIsTotal = 0;
m_cumOosCorrect = 0;
m_cumOosTotal = 0;
//--- discard any in-progress chunked run/tuning state - it references buffers/checkpoints from
//--- before this reset and must never be resumed into the freshly rebuilt topology below
m_trainRunActive = false;
m_eraResumePending = false;
m_haveOosCheckpoint = false;
m_oosWindow.Clear();
m_syncWaitStartTick = 0;
m_tuneTrialIndex = -1;
//--- an explicit reset restarts from era 0 against a fresh topology - re-verify history sync and
//--- rebuild the label cache too, and drop any in-flight continual-learning OOS simulation, since
//--- both would otherwise reference bars/weights from before this reset.
m_warmupPassesRemaining = 3;
m_labelCacheBars = 0;
m_labelCacheAnchorTime = 0;
m_labelCachePrebuilt = false;
m_labelPrebuildActive = false;
m_prebuildSeedPending = false;
if(m_simOosRunActive)
{
delete m_simOosNet;
m_simOosNet = NULL;
m_simOosRunActive = false;
}
//--- Re-seed before building a fresh topology so weight init is genuinely random, not dominated
//--- by whatever fixed/deterministic seed the genetic tuner's last candidate evaluation left in
//--- the MQL5 RNG state (GA_SEED_BASE + m_gaSeedIdx). Matches the tuner's own final-retrain path
//--- (line ~5071) and Warrior_EA.mq5's OnInit.
MathSrand(GetTickCount());
bool rebuilt = BuildFreshTopology();
if(!rebuilt)
Print(ID + ": ERROR - failed to rebuild fresh topology after weights reset");
else
{
//--- Stamp the config fingerprint with isInitialized=FALSE, NOT m_isInitialized. The compare in
//--- InitNeuralNetwork (and its own fresh-start .cfg write) always run before m_isInitialized is
//--- set true at the end of init, so they always use false. ResetWeights is invoked from the panel
//--- AFTER init, where m_isInitialized is true - passing it here would make this the ONLY .cfg on
//--- disk with isInitialized=true, so the very next attach spuriously fails the compare
//--- ("Configuration mismatch. Deleting file") and needlessly discards the weights we just reset.
//--- This flag is runtime lifecycle state, not a topology/input parameter, so it must never gate reuse.
SaveTopologyConfiguration(m_activeFileName, m_initialNeuronsCount, m_hiddenLayersCount, m_neuronsReduction, m_minNeuronsCount, m_optimizationAlgo, m_historyBars, m_outputNeuronsCount, m_neuronsCount, m_studyPeriod, m_minTrainYear, false, LEGACY_CONVERGE_WR_SLOT, m_fractalPeriods, m_activeFileCommon);
Print(ID + ": weights reset - training will restart from era 0 (current config only: " + m_activeFileName + ")");
}
m_trainingStopRequested = stopped;
if(!stopped && !bEventStudy)
bEventStudy = EventChartCustom(ChartID(), 1, 0, 0, "Reset");
return rebuilt;
}
};
//+------------------------------------------------------------------+
//| Constructor |
//+------------------------------------------------------------------+
//--- These are only fallback defaults for a fresh object before the EA's OnInit() applies the
//--- active input values via the public setters in Warrior_EA.mq5. The input-driven values are the
//--- source of truth for the actual run configuration.
CExpertSignalAIBase::CExpertSignalAIBase(void) :
ID("NULL"),
m_neuronsCount(0),
m_studyPeriod(10),
m_minTrainYear(1970),
m_optimizationAlgo(TrainingOptimizer), // see the member declaration comment
//--- Placeholder only; InitNeuralNetwork() replaces it with ComputeFirstLayerWidth() before anything
//--- reads it. Deliberately the floor rather than 0, so a hypothetical path that built a topology
//--- without going through init would produce a small usable net instead of a zero-width layer.
m_initialNeuronsCount(FIRST_LAYER_MIN_WIDTH),
m_outputNeuronsCount(OUTPUT_CLASSIFICATION),
//--- Frozen. Nothing reads these to build a topology any more - the taper derives its own endpoints
//--- (BuildFreshTopology) - but they still occupy positional slots in the .cfg sidecar and the weights
//--- fingerprint. Held at their historical defaults so both stay byte-stable; changing either value
//--- would re-key every model on disk for no behavioural reason whatsoever.
m_minNeuronsCount(MIN_NEURONS_20),
m_neuronsReduction(RF_70),
m_hiddenLayersCount(3),
m_lstmHiddenSize(32),
m_convFilterCount(16),
m_historyBars(14),
m_fractalPeriods(5),
m_pattern_0(25),
m_pattern_1(50),
m_pattern_2(75),
m_pattern_3(100),
m_useVolumes(true),
m_useTime(true),
m_useATR(true),
m_useMA(false),
m_useRSI(false),
m_useMACD(false),
m_useIchimoku(false),
m_useSwingContext(false),
m_useNews(false),
m_newsFeatureWindowMinutes(60),
m_useADCumulativeDelta(false),
m_useADShorteningOfThrust(false),
m_useADWyckoffEventStream(false),
m_useADWyckoffFailedStructure(false),
m_useADWyckoffSignificantBarInversion(false),
m_autoTuneIndicators(false),
m_indicatorTuneTrials(8),
m_indicatorsPtr(NULL),
Net(NULL),
m_shadowNet(NULL),
TempData(NULL),
dError(-1),
dUndefine(0),
dForecast(0),
dPrevSignal(0),
m_lastNonNeutralSignal(Neutral),
m_gateSnapshot(Neutral),
m_signalClusterWindow(6),
m_nmsLiveBuyTime(0),
m_nmsLiveSellTime(0),
m_nmsLiveBuyAccept(false),
m_nmsLiveSellAccept(false),
m_nmsLiveKeptTime(0),
m_nmsLiveKeptDir(Neutral),
m_nmsLiveKeptConf(0),
dtStudied(0),
m_eraCount(0),
m_trainingComplete(false),
m_inferenceOnly(false),
m_modelLoadedFromDisk(false),
m_mqlInferenceValidated(false),
m_shadowBootstrapAttempted(false),
m_enableOnlineLearning(true),
m_enableMinorityReplay(true),
m_constrainReplay(true),
m_freezePriorCalibration(false),
m_useStaticPrior(false),
m_onlineLearnedUpToTime(0),
m_onlineRollingAcc(-1.0),
m_onlineSamples(0),
m_onlineBarsSincePersist(0),
m_onlineBlendFrozen(false),
bEventStudy(false),
m_oosSplitPct(30),
dOosError(-1),
dOosForecast(0),
m_oosSamples(0),
m_cumIsCorrect(0),
m_cumIsTotal(0),
m_cumOosCorrect(0),
m_cumOosTotal(0),
m_oosOutSpreadSum(0),
m_oosOutCount(0),
m_countBuySignals(0),
m_countSellSignals(0),
m_countNeutralSignals(0),
m_trueBuyCount(0),
m_trueSellCount(0),
m_trueNeutralCount(0),
m_useLogitAdjustedLoss(true),
m_logitAdjustTau(1.0),
m_prevEraTrueBuyCount(0),
m_prevEraTrueSellCount(0),
m_prevEraTrueNeutralCount(0),
m_oosBuyHits(0),
m_oosBuyTotal(0),
m_oosSellHits(0),
m_oosSellTotal(0),
m_oosNeutralHits(0),
m_oosNeutralTotal(0),
m_oosBuyPredicted(0),
m_oosBuyPredictedHits(0),
m_oosSellPredicted(0),
m_oosSellPredictedHits(0),
m_oosNeutralPredicted(0),
m_oosNeutralPredictedHits(0),
m_oosConfidenceSum(0),
m_confidenceCalScale(1.0),
m_minDirectionalRecallPct(40),
m_oversampleParity(OVERSAMPLE_PARITY_FRACTION),
m_logitPriorStrength(0.25),
m_priorBuy(0.0),
m_priorSell(0.0),
m_priorNeutral(0.0),
m_oosBuyFired(0),
m_oosBuyFiredHits(0),
m_oosSellFired(0),
m_oosSellFiredHits(0),
m_lastBuyFiredPrecPct(-1),
m_lastSellFiredPrecPct(-1),
m_lastBuyFired(0),
m_lastSellFired(0),
m_maxClassSampleWeight(1.5),
m_focalGamma(1.0),
m_swingConfirmationBars(100),
m_maxErasPerRun(300),
m_arrowRestoreIndex(0),
m_arrowRestorePending(false),
m_arrowRestoreStartMs(0),
m_rescanIndex(0),
m_rescanHi(0),
m_rescanBarsNow(0),
m_rescanPending(false),
m_rescanStartMs(0),
m_rescanRawBuy(0),
m_rescanRawSell(0),
m_rescanRawNeutral(0),
m_trainRunActive(false),
m_eraResumePending(false),
m_resumeBars(0),
m_resumeTotalIter(0),
m_resumeOosCutoff(0),
m_resumeBarIndex(0),
m_resumeAddLoop(false),
m_isTrainQueueCount(0),
m_isTrainCursor(0),
m_isPass2Active(false),
m_isPass2Done(false),
m_isPass3Active(false),
m_oosScoreIndex(0),
m_oosScoreStartIndex(0),
m_lastStatusLabelUpdateTick(0),
m_lastBuyRecallPct(-1),
m_lastSellRecallPct(-1),
m_lastDisplayNeuron0(0),
m_lastDisplayNeuron1(0),
m_lastDisplayNeuron2(0),
m_lastDisplaySignal(0),
m_lastBarTime(0),
m_modelEta(InitialEtaForOptimizer()),
m_etaCeiling(InitialEtaForOptimizer()),
m_erasSinceCooldown(0),
m_bestOosForecast(-1),
m_bestBalancedOos(-1),
m_bestPassedRecall(false),
m_haveOosCheckpoint(false),
m_oosStable(false),
m_objectiveMet(false),
m_erasSinceBestBalanced(0),
m_plateauStage(0),
m_focalGammaRuntime(2.0),
m_syncWaitStartTick(0),
m_warmupPassesRemaining(0),
m_labelCacheBars(0),
m_labelCacheAnchorTime(0),
m_labelCachePrebuilt(false),
m_labelPrebuildActive(false),
m_prebuildSeedPending(false),
m_labelPrebuildBars(0),
m_labelPrebuildOosCutoff(0),
m_labelPrebuildIndex(-1),
m_labelPrebuildBuyCount(0),
m_labelPrebuildSellCount(0),
m_labelPrebuildNeutralCount(0),
m_simOosNet(NULL),
m_simOosRunActive(false),
m_simOosCutoff(0),
m_simOosBarIndex(-1),
m_simOosForecast(0),
m_simOosSamples(0),
m_tuneTrialIndex(-1),
m_tuneBestOosForecast(-1),
m_tuneLastTrialWasWin(true),
m_tuneHaveBestCheckpoint(false),
m_tuneStartTrainBar(0),
m_gaActive(false),
m_gaFinalRetrain(false),
m_evalMode(false),
m_evalEraBudget(0),
m_gaPopSize(0),
m_gaGen(0),
m_gaMaxGen(0),
m_gaRung(0),
m_gaCandPos(0),
m_gaSeedIdx(0),
m_gaAliveCount(0),
m_gaBestScore(-1),
m_gaHaveBest(false),
m_trainingPaused(false),
m_trainingStopRequested(false),
m_activeFileCommon(true),
m_configLockName(""),
m_isInitialized(false),
m_purgeChartOnDestruct(true)
{
//--- indicator tuning defaults live in CADIndicatorTuner's own constructor (Expert\ADIndicatorTuner.mqh),
//--- which runs automatically for the m_indicatorTuner member above.
}
//+------------------------------------------------------------------+
//| Destructor |
//+------------------------------------------------------------------+
CExpertSignalAIBase::~CExpertSignalAIBase(void)
{
//--- deliberately NOT calling PersistOnShutdown() here: OnDeinit() (Warrior_EA.mq5) already calls
//--- it explicitly for every signal, one call stack frame shallower, BEFORE Expert.Deinit() tears
//--- these objects down. Doing it again here nested inside that same teardown cascade doubled the
//--- stack depth of an already-deep recursive save (layers -> neurons -> connections) right at the
//--- point in MT5's lifecycle (EA recompile while attached) that has the least stack headroom, and
//--- reliably crashed the terminal with a stack overflow. Keep this destructor cheap.
if(CheckPointer(Net) != POINTER_INVALID)
delete Net;
if(CheckPointer(m_shadowNet) != POINTER_INVALID)
delete m_shadowNet;
if(CheckPointer(TempData) != POINTER_INVALID)
delete TempData;
if(CheckPointer(m_simOosNet) != POINTER_INVALID)
delete m_simOosNet;
if(m_purgeChartOnDestruct)
PurgeChart();
else
ClearStatusLabel();
//--- Last, and cheap by design (one global-variable delete): the claim must outlive every save above
//--- it, or a chart re-attaching during this teardown could start writing the same files mid-save.
ReleaseConfigLock();
}
//+------------------------------------------------------------------+
//| Sets the file/id identity a subclass constructor would otherwise |
//| repeat verbatim (ID, m_id, m_folderPath, m_fileName, pattern count)|
//+------------------------------------------------------------------+
void CExpertSignalAIBase::SetIdentity(string id, string shortId, int patternCount = 4)
{
ID = id;
m_id = shortId;
m_folderPath = eaName + "\\" + "Neural Networks" + "\\" + "State" + "\\" + m_id + "\\";
m_fileName = m_folderPath + _Symbol + "_" + IntegerToString(_Period);
m_pattern_count = patternCount;
}
//+------------------------------------------------------------------+
//| "Voting" that price will grow. |
//+------------------------------------------------------------------+
int CExpertSignalAIBase::LongCondition(void)
{
int result = 0;
//--- Readiness gate: live trading still requires a converged model, but an inference-only tester run
//--- may replay a model that was ACTUALLY loaded from disk even if its persisted trainingComplete flag
//--- is false. Without that exception the tester could seed/refresh dPrevSignal from the deployed model
//--- and draw chart arrows from those weights, yet this gate would still hard-zero the trading vote.
//--- A fresh random topology still cannot trade in the tester because m_modelLoadedFromDisk stays false.
if(!m_trainingComplete && !(m_inferenceOnly && m_modelLoadedFromDisk))
return 0;
//--- "not yet studied" sentinel - dPrevSignal == -2 is not a real Sell. Its MAGNITUDE is 2, so it
//--- passed straight through the confidence floor that used to sit here (|-2| exceeds any 0..1
//--- threshold); only the m_trainingComplete gate above was keeping it out. Checked explicitly now
//--- that the floor is gone, rather than left resting on that.
if(dPrevSignal == -2)
return 0;
//--- NO confidence floor here, by design - see m_minSignalConfidence's former declaration site. A
//--- weak call is not blocked at the AI's own boundary; it votes at its tier weight (as low as
//--- m_pattern_0) and is then filtered by Min vote to open, exactly like a weak classic vote.
//--- alternation gate - see m_lastNonNeutralSignal's declaration comment. Suppressed until a Sell
//--- has been seen since the last Buy we fired.
if(DoubleToSignal(dPrevSignal) == Buy && m_lastNonNeutralSignal != Buy)
{
int tier = ConfidenceTier();
result = PatternWeightForTier(tier);
m_active_pattern = "Pattern_" + IntegerToString(tier);
m_active_direction = "Buy";
m_lastNonNeutralSignal = Buy;
}
return(result);
}
//+------------------------------------------------------------------+
//| "Voting" that price will fall. |
//+------------------------------------------------------------------+
int CExpertSignalAIBase::ShortCondition(void)
{
int result = 0;
//--- Readiness gate - see LongCondition's matching comment.
if(!m_trainingComplete && !(m_inferenceOnly && m_modelLoadedFromDisk))
return 0;
//--- "not yet studied" sentinel, and no confidence floor - see LongCondition's matching comments.
if(dPrevSignal == -2)
return 0;
//--- alternation gate - see m_lastNonNeutralSignal's declaration comment. Suppressed until a Buy
//--- has been seen since the last Sell we fired.
if(DoubleToSignal(dPrevSignal) == Sell && m_lastNonNeutralSignal != Sell)
{
int tier = ConfidenceTier();
result = PatternWeightForTier(tier);
m_active_pattern = "Pattern_" + IntegerToString(tier);
m_active_direction = "Sell";
m_lastNonNeutralSignal = Sell;
}
return result;
}
//+------------------------------------------------------------------+
//| Buckets the live confidence magnitude into one of 4 equal bands |
//| between the head's own structural floor and 1.0 - see m_pattern_0's|
//| declaration comment for the resulting tier/weight table. Neither |
//| head has a configurable floor: the boundary is 1/3 for the 3-class|
//| softmax and 0.5 for regression (DoubleToSignal's own threshold), |
//| both arithmetic properties of the head rather than settings. |
//+------------------------------------------------------------------+
int CExpertSignalAIBase::ConfidenceTier(void)
{
//--- The head's own STRUCTURAL decision boundary - the lowest confidence magnitude that head can
//--- possibly report for a directional call - not a user setting:
//--- - 3-class classification: the winning class of a 3-way softmax is arithmetically >= 1/3, since
//--- three probabilities summing to 1 cannot all be below it. Nothing can ever be read below this.
//--- - single-neuron regression: 0.5, DoubleToSignal()'s own decision boundary.
//--- Quartiling from HERE (rather than from an input, as the classification branch used to) is what
//--- makes the tier boundaries a fixed property of the model instead of something that silently moves
//--- whenever the trader adjusts an unrelated vote threshold - and it is what lets the same tier
//--- weights mean the same thing on both heads. See m_pattern_0's declaration comment for the weights.
double floorConf = (m_outputNeuronsCount == 3) ? (1.0 / 3.0) : 0.5;
double span = MathMax(1.0 - floorConf, 0.0001);
double t = (CalibratedConfidenceMagnitude() - floorConf) / span;
int tier = (int)MathFloor(t * 4.0);
return MathMax(0, MathMin(tier, 3));
}
//+------------------------------------------------------------------+
//| Returns the given tier's current pattern weight (0-100) |
//+------------------------------------------------------------------+
int CExpertSignalAIBase::PatternWeightForTier(int tier)
{
switch(tier)
{
case 0:
return m_pattern_0;
case 1:
return m_pattern_1;
case 2:
return m_pattern_2;
default:
return m_pattern_3;
}
}
//+------------------------------------------------------------------+
//| Set the specified pattern's weight to the specified value |
//+------------------------------------------------------------------+
void CExpertSignalAIBase::ApplyPatternWeight(int patternNumber, int weight)
{
switch(patternNumber)
{
case 0:
Pattern_0(weight);
break;
case 1:
Pattern_1(weight);
break;
case 2:
Pattern_2(weight);
break;
case 3:
Pattern_3(weight);
break;
default:
break;
}
}
//+------------------------------------------------------------------+
//| OnTick function |
//+------------------------------------------------------------------+
void CExpertSignalAIBase::OnTickHandler(void)
{
ScheduleTrainingIfNeeded();
}
//+------------------------------------------------------------------+
//| Schedules the next training pass (if one is due) and refreshes |
//| the per-tick status label. Factored out of OnTickHandler() so |
//| Warrior_EA.mq5's always-on timer (see PollTraining()) can drive |
//| this on a fixed wall-clock schedule too - training must not stall |
//| just because the market is closed and no ticks are arriving. |
//+------------------------------------------------------------------+
void CExpertSignalAIBase::ScheduleTrainingIfNeeded(void)
{
//--- stopped: no new training passes get scheduled at all (StartTraining() re-arms this).
//--- paused: still schedule so bEventStudy/dtStudied bookkeeping stays current, but Train() itself
//--- blocks at the next era boundary until resumed - keeps in-memory state coherent either way.
//--- complete: training already converged - a plain new bar must NOT re-enter Train()'s full era
//--- loop, which would otherwise reset the best-checkpoint/eta-decay tracking and run real
//--- Net.backProp() passes again, forever, once per bar, on an already-converged model (see
//--- RefreshConvergedSignal()'s declaration comment). Just keep the live signal current instead.
//--- publish this signal's current signed confidence for the intelligent trailing (and any other
//--- live-confidence consumer) - see g_LiveAISignedConfidence in Variables\ConfidenceBridge.mqh.
//--- Cheap: SignedAIConfidence() just reads the already-computed dPrevSignal.
g_LiveAISignedConfidence = SignedAIConfidence();
datetime lastBarDate = (datetime)SeriesInfoInteger(m_symbol.Name(), m_period, SERIES_LASTBAR_DATE);
// A failed lookup (0) must not silently read as "dtStudied is already caught up, nothing pending" -
// that would freeze this function into never re-triggering training/signal refresh again until some
// other path happens to bump dtStudied. Treat a failed lookup as pending instead (same >0-guard
// philosophy as the SERIES_FIRSTDATE lookup elsewhere in this class) so a transient history-sync
// hiccup costs one extra harmless check, not a silent stall.
bool newBarPending = (dPrevSignal == -2 || lastBarDate <= 0 || ((m_inferenceOnly ? m_lastBarTime : dtStudied) < lastBarDate));
//--- m_inferenceOnly (single backtest) takes the converged/inference branch even if the seeded model
//--- wasn't flagged complete, so a backtest never drops into Train()'s era loop - see m_inferenceOnly.
if((m_trainingComplete || m_inferenceOnly) && !m_trainingStopRequested && !m_trainRunActive)
{
if(newBarPending)
RefreshConvergedSignal();
}
else
if(!m_trainingStopRequested && !bEventStudy && newBarPending)
bEventStudy = EventChartCustom(ChartID(), 1, (long)MathMax(0, MathMin(iTime(m_symbol.Name(), PERIOD_CURRENT, (int)(100 * Net.recentAverageSmoothingFactor * (m_trainingComplete ? 1 : 10))), dtStudied)), 0, "New Bar");
//--- Train() (see its declaration comment) now yields every ~TRAIN_TIME_BUDGET_MS instead of
//--- blocking for a whole era, so while a run is active this per-tick line would otherwise
//--- overwrite Train()'s own full-detail status label on every single tick between chunks -
//--- flickering between the two instead of showing one steady picture. Only write this terse
//--- summary when nothing else is actively updating the status label (idle/stopped/paused/cooldown).
if(!m_trainRunActive)
{
//--- Compact, accurate end-state text. The completed state distinguishes a model that is genuinely
//--- adapting live (online learning active - a live chart with EnableOnlineLearning, not the tester)
//--- from one running pure inference (the Strategy Tester, or online learning off), so the label is
//--- literally true either way and never over-promises "keeps learning" when it doesn't - see
//--- OnlineLearnStep()'s gate for exactly when adaptation runs.
bool onlineActive = m_enableOnlineLearning && !m_inferenceOnly
&& !MQLInfoInteger(MQL_TESTER) && !MQLInfoInteger(MQL_OPTIMIZATION) && !MQLInfoInteger(MQL_FORWARD)
&& CheckPointer(Net) != POINTER_INVALID && !Net.CpuInference();
//--- Simple end-state panel (default, VerboseMode off): plain-language status + the model's
//--- compounded/persistent Buy/Sell win-rate (directional accuracy, Neutral excluded - persisted in
//--- .stats WST5, so it survives a fresh chart reload and is meaningful the moment a drop-and-go user
//--- attaches the EA) + the current call. The verbose era/forecast dump below stays for power users.
if(!VerboseMode)
{
string statusPlain;
if(m_trainingComplete)
statusPlain = onlineActive ? "Live - learning from new bars" : "Ready for live trading";
else if(m_trainingStopRequested)
statusPlain = "Paused - progress saved";
else if(m_trainingPaused)
statusPlain = "Paused";
else
statusPlain = "Getting ready...";
ENUM_SIGNAL liveSig = DoubleToSignal(dPrevSignal);
string liveSigPlain = (liveSig == Buy) ? "Buy" : (liveSig == Sell) ? "Sell" : "Neutral (no trade)";
string simpleLive = ID + " - " + statusPlain + "\n";
//--- Only show the accuracy line once at least one signal has been validated (compounded counts
//--- persist across restarts, so a deployed model shows real numbers immediately, not "measuring").
if(m_cumIsTotal > 0 || m_cumOosTotal > 0)
simpleLive += ComputeCompoundedAccuracyLine() + "\n";
simpleLive += "Current signal: " + liveSigPlain;
SetStatusLabel(simpleLive);
return;
}
string completeText = onlineActive ? "Complete - live (adapting to new bars)" : "Complete - ready for live (inference)";
string trainingState = m_trainingStopRequested
? (m_trainingComplete ? completeText : "Stopped - resumable (weights kept)")
: (m_trainingPaused ? "Paused" : (m_trainingComplete ? completeText : "In progress"));
//--- same "Forecast: <signal> -> <value>" line the active training loop's status label ends on
//--- (see the classLine-terminated StringFormat below), instead of a raw bEventStudy/dPrevSignal/
//--- dtStudied debug dump - this is what stays on screen once training stops/pauses/completes.
SetStatusLabel(StringFormat(
ID + " : Era %d -> Training %s\n" +
"Forecast: %s -> %.2f",
m_eraCount, trainingState,
EnumToString(DoubleToSignal(dPrevSignal)), dPrevSignal));
}
}
//+------------------------------------------------------------------+
//| Timer-driven equivalent of OnTickHandler()'s scheduling, called |
//| from Warrior_EA.mq5's always-on OnTimer() so training keeps |
//| progressing purely on wall-clock time - no dependency on ticks, |
//| which simply don't arrive while the market is closed. |
//+------------------------------------------------------------------+
void CExpertSignalAIBase::PollTraining(void)
{
//--- Drain a slice of the queued chart-arrow restore FIRST, and skip training work on any tick where
//--- restoring is still in flight. Both compete for the one MQL5 thread; letting the arrows finish
//--- quickly (a few hundred ms of slices) means the user sees a complete chart almost immediately,
//--- whereas interleaving them with 80ms training chunks would stretch the restore over minutes.
if(m_arrowRestorePending)
{
AdvanceChartSignalRestore();
return;
}
//--- Same one-thread reasoning as the arrow restore above: a manual rescan (Show Signals) also competes
//--- for the single MQL5 thread, and its per-bar feedForward is real compute rather than a cheap object
//--- write, so it must finish its own slices before training resumes rather than interleaving with it.
if(m_rescanPending)
{
AdvanceChartSignalRescan();
return;
}
if(m_isInitialized)
ScheduleTrainingIfNeeded();
}
//+------------------------------------------------------------------+
//| |
//+------------------------------------------------------------------+
void CExpertSignalAIBase::OnChartEventHandler(const int id,
const long &lparam,
const double &dparam,
const string &sparam)
{
if(id == 1001)
{
TuneIndicatorsAndTrain(lparam);
bEventStudy = false;
OnTickHandler();
}
}
//+------------------------------------------------------------------+
//| Claim m_activeFileName for this chart, terminal-wide. |
//| |
//| Two charts running the same AIType with the same retrain-affecting|
//| inputs resolve to the SAME .nnw/.cfg/.stats/checkpoint set. Both |
//| then train independently and save over each other, so whichever |
//| writes last wins and the other's eras are discarded - silently, |
//| because every individual file operation succeeds. A five-chart |
//| comparison run on 2026-07-29 lost both its HYBRID models this way |
//| (one chart left at the AIType default), and the only evidence |
//| anywhere was that model path appearing twice as often in the log. |
//| |
//| A terminal-wide global variable is the right lock rather than a |
//| lock FILE: GlobalVariableTemp() is an atomic create-if-absent, |
//| and a TEMPORARY variable dies with the terminal, so a crash can |
//| never leave a stale lock that blocks the next start. Within one |
//| session a stale entry is still possible (an EA removed without a |
//| clean deinit), so the owner's chart id is stored and revalidated. |
//+------------------------------------------------------------------+
bool CExpertSignalAIBase::AcquireConfigLock(void)
{
//--- FNV-1a over the resolved filename: every retrain-affecting input is already folded into that
//--- name, so equal names mean genuinely equal configs and nothing else has to be compared. Hashed
//--- because MQL5 caps global-variable names at 63 characters and the path alone exceeds that.
uint h = 2166136261;
int len = StringLen(m_activeFileName);
for(int i = 0; i < len; i++)
{
h ^= (uint)StringGetCharacter(m_activeFileName, i);
h *= 16777619;
}
string name = "WarriorAI_" + m_id + "_" + StringFormat("%08x", h);
long self = ChartID();
//--- Atomic: true means it did not exist and is now ours.
if(GlobalVariableTemp(name))
{
GlobalVariableSet(name, (double)self);
m_configLockName = name;
return true;
}
long owner = (long)GlobalVariableGet(name);
//--- Our own entry: this chart is re-initializing after a parameter change or a recompile whose
//--- OnDeinit never reached ReleaseConfigLock(). Reclaim it instead of refusing to start.
if(owner == self)
{
m_configLockName = name;
return true;
}
//--- Owner recorded but its chart no longer runs an expert - take the claim over. owner == 0 is
//--- deliberately NOT treated as stale: it means another instance created the variable microseconds
//--- ago and has not stamped its id yet, which is a live claim, not a dead one.
bool ownerAlive = false;
if(owner != 0)
{
long id = ChartFirst();
while(id >= 0)
{
if(id == owner)
{
ownerAlive = (StringLen(ChartGetString(id, CHART_EXPERT_NAME)) > 0);
break;
}
id = ChartNext(id);
}
}
if(owner != 0 && !ownerAlive)
{
GlobalVariableSet(name, (double)self);
m_configLockName = name;
return true;
}
Print(ID + ": REFUSED to start - another chart is already training this exact configuration. Both" +
" would save into the same files (" + m_activeFileName + ".nnw plus its .cfg/.stats/checkpoints)" +
" and overwrite each other's progress with no error reported anywhere. Owner: " +
(owner != 0 ? "chart " + IntegerToString(owner) + " (" + ChartSymbol(owner) + " " +
EnumToString((ENUM_TIMEFRAMES)ChartPeriod(owner)) + ")" : "another chart, still initializing") +
". Change AIType or any retrain-affecting input on THIS chart so it trains its own model, or" +
" remove one of the two charts. Note AIType defaults to " + EnumToString(HYBRID_2L) +
" - a chart whose AIType was never actually changed lands here.");
return false;
}
//+------------------------------------------------------------------+
//| Drop this instance's claim (see AcquireConfigLock). |
//+------------------------------------------------------------------+
void CExpertSignalAIBase::ReleaseConfigLock(void)
{
if(StringLen(m_configLockName) == 0)
return;
//--- Only delete a claim we still hold: if a later instance took this entry over via the stale-owner
//--- path above, deleting it here would silently hand the config to a third chart.
if((long)GlobalVariableGet(m_configLockName) == ChartID())
GlobalVariableDel(m_configLockName);
m_configLockName = "";
}
//+------------------------------------------------------------------+
//| Common network bootstrap shared by every AI signal: sets up |
//| indicators, then loads a saved network or builds a fresh one |
//| whose only per-signal-type difference is AddCustomLayers(). |
//+------------------------------------------------------------------+
bool CExpertSignalAIBase::InitNeuralNetwork(CIndicators *indicators)
{
if(m_isInitialized)
return true;
if(indicators == NULL)
return false;
//--- Disambiguate the log/panel name for concurrent charts of the same family. MLP_3L and MLP_4L both
//--- call themselves "Perceptron", so running them side by side produces two interleaved streams of
//--- identically-prefixed lines that cannot be separated afterwards - which silently costs you half a
//--- comparison run. The dense layer count is exactly what AIType varies between them, so it is what
//--- the name has to carry. Display only: m_id (the State\<id>\ folder) and the config fingerprint are
//--- deliberately untouched, so this cannot re-key a single model file. Idempotent, because a failed
//--- init leaves m_isInitialized false and this can be reached twice on the same object.
string depthTag = " " + IntegerToString(m_hiddenLayersCount) + "L";
if(StringFind(ID, depthTag) < 0)
ID += depthTag;
m_indicatorsPtr = indicators;
if(!CExpertSignalCustom::InitIndicators(indicators))
return false;
if(!CExpertSignalAIBase::InitIndicators(indicators))
return false;
Net = new CNet(NULL);
if(CheckPointer(Net) == POINTER_INVALID)
return false;
//--- Size the first dense layer to the data. HERE and only here: it must be settled before the
//--- fingerprint below (which hashes it) and must never move afterwards - see ComputeFirstLayerWidth()
//--- and the note on fingerprint-feeding members at the top of this file. InitIndicators() above is
//--- what finalises m_neuronsCount, so this is the earliest point the input width is actually known.
m_initialNeuronsCount = ComputeFirstLayerWidth();
PrintVerbose(ID + ": first dense layer sized to " + IntegerToString(m_initialNeuronsCount) +
" units for a " + IntegerToString((int)m_historyBars * m_neuronsCount) +
"-wide input over " + IntegerToString(m_studyPeriod) + " study years");
//--- Per-configuration fingerprint appended to the weights filename so that every distinct
//--- combination of RETRAIN-AFFECTING inputs gets its OWN persistent .nnw/.cfg, instead of all
//--- combinations sharing one file keyed only on symbol/period/output/optimizer. This is what lets a
//--- genetic/complete optimization that sweeps network params (neuron counts, layers, reduction,
//--- history bars, study period, feature set, focal gamma, OOS split, recall/WR targets, ...) build
//--- each combo's model exactly ONCE and then reuse it on every later pass that revisits that combo -
//--- previously each differing pass overwrote the single shared cache and retrained from scratch, so
//--- there was no cross-combination reuse at all. The topology .cfg check further below still runs as
//--- a secondary guard (and catches a rare hash collision by mismatching and retraining).
//--- Deliberately covers ONLY params that change the trained weights. Inference-only gates
//--- (Min_Vote_Open's confidence floor, SignalClusterWindow) and all post-training/live settings (money
//--- management, SL/TP, trailing, entry, filters) are excluded, so changing those still reuses the
//--- exact same model - matching the pre-existing behavior the optimizer already relied on.
string fp = StringFormat("%d|%d|%d|%d|%.4f|%d|%d|%d|%d|%d|%d|%d|%d|%.2f|%d|%d|%d|%d|%d|%d|%d|%d",
m_initialNeuronsCount, m_hiddenLayersCount, m_lstmHiddenSize, m_convFilterCount, m_neuronsReduction,
m_minNeuronsCount, m_optimizationAlgo, m_historyBars, m_outputNeuronsCount,
m_neuronsCount, m_studyPeriod, m_minTrainYear, LEGACY_CONVERGE_WR_SLOT, m_fractalPeriods,
m_minDirectionalRecallPct, m_focalGamma, m_oosSplitPct, m_swingConfirmationBars,
(int)m_useVolumes, (int)m_useTime, (int)m_useATR, (int)m_useSwingContext,
(int)m_useNews, m_newsFeatureWindowMinutes);
//--- MA/RSI + AD feature flags appended separately to keep each StringFormat call's arg list modest.
//--- m_useMA/m_useRSI belong here for the same reason every other feature flag does: they change the
//--- input-vector width (see InitIndicators()'s m_neuronsCount += 5/+1), so a model trained with them
//--- on must never silently reuse a cache trained with them off. m_neuronsCount alone (listed above)
//--- captured the WIDTH but not the composition, so two different feature sets summing to the same
//--- width could have collided onto one cache file - these two flags close that gap.
fp += StringFormat("|%d|%d|%d|%d|%d|%d|%d|%d",
(int)m_useMA, (int)m_useRSI,
(int)m_useADCumulativeDelta, (int)m_useADShorteningOfThrust,
(int)m_useADWyckoffEventStream, (int)m_useADWyckoffFailedStructure,
(int)m_useADWyckoffSignificantBarInversion,
//--- starting MA TYPE (MA_Type input): changes the MA feature's values, so a change
//--- must invalidate the cache. The auto-tuned type/period themselves live in the
//--- .nnw indicator-param block (Flatten/Unflatten), not here - this is the seed only.
(int)MA_Type);
//--- MACD/Ichimoku feature flags, appended ONLY WHEN ENABLED rather than unconditionally like every
//--- flag above. Both spellings are equally correct as a fingerprint (deterministic either way, and a
//--- model with these on can never collide with one that has them off), but appending them
//--- unconditionally would have changed the hash of EVERY existing config the moment this feature
//--- shipped - re-keying and forcing a full retrain of already-converged models that don't use MACD or
//--- Ichimoku at all. Conditional append leaves those fingerprints byte-identical. The seed periods go
//--- in for the same reason MA_Type does above: they change the feature's values. Anything added here
//--- in future should follow the same rule.
if(m_useMACD)
fp += StringFormat("|MACD:%d:%d:%d", (int)MACD_PeriodFast, (int)MACD_PeriodSlow, (int)MACD_PeriodSignal);
if(m_useIchimoku)
fp += StringFormat("|ICHI:%d:%d:%d", (int)Ichimoku_PeriodTenkan, (int)Ichimoku_PeriodKijun, (int)Ichimoku_PeriodSenkou);
//--- Batch normalization changes the LAYER COUNT, not just the weights, so a model trained with it
//--- must never load into a topology built without it (and vice versa) - the .cfg guard would catch
//--- the mismatch and retrain, but only after a confusing failure. Appended conditionally, following
//--- the same rule as MACD/Ichimoku above: a config with batch norm off keeps the fingerprint it
//--- already had, so shipping this does not re-key and force a retrain of every existing model.
if(EnableBatchNorm && BatchNormWindow > 1)
fp += StringFormat("|BN:%d", BatchNormWindow);
//--- Changes the training gradient, so a model trained with it must never load into a run
//--- without it. Conditional append, same rule as MACD/Ichimoku/BN above: a config with
//--- this OFF keeps the fingerprint it already had, so the already-converged models on
//--- disk stay untouched and remain loadable as the fallback if this regresses.
if(m_useLogitAdjustedLoss && m_logitAdjustTau > 0.0)
fp += StringFormat("|LA:%d", (int)MathRound(m_logitAdjustTau * 100.0));
//--- 2026-07-29 audit of every input in Variables\Inputs.mqh against this hash. Five were changing the
//--- trained weights without changing the filename, so switching any of them re-adopted a model trained
//--- under the OLD value - the exact trap that the .nnw architecture incident already cost a day to
//--- (see EnforceTopologyContract): the .cfg guard would eventually mismatch and retrain, but only
//--- after a confusing failure, and a matching topology would not mismatch at all.
//--- Oversampling regime: m_enableMinorityReplay gates the replay loop AND scales the focal gamma
//--- (Training.mqh), so it always changes the weights. The other two only bite while replay is on, so
//--- they are nested - turning replay off must not re-key a model over a parity value nothing reads.
//--- Parity is stored as an integer percent: it arrives as a percentage preset divided by 100, and a
//--- float in a hash input would make the fingerprint depend on formatting rather than on the value.
fp += StringFormat("|MR:%d", (int)m_enableMinorityReplay);
if(m_enableMinorityReplay)
fp += StringFormat(":%d:%d", (int)MathRound(m_oversampleParity * 100.0), (int)m_constrainReplay);
//--- Feature-value inputs, each conditional on the feature that reads it actually being on - the same
//--- rule the MACD/Ichimoku blocks above follow. Tick vs real volume feeds different numbers into the
//--- same input slot (Features.mqh's m_Volumes.Create), and PeriodMA/PeriodRSI seed the tuner
//--- (ADIndicatorTuner.mqh) exactly as MA_Type does - MA_Type was already hashed, these two were not.
//--- All three also feed the CLASSIC MA/RSI votes, which are inference-only; gating on the AI feature
//--- flag is what keeps a classic-signal tweak from re-keying a model that never saw it.
if(m_useVolumes)
fp += StringFormat("|VOL:%d", (int)VolumeData);
if(m_useMA)
fp += StringFormat("|MAP:%d", (int)PeriodMA);
if(m_useRSI)
fp += StringFormat("|RSIP:%d", (int)PeriodRSI);
//--- FNV-1a 32-bit -> 8 hex chars: compact, deterministic, order-stable, collision-safe enough for
//--- the small optimizer grids in play (a collision would merely fail the .cfg guard and retrain).
uint fpHash = 2166136261;
int fpLen = StringLen(fp);
for(int fpi = 0; fpi < fpLen; fpi++)
{
fpHash ^= (uint)StringGetCharacter(fp, fpi);
fpHash *= 16777619;
}
m_fileName += "_" + DoubleToString(MathRound(m_outputNeuronsCount)) + "_" + DoubleToString(MathRound(m_optimizationAlgo)) + "_" + StringFormat("%08x", fpHash);
//--- Finish the display name with the leading 4 hex digits of that same fingerprint, so every log line
//--- and panel names the model file it belongs to. The dense-depth tag added at the top of this
//--- function separates MLP_3L from MLP_4L, but NOT two charts that differ by anything else - the
//--- batch-norm control is 3L on both sides, which put two identical "Perceptron 3L" streams in the
//--- log the first time this was tried. Any config difference at all changes this hash, by
//--- construction, so it is the only discriminator that cannot go stale as inputs are added. Matches
//--- the filename's first 4 characters, so a log line greps straight to its .nnw.
string cfgTag = " [" + StringSubstr(StringFormat("%08x", fpHash), 0, 4) + "]";
if(StringFind(ID, cfgTag) < 0)
ID += cfgTag;
//--- One self-verifying config line per chart, deliberately NOT gated on VerboseMode. A multi-chart
//--- comparison is only valid if every chart is identical except the axis under test, and until now
//--- a drifted setting was invisible: the filename carries a HASH, so two charts that should match
//--- and do not look merely "different" with no indication of WHICH field moved. Printing the raw
//--- fingerprint string makes the six lines directly diffable - any accidental divergence in study
//--- period, feature set, focal gamma or anything else that feeds training shows up as a textual
//--- difference at startup instead of an unexplained result three hours later.
Print(ID + ": config - " + IntegerToString(m_hiddenLayersCount) + " dense from " +
IntegerToString(m_initialNeuronsCount) + " units | batchnorm " +
((EnableBatchNorm && BatchNormWindow > 1) ? "ON(" + IntegerToString(BatchNormWindow) + ")" : "OFF") +
" | logit-adjust " + ((m_useLogitAdjustedLoss && m_logitAdjustTau > 0.0)
? "ON(tau " + DoubleToString(m_logitAdjustTau, 2) + ")" : "OFF") +
" | replay " + ((m_enableMinorityReplay && !m_useLogitAdjustedLoss) ? "ON" : "OFF") +
" | input " + IntegerToString((int)m_historyBars * m_neuronsCount) +
" (" + IntegerToString((int)m_historyBars) + " bars x " + IntegerToString(m_neuronsCount) + ")");
Print(ID + ": fingerprint - " + fp);
//--- Strategy Tester / optimizer: target a LOCAL (agent-sandboxed, non-FILE_COMMON) cache file
//--- instead of the shared production weights, so genetic/complete optimization passes on this
//--- same agent can reuse an already-trained model whenever the topology-relevant inputs
//--- (neuron counts, layers, history bars, output count, opt algo, study period, ...) are
//--- unchanged from a previous pass, instead of re-running every training era from scratch each
//--- pass. The live/manual-chart production .nnw/.cfg under FILE_COMMON are never touched by
//--- this path, so a backtest can never corrupt or overwrite the deployed live model.
bool inTesterOrOpt = MQLInfoInteger(MQL_TESTER) || MQLInfoInteger(MQL_OPTIMIZATION) || MQLInfoInteger(MQL_FORWARD);
m_activeFileName = inTesterOrOpt ? (m_fileName + "_optcache") : m_fileName;
m_activeFileCommon = !inTesterOrOpt;
//--- Claim these files before anything reads or writes them, and refuse to start if another chart in
//--- this terminal already holds them (see AcquireConfigLock). Deliberately placed here: this is the
//--- first moment the resolved filename - i.e. the config's true identity - is known, and it is still
//--- ahead of every load, seed and save. Tester/optimizer agents are exempt: each is a separate
//--- process writing its own sandboxed _optcache, and running one config across many agents in
//--- parallel is the entire point of an optimization.
if(!inTesterOrOpt && !AcquireConfigLock())
return false;
//--- Any Strategy-Tester run - a single backtest OR an optimization pass - runs pure inference on the
//--- deployed model, never trains. A user optimizing TRADING parameters (SL/TP, filters, MM, ...) wants
//--- the AI held fixed at the deployed weights so passes are fast and comparable; retraining the net
//--- per config would be slow and make every pass a different model. AI hyperparameters are tuned on a
//--- chart (the internal auto-tuner / a real training run), not via MT5 optimization. Training and the
//--- new online continual-learning step therefore run ONLY on a live chart (see OnlineLearnStep).
m_inferenceOnly = MQLInfoInteger(MQL_TESTER);
//--- Seed the agent-local optcache from the deployed production model on the first tester/opt pass.
//--- Without this, the tester's separate _optcache file starts empty and the run retrains from zero -
//--- so a buyer who loads a .set and hits "backtest" waits through a full training run instead of a
//--- backtest of the model they deployed. The optcache shares the production model's exact config
//--- fingerprint (same m_fileName base), so the copied weights are guaranteed topology-compatible.
//--- Copies FROM FILE_COMMON (the live/manual-chart model) INTO the agent-local sandbox only; the
//--- production files are read, never written, so a backtest still can't corrupt the deployed model.
//--- Re-seeds when the cache is MISSING *or* STALE. Staleness matters because a tester run no longer
//--- writes this file at all (see PersistWeightsOnShutdown's inference-only skip), so without a
//--- freshness check the very first seeded copy would be reused forever - meaning the obvious workflow
//--- "retrain/redeploy on the chart, then backtest" would silently keep testing the OLD model. The
//--- config fingerprint in the filename can't catch this: retraining changes the WEIGHTS, not the
//--- topology inputs the fingerprint hashes, so the name stays identical.
bool cacheMissing = !FileIsExist(m_activeFileName + ".nnw");
bool cacheStale = false;
if(inTesterOrOpt && !cacheMissing && FileIsExist(m_fileName + ".nnw", FILE_COMMON))
{
datetime prodModified = (datetime)FileGetInteger(m_fileName + ".nnw", FILE_MODIFY_DATE, true);
datetime cacheModified = (datetime)FileGetInteger(m_activeFileName + ".nnw", FILE_MODIFY_DATE, false);
//--- both timestamps must be readable before trusting the comparison; a 0 means "couldn't tell",
//--- and re-seeding on an unreadable timestamp every single pass would be worse than not checking.
cacheStale = (prodModified > 0 && cacheModified > 0 && prodModified > cacheModified);
if(cacheStale)
Print(__FUNCTION__ + ": the deployed model is newer than this agent's cached copy - re-seeding so the backtest runs the CURRENT model, not the previously cached one.");
}
if(inTesterOrOpt && (cacheMissing || cacheStale))
{
if(FileIsExist(m_fileName + ".nnw", FILE_COMMON))
{
//--- The .nnw is the only copy that MUST succeed - retried (see CopyFileWithRetry's declaration
//--- comment) because a live chart's own atomic Save() can be mid-rename on this exact file.
//--- Its return value used to be ignored entirely, so a failed copy still logged "seeded tester
//--- cache..." as if it had worked, and the run silently trained from scratch instead.
if(CopyFileWithRetry(m_fileName + ".nnw", m_activeFileName + ".nnw"))
{
//--- Best-effort sidecars: not retried - losing one just means a cold calibration/shadow-blend
//--- start rather than a wrong/untrained model, which the .nnw copy above already guards against.
//--- Still share-aware (CopySharedFile, not FileCopy): the live chart holds these open too, so
//--- plain FileCopy would fail on them for exactly the same reason it failed on the .nnw.
if(FileIsExist(m_fileName + ".cfg", FILE_COMMON))
CopySharedFile(m_fileName + ".cfg", m_activeFileName + ".cfg", false);
if(FileIsExist(m_fileName + "_shadow.nnw", FILE_COMMON))
CopySharedFile(m_fileName + "_shadow.nnw", m_activeFileName + "_shadow.nnw", false);
//--- carry the calibration sidecar into the agent sandbox too, so a seeded backtest calibrates its
//--- live decisions with the deployed model's priors instead of the un-adjusted cold defaults.
if(FileIsExist(m_fileName + ".stats", FILE_COMMON))
CopySharedFile(m_fileName + ".stats", m_activeFileName + ".stats", false);
Print(__FUNCTION__ + ": seeded tester cache from the deployed production model (" + m_fileName + ") - this run reuses the deployed weights instead of retraining");
}
//--- else: CopyFileWithRetry already logged why. Fall through - the Net.Load() below will
//--- correctly report "no file" and BuildFreshTopology() takes over, same as a genuine first pass.
}
else if(m_inferenceOnly)
//--- Name the exact file (symbol + timeframe + config fingerprint) it looked for: the model is
//--- keyed on the CHART TIMEFRAME, so the #1 cause of this is running the tester on a different
//--- timeframe than the model was trained on (e.g. an H4 model, tester set to H1) - which reads
//--- as "no model" when one exists under a different timeframe. Spelling out the filename makes
//--- that mismatch obvious instead of looking like the deploy silently failed.
Print(__FUNCTION__ + ": WARNING - no deployed production model found at '" + m_fileName +
".nnw' (shared folder) for " + _Symbol + " " + EnumToString((ENUM_TIMEFRAMES)_Period) +
". A single backtest runs inference only and will NOT train. Most common cause: the tester" +
" timeframe differs from the one the model was trained on (the filename is keyed on timeframe)." +
" Otherwise, train this configuration on a chart first, then re-run the backtest.");
}
if(!LoadAndCompareTopologyConfiguration(m_activeFileName, m_initialNeuronsCount, m_hiddenLayersCount, m_neuronsReduction, m_minNeuronsCount, m_optimizationAlgo, m_historyBars, m_outputNeuronsCount, m_neuronsCount, m_studyPeriod, m_minTrainYear, m_isInitialized, LEGACY_CONVERGE_WR_SLOT, m_fractalPeriods, m_activeFileCommon))
{
// Topology/input params diverged from what produced the saved .nnw (or no .cfg exists yet;
// for inTesterOrOpt this is also the normal "first pass on this agent" case). The stale .cfg
// was already deleted on a mismatch, but the .nnw weights themselves are shaped for the OLD
// topology - loading them into a network built to the NEW shape would corrupt state or crash.
// Drop the incompatible weights/checkpoint too so the Net.Load() below cleanly misses and
// BuildFreshTopology() takes over (i.e. this pass pays the training cost once, and the result
// gets cached below for the NEXT pass to reuse, same as a live topology change would).
if(FileIsExist(m_activeFileName + ".nnw", m_activeFileCommon ? FILE_COMMON : 0))
{
Print(__FUNCTION__ + ": " + m_activeFileName + " - topology/input params changed since last save; discarding incompatible saved weights and starting fresh");
FileDelete(m_activeFileName + ".nnw", m_activeFileCommon ? FILE_COMMON : 0);
//--- Reaching here means a TRAINED model was just thrown away, so its drawn signals are stale for
//--- exactly the same reason ResetWeights() clears them: they would otherwise be restored moments
//--- later (LoadChartSignals runs at the end of this function) and shown as if they belonged to the
//--- model about to be trained. The sibling "no .cfg yet" case does not reach here (there are no
//--- weights to delete), so it is handled separately at the fresh-topology branch below.
ClearPersistedChartSignals("saved weights discarded - topology/input params changed");
}
if(FileIsExist(m_activeFileName + "_ckpt.tmp", m_activeFileCommon ? FILE_COMMON : 0))
FileDelete(m_activeFileName + "_ckpt.tmp", m_activeFileCommon ? FILE_COMMON : 0);
// Same reasoning applies to the EMA shadow-weight file (see m_shadowNet's declaration comment) -
// it's shaped for the OLD topology too, and EnsureShadowNet() has no independent way to detect
// that mismatch on Load() (CNet::Load() doesn't cross-validate against an expected shape). Drop
// it so EnsureShadowNet() cleanly misses and re-bootstraps from the fresh Net instead.
if(FileIsExist(m_activeFileName + "_shadow.nnw", m_activeFileCommon ? FILE_COMMON : 0))
FileDelete(m_activeFileName + "_shadow.nnw", m_activeFileCommon ? FILE_COMMON : 0);
//--- the calibration sidecar is tied to the discarded weights - drop it too so a fresh run
//--- re-measures priors from scratch instead of adjusting with a stale model's base rates.
if(FileIsExist(m_activeFileName + ".stats", m_activeFileCommon ? FILE_COMMON : 0))
FileDelete(m_activeFileName + ".stats", m_activeFileCommon ? FILE_COMMON : 0);
SaveTopologyConfiguration(m_activeFileName, m_initialNeuronsCount, m_hiddenLayersCount, m_neuronsReduction, m_minNeuronsCount, m_optimizationAlgo, m_historyBars, m_outputNeuronsCount, m_neuronsCount, m_studyPeriod, m_minTrainYear, m_isInitialized, LEGACY_CONVERGE_WR_SLOT, m_fractalPeriods, m_activeFileCommon);
}
double loadedIndicatorParams[];
//--- Inference-only backtest: if this deployed model was validated MQL5-inference-safe at deploy
//--- (marker in its .stats), load it host-only and run the pure-MQL5 forward path so the backtest
//--- never loads WarriorDML/WarriorCPU.dll - no DLL file-lock class of failure, and the exact math
//--- the Market build ships. Falls back to a compute backend just below if that load fails.
if(m_inferenceOnly && CheckPointer(Net) != POINTER_INVALID)
{
LoadModelStats(m_activeFileName, m_activeFileCommon); // reads m_mqlInferenceValidated (and priors)
if(m_mqlInferenceValidated)
{
Net.SetCpuInference(true);
PrintVerbose(__FUNCTION__ + ": " + ID + " - inference-only backtest running pure-MQL5 (DLL-free): the deployed model is validated MQL5-inference-safe");
}
}
bool netLoaded = LoadNetWithRetry(loadedIndicatorParams);
//--- Pure-MQL5 load failed unexpectedly (should not happen for a validated model) - drop back to a
//--- compute backend and retry once so the backtest still runs via the DLL rather than on a fresh net.
if(!netLoaded && CheckPointer(Net) != POINTER_INVALID && Net.CpuInference())
{
Print(__FUNCTION__ + ": " + ID + " - pure-MQL5 load failed; retrying with a compute backend (DLL)");
Net.SetCpuInference(false);
netLoaded = LoadNetWithRetry(loadedIndicatorParams);
}
//--- the file may carry a superseded architecture - correct it before anything reads the net
if(netLoaded)
EnforceTopologyContract();
//--- restore the calibration sidecar (priors + confidence scale) that pairs with these weights, so a
//--- restart - including a buyer's inference-only backtest - calibrates live decisions exactly as the
//--- saved model did instead of running with cold defaults (priors 0 => no adjustment). See LoadModelStats().
if(netLoaded)
LoadModelStats(m_activeFileName, m_activeFileCommon);
m_modelLoadedFromDisk = netLoaded;
//--- Make a successful resume visible (the counterpart to the fresh-start / mismatch messages below):
//--- on a live chart this confirms the saved model was found and loaded rather than silently retrained.
if(netLoaded && !inTesterOrOpt)
Print(ID + ": resumed saved model from era " + IntegerToString(m_eraCount) + " (trainingComplete=" + (string)m_trainingComplete + ") - continuing, not retraining from era 0.");
//--- a restart that recovers existing weights already has a proven-synced history and (since it
//--- already has at least one trained era behind it) doesn't have era 0's cold-start oversampling
//--- problem either - only a genuinely fresh start needs the 3 warm-up passes (see Train()'s
//--- m_warmupPassesRemaining gate).
//--- The label cache itself, however, is NEVER restored from the .nnw checkpoint - it lives only in
//--- the in-memory m_labelCacheBuy/Sell/HasValue arrays, which start empty every process start
//--- regardless of netLoaded. Previously this was set to netLoaded, which on a successful checkpoint
//--- load skipped the eager StartLabelCachePrebuild()/AdvanceLabelCachePrebuild() scan (Train()'s
//--- !m_labelCachePrebuilt gate) - every bar then fell through to the lazy per-bar fallback in the
//--- era loop, which calls ComputeLabelForBar(), a dead stub that unconditionally returns
//--- buy=false/sell=false (the real ZigZag-based labeling logic lives ONLY in
//--- AdvanceZigZagLabelState(), reachable exclusively from the eager prebuild). The result: every
//--- restart that loaded a checkpoint silently force-labeled the entire era Neutral until something
//--- else (a topology mismatch, a fresh start) triggered a real prebuild. Always eager-prebuilding
//--- now, checkpoint or not, closes this at the root - the dead stub fallback then never matters.
m_warmupPassesRemaining = netLoaded ? 0 : 3;
m_labelCachePrebuilt = false;
if(inTesterOrOpt && netLoaded)
Print(__FUNCTION__ + ": " + ID + " - reused cached weights from a previous optimization/tester pass on this agent (era " + IntegerToString(m_eraCount) + ", trainingComplete=" + (string)m_trainingComplete + ") - skipping redundant training for this unchanged config");
if(netLoaded && ArraySize(loadedIndicatorParams) == AD_TUNE_PARAM_COUNT)
{
// Restart deploying previously AutoTune-d indicator params even with AutoTuneIndicators=false now -
// indicators above were already created with today's defaults, so rebuild them once with the
// restored values before any training/signal work happens.
m_indicatorTuner.Unflatten(loadedIndicatorParams);
ReInitADIndicators(indicators);
}
if(!netLoaded)
{
int error_code = GetLastError();
//--- Do NOT present error_code as the cause: on a no-GPU/CPU-DLL box it is the harmless 5100
//--- (OpenCL-not-found) left by the compute probe inside CNet::Load, NOT the reason the file was
//--- rejected. CNet::Load now prints the precise reason (bad marker / type mismatch / 0-layer stub /
//--- partial layer load) itself. Only clear the stale code here; the "rebuilding fresh" line below is
//--- the user-facing summary.
if(error_code != 5004) // not "file not found"
ResetLastError();
//--- CRITICAL: a failed load may have ALREADY overwritten the training-state out-params from the
//--- bad file's header before it was rejected - notably a corrupt/empty 0-layer stub whose header
//--- still says trainingComplete=1 (see CNet::Load's 0-layer guard). Left as-is, the freshly built,
//--- untrained topology below would be treated as an already-deployed converged model: it would
//--- never train, run inference on random weights (every bar scores Neutral, so the end-of-era NMS
//--- sweep deletes every chart arrow), and "save weights" would just re-persist that empty net.
//--- Force the state back to a genuine fresh start so BuildFreshTopology() actually gets trained.
m_trainingComplete = false;
m_eraCount = 0;
dtStudied = 0;
dForecast = 0;
//--- Cold the in-memory calibration so the freshly-rebuilt (untrained) topology below runs with no
//--- stale prior-correction until a retrain re-measures it (priors 0 => AdjustedSignalFromSoftmax is a
//--- no-op; scale 1.0 = the constructor default). In-memory ONLY - deliberately does NOT touch any
//--- file. (Earlier this session I also deleted the .stats/_shadow.nnw sidecars here; that was too
//--- destructive - a load failure can be transient/spurious (a not-yet-ready compute backend, a
//--- momentary file lock, or - on this CPU-DLL machine - GetLastError() being polluted with the
//--- harmless OpenCL-not-found 5100 from the probe inside CNet::Load), and wiping a user's calibration
//--- and deployed shadow on any such hiccup is the wrong default. The sidecars self-heal anyway: the
//--- shadow re-blends toward the retrained Net and .stats is overwritten on the next save.)
m_priorBuy = 0.0;
m_priorSell = 0.0;
m_priorNeutral = 0.0;
m_confidenceCalScale = 1.0;
//--- Accurate diagnostic (do NOT cite GetLastError() - inside CNet::Load the OpenCL probe leaves 5100
//--- there on a no-GPU/CPU-DLL box, which has nothing to do with the file). Distinguish an ordinary
//--- fresh start (no file yet) from a real read failure of an existing file by testing existence.
if(!inTesterOrOpt)
{
int loadFlags = m_activeFileCommon ? FILE_COMMON : 0;
if(FileIsExist(m_activeFileName + ".nnw", loadFlags))
Print(ID + ": could not read the existing model file " + m_activeFileName + ".nnw - rebuilding a fresh topology to retrain from era 0. Existing .stats/_shadow.nnw are KEPT (they refresh as training runs). If this recurs, that .nnw is likely corrupt - back it up, then use the panel's reset-weights to start clean.");
else
Print(ID + ": no saved model for this config yet - starting a fresh training run from era 0.");
}
//--- Re-seed before building a fresh topology so weight init is genuinely random. A prior
//--- genetic tuner sweep (TuneIndicatorsAndTrain's candidate eval loop, line ~5006) left the
//--- MQL5 RNG at GA_SEED_BASE + m_gaSeedIdx; if this load-fail path then builds a production
//--- topology without re-seeding, the deployed model's weights would be deterministic/repeatable
//--- from whatever the last candidate's seed was — silently reproducible, not genuinely random.
//--- Matches ResetWeights() and Warrior_EA.mq5's OnInit.
MathSrand(GetTickCount());
//--- Era 0 with no weights behind it, so any arrow currently on this chart was drawn by a
//--- DIFFERENT model - the previous fingerprint's, or a corrupt .nnw's. Neither the panel reset
//--- nor the topology-mismatch discard above covers this path: both are gated on there being a
//--- saved .nnw to delete, and here there is none (a changed config produces a new m_fileName,
//--- so the old model's files are not "discarded", they are simply not this model's files).
//--- Left alone the stale arrows do NOT just look wrong - the chart objects survive deploys and
//--- restarts on their own, and SaveChartSignals() rebuilds the sidecar by scanning the chart,
//--- so the first save of this fresh run would adopt the dead model's calls as its own history.
//--- Deliberately at this call site rather than inside BuildFreshTopology(): the genetic tuner
//--- calls that for every throwaway candidate (AutoTune.mqh) and must not touch the chart.
ClearPersistedChartSignals("fresh topology at era 0 - arrows belong to a previous model");
if(!BuildFreshTopology())
return false;
}
TempData = new CArrayDouble();
if(CheckPointer(TempData) == POINTER_INVALID)
return false;
if(netLoaded)
// Populate dPrevSignal from the just-loaded weights immediately, rather than leaving it at
// its blank constructor default until the next (asynchronous, queued) training pass happens
// to run - matters most for the tester cache-reuse path above, where training may be skipped
// entirely for this run because dtStudied already covers the whole backtest window.
RefreshLatestSignal();
//--- Status line must match what the gate below (if(!m_trainingComplete && !m_inferenceOnly)) will
//--- actually do - otherwise an inference-only single backtest logs "resuming full training now" right
//--- under the "runs inference only and will NOT train" warning, which reads as a contradiction.
string trainState = m_trainingComplete
? "already complete - staying converged, no full retrain on this restart"
: (m_inferenceOnly
? "NOT complete, but this is an inference-only backtest - NOT training (see warning above); deploy a trained model for meaningful results"
: "NOT complete (interrupted or never converged) - resuming full training now");
Print(__FUNCTION__ + ": " + m_activeFileName + " - training " + trainState);
//--- Only kick off a full Train() run here if the loaded model genuinely isn't converged yet - an
//--- already-complete model used to get one full era-loop retrain (real Net.backProp() over the
//--- whole IS window) on every single EA restart/reattach for no reason, since this "Init" event
//--- bypassed ScheduleTrainingIfNeeded()'s m_trainingComplete gate entirely. dPrevSignal is already
//--- fresh from RefreshLatestSignal() above; ScheduleTrainingIfNeeded()'s normal per-tick check
//--- will call RefreshConvergedSignal() itself once a genuinely new bar closes.
if(!m_trainingComplete && !m_inferenceOnly)
bEventStudy = EventChartCustom(ChartID(), 1, (long)MathMax(0, MathMin(iTime(_Symbol, PERIOD_CURRENT, (int)(100 * Net.recentAverageSmoothingFactor * (m_trainingComplete ? 1 : 10))), dtStudied)), 0, "Init");
//--- Restore arrows persisted from a previous session (see SaveChartSignals). MUST run here, not in
//--- InitIndicators(): the arrows file is keyed on the FULL m_fileName including the per-config
//--- fingerprint, which is only appended above - see the note left at InitIndicators()'s old call site.
LoadChartSignals();
//--- bootstrap (or restore) the EMA shadow net now rather than waiting for the first
//--- RefreshLatestSignal()/era-blend call to lazily trigger it - see m_shadowNet's declaration
//--- comment.
EnsureShadowNet();
m_isInitialized = true;
return true;
}
//+------------------------------------------------------------------+
//| Capacity budget for the first dense layer - see the declaration. |
//+------------------------------------------------------------------+
int CExpertSignalAIBase::ComputeFirstLayerWidth(void) const
{
int inputWidth = (int)m_historyBars * m_neuronsCount;
if(inputWidth <= 0)
return FIRST_LAYER_MIN_WIDTH;
//--- Expected in-sample bars. Deliberately estimated from the configured study period and the chart
//--- timeframe, NOT from Bars() - what is downloaded right now grows over a terminal's lifetime, and
//--- a topology that silently widens as history fills in would re-key its own weights file and throw
//--- away a trained model. MARKET_OPEN_FRACTION is a rough allowance for closed hours/weekends; it
//--- does not need to be accurate, because the ladder below quantizes the result heavily.
int secs = PeriodSeconds(m_period);
if(secs <= 0)
secs = PeriodSeconds(PERIOD_H1);
double barsPerYear = (SECONDS_PER_YEAR / (double)secs) * MARKET_OPEN_FRACTION;
double isBars = (double)m_studyPeriod * barsPerYear * (100.0 - (double)m_oosSplitPct) / 100.0;
//--- One first-layer weight per in-sample bar. That layer is (inputWidth+1) x width and dominates the
//--- model, so this is effectively a whole-model capacity budget. One parameter per sample is already
//--- generous for a signal this weak; it is a ceiling, not a target.
int budget = (int)(isBars / (double)(inputWidth + 1));
//--- Snap DOWN to the ladder: the estimate above is approximate, and a value that moves with every
//--- small change would re-key the weights file for no benefit. Rungs are far enough apart that the
//--- estimate would have to be wrong by ~2x to land on a different one.
int ladder[] = {16, 32, 64, 128, 256, 512, 1024};
int chosen = FIRST_LAYER_MIN_WIDTH;
for(int i = 0; i < ArraySize(ladder); i++)
if(ladder[i] <= budget)
chosen = ladder[i];
//--- Budget below the floor means this configuration cannot support even the narrowest usable layer -
//--- the model will be over-parameterized no matter what is chosen here, and no amount of
//--- regularization fixes having more weights than examples. Typical cause is a high timeframe
//--- (D1 over 10 years is under 2,000 bars) or too many features for the history available. Say so:
//--- the fix is fewer HistoryBars / fewer feature groups / a longer study period, none of which this
//--- function can choose on the user's behalf.
if(budget < FIRST_LAYER_MIN_WIDTH)
Print(ID + ": WARNING - " + IntegerToString((int)isBars) + " estimated in-sample bars cannot support a " +
IntegerToString(inputWidth) + "-wide input. The first layer is being floored at " +
IntegerToString(FIRST_LAYER_MIN_WIDTH) + " units, which is still roughly " +
DoubleToString((double)(inputWidth + 1) * FIRST_LAYER_MIN_WIDTH / MathMax(1.0, isBars), 1) +
" weights per training bar - expect overfitting. Reduce HistoryBars or the feature set," +
" lengthen the study period, or train on a lower timeframe.");
return MathMax(FIRST_LAYER_MIN_WIDTH, chosen);
}
//+------------------------------------------------------------------+
//| Batch-normalization layer - see the declaration comment. |
//+------------------------------------------------------------------+
bool CExpertSignalAIBase::AddBatchNormStage(CArrayObj *topology, int units)
{
if(CheckPointer(topology) == POINTER_INVALID)
return false;
//--- Not an error: the input is off, so the topology simply has no normalization layers. Returning
//--- true keeps every call site a plain `if(!Add...) return false;` with no extra branching.
if(!EnableBatchNorm)
return true;
//--- A window of 1 makes the layer a no-op passthrough (mean==x, variance==0), which is a silently
//--- useless layer rather than an obviously absent one. Refuse to build it instead.
if(BatchNormWindow <= 1)
return true;
CLayerDescription *desc = new CLayerDescription();
if(CheckPointer(desc) == POINTER_INVALID)
return false;
desc.count = units;
desc.type = defNeuronBatchNorm;
desc.batch = BatchNormWindow;
//--- Identity forward transform. The non-linearity belongs to the dense layer stacked on top of this
//--- one; normalizing and then squashing in the same step would undo the normalization.
desc.activation = NONE;
desc.optimization = (ENUM_OPTIMIZATION)m_optimizationAlgo;
if(!topology.Add(desc))
{
delete desc;
return false;
}
return true;
}
//+------------------------------------------------------------------+
//| Convolution front-end (no pooling). Shared by CSignalCONV and |
//| and CSignalHYBRID - see the declaration comment. |
//+------------------------------------------------------------------+
bool CExpertSignalAIBase::AddConvStage(CArrayObj *topology)
{
if(CheckPointer(topology) == POINTER_INVALID)
return false;
//--- Convolutional Layer
CLayerDescription *desc = new CLayerDescription();
if(CheckPointer(desc) == POINTER_INVALID)
return false;
//--- desc.count here is the conv layer's own output-filter count (CNeuronConvOCL::Init's window_out
//--- param, AI\Network.mqh) - was m_hiddenLayersCount (an unrelated dense-taper-depth setting,
//--- defaulting to 4), bottlenecking every sliding position to just 4 filters regardless of how wide
//--- the rest of the network was. See ConvFilterCount's declaration comment (Variables\Inputs.mqh).
desc.count = m_convFilterCount;
desc.type = defNeuronConv;
// PRELU, not TANH: matches what CNeuronConv's CPU path (Network.mqh) has always hardcoded
// regardless of this setting (its activationFunction() override ignores `activation` entirely) -
// this used to silently diverge from the GPU/DirectML tier, which DOES honor this field and was
// therefore actually running tanh instead of the intended PReLU whenever hardware accel was active.
desc.activation = PRELU;
desc.optimization = (ENUM_OPTIMIZATION)m_optimizationAlgo;
desc.window = m_neuronsCount;
desc.step = m_neuronsCount;
if(!topology.Add(desc))
{
delete desc;
return false;
}
//--- NO POOLING LAYER. Removed 2026-07-29 - it was MISCONFIGURED against the conv output's memory
//--- layout, and the correct configuration is not worth having at this filter count.
//--- FeedForwardConv (AI\Network.cl) emits POSITION-MAJOR output - matrix_o[out + window_out * i] - so
//--- one bar's window_out filter responses are CONTIGUOUS and consecutive bars sit window_out apart.
//--- Both pooling implementations (FeedForwardProof, CPU_FeedForwardProof) slide FLAT over that buffer:
//--- pos = i * step, reducing `window` CONSECUTIVE elements. Those neighbours are therefore FILTERS of
//--- one bar, never one filter across time.
//--- The NeuroNet_DNG reference (references\MQL5\Experts\, kernels byte-identical to ours) handles that
//--- by tying the pool to the filter count: window = step = window_out, giving a clean non-overlapping
//--- max-over-channels that emits exactly one value per bar. Deliberate and well-formed.
//--- This code instead shipped window=3 / step=2 against 16 filters - untied to window_out - so windows
//--- OVERLAPPED across the filter axis and every 8th straddled a bar boundary (bar0_f14, bar0_f15,
//--- bar1_f0). It collapsed unrelated feature detectors into whichever fired hardest, passed gradient to
//--- that winner alone, and halved the feature map - all BELOW every learnable layer, where nothing
//--- above can recover it. Measured: CONV pinned at ~40% balanced accuracy for 510 eras with Sell recall
//--- 0% while plain MLPs reached 57-61%; HYBRID, carrying the same stage, was second-worst of the
//--- batch-norm group.
//--- Why not simply adopt the reference contract: max-over-channels at OUR 16 filters reduces 320 conv
//--- outputs to 20 - one scalar per bar for a 420-wide input - and the first dense layer would then FAN
//--- OUT 20 -> 64 instead of funnelling. The reference could afford that at window_out=4; here it is a
//--- bottleneck, so the whole feature map goes to the dense stack instead.
//--- TIME-axis pooling (what the removed inputs' "N Bars" labels described) is not expressible either
//--- way: it needs a stride of window_out BETWEEN samples inside one window, which a consecutive-window
//--- kernel cannot do at any window/step. That would need a stride-aware kernel in Network.cl +
//--- WarriorCPU.cpp + WarriorDML.cpp and a DLL rebuild.
return true;
}
//+------------------------------------------------------------------+
//| LSTM sequence stage. Shared by CSignalLSTM and CSignalHYBRID - |
//| see the declaration comment. |
//+------------------------------------------------------------------+
bool CExpertSignalAIBase::AddLstmStage(CArrayObj *topology)
{
if(CheckPointer(topology) == POINTER_INVALID)
return false;
CLayerDescription *desc = new CLayerDescription();
if(CheckPointer(desc) == POINTER_INVALID)
return false;
desc.count = m_lstmHiddenSize;
desc.type = defNeuronLSTM;
desc.activation = TANH;
//--- CNeuronLSTMOCL now has an accelerated SGD+momentum kernel (LSTM_UpdateWeightsMomentum,
//--- AI\Network.mqh/Network.cl/DirectML\WarriorCPU.cpp/WarriorDML.cpp) alongside the original
//--- Adam one, so this layer honors the same TrainingOptimizer input as PAI/CONV - see
//--- m_optimizationAlgo's declaration comment.
desc.optimization = (ENUM_OPTIMIZATION)m_optimizationAlgo;
desc.window = (int)m_historyBars * m_neuronsCount;
//--- MathMax(1,...) guard taken from the HYBRID copy: the CSignalLSTM copy divided unguarded, so a
//--- historyBars of 1 produced step 0 there and step 1 here for what is meant to be the same layer.
desc.step = MathMax(1, (int)m_historyBars / 2);
if(!topology.Add(desc))
{
delete desc;
return false;
}
return true;
}
//+------------------------------------------------------------------+
//| Builds a fresh, untrained topology into Net - the exact layer |
//| construction InitNeuralNetwork() used to inline for the |
//| "no saved .nnw" case; factored out so TuneIndicatorsAndTrain() can|
//| get a clean-slate Net per trial without touching indicator init. |
//+------------------------------------------------------------------+
bool CExpertSignalAIBase::BuildFreshTopology()
{
CArrayObj *Topology = new CArrayObj();
if(CheckPointer(Topology) == POINTER_INVALID)
return false;
//--- Input Layer
CLayerDescription *desc = new CLayerDescription();
if(CheckPointer(desc) == POINTER_INVALID)
{
delete Topology;
return false;
}
desc.count = m_historyBars * m_neuronsCount;
desc.type = defNeuron;
desc.activation = NONE;
desc.optimization = (ENUM_OPTIMIZATION)m_optimizationAlgo;
if(!Topology.Add(desc))
{
delete Topology;
return false;
}
//--- neuron-type-specific layers (Conv+Pool, LSTM, or none for a plain perceptron)
if(!AddCustomLayers(Topology))
{
delete Topology;
return false;
}
//--- Hidden Layers, tapering from m_initialNeuronsCount down to m_minNeuronsCount, each preceded by
//--- a batch-normalization layer (no-op when EnableBatchNorm is off). Placed BETWEEN layers rather
//--- than inside them because every layer here computes activation(W.x+b) in a single kernel - there
//--- is no seam between the matmul and the non-linearity to insert anything into. Normalizing the
//--- previous layer's OUTPUT is the equivalent formulation and is exactly what the NeuroNet_DNG
//--- reference's own worked example does (input -> BatchNorm -> hidden -> output).
//--- The first one also normalizes whatever the conv/pool/LSTM stage produced, which is the widest
//--- unbounded stage in the whole network and the one whose scale drift hurts most.
//--- GEOMETRIC taper from the derived first-layer width down to a final hidden width tied to the
//--- output count, spread evenly over however many layers the architecture asks for. This replaces a
//--- pair of inputs (NeuronsReduction, MinNeuronsCount) that were calibrated when the first layer was
//--- a hand-picked 500: they produced a genuine 500 -> 150 -> 45 funnel there, but against the derived
//--- width they degenerate. At 64 units, "keep 30% with a floor of 20" gives 64 -> 20 -> 20 - the
//--- reduction stops mattering after one step and the "minimum" silently becomes the width of every
//--- layer but the first. Deriving the ratio from the endpoints keeps the funnel shape correct at any
//--- width, which is the whole point of having derived the width in the first place.
int lastHidden = MathMax(HIDDEN_TAPER_OUTPUT_MULTIPLE * m_outputNeuronsCount, HIDDEN_TAPER_MIN_WIDTH);
//--- Never wider than where the taper starts: a narrow first layer (see the D1 case in
//--- ComputeFirstLayerWidth) must still funnel DOWN, not fan back out.
lastHidden = MathMin(lastHidden, m_initialNeuronsCount);
double taperRatio = (m_hiddenLayersCount > 1)
? MathPow((double)lastHidden / (double)m_initialNeuronsCount, 1.0 / (double)(m_hiddenLayersCount - 1))
: 1.0;
//--- Width of the layer immediately below the next batch-norm layer. Only advisory (CNet sizes each
//--- batch-norm layer from whatever it actually sits on), but kept honest so the descriptor list
//--- reads correctly. Seeded with the input width - which is also a lie for CONV/LSTM/HYBRID, where
//--- the custom stage in between has resized things; that is exactly why CNet does not trust it.
int prevWidth = (int)(m_historyBars * m_neuronsCount);
bool result = true;
for(int i = 0; (i < m_hiddenLayersCount && result); i++)
{
int n = (i == 0)
? m_initialNeuronsCount
: MathMax(lastHidden, (int)MathRound(m_initialNeuronsCount * MathPow(taperRatio, (double)i)));
result = (AddBatchNormStage(Topology, prevWidth) && result);
if(!result)
break;
prevWidth = n;
desc = new CLayerDescription();
if(CheckPointer(desc) == POINTER_INVALID)
{
delete Topology;
return false;
}
desc.count = n;
desc.type = defNeuron;
desc.activation = HiddenLayerActivation();
desc.optimization = (ENUM_OPTIMIZATION)m_optimizationAlgo;
result = (Topology.Add(desc) && result);
}
if(!result)
{
delete Topology;
return false;
}
//--- Batch norm immediately before the head. This is the one placement that matters most: it is what
//--- keeps the logit spread from decaying as the weights below it shrink, and it is the precondition
//--- for ever running an UNBOUNDED head here (see the 2026-07-28 note on desc.activation below).
if(!AddBatchNormStage(Topology, prevWidth))
{
delete Topology;
return false;
}
//--- Output Layer
desc = new CLayerDescription();
if(CheckPointer(desc) == POINTER_INVALID)
{
delete Topology;
return false;
}
desc.count = m_outputNeuronsCount;
desc.type = defNeuron;
// The activation itself comes from OutputLayerActivation() - see its declaration comment. It is NOT
// written as a literal here any more: this line only ever reached a brand-new topology, so when the
// head was changed here the change never touched a single existing .nnw (2026-07-29 incident).
// EnforceTopologyContract() now re-asserts the same accessor after every Load().
// Regression (1 output): TANH - its native [-1,1] range maps directly onto the -1/0/1
// Sell/Neutral/Buy target convention, without SIGMOID's [0,1] offset or ReLU's clipped
// negative half.
// Classification (3 outputs): SIGMOID, not NONE. The FORWARD activation is still SIGMOID (bounded,
// 0..1 per neuron) - NONE (raw, unbounded identity) used to sit here on the theory that softmax
// "needs" unbounded logits to work against, but with nothing to saturate it, class-balance
// oversampling replaying the same rare-class bar up to 5x in a row (see Train()'s reps loop) let
// Adam's momentum walk one neuron's raw output arbitrarily far in one direction - observed in
// practice as the reported IS error growing to ~2.5e29 within two eras and training permanently
// collapsing onto whichever class's logit happened to run away first. SIGMOID reuses the exact
// same bounded, clamped weight-update math (native MQL5, OpenCL, WarriorCPU.dll, WarriorDML.dll)
// the regression path already relies on, just applied per-neuron.
// The BACKWARD gradient, however, is NOT 3 independent per-neuron sigmoid deltas anymore -
// CNet::backProp()/backPropOCL() (AI\Network.mqh) detect the 3-output classification case and
// compute a true softmax+categorical-cross-entropy gradient (softmax_i - target_i) across all 3
// neurons jointly before the per-sample class-balance/focal-loss weighting is applied. This is
// what ties Buy/Sell/Neutral together during training - raising one class's probability now
// structurally lowers the other two via the shared softmax normalizer - closing the gap between
// "3 parallel binary regressions" (the old behavior, prone to jointly drifting toward all-Neutral)
// and a real mutually-exclusive multi-class objective, while keeping the bounded SIGMOID forward
// pass that fixed the logit-runaway collapse above. ApplyClassificationSoftmax() at read time now
// reproduces exactly the normalization the loss was trained against (previously it was purely a
// post-hoc display transform, decoupled from the actual training signal).
// 2026-07-27: changed SIGMOID -> NONE (identity/logit passthrough) to cure a fp32-OpenCL "neutral
// collapse from era 0", on the theory that softmax competition structurally prevents logit runaway.
// 2026-07-28: REVERTED to SIGMOID. That theory does not hold in this codebase, because three
// constants downstream are calibrated for a BOUNDED head and none of them moved with it:
// 1. CLASS_LOGIT_SCALE = 6.0 (AI\Network.mqh) is a temperature gain that exists to stretch a
// [0,1] sigmoid into a usable logit span. Applied to an unbounded head it is a 6x amplifier
// on an already-free logit, and every softmax in the system uses it - the training gradient
// (backProp/backPropOCL) and ApplyClassificationSoftmax() alike.
// 2. BIAS_MAGNITUDE = 3.0 in AdvanceLabelCachePrebuild()'s cold-start seed, whose own comment
// reads "sigmoid(+-3) ~= 0.95/0.05". With NONE the net STARTS at logits +3/-3/-3, so the
// softmax sees exp(6*3) vs exp(6*-3) - a 36-nat gap, bit-exactly (1,0,0) in fp32 before a
// single weight update.
// 3. HiddenLayerActivation() = PRELU is unbounded. The SIGMOID head was the ONLY bounded stage
// in the whole forward path. MAX_WEIGHT clamps weights but not activations: at fan-in 500
// with weights pinned at +-100 each layer multiplies magnitude by up to 5e4.
// Measured result of the NONE build (opencl.log, 2026-07-28 13:23, SP500 H1): IS error 5.6e15,
// "OOS raw out B:5547071595610112.000..5547071595610112.000" - note min == max, i.e. the outputs
// had become completely INPUT-INDEPENDENT. That is the signature: a saturated softmax makes the
// gradient (target - smax) a constant with no input correlation, so weights drift until they pin
// at +-MAX_WEIGHT, which saturates harder - a closed positive-feedback loop armed at era 0 by the
// bias seed. All four architectures degenerated to one- or two-class output.
// The fp32 saturation this replaced is a real but SEPARATE problem and is already addressed by
// MIN_ACTIVATION_DERIVATIVE (raised 1e-4 -> 1e-3 in AI\Network.cl and AI\Network.mqh, same day).
// If it resurfaces, raise that floor or seed smaller output biases - do NOT unbound the head
// again without simultaneously setting CLASS_LOGIT_SCALE to 1.0 and BIAS_MAGNITUDE to ~0.5.
desc.activation = OutputLayerActivation();
desc.optimization = (ENUM_OPTIMIZATION)m_optimizationAlgo;
if(!Topology.Add(desc))
{
delete Topology;
return false;
}
if(CheckPointer(Net) != POINTER_INVALID)
delete Net;
Net = new CNet(Topology);
delete Topology;
if(CheckPointer(Net) == POINTER_INVALID)
return false;
// A fresh topology invalidates any existing shadow (see m_shadowNet's declaration comment) -
// its weights, if any, are shaped for the OLD Net and would either mismatch dimensionally or,
// worse, silently blend unrelated weight spaces if the shape happens to coincide. Reset to NULL
// here; EnsureShadowNet() lazily re-bootstraps a fresh clone of the new Net on first use.
if(CheckPointer(m_shadowNet) != POINTER_INVALID)
{
delete m_shadowNet;
m_shadowNet = NULL;
}
//--- Let EnsureShadowNet() re-attempt the clone bootstrap once for this new topology (see the latch's
//--- declaration comment) - the old shadow, and any prior failed-bootstrap verdict, no longer apply.
m_shadowBootstrapAttempted = false;
//--- A brand-new untrained net has NO online continual-learning history (see OnlineLearnStep): reset
//--- the watermark/guardrail/counters so a fresh start or a ResetWeights()-then-retrain never resumes
//--- from a superseded model's learned-up-to point or its stale rolling accuracy. Restored (not reset)
//--- on a normal reload of an existing model - that path loads them via LoadModelStats() and never
//--- calls BuildFreshTopology(). Harmless during m_evalMode candidate rebuilds (online learning is
//--- gated off there, and the deployed final retrain rebuilds and resets again before deployment).
m_onlineLearnedUpToTime = 0;
m_onlineRollingAcc = -1.0;
m_onlineSamples = 0;
m_onlineBarsSincePersist = 0;
m_onlineBlendFrozen = false;
return true;
}
//+------------------------------------------------------------------+
//| |
//+------------------------------------------------------------------+
bool CExpertSignalAIBase::InitIndicators(CIndicators *indicators)
{
//--- Reset only the status label on (re-)init; deliberately do NOT PurgeChart() here so previously drawn
//--- signal arrows survive an EA re-init (recompile / param change / timeframe switch) instead of
//--- vanishing every time - see SIG_ARROW_PREFIX. Full cleanup still happens in the destructor.
ClearStatusLabel();
//--- NOTE: LoadChartSignals() is deliberately NOT called here any more. This method runs from
//--- InitNeuralNetwork() BEFORE the per-config fingerprint is appended to m_fileName, so at this point
//--- m_fileName is only "<folder>\<symbol>_<period>" - the load looked for e.g. "SP500_16385.arrows"
//--- while SaveChartSignals() (which only ever runs post-init, with the finished name) had written
//--- "SP500_16385_3.00000000_1.00000000_<hash>.arrows". The mismatch made the restore silently no-op on
//--- every restart from the moment the fingerprint was introduced. It is now called at the END of
//--- InitNeuralNetwork(), once m_fileName is final. Same family as the fingerprint trap documented at
//--- BuildConfigFingerprint: anything keyed on m_fileName must run AFTER it is fully built.
if(!InitOpen(indicators))
return false;
if(!InitClose(indicators))
return false;
if(!InitLow(indicators))
return false;
if(!InitHigh(indicators))
return false;
//--- label source, always created unconditionally, same as the OHLC indicators above - see
//--- m_ADZigZag's declaration comment. Optionally ALSO read as an input feature (m_useSwingContext,
//--- below) using the same already-running indicator instance - no separate init needed for that.
if(!InitADZigZag(indicators))
return false;
m_neuronsCount = 4; // (close-open)/atr, (high-open)/atr, (low-open)/atr, bullish/bearish flag
if(m_useVolumes)
{
m_neuronsCount++;
if(!InitVolumes(indicators))
return false;
}
// Unconditional, same reasoning as m_ATR/m_ADZigZag below: m_Time.GetData() is read
// unconditionally elsewhere (label-eligibility gate, cache anchor, online-learning watermark,
// arrow timestamps) regardless of whether the cyclical time-of-day/day-of-week values are also
// opted into as an explicit feature via m_useTime - so the indicator itself must always exist.
if(!InitTime(indicators))
return false;
if(m_useTime)
{
m_neuronsCount += 6;
}
if(m_useATR)
{
//already init in the base class
m_neuronsCount++;
}
if(m_useMA)
{
if(!InitMA(indicators))
return false;
m_neuronsCount += 5; // (open-MA)/atr, (high-MA)/atr, (low-MA)/atr, (close-MA)/atr, (MA-MA[1])/atr
}
if(m_useRSI)
{
if(!InitRSI(indicators))
return false;
m_neuronsCount++; // RSI/100
}
if(m_useMACD)
{
if(!InitMACDFeature(indicators))
return false;
m_neuronsCount += 3; // main/atr, signal/atr, histogram/atr
}
if(m_useIchimoku)
{
if(!InitIchimoku(indicators))
return false;
// (close-Tenkan)/atr, (close-Kijun)/atr, (Tenkan-Kijun)/atr, (close-SpanA)/atr, (close-SpanB)/atr,
// signed cloud thickness at this bar, signed PROJECTED cloud thickness, Chikou displacement
m_neuronsCount += 8;
}
if(m_useSwingContext)
m_neuronsCount += 9; // 5 confirmed-pivot features (direction, distance-since-pivot, prior-leg magnitude, retracement ratio, bars-since-pivot) + 4 recent-context features (Donchian pos 20/50, 20-bar return, 20-bar SMA extension) - see BufferTempDataCompute()'s matching block
if(m_useNews)
m_neuronsCount += 2; // NewsRecency, NewsProximity - see BufferTempDataCompute()'s matching block
if(m_useADCumulativeDelta)
{
if(!InitADCumulativeDelta(indicators))
return false;
m_neuronsCount += 6; // Pressure, CumulativeDelta, BullishPressure, BearishPressure, Absorption, Initiative
}
if(m_useADShorteningOfThrust)
{
if(!InitADShorteningOfThrust(indicators))
return false;
m_neuronsCount += 4; // SOT, SOTEffortRegime, SOTConfirmation, SOTPushRegime
}
if(m_useADWyckoffEventStream)
{
if(!InitADWyckoffEventStream(indicators))
return false;
// 12, not 14 - EventPrice (buffer 4) and EventPhase (buffer 1) are deliberately excluded, see
// BufferTempData()'s comment
m_neuronsCount += 12; // EventCode, ZoneTop, ZoneBottom, StructuralPhase, CHoCHTrendToRange, CHoCHRangeToTrend, SlopeAccumulationBullish, SlopeAccumulationBearish, SlopeDistributionBullish, SlopeDistributionBearish, Reaccumulation, Redistribution
}
if(m_useADWyckoffFailedStructure)
{
if(!InitADWyckoffFailedStructure(indicators))
return false;
m_neuronsCount += 5; // Value, BullishStructuralFailure, BearishStructuralFailure, FailedAccumulation, FailedDistribution
}
if(m_useADWyckoffSignificantBarInversion)
{
if(!InitADWyckoffSignificantBarInversion(indicators))
return false;
m_neuronsCount += 5; // SignificantBarQuality, BullishSignificantBar, BearishSignificantBar, BullishControlFlip, BearishControlFlip
}
if(!FolderCreate(m_folderPath, FILE_COMMON))
{
if(GetLastError() != 5010) // If the error is not because the folder already exists
{
Print("Failed to create folder: " + m_folderPath);
}
else
{
ResetLastError(); // Reset the error code
}
}
return true;
}
//+------------------------------------------------------------------+
//+------------------------------------------------------------------+
//| IMPLEMENTATION |
//| |
//| CExpertSignalAIBase's method bodies live in these partial files. |
//| They MUST be included here, after the class declaration above, |
//| and nowhere else. Order between them does not matter - they are |
//| all out-of-class definitions of an already-declared class. |
//| |
//| The class was ~8200 lines in one file; this splits the bodies by |
//| responsibility so a change to, say, chart drawing no longer means |
//| scrolling past the era loop. Nothing was rewritten in the move. |
//+------------------------------------------------------------------+
#include "AIBase\Training.mqh"
#include "AIBase\Labels.mqh"
#include "AIBase\OnlineLearning.mqh"
#include "AIBase\AutoTune.mqh"
#include "AIBase\Inference.mqh"
#include "AIBase\Persistence.mqh"
#include "AIBase\ChartUI.mqh"
#include "AIBase\Features.mqh"
//+------------------------------------------------------------------+
View git blame Copy permalink