forked from animatedread/Warrior_EA
bfc1da9de1
BuildFeatureWindow() replaces eight hand-rolled copies of the same loop
and feeds the window OLDEST BAR FIRST. Every copy fed it newest-first,
because MQL5 timeseries indices run backwards and `r + b` with b ascending
walks into the past.
Harmless for PAI and CONV - a dense layer learns a weight per position
either way, a conv learns time-mirrored kernels. Not harmless for the
recurrent stacks:
- LSTM_SeqStepForward reads `inputs + t*Iw`, so step t is block t.
- It writes output[] only when t == steps-1: the visible output IS the
last hidden state.
- c_t = f*c_{t-1} + i*g decays toward the start of the sequence.
lstm_seq_flowcheck.cpp measured block 0's influence on the output at
1.2e-2 of block T-1's, at the shipped forget bias of 1.0.
So the bar being PREDICTED sat at the far end of the decay and the output
was handed to the OLDEST bar in the window - the exact inverse of what the
window is for. ~80x backwards on LSTM and HYBRID, on all three tiers
(OpenCL kernel, CPU DLL, pure-MQL5 inference), which is why it never
surfaced as a backend discrepancy.
This does not create edge - the MI diagnostics read at the noise floor
(p=0.4975) with a working positive control. It makes the one hypothesis
those diagnostics explicitly do NOT cover testable: they are marginal and
per-bar, and state they "cannot rule out one that only exists in
combination or across time". The sequence model is the instrument for
across-time structure and it has been crippled, so that hypothesis has
never been honestly tested.
Fingerprint gets an unconditional |WIN:2 - the vector keeps its shape and
its features, so a stale .nnw would load cleanly and run a model fitted to
one ordering against the other, silently. Re-keying every config is the
point, not collateral damage. FORCES A FULL RETRAIN.
Also: the now-relative bar caches are re-keyed on the two live paths.
EnsureBarCachesCapacity() was only ever called from training paths, but
once m_trainingComplete is set ScheduleTrainingIfNeeded() routes every bar
to RefreshConvergedSignal() and Train() is never re-entered - so nothing
cleared the feature cache again for the life of the process. A chart that
trained to convergence kept replaying the rows computed for the last
training era's bar grid: the live signal froze at its convergence-time
value, and OnlineLearnStep() backpropped those stale features against
freshly resolved labels. Backtests were never affected (an inference-only
process never allocates the arrays, so every read recomputes).
Compiles clean: 0 errors, 0 warnings.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2454 lines
190 KiB
MQL5
2454 lines
190 KiB
MQL5
//+------------------------------------------------------------------+
|
|
//| Warrior_EA |
|
|
//| AnimateDread |
|
|
//| |
|
|
//+------------------------------------------------------------------+
|
|
#include "ExpertSignalCustom.mqh"
|
|
#include "..\AI\Network.mqh"
|
|
#include "..\Variables\IndicatorResources.mqh"
|
|
#include "..\Variables\IndicatorTuneRanges.mqh"
|
|
#include "..\System\StatusLabel.mqh"
|
|
#include "..\System\NewsRelevance.mqh"
|
|
#include "..\System\CrossAsset.mqh"
|
|
#include "ADIndicatorTuner.mqh"
|
|
//--- Hard 0/1 one-hot targets for the 3-neuron classification head - matches the book's
|
|
//--- (nnbook.txt sec. 1.4 "Cross-entropy") cross-entropy formulation, which defines the reference
|
|
//--- distribution's occurring-event probability as exactly 1.0 (missing event exactly 0.0), no
|
|
//--- smoothing. This used to be softened to 0.9/0.1 (see git history) as a workaround for the OLD
|
|
//--- per-neuron independent-sigmoid backward gradient: since that gradient was each neuron's own
|
|
//--- raw (target-sigmoid_output) delta with nothing else driving it to zero, a literal 1.0/0.0
|
|
//--- target was only ever approached asymptotically, so weights kept growing (up to the MAX_WEIGHT
|
|
//--- clamp) chasing it forever. Now that CNet::backProp()/backPropOCL() (AI\Network.mqh) compute a
|
|
//--- joint softmax+categorical-cross-entropy gradient (softmax_i - target_i) across all 3 neurons
|
|
//--- instead, the winning class's SOFTMAX probability - not any one neuron's raw sigmoid value -
|
|
//--- is what needs to approach the target, and softmax can reach very close to 1.0 for the winning
|
|
//--- class from ordinary (non-saturated) logit separation, so the old runaway-weight failure mode
|
|
//--- is no longer expected to require smoothed targets to avoid. If it resurfaces in practice (the
|
|
//--- per-neuron SIGMOID forward pass can still individually saturate before softmax normalizes),
|
|
//--- that's the first thing to check before reintroducing smoothing.
|
|
// Soft labels keep the classification head from overfitting to hard one-hot targets while still
|
|
// preserving a clear target for the true class. The true class gets 0.9 and the others 0.05 each.
|
|
#define LABEL_SMOOTH_HIGH 0.9
|
|
#define LABEL_SMOOTH_LOW 0.05
|
|
//--- Namespace prefix for the directional signal arrows this class draws (DrawObject/DeleteObject). Two
|
|
//--- reasons it exists: (1) so PurgeChart() can delete ONLY our arrows and never the user's own manual
|
|
//--- chart drawings (a full ObjectsDeleteAll(0) on a client's chart is not acceptable for a commercial
|
|
//--- product), and (2) so arrows can survive an EA re-init instead of being wiped on every InitIndicators
|
|
//--- - the whole point of keeping them on screen (the panel has a hide toggle if the user wants them gone).
|
|
#define SIG_ARROW_PREFIX "WarSig_"
|
|
//--- Upper bound on arrows restored from a .arrows file (see LoadChartSignals). Purely a guard against a
|
|
//--- corrupt header declaring a garbage count - a long converged run legitimately accumulates a few
|
|
//--- thousand, so this sits well above that. Restoring is chunked across timer calls regardless, so a
|
|
//--- large-but-valid count costs progressive fill-in, never a frozen OnInit.
|
|
#define MAX_RESTORED_ARROWS 50000
|
|
//--- How many of the MOST RECENT arrows are kept on the chart and in the .arrows sidecar. Arrows would
|
|
//--- otherwise accumulate for the life of the model (a converged SP500 H1 run had reached 2896), which
|
|
//--- clutters the chart, slows every save/restore, and preserves history nobody scrolls back to. Both
|
|
//--- SaveChartSignals (which also DELETES the pruned objects from the chart) and LoadChartSignals select
|
|
//--- by TIME, not by scan order - ObjectsTotal() order is arbitrary, so "the last N scanned" would keep a
|
|
//--- random subset rather than the newest.
|
|
#define MAX_PERSISTED_ARROWS 1000
|
|
//--- Manual "rescan" window (see RescanChartSignals): how many of the MOST RECENT bars get re-inferred
|
|
//--- from the currently deployed weights when the operator asks for a fresh signal set. Bounded well
|
|
//--- below a full StudyPeriods re-render (which can be years of bars and would freeze the one MQL5 chart
|
|
//--- thread, same doctrine as MAX_RESTORED_ARROWS/ARROW_RESTORE_BUDGET_MS) - a rescan is meant to replace
|
|
//--- stale old arrows with what the model calls on RECENT history, not reproduce the whole training run.
|
|
#define SIGNAL_RESCAN_LOOKBACK_BARS 5000
|
|
//--- Wall-clock budget per chunk of the deferred arrow restore. Same doctrine as TRAIN_TIME_BUDGET_MS:
|
|
//--- MQL5 gives a chart ONE thread, so "async" here means small time-boxed slices between which the
|
|
//--- terminal can service the panel, the chart and the journal - never a single long blocking pass.
|
|
//--- 50ms sits between the training chunk (120ms, already tolerated) and the 500ms timer period, so the
|
|
//--- restore completes in a handful of slices while leaving ~90% of each timer window free for the UI.
|
|
#define ARROW_RESTORE_BUDGET_MS 50
|
|
//--- Max allowed |Δ| between the compute backend's and the pure-MQL5 path's outputs for a model to be
|
|
//--- marked MQL5-inference-safe (see ValidateCpuInference). The outputs are bounded sigmoid values;
|
|
//--- backend-vs-double summation-order noise is ~1e-6 on the CPU-DLL path (double) and stays well under
|
|
//--- this even on a float32 OpenCL backend, while a genuine math/port bug shows up as >0.01. Fails safe.
|
|
#define CPU_INFERENCE_MAX_DIFF 1.0e-3
|
|
extern bool g_signalsVisible;
|
|
//--- FILTER-BASED auto-tuner constants (see TuneIndicatorsByFilter). The GA_* / TUNE_POP_* knobs that
|
|
//--- lived here are gone with the genetic search they configured.
|
|
//--- MI_BINS: equal-frequency bins the feature column is discretised into before the joint histogram.
|
|
//--- Mutual information is biased upward as bins increase (each bin holds fewer samples, so noise looks
|
|
//--- like structure); 8 bins against MI_SAMPLE_BARS samples keeps ~250 samples per bin per class, which
|
|
//--- is comfortably in the regime where that bias is small and equal across candidates - and equal is
|
|
//--- what matters, since this score is only ever used to RANK.
|
|
#define MI_BINS 8
|
|
//--- Bars sampled (evenly spaced across the in-sample window) per candidate evaluation. The whole cost of
|
|
//--- tuning is candidates x this x features, so it is the one number that trades accuracy for time.
|
|
#define MI_SAMPLE_BARS 2000
|
|
#define MI_MIN_SAMPLES 200
|
|
//--- Eras the MI diagnostics may wait for the cross-asset panel before reporting without it. Three is
|
|
//--- enough for the terminal to finish synchronising auxiliary symbols on a live chart, and short
|
|
//--- enough that a tester run - where an un-downloaded reference symbol is permanently absent - is not
|
|
//--- left without diagnostics at all.
|
|
#define MI_REPORT_MAX_DEFERRALS 3
|
|
//--- Largest |k| the label-alignment scan uses. Every MI sample is padded by MiShiftPad() - which is at
|
|
//--- least this - at BOTH ends REGARDLESS of the offset being requested, so that every build enumerates
|
|
//--- the IDENTICAL bar set with the IDENTICAL stride. That is what makes two builds comparable row by
|
|
//--- row. Padding by |offset| instead (as this did until 2026-08-02) shifted the offset build's starting
|
|
//--- bar, so the positive control paired rows that were offset+pad apart rather than offset apart: it
|
|
//--- reported 0.00307 nats for a pair it called "24 bars apart", which is the value for 48 bars, failed
|
|
//--- its own 5x gate, and printed "every mutual-information figure above is void" over perfectly sound
|
|
//--- measurements. Verified against an independent computation in research/test_mi_control.py.
|
|
#define MI_ALIGN_MAX_SHIFT 5
|
|
//--- Coordinate-descent passes. The second pass lets a parameter re-optimise against what the others
|
|
//--- moved to; the loop breaks early as soon as a pass changes nothing, so this is a ceiling, not a cost.
|
|
#define MI_TUNE_PASSES 2
|
|
//--- MI_NOISE_PERMUTATIONS: draws from the null distribution used to test the observed score. Sets the
|
|
//--- resolution of the empirical p-value, which can never go below 1/(B+1) - so 200 draws can report
|
|
//--- "p<=0.005" and no finer, which is ample for a yes/no on whether a feature set carries signal.
|
|
//--- Cheap because BuildMiSample runs ONCE and every draw reuses it (see ScoreMiSample); the cost is a
|
|
//--- relabel and 26 histogram passes, not 2000 feature extractions.
|
|
//--- Two earlier values were wrong and both are instructive. ONE shuffle answers "is this above a coin
|
|
//--- flip's worth of noise" rather than "is this above noise". FIVE looked principled but was not: on
|
|
//--- 2026-08-01 all four charts scored the identical 0.00401 nats on identical features and identical
|
|
//--- labels, and reported z of +1.3, +2.0, +4.0 and +4.7 - two "noise floor", two "real". With five
|
|
//--- draws the standard deviation of the standard-deviation estimate is ~35%, so the denominator of that
|
|
//--- z was noisier than the effect it was judging. Counting ranks avoids estimating a spread at all.
|
|
#define MI_NOISE_PERMUTATIONS 200
|
|
//--- Lag-profile draws, deliberately far below MI_NOISE_PERMUTATIONS: the profile redraws its null at
|
|
//--- EVERY lag (see ReportFeatureLagProfile for why a shared floor would be wrong), so the cost is
|
|
//--- draws x historyBars, not draws. 40 resolves a p of 0.05 to within about one draw, which is all a
|
|
//--- lookback decision needs - this figure never gates a trade.
|
|
#define MI_LAG_PERMUTATIONS 40
|
|
//--- Per-lag significance, applied against the null of the MAXIMUM over lags rather than against each
|
|
//--- lag's own null. The first version did the latter and it was wrong: ~21 lags at 0.05 stars one lag
|
|
//--- per run before any signal exists, and on SP500 H1 that produced two opposite verdicts on identical
|
|
//--- data hours apart. The alpha stays 0.05; what changed is the null it is measured against.
|
|
#define MI_LAG_ALPHA 0.05
|
|
//--- WHICH TARGET BuildMiSample() scores the features against. The barrier class is the shipped training
|
|
//--- target; the excursion targets exist to answer a question the barrier label cannot, and that every MI
|
|
//--- verdict in this project so far has silently conflated.
|
|
//---
|
|
//--- "Optimal SL/TP" decomposes into two predictions that behave nothing alike:
|
|
//--- HOW FAR price travels (the excursions) - essentially a volatility question, and volatility
|
|
//--- clustering is about the most robust regularity there is, so expect this to be predictable.
|
|
//--- WHICH barrier is reached first (the asymmetry) - direction, which is what the barrier label
|
|
//--- measures and what has come back at the noise floor every time.
|
|
//--- Expectancy comes ONLY from the second. The first buys position sizing and drawdown control, which
|
|
//--- is worth having under prop-firm limits but is not an edge - exit management tested against RANDOM
|
|
//--- entries moved the payoff ratio 0.92 -> 5.72 with expectancy FLAT.
|
|
//--- Measuring them separately is the point: if size clears and asymmetry does not, the deliverable is
|
|
//--- risk control and we should stop looking for edge in the exit.
|
|
//--- BARRIER DERIVATION. The stop sits at a HIGH quantile of adverse travel, so only the minority of bars
|
|
//--- whose adverse excursion exceeds it ever reach it; the target at the MEDIAN of favourable travel, so
|
|
//--- it is reached about half the time inside the horizon by construction. Neither number creates
|
|
//--- expectancy (chance precision equals break-even at every geometry) - they make the target reachable
|
|
//--- and the stop survivable, which the enum grid {2,3} x {2,3,4,6,8,10} could only do by luck.
|
|
//---
|
|
//--- THE STOP QUANTILE WAS 0.25 AND THAT WAS BACKWARDS. q25 means 75% of bars exceed the stop, i.e. it is
|
|
//--- hit three times in four - the opposite of "ordinary noise does not reach it". Caught by the measured
|
|
//--- reachability line the derivation prints ("stop on 75.0% of bars"), which is the entire reason that
|
|
//--- figure is reported instead of assumed. A quantile is a threshold, not a rate: to be rarely reached a
|
|
//--- stop must sit ABOVE most of the distribution.
|
|
#define BARRIER_SL_QUANTILE 0.75
|
|
#define BARRIER_TP_QUANTILE 0.50
|
|
//--- Below this many resolved excursions the quantiles are too noisy to key a training target on, and the
|
|
//--- configured multiples stand.
|
|
#define BARRIER_DERIVE_MIN_SAMPLES 500
|
|
//--- The derivation is a FIXED-POINT ITERATION, not a one-shot. ComputeBarrierHorizonBars scales the
|
|
//--- horizon with the target (first-passage time grows with the band), and the excursions are measured
|
|
//--- OVER that horizon - so target -> horizon -> excursions -> target is a loop. Deriving once would set
|
|
//--- the target from travel measured under the OLD horizon and quietly mis-state it. Re-measure until the
|
|
//--- multiples stop moving, capped so a pathological oscillation cannot spin forever.
|
|
#define BARRIER_DERIVE_MAX_PASSES 5
|
|
#define BARRIER_DERIVE_TOLERANCE 0.05
|
|
//--- Warn when the reward:risk floor forces a target this market rarely reaches. Not a hard error: the
|
|
//--- ratio is the user's risk policy, and the honest response is to say what it costs, not to override it.
|
|
#define BARRIER_MIN_TP_REACH_PCT 20.0
|
|
#define MI_TARGET_BARRIER 0 // shipped 3-class triple-barrier label
|
|
#define MI_TARGET_EXC_UP 1 // (maxHigh - entry)/ATR over the horizon, 3 equal-frequency bins
|
|
#define MI_TARGET_EXC_DOWN 2 // (entry - minLow)/ATR
|
|
#define MI_TARGET_EXC_RANGE 3 // up + down: pure realised volatility, the control that SHOULD clear
|
|
#define MI_TARGET_EXC_ASYM 4 // up - down: RAW asymmetry - CONFOUNDED BY VOLATILITY, see below
|
|
//--- SCALE-FREE asymmetry, and the only one of the two that can support a directional claim.
|
|
//--- (up-dn) is NOT scale-free: if sigma is predictable - and RANGE clears at ~4x its null on every
|
|
//--- instrument tested - and the directional part is symmetric noise eps, then up-dn ~ sigma*eps, so a
|
|
//--- large sigma pushes the value into BOTH outer terciles. A pure volatility predictor therefore scores
|
|
//--- positive MI against a 3-bin (up-dn) while carrying no directional information whatsoever, and it
|
|
//--- does so consistently across instruments - so replication does not rule it out. Measured 2026-08-07:
|
|
//--- raw ASYM cleared on EURUSD and USDCAD at p=0.0050 exactly where RANGE was strongest.
|
|
//--- Dividing by (up+dn) removes the scale factor and leaves the question actually being asked: given
|
|
//--- that price moved, WHICH WAY did it move further. Bounded in [-1,+1] by construction.
|
|
#define MI_TARGET_EXC_ASYM_NORM 5
|
|
//--- Significance the indicator tuner's winner must reach, AFTER correcting for having been chosen out of
|
|
//--- N candidates. Same 0.05 as elsewhere; what matters is that a gate exists at all, since this selector
|
|
//--- overwrites the user's configured indicator settings and forces a fresh topology.
|
|
#define MI_TUNE_ALPHA 0.05
|
|
//--- Ceiling on profiled lags, sizing the retained-draw matrix. m_historyBars is derived and could in
|
|
//--- principle exceed this; the profile then covers the first MI_LAG_MAX_PROFILE-1 lags and says so via
|
|
//--- the lag count it prints, rather than overrunning the buffer.
|
|
#define MI_LAG_MAX_PROFILE 32
|
|
//--- Draws per candidate in the barrier-geometry scan. Raised from the ranking-only 20 because these draws
|
|
//--- now do a second job: they build the FAMILY-WISE null that decides whether the winner is real (see
|
|
//--- ReportBarrierGeometryScan). A max-statistic lives in the upper tail of the null, and a tail is exactly
|
|
//--- where 20 draws are thinnest. Cost is draws x candidates on an already-extracted sample - the whole
|
|
//--- scan ran in 3.2s at 20 draws, so this is single-digit seconds, once, before era 0.
|
|
#define MI_GEOMETRY_PERMUTATIONS 40
|
|
//--- Family-wise significance for the geometry winner. Stricter than MI_LAG_ALPHA because the two decisions
|
|
//--- are not comparable: a lag profile that guesses wrong costs some input width, whereas acting on this
|
|
//--- one means RELABELLING and retraining every topology from era 0. The gate must be hard to pass.
|
|
#define MI_GEOMETRY_ALPHA 0.05
|
|
//--- Ceiling on scanned candidates, sized to the shipped grid (2 stop multiples x 6 target multiples). Only
|
|
//--- used to size the fixed draw matrix below; the loop still skips ineligible pairings.
|
|
#define MI_GEOMETRY_MAX_CANDIDATES 12
|
|
//--- How many standard errors a checkpoint's directional precision must clear chance by before it is
|
|
//--- considered deployable - see the edgeFloorPct block in Train(). Two sigma is the conventional
|
|
//--- "not a fluke" bar and lands near 1pp at the ~11,000 directional calls a full OOS era produces.
|
|
//--- CRITICAL CONTEXT, and the reason chance is the right reference at all: under a driftless random
|
|
//--- walk the probability of touching +k*ATR before -m*ATR is m/(m+k), and the break-even win rate for a
|
|
//--- k:m reward:risk trade is ALSO m/(m+k). The label's own base rate IS the break-even rate, at every
|
|
//--- SL/TP setting. So "beats chance" and "is profitable" are the same test, no SL/TP choice can
|
|
//--- manufacture an edge, and this margin is measuring expectancy directly.
|
|
#define EDGE_MIN_SIGMAS 2.0
|
|
//--- OVERSAMPLING CONSTANTS REMOVED 2026-07-31 (MAX_OVERSAMPLE_REPLICAS, OVERSAMPLE_PARITY_FRACTION).
|
|
//--- Data-level class-balance oversampling is gone; every bar is queued exactly once and the imbalance
|
|
//--- is corrected analytically in the gradient by the logit-adjusted loss. See the queueing block in
|
|
//--- Expert\AIBase\Training.mqh for the four successive oversampling designs that collapsed before it,
|
|
//--- and the class-imbalance audit in Variables\Inputs.mqh for the nine inputs this consolidated.
|
|
//--- TRIPLE-BARRIER LABELS (Lopez de Prado, "Advances in Financial Machine Learning", ch. 3).
|
|
//--- 2026-08-01: replaced exact-pivot ZigZag labels. ZigZag REMAINS the input-feature source
|
|
//--- (m_useSwingContext) and the horizon source below - only the TARGET changed.
|
|
//---
|
|
//--- Why. The old label marked the single exact bar where a ZigZag pivot confirmed: ~1,164 Buy, ~1,164
|
|
//--- Sell, ~35,841 Neutral - a 31:1 imbalance, and every correction mechanism in this file (logit
|
|
//--- adjustment, prior EMA, bias seed, recall floor, alternation gate, NMS, four oversampling designs)
|
|
//--- was downstream of that one choice. The imbalance was self-inflicted by the target, not a property
|
|
//--- of the market: the reference book (references\neuronetworksbook.pdf ch. 3.1/3.3) also uses ZigZag
|
|
//--- but targets the DIRECTION TO THE NEXT EXTREMUM on every bar - ~50/50, nothing to correct.
|
|
//---
|
|
//--- What replaces it. For each bar, place the EA's OWN stop and target around a hypothetical entry at
|
|
//--- that bar's close and ask which barrier a real trade would touch first, within a horizon:
|
|
//--- long reaches TP before SL, short does not -> Buy
|
|
//--- short reaches TP before SL, long does not -> Sell
|
|
//--- neither resolves in the model's favour -> Neutral
|
|
//--- The barriers come from SL_Mode/TP_Mode (m_sl_mode/m_tp_mode, inherited from CExpertSignalCustom),
|
|
//--- so the label and the trade CANNOT drift apart, and `dir-precision` in the era line stops being a
|
|
//--- proxy and literally becomes the win rate of the strategy under its own exit rules. That is the
|
|
//--- number this project never had. No new user input: the two that shape the label already exist.
|
|
//---
|
|
//--- Expected balance. At the shipped SL_ATR_x1 / TP_ATR_x3 the gambler's-ruin probability of touching
|
|
//--- +3 ATR before -1 ATR is 1/(1+3) = 25% per side, so roughly 25/25/50 - about 2:1 rather than 31:1.
|
|
//--- Measured for real at the end of the prebuild; do not assume it.
|
|
//---
|
|
//--- INTRABAR AMBIGUITY IS RESOLVED PESSIMISTICALLY. When one bar's range spans both barriers, OHLC
|
|
//--- cannot say which came first, so the label counts it as the STOP. A win rate built on the
|
|
//--- optimistic reading is exactly the kind of number that evaporates live.
|
|
#define BARRIER_TIE_GOES_TO_STOP 1
|
|
//--- Vertical (time) barrier, in bars. DERIVED, never configured: the median distance between confirmed
|
|
//--- ZigZag pivots over the training window - i.e. this symbol/timeframe's own natural swing horizon,
|
|
//--- measured from the same indicator the features already read. Snapped to the ladder below so the
|
|
//--- estimate has to move ~30% to change the answer; without that quantization a horizon that drifted as
|
|
//--- history downloaded would silently relabel a partially-trained model's targets mid-run. Same
|
|
//--- measure-once-then-quantize contract as ComputeFirstLayerWidth().
|
|
//--- It is deliberately NOT in the weights-filename fingerprint - a filename keyed on a measured
|
|
//--- quantity orphans a trained model the moment the measurement moves. See BuildConfigFingerprint.
|
|
//--- 2026-08-01: the ladder used to stop at 128 and that ceiling, not the barrier geometry, was the
|
|
//--- binding constraint. First-passage time for a driftless walk leaving [-m, +k] is proportional to m*k,
|
|
//--- and the measured swing median here is ~12 bars at m*k=1 - so EVERYTHING from 2:6 upward wanted more
|
|
//--- than 128 bars and got truncated, including the SHIPPED 2:6 configuration. A truncated label stops
|
|
//--- meaning "does the target come before the stop" and quietly becomes "...within 128 bars", while the
|
|
//--- deployed EA holds until SL or TP with no bar limit. That is a train/deploy mismatch in the target
|
|
//--- itself, and it silently converted the slow winners - exactly the trades a 1:3 barrier exists to
|
|
//--- catch - into Neutral. Extended to cover the whole selectable grid: 3:10 needs ~360 bars.
|
|
//--- Cost is one embargo of BARRIER_HORIZON_MAX bars out of ~38k, i.e. nothing.
|
|
#define BARRIER_HORIZON_LADDER_COUNT 11
|
|
#define BARRIER_HORIZON_MIN 12
|
|
#define BARRIER_HORIZON_MAX 384
|
|
//--- Fallback when the ZigZag scan finds too few pivots to take a median from (a cold history buffer, or
|
|
//--- a symbol so quiet that Depth-12 emits almost nothing). Mid-ladder, and it logs when it fires.
|
|
#define BARRIER_HORIZON_FALLBACK 32
|
|
//--- Minimum confirmed pivots required before the median is trusted rather than the fallback.
|
|
#define BARRIER_HORIZON_MIN_SAMPLES 20
|
|
//--- Share of bars one class must hold before the era-0 output-bias cold-start seed fires - see the
|
|
//--- trigger in AdvanceLabelCachePrebuild(). A +-3.0 bias seed is a correction at a 94%-Neutral prior
|
|
//--- and a distortion at a 50% one, so the threshold marks where "dominant" actually starts.
|
|
#define COLD_START_SEED_MIN_DOMINANCE 0.70
|
|
//--- Reduce-on-regression learning-rate decay. `eta` (AI\Network.mqh) is a plain global read fresh
|
|
//--- by every weight-update call on every backend (native/OpenCL/CPU-DLL/DirectML-DLL alike), so
|
|
//--- shrinking it here takes effect on the very next backProp() everywhere at once. A fixed step
|
|
//--- size that is large enough to escape a bad random init quickly (see the sharp era 5->14 OOS
|
|
//--- accuracy climb this was tuned against) is, by the same token, large enough to overshoot once
|
|
//--- training gets close to a good solution - the era 14->16 regression right after that climb is
|
|
//--- the classic signature of that overshoot, not a structural bug. The existing best-checkpoint
|
|
//--- mechanism just below (m_bestOosForecast + CNet::CaptureWeights) already guarantees the FINAL deployed
|
|
//--- model can't regress; this constant lets the live training process itself settle down instead
|
|
//--- of continuing to oscillate around a good solution once it finds one.
|
|
#define ETA_DECAY_REGRESSION_PCT 5.0 // only decay after a real regression, not per-era noise
|
|
#define ETA_DECAY_FACTOR 0.7
|
|
#define ETA_MIN 0.0001
|
|
//--- Balanced accuracy (macro-recall) of a model that puts every bar in ONE class: (0+0+100)/3. It is
|
|
//--- the FLOOR of the balanced metric, not a midpoint - any genuinely multi-class model scores above it.
|
|
//--- ComputeFirstLayerWidth() constants. SECONDS_PER_YEAR is the mean Julian year (365.25 days), the
|
|
//--- same convention MQL5's own date arithmetic uses. MARKET_OPEN_FRACTION allows for closed hours and
|
|
//--- weekends - ~0.72 is right for both a 24/5 FX week and an index with extended sessions, and the
|
|
//--- ladder in that function makes anything in the 0.6-0.85 range land on the same rung anyway.
|
|
//--- FIRST_LAYER_MIN_WIDTH is a floor for degenerate configs (a very short study period, or a symbol
|
|
//--- whose history cannot support the requested window) - below this the taper has nothing to work with.
|
|
#define SECONDS_PER_YEAR 31557600.0
|
|
#define MARKET_OPEN_FRACTION 0.72
|
|
//--- Ceiling on how much of the classification head's LOGIT RANGE the logit-adjustment offsets may
|
|
//--- consume, as a fraction. The head is SIGMOID, so each output is bounded to [0,1] and the widest
|
|
//--- logit difference the net can express between two classes is CLASS_LOGIT_SCALE * (1 - 0) - six,
|
|
//--- at the shipped scale. The offsets are tau*log(prior_c), whose spread on a 30:1 imbalance is
|
|
//--- log(0.939) - log(0.031) = 3.42, so an untamed tau=1.0 spends 57% of the ENTIRE expressible range
|
|
//--- on the prior correction alone. Measured 2026-07-29: every chart did the only thing it could -
|
|
//--- saturate its Buy/Sell outputs to 1.0 to overcome a -3.42 handicap during training - and since the
|
|
//--- offsets are absent at inference, that surplus made EVERY bar directional. Neutral recall 0%, calls
|
|
//--- on ~100% of bars, win rate 5-7% against a ~6% base rate: no information at all, while balanced
|
|
//--- accuracy read a flattering 64% because two of its three terms were ~95%.
|
|
//--- Menon et al. assume an unbounded logit head, where a 3.42 shift is negligible against the range
|
|
//--- the network can reach. It is not negligible here, so the strength has to be expressed relative to
|
|
//--- the range actually available. Deliberately a FRACTION rather than a tau ceiling: it stays correct
|
|
//--- if CLASS_LOGIT_SCALE changes, if the head becomes unbounded, or on any other symbol/timeframe
|
|
//--- whose class imbalance differs - none of which a hardcoded tau would survive.
|
|
#define LOGIT_ADJUST_MAX_RANGE_FRACTION 0.20
|
|
//--- Minimum directional call rate a checkpoint must reach to be considered deployable, expressed as a
|
|
//--- FRACTION OF THE TRUE DIRECTIONAL BASE RATE rather than an absolute percentage - a model that calls
|
|
//--- a direction a quarter as often as one actually occurs is sparse but usable; one that calls ten
|
|
//--- times a decade is not, however precise those ten calls were. Derived rather than configured so it
|
|
//--- adapts to any symbol/timeframe/label rule without a second input to keep in sync.
|
|
#define MIN_COVERAGE_FRACTION_OF_BASE_RATE 0.25
|
|
#define FIRST_LAYER_MIN_WIDTH 16
|
|
//--- Where the dense taper ENDS. The last hidden layer wants to be small enough to force the network to
|
|
//--- commit to a compressed representation, but comfortably wider than the decision itself so it is not
|
|
//--- the bottleneck - a few units per class is the usual heuristic. The absolute floor covers the
|
|
//--- single-output regression head, where 4x1 would be absurdly narrow.
|
|
//--- Conv receptive field, in BARS. The reference (references\MQL5\Experts\EDL\Trajectory.mqh) uses a
|
|
//--- window of 2 positions with step 1, stacked twice for an effective field of 3; a value of 2 here
|
|
//--- matches it at our bar granularity. Not an input: it is a structural property of the front-end, and
|
|
//--- a user who picks it is choosing against the pool and second-conv shapes derived around it.
|
|
//---
|
|
//--- RE-LANDED 2026-07-31 at 3, WITHOUT the channel pool that came with it the first time.
|
|
//--- Why the 34d6aa4 attempt failed, established by reading the reference kernels rather than guessing:
|
|
//--- CNeuronConvOCL emits POSITION-MAJOR output - `matrix_o[out + window_out * i]`, i.e.
|
|
//--- [pos0 f0..fN][pos1 f0..fN]... (References\MQL5\Experts\NeuroNet_DNG\NeuroNet.cl, FeedForwardConv).
|
|
//--- The reference pool (FeedForwardProof) is a flat contiguous max over `window` elements at stride
|
|
//--- `step`. On position-major data ANY window <= window_out therefore maxes ACROSS FILTERS INSIDE ONE
|
|
//--- POSITION - it cannot pool over time at all. The stage we built used window = step = filterCount,
|
|
//--- which is exactly one max over all 8 filters per position: 87.5% of the conv's output discarded, and
|
|
//--- only the argmax filter receiving gradient at each position. That is the measured regression (era 5:
|
|
//--- 18% dir-precision and 45 live fires at RF=1, versus "no directional calls" and 0 fires at RF=2).
|
|
//--- It is a property of the reference's own layout, not a porting mistake, so there is no "correct pool"
|
|
//--- to swap in here: pooling over time is not expressible on this layout without a transpose.
|
|
//--- The literature answer, already cited elsewhere in this file, is Springenberg et al. ICLR 2015
|
|
//--- ("Striving for Simplicity: The All Convolutional Net"): drop pooling, get the hierarchy from strided
|
|
//--- convolution instead. So this stage is now a single TRUE convolution - a CONV_RECEPTIVE_FIELD_BARS-bar
|
|
//--- window sliding one bar at a time - and nothing else. 3 bars is the smallest window that can express
|
|
//--- a turning point (before/at/after), which is what the ZigZag labels mark.
|
|
//--- Verify a change here against CNet::LayerLearningReport's per-era norm(d|W|%/|dW|%) line, not against
|
|
//--- accuracy: |dW| >> d|W| means the layer is rotating (learning), the two roughly equal with the norm
|
|
//--- falling means it is only being decayed away. Reading accuracy is what made the first regression take
|
|
//--- a full retrain cycle to spot.
|
|
#define CONV_RECEPTIVE_FIELD_BARS 3
|
|
//--- Master switch for the sequence-LSTM front-end (AI\Network.mqh CNeuronLSTMOCL).
|
|
//--- 1 = the layer is a real recurrence over bars: one shared gate block applied at every timestep,
|
|
//--- with backpropagation-through-time. Gradient-checked to 2.3e-10 (DirectML\lstm_seq_gradcheck.cpp)
|
|
//--- and signal/gradient-reach-checked (DirectML\lstm_seq_flowcheck.cpp).
|
|
//--- 0 = the pre-2026-07-30 layer: ONE gate step over the whole flattened input, no recurrence.
|
|
//---
|
|
//--- RE-LANDED 2026-07-31, together with a corrected LSTM_FORGET_BIAS_INIT - see that constant, which is
|
|
//--- the part that was actually wrong.
|
|
//--- The 2026-07-30 attempt reported "flat IS error + Neutral:100%", which reads as "no gradient" but is
|
|
//--- equally the signature of an output that does not VARY with the input. The forward path rules the
|
|
//--- first out: CPU_LSTMSeqForward starts every sample at h_0 = c_0 = 0 (t==0 takes the nullptr branch),
|
|
//--- unrolls T = m_historyBars steps over that sample's own window, and emits h_T. State does not leak
|
|
//--- between shuffled samples, so shuffling is not the problem either. What DOES depend on the constant
|
|
//--- is saturation: with forget bias b the cell tends to c ~ i*g/(1-sigmoid(b)) over T steps, so b = 2.0
|
|
//--- gives c ~ 8*i*g, tanh(c) pins to +/-1, and h_T = o*tanh(c) becomes near-binary and set by the gate
|
|
//--- biases rather than by the bars. That is exactly "input-independent output".
|
|
//--- The tell in the era line is "OOS raw out B:min..max": a collapsed span there is this failure, not a
|
|
//--- gradient failure. Check it and the norm(d|W|%/|dW|%) line together before concluding anything.
|
|
#define LSTM_SEQUENCE_MODE 1
|
|
#define HIDDEN_TAPER_OUTPUT_MULTIPLE 4
|
|
#define HIDDEN_TAPER_MIN_WIDTH 8
|
|
//--- ComputeConvFilterCount()/ComputeLstmHiddenSize() bounds. Both stages used to be inputs with a
|
|
//--- hand-picked constant default (16 filters, 32 hidden units) chosen with no reference to how wide the
|
|
//--- input actually ended up or how much data there is to fit them - the same defect the first-layer
|
|
//--- width had before it was derived. CONV_COMPRESSION_DIVISOR is the ratio the conv stage should
|
|
//--- compress a bar's feature vector by: the layer is a per-bar projection (window = step = one bar's
|
|
//--- features, see AddConvStage), so filters > features EXPANDS a correlated input at the very bottom of
|
|
//--- the stack, which is over-parameterization in its purest form. Halving is the conventional bottleneck
|
|
//--- choice and holds at any feature count.
|
|
//--- ComputeHiddenLayerCount() bounds. HIDDEN_TAPER_TARGET_RATIO is the per-layer compression the taper
|
|
//--- aims for - halving, the conventional funnel - and depth is however many such steps it takes to get
|
|
//--- from the derived first-layer width to the output-tied final width. Two layers is the floor because
|
|
//--- one dense layer plus the head is a linear model with a single non-linearity; five is the ceiling
|
|
//--- because beyond it the per-step compression is so mild that the extra depth adds vanishing-gradient
|
|
//--- risk without adding abstraction.
|
|
#define HIDDEN_TAPER_TARGET_RATIO 2.0
|
|
#define MIN_HIDDEN_LAYERS 2
|
|
#define MAX_HIDDEN_LAYERS 5
|
|
//--- EstimatedInSampleBars() fallback for a chart whose history has not finished downloading. Below the
|
|
//--- trusted-bar floor the measurement says more about the sync state than about the symbol.
|
|
#define TOPOLOGY_BUDGET_MIN_TRUSTED_BARS 500
|
|
#define TOPOLOGY_BUDGET_FALLBACK_YEARS 10
|
|
#define CONV_COMPRESSION_DIVISOR 2
|
|
#define CONV_FILTERS_MIN 4
|
|
#define CONV_FILTERS_MAX 32
|
|
#define LSTM_HIDDEN_MIN 8
|
|
#define LSTM_HIDDEN_MAX 128
|
|
#define BALANCED_COLLAPSE_PCT (100.0 / 3.0)
|
|
//--- How far above that floor a best-so-far checkpoint must sit before the regression handler is
|
|
//--- allowed to defend it during the pre-recall-pass phase. See the guard's comment in Train(): the
|
|
//--- point is to distinguish "the best we have is still basically a collapse, keep exploring freely"
|
|
//--- from "we found a real multi-class state and are now sliding off it", which is the case that ran
|
|
//--- unchecked for 228 eras on SP500 H1 (2026-07-29).
|
|
#define BALANCED_WORTH_DEFENDING_MARGIN_PCT 5.0
|
|
//--- PLATEAU LADDER ------------------------------------------------------------------------------
|
|
//--- The two branches above only fire on a NEW BEST (isBetterEra) or on a real REGRESSION
|
|
//--- (isWorseEra, a drop of more than ETA_DECAY_REGRESSION_PCT below the best). Between them sits a
|
|
//--- dead zone - "not better, not meaningfully worse" - where NOTHING happened: no checkpoint, no
|
|
//--- restore, no eta change. A run that settles into that band is stuck, and before this ladder
|
|
//--- existed it stayed stuck until the era cap (observed 2026-07-25: balanced accuracy pinned in a
|
|
//--- 64-69% band for 15 consecutive eras while eta sat frozen and the softmax outputs slowly
|
|
//--- compressed toward uniform, with ~970 eras still to burn before the cap would end it).
|
|
//---
|
|
//--- So: count eras since the last NEW BEST, and escalate when that count says the run has stopped
|
|
//--- improving on its own. Two ideas from the literature, in this order:
|
|
//--- 1. WARM RESTART (Loshchilov & Hutter, SGDR, ICLR 2017). On a plateau the correct move is a
|
|
//--- BIGGER step, not a smaller one - decaying eta into a plateau just entrenches whatever basin
|
|
//--- the model is sitting in. Note this is the opposite of the isWorseEra branch's decay, and
|
|
//--- deliberately so: decay answers overshoot, restart answers stagnation.
|
|
//--- 2. GAMMA ANNEALING (Mukhoti et al., "Calibrating Deep Neural Networks using Focal Loss",
|
|
//--- NeurIPS 2020, which schedules gamma DOWN over training rather than fixing it). Focal loss's
|
|
//--- (1-pt)^gamma modulator goes to ~0 on everything the model already classifies well, so late
|
|
//--- in a run the surviving gradient comes almost entirely from genuinely ambiguous bars - and
|
|
//--- near a pivot, ZigZag labels ARE ambiguous. Meanwhile WEIGHT_DECAY keeps pulling every
|
|
//--- weight toward zero on every step regardless (see AI\Network.mqh's note that a weight's
|
|
//--- sustainable magnitude is ~ its gradient SNR / WEIGHT_DECAY). Annealing gamma hands the
|
|
//--- easy-but-correct bars their gradient back, restoring the signal side of that ratio.
|
|
//--- Annealing is MONOTONE (gamma only ever decreases within a run), matching the scheduled-gamma
|
|
//--- literature; eta may still be bumped back up by the existing recovery bump.
|
|
//---
|
|
//--- Escalation is per-stage, every PLATEAU_PATIENCE_ERAS eras without a new best. ANY new best
|
|
//--- resets the counter and the stage to 0 (the ladder is a response to stagnation, so evidence the
|
|
//--- run is moving again retires it) - except the annealed gamma, which stays where it got to.
|
|
#define PLATEAU_PATIENCE_ERAS 8 // eras with no new best balanced accuracy before escalating a stage
|
|
#define PLATEAU_STAGE_RESTART 1 // warm-restart eta to its ceiling + first gamma step down
|
|
#define PLATEAU_STAGE_ANNEAL 2 // gamma to 0 (plain class-balanced CE) + one more warm restart
|
|
#define PLATEAU_STAGE_DEPLOY 3 // exhausted: deploy the best checkpoint and finish the run
|
|
//--- PLATEAU_GAMMA_STEP removed 2026-07-31 with focal loss - the ladder escape is the warm restart.
|
|
//--- FILE-COMPATIBILITY SHIM for the removed "Min OOS accuracy % (converge)" input (was MinWR, default
|
|
//--- PCT_80). Convergence is decided by the plateau ladder now, so the value has no behavioural effect
|
|
//--- anywhere - but it still occupies a slot in TWO persisted identities that must not shift:
|
|
//--- 1. the topology .cfg layout (SaveTopologyConfiguration/LoadAndCompareTopologyConfiguration), and
|
|
//--- 2. the weights-FILENAME fingerprint (BuildConfigFingerprint) - change the hash and every existing
|
|
//--- model silently becomes unreachable and retrains from era 0.
|
|
//--- So keep writing/hashing the old default, and stop COMPARING the .cfg field (see that function) so a
|
|
//--- model saved under ANY previous MinWR still loads. Models trained with the shipped default 80 keep
|
|
//--- their exact filename and resume normally; one trained under a non-default MinWR gets a new filename
|
|
//--- and retrains once, which is unavoidable when a fingerprint field stops being a variable.
|
|
#define LEGACY_CONVERGE_WR_SLOT 80
|
|
//--- Same treatment for the retired StudyPeriods input (removed 2026-07-30 - training now covers all
|
|
//--- available history). Its .cfg slot is positional and cannot be deleted without invalidating every
|
|
//--- deployed file, so a constant goes in and the field is no longer compared on load.
|
|
#define LEGACY_STUDY_PERIOD_SLOT 0
|
|
//--- Minimum true OOS samples a class needs this era before its recall is trusted as a real pass -
|
|
//--- see directionalRecallOK's declaration comment for the era-44-46 false-convergence this prevents.
|
|
//--- 10 is a low bar (still lets a genuinely thin early-run OOS window fall back to "not blocking"
|
|
//--- via recallPct==-1), just enough to rule out the zero/near-zero-sample degenerate case.
|
|
#define MIN_OOS_CLASS_SAMPLES_FOR_GATE 10
|
|
//--- Bound on FindConfirmedZigZagPivot()'s backward scan (m_useSwingContext feature block) - ADZigZag's
|
|
//--- own stock defaults (Depth=12) produce pivots frequently enough in normal conditions that this cap
|
|
//--- should rarely bind, but a long, unusually strong single-direction run could genuinely go this long
|
|
//--- without a confirmed opposite-type pivot. Generous rather than tight: the scan is cheap (plain array
|
|
//--- reads, no indicator recompute) and its result is cached per-bar by BufferTempData()'s feature cache,
|
|
//--- so the one-time cost of a long scan is paid at most once per unique bar, not per training pass.
|
|
#define SWING_SCAN_CAP_BARS 750
|
|
//--- EMA shadow-weight deployment blend rate - see m_shadowNet's declaration comment for the full
|
|
//--- rationale. 0.01 matches the Tau range (0.001-0.01) used for target-network soft updates in
|
|
//--- Dmitriy Gizlyk's reference NeuroNet_DNG-based RL algorithms (references\MQL5\Experts\*\Study.mq5) -
|
|
//--- small enough that no single era's raw weights can move the deployed model far, large enough that
|
|
//--- the shadow still tracks real, sustained learning within a few dozen eras rather than lagging
|
|
//--- forever.
|
|
#define SHADOW_WEIGHT_TAU 0.01
|
|
//--- Online continual-learning tunables (see OnlineLearnStep()). Live-chart-only: once a model is
|
|
//--- deployed (m_trainingComplete) it keeps adapting to newly-RESOLVED bars as their barrier outcome
|
|
//--- becomes known - the same supervised triple-barrier task it was trained on, never trade P&L
|
|
//--- (that would be RL).
|
|
//--- ONLINE_LEARN_MAX_CATCHUP caps how many newly-confirmed bars one step may backprop, so a weekend
|
|
//--- gap or a restart can't stall a tick with an unbounded catch-up loop (excess is picked up over the
|
|
//--- next few bars). ONLINE_ACC_SMOOTH is the guardrail EMA horizon (bars) for the rolling predict-
|
|
//--- before-learn accuracy - far shorter than CNet::recentAverageSmoothingFactor (10000) so the
|
|
//--- guardrail actually reacts within a realistic live sample count. The deployed shadow is only blended
|
|
//--- toward Net while that rolling accuracy holds at/above max(ONLINE_LEARN_MIN_ACC, deploy_baseline -
|
|
//--- ONLINE_LEARN_ACC_MARGIN); if it drops, Net keeps adapting (so it can recover) but the blend is
|
|
//--- FROZEN so drift can never reach live - the conservative "reject the deployment of a bad update"
|
|
//--- guardrail (no weight-snapshot/revert needed). Persist every ONLINE_LEARN_PERSIST_EVERY updates so
|
|
//--- a crash loses at most that many bars of adaptation.
|
|
//---
|
|
//--- CLASS IMBALANCE. This path streams the RAW live class distribution one bar at a time, so calling
|
|
//--- backProp() at the default sampleWeight of 1.0 makes it a majority-class drift vector by
|
|
//--- construction - it would pull a balanced, converged model back toward Neutral. The streaming-safe
|
|
//--- correction is COST-level: alpha-balanced focal loss (Lin et al. 2017, "Focal Loss for Dense Object
|
|
//--- Detection", eq. 5), weight = alpha_c * (1 - p_t)^gamma, a SINGLE loss designed for exactly this
|
|
//--- ratio. alpha_c is inverse class frequency from the persisted priors (m_priorBuy/Sell/Neutral),
|
|
//--- majority normalised to 1.0, scaled by ONLINE_LEARN_PARITY and capped; gamma is
|
|
//--- ONLINE_LEARN_FOCAL_GAMMA. ONLINE_LEARN_MAX_CLASS_WEIGHT is the uncapped ceiling,
|
|
//--- ONLINE_LEARN_ALPHA_CAP the tighter one actually applied.
|
|
//--- The two engines deliberately use DIFFERENT corrections. Train() corrects analytically in the
|
|
//--- gradient via the logit-adjusted loss, which is NOT available here: ApplyLogitAdjustment() only
|
|
//--- runs inside a training run, so a deployed-then-reloaded model carries no offsets and would
|
|
//--- otherwise stream skewed data in with no correction at all. These were shared inputs until
|
|
//--- 2026-07-31 and are now constants at those inputs' shipped defaults - behaviour is unchanged.
|
|
//--- ONLINE_LEARN_ETA_SCALE: this path previously inherited whatever value the GLOBAL `eta` happened to
|
|
//--- be left at by the last Train() chunk of ANY model instance (PAI/CONV/LSTM/HYBRID share it), which
|
|
//--- is arbitrary. eta is now pinned to this model's own converged rate scaled down for the duration of
|
|
//--- the loop, then restored, so a live adaptation step is deliberately gentler than a training step.
|
|
#define ONLINE_LEARN_MAX_CATCHUP 64
|
|
#define ONLINE_ACC_SMOOTH 50.0
|
|
#define ONLINE_LEARN_WARMUP 20
|
|
#define ONLINE_LEARN_MIN_ACC 40.0
|
|
#define ONLINE_LEARN_ACC_MARGIN 10.0
|
|
#define ONLINE_LEARN_PERSIST_EVERY 32
|
|
#define ONLINE_LEARN_MAX_CLASS_WEIGHT 5.0
|
|
//--- Pinned to the shipped defaults of the removed OversampleParity (90%), ConstrainReplay (true ->
|
|
//--- cap 3.0) and FocalLossGamma (1.0) inputs - see the CLASS IMBALANCE note above.
|
|
#define ONLINE_LEARN_PARITY 0.9
|
|
#define ONLINE_LEARN_ALPHA_CAP 3.0
|
|
#define ONLINE_LEARN_FOCAL_GAMMA 1.0
|
|
#define ONLINE_LEARN_ETA_SCALE 0.25
|
|
//--- Free function, not a class method: needed by CExpertSignalAIBase's constructor init list for
|
|
//--- both m_modelEta and m_etaCeiling (see their declaration comments), which runs before member-init
|
|
//--- order could safely make one depend on another - this only touches the TrainingOptimizer input
|
|
//--- (Variables\Inputs.mqh) and the SgdLearningRate/AdamLearningRate inputs (AI\Network.mqh's `lr`
|
|
//--- macro resolves to AdamLearningRate), never class members.
|
|
double InitialEtaForOptimizer(void)
|
|
{
|
|
return (TrainingOptimizer == SGD) ? SgdLearningRate : lr;
|
|
}
|
|
class CExpertSignalAIBase : public CExpertSignalCustom
|
|
{
|
|
protected:
|
|
string ID;
|
|
//--- ID with the bracketed config tag stripped - "Hybrid 3L [HYB-9369]" -> "Hybrid 3L". The tag is a
|
|
//--- topology prefix plus a fingerprint hash: it exists to tell one CHART's model files from another's
|
|
//--- when reading the journal or the State\ folders, which is a developer's problem, not an owner's.
|
|
//--- Logs and the VerboseMode panels keep the full ID; the plain-language panels use this. Strips from
|
|
//--- the LAST " [" so a model name containing a bracket could not truncate the whole label.
|
|
string DisplayName(void) const
|
|
{
|
|
int cut = StringFind(ID, " [");
|
|
int next = cut;
|
|
while(next >= 0)
|
|
{
|
|
cut = next;
|
|
next = StringFind(ID, " [", cut + 1);
|
|
}
|
|
return (cut >= 0 ? StringSubstr(ID, 0, cut) : ID);
|
|
}
|
|
CiOpen m_Open;
|
|
CiClose m_Close;
|
|
CiHigh m_High;
|
|
CiLow m_Low;
|
|
CiVolumes m_Volumes;
|
|
CiTime m_Time;
|
|
//--- optional classic-indicator input features (m_useMA/m_useRSI/m_useMACD/m_useIchimoku below) -
|
|
//--- independent instances from the ones Signals\SignalMA.mqh/SignalRSI.mqh/SignalMACD.mqh/
|
|
//--- SignalIchimoku.mqh use for trade voting, since this class and those CSignal* classes are
|
|
//--- unrelated hierarchies (feature engineering vs. signal vote). The MA
|
|
//--- feature now uses the SAME unified indicator as the classic vote (CustomIndicators\ADMovingAverage.mq5)
|
|
//--- via CiCustom, so its type/period are tunable (see ADIndicatorTuner maType/maPeriod).
|
|
CiCustom m_MA;
|
|
CiRSI m_RSI;
|
|
//--- built-in Ci* wrappers, not CiCustom: MACD is defined on plain EMAs and Ichimoku on plain
|
|
//--- highest/lowest midpoints, so there is no unified AD* custom indicator to route them through the
|
|
//--- way m_MA goes through ADMovingAverage. Periods come from the tuner (macdFast/macdSlow/
|
|
//--- macdSignal, ichiTenkan/ichiKijun/ichiSenkou).
|
|
CiMACD m_MACDFeature;
|
|
CiIchimoku m_Ichimoku;
|
|
//--- custom price-action/volume indicators (CustomIndicators\*.mq5), loaded via iCustom/CiCustom
|
|
CiCustom m_ADCumulativeDelta;
|
|
CiCustom m_ADShorteningOfThrust;
|
|
CiCustom m_ADWyckoffEventStream;
|
|
CiCustom m_ADWyckoffFailedStructure;
|
|
CiCustom m_ADWyckoffSignificantBarInversion;
|
|
//--- ground truth for the training labels (see m_swingConfirmationBars' declaration comment) -
|
|
//--- CustomIndicators\ADZigZag.mq5, a renamed/rebranded copy of the stock MQL5 ZigZag indicator,
|
|
//--- always created (not gated behind an Enable* input like the AD* feature indicators above,
|
|
//--- since it isn't an optional input feature - it IS the label) and always run at its own stock
|
|
//--- defaults (Depth=12, Deviation=5, Backstep=3) - never touched by AutoTuneIndicators, since
|
|
//--- tuning the ground truth itself alongside the model being scored against it would let a trial
|
|
//--- "improve" its OOS score by cherry-picking an easier target rather than a better model.
|
|
CiCustom m_ADZigZag;
|
|
//--- currently-active tunable input values for each AD indicator (AutoTuneIndicators search space)
|
|
//--- plus their own flatten/unflatten/perturb/best-tracking logic - see CADIndicatorTuner's
|
|
//--- declaration comment (Expert\ADIndicatorTuner.mqh) for why this is a separate collaborator
|
|
//--- rather than loose fields/methods here: it's entirely self-contained (never touches Net,
|
|
//--- Train()'s state machine, or anything else in this class), unlike TuneIndicatorsAndTrain()
|
|
//--- below, which orchestrates Train()/Net/checkpointing around it and stays here for exactly that
|
|
//--- reason. InpContextMode/InpSessionType/InpSessionCount are session choices, not accuracy
|
|
//--- knobs, and are hardcoded to the indicators' own defaults in InitAD*() rather than stored/
|
|
//--- tuned in the collaborator.
|
|
CADIndicatorTuner m_indicatorTuner;
|
|
bool m_autoTuneIndicators;
|
|
//--- rebuilds only the AD* indicator handles (in place) so ReInit picks up updated param structs
|
|
bool ReInitADIndicators(CIndicators *indicators);
|
|
//--- builds a fresh, untrained topology into Net (assumes Net is currently NULL); factored out of
|
|
//--- InitNeuralNetwork() so TuneIndicatorsAndTrain() can rebuild weights per trial without
|
|
//--- re-running indicator init (which would re-Add() the base indicators into indicators a 2nd time)
|
|
bool BuildFreshTopology();
|
|
//--- CIndicators collection passed into InitNeuralNetwork(), retained so
|
|
//--- TuneIndicatorsAndTrain() can call ReInitADIndicators() between trials
|
|
CIndicators *m_indicatorsPtr;
|
|
CNet *Net;
|
|
//--- EMA "shadow" copy of Net, blended a small step (SHADOW_WEIGHT_TAU) toward Net's weights at
|
|
//--- the end of every era (see Train()'s era-end block) rather than replaced outright. Live
|
|
//--- trading/inference (RefreshLatestSignal()) reads from this, not from Net directly, so any
|
|
//--- single era's raw weights - including an Adam overshoot event - can only ever nudge what's
|
|
//--- actually deployed by SHADOW_WEIGHT_TAU, not overwrite it wholesale. Bootstrapped as a clone of
|
|
//--- Net (via the same Save()/Load() pattern m_simOosNet uses) the first time InitNeuralNetwork()
|
|
//--- runs with no prior shadow file, and persisted alongside Net.Save() thereafter. NULL only
|
|
//--- during the brief window before that first bootstrap completes - every read site falls back to
|
|
//--- Net in that case, never blocks on it.
|
|
CNet *m_shadowNet;
|
|
//--- One-shot latch for EnsureShadowNet()'s clone bootstrap. Cloning a second full net can fail on the
|
|
//--- tester's CPU-DLL compute fallback (the chart's GPU/DirectML path succeeds); without this the
|
|
//--- bootstrap would retry on EVERY bar, re-initialising the compute backend each time - log spam and a
|
|
//--- multi-second-per-bar crawl through an inference-only backtest. A failed/skipped bootstrap is
|
|
//--- harmless: every read site falls back to Net, and a bootstrapped shadow is identical to Net until
|
|
//--- era-end blending diverges it (which never happens in a no-training run). Reset with m_shadowNet.
|
|
bool m_shadowBootstrapAttempted;
|
|
//--- Online continual-learning state (see OnlineLearnStep(); tunables at ONLINE_LEARN_* above). Live-
|
|
//--- chart-only - a deployed (m_trainingComplete) model keeps learning from newly-confirmed ZigZag
|
|
//--- structure as bars mature, waiting the full m_swingConfirmationBars confirmation delay just like
|
|
//--- training so a still-provisional (repainting) recent bar is NEVER backpropped. m_enableOnlineLearning
|
|
//--- is the input gate. m_onlineLearnedUpToTime is the bar-TIME watermark of the newest bar already
|
|
//--- learned from (indexed by time, not now-relative index, so it survives the per-bar index-frame
|
|
//--- shift); persisted in the .stats sidecar (WST3). m_onlineRollingAcc is the guardrail EMA (0-100)
|
|
//--- of predict-before-learn hits (seeded from the deploy OOS baseline), m_onlineSamples the cumulative
|
|
//--- update count (both persisted). m_onlineBarsSincePersist drives periodic saves (in-memory only).
|
|
bool m_enableOnlineLearning;
|
|
datetime m_onlineLearnedUpToTime;
|
|
double m_onlineRollingAcc;
|
|
long m_onlineSamples;
|
|
int m_onlineBarsSincePersist;
|
|
//--- Latched log state so the guardrail freeze/resume transition prints once per flip, not per bar.
|
|
bool m_onlineBlendFrozen;
|
|
CArrayDouble *TempData;
|
|
double dError;
|
|
double dUndefine;
|
|
double dForecast;
|
|
double dPrevSignal;
|
|
//--- ALTERNATION GATE REMOVED 2026-08-01 with the triple-barrier relabel. m_lastNonNeutralSignal
|
|
//--- suppressed any live Buy following another Buy with no Sell between. That was CORRECT under the
|
|
//--- exact-pivot target (a ZigZag can only emit a new pivot by FLIPPING type, so a repeat was
|
|
//--- provably a false fire) and the premise died with the target: a triple-barrier label is answered
|
|
//--- independently at every bar, so ten consecutive Buy setups inside one trend are simply correct.
|
|
//--- Do not reinstate it. It also meant a one-sided (`Sell:0%`) model got ONE trade per backtest,
|
|
//--- because the awaited opposite signal that reopens the gate never came.
|
|
//--- Inference-path census counters - see PrintInferenceTally() for why these exist. Cheap enough to
|
|
//--- keep unconditionally: three increments per bar against a full feedForward.
|
|
long m_refreshOk;
|
|
long m_refreshFailFeatures;
|
|
long m_refreshFailShort;
|
|
long m_refreshBuy;
|
|
long m_refreshSell;
|
|
long m_refreshNeutral;
|
|
//--- VOTE-GATE census. RefreshLatestSignal() can answer Buy on hundreds of bars while
|
|
//--- LongCondition()/ShortCondition() still return 0 on every one, because those open with a
|
|
//--- readiness gate the refresh path never consults:
|
|
//--- if(!m_trainingComplete && !(m_inferenceOnly && m_modelLoadedFromDisk)) return 0;
|
|
//--- In the tester that reduces to "the seeded _optcache.nnw must have LOADED"; when it has not, the
|
|
//--- run silently produces direction 0.00 on every bar. Without these two the census reads as "the
|
|
//--- model answers Neutral" - false, and it points at a completely different fix. They separate the
|
|
//--- model's ANSWER from whether that answer was allowed to become a vote.
|
|
long m_voteGateBlocked; // directional decisions the readiness gate discarded
|
|
long m_voteGatePassed; // directional decisions that became a real vote
|
|
//--- Flag pair as of the first vote attempt, latched so the tally can name WHICH half of the gate
|
|
//--- failed rather than just reporting that it did. -1 = no vote was ever attempted.
|
|
int m_voteGateCompleteAtFirst;
|
|
int m_voteGateLoadedAtFirst;
|
|
//--- m_gateSnapshot removed with the alternation gate above - it existed only to roll that gate back
|
|
//--- when the parent discarded this filter's vote. The AI signal now holds no one-shot vote state, so
|
|
//--- BeginVote()/RevokeVote() fall back to the base class's empty implementations.
|
|
//--- Non-max suppression (NMS) of directional signals. Consecutive H4 bars near a real turn share
|
|
//--- almost their entire feature vector, so a single reversal fires the same class on a whole run of
|
|
//--- adjacent bars - a visual/emission cluster around one event, not several distinct turns.
|
|
//--- DEFAULT OFF since 2026-08-01: suppressing repeats was justified by the exact-pivot target, and
|
|
//--- under triple-barrier labels a run of same-direction setups is real signal, so collapsing it
|
|
//--- DISCARDS trades rather than de-duplicating them. Kept opt-in for one marker per move.
|
|
//--- Keeps only the FIRST (earliest) bar of a same-direction run, dropping same-direction neighbours
|
|
//--- within m_signalClusterWindow bars; 0 disables. Per-direction, so a missed opposite signal never
|
|
//--- blocks a later reversal (unlike the alternation gate), and causal, so the live signal and the
|
|
//--- drawn history declutter identically. Post-processing ON TOP of the model - the raw per-bar
|
|
//--- recall/precision/accuracy stats stay un-NMS'd so they keep measuring the network itself.
|
|
//--- The cluster extends from the last SEEN same-direction bar, not the last KEPT one: measuring
|
|
//--- from the kept bar re-emitted an arrow every window+1 bars inside a long run.
|
|
int m_signalClusterWindow;
|
|
//--- Per-bar predicted signed signal for THIS era (index = now-relative bar index; -2 = not scored
|
|
//--- this era). When NMS is on, the scan passes (1/2/3) ONLY record into this cache and draw nothing;
|
|
//--- PruneDirectionalClusters() then renders the whole declustered set once at era end. That is what
|
|
//--- keeps the chart from ever showing the raw mid-era clusters (the passes would otherwise draw every
|
|
//--- above-threshold bar as it scanned, and the sweep only cleaned up at era end). Recording (not
|
|
//--- drawing) also decouples NMS from pass 2's SHUFFLED order, which no inline cursor could dedup.
|
|
//--- Sized to bar count each fresh era.
|
|
double m_arrowSignalCache[];
|
|
//--- Live-side NMS state: bar TIME of the last SEEN signal per direction (advances on every same-
|
|
//--- direction bar, kept or suppressed, so a contiguous live run collapses to one) plus the cached
|
|
//--- accept/suppress decision for that exact bar (keeps repeated same-bar RefreshLatestSignal calls
|
|
//--- idempotent - re-evaluating the same bar returns its first decision, not a flipped one). 0 = none.
|
|
datetime m_nmsLiveBuyTime;
|
|
datetime m_nmsLiveSellTime;
|
|
bool m_nmsLiveBuyAccept;
|
|
bool m_nmsLiveSellAccept;
|
|
//--- Last KEPT live signal of either direction, for cross-direction resolution: a Buy and a Sell
|
|
//--- within m_signalClusterWindow bars are flicker at one turn zone (real opposite pivots are a whole
|
|
//--- leg apart), so only the higher-confidence side is kept. Confidence = |signed signal| = winning
|
|
//--- softmax probability. Same rule runs in PruneDirectionalClusters() for the historical chart.
|
|
datetime m_nmsLiveKeptTime;
|
|
ENUM_SIGNAL m_nmsLiveKeptDir;
|
|
double m_nmsLiveKeptConf;
|
|
datetime dtStudied;
|
|
long m_eraCount; // cumulative era counter, persisted in the .nnw so restarts don't look like they reset progress
|
|
bool m_trainingComplete; // persisted: true only once Train() converged (objective+stability), not just interrupted
|
|
bool bEventStudy;
|
|
//--- out-of-sample holdout: share (%) of the study period never trained on, used only to
|
|
//--- measure genuine forward accuracy so overfitting shows up in the stats, not just live/OOS trading
|
|
int m_oosSplitPct;
|
|
double dOosError; // smoothed OOS mismatch rate (0..100), lower is better
|
|
double dOosForecast; // smoothed OOS accuracy (0..100)
|
|
int m_oosSamples; // count of OOS predictions evaluated this Train() call
|
|
//--- per-era raw (pre-softmax) output-neuron stats over pass 3's OOS scan, reset at pass 3
|
|
//--- start; surfaced in the era-end log line. The per-bar logit spread (max-min across the 3
|
|
//--- outputs) is the collapse fingerprint: a healthy net differentiates bars (avg spread well
|
|
//--- above 0), a saturated all-one-class net pins all three outputs to the same value on every
|
|
//--- bar (avg spread ~0, mins/maxes flat) - see CLASS_LOGIT_SCALE's comment (AI\Network.mqh).
|
|
double m_oosOutMin[3];
|
|
double m_oosOutMax[3];
|
|
double m_oosOutSpreadSum;
|
|
int m_oosOutCount;
|
|
//--- per-era counts of the network's own classification of each bar it fed forward (IS+OOS),
|
|
//--- reset at the start of every era; surfaced in the status label text so class imbalance
|
|
//--- (e.g. the network collapsing to all-Neutral) is visible while training runs
|
|
int m_countBuySignals;
|
|
int m_countSellSignals;
|
|
int m_countNeutralSignals;
|
|
//--- per-era counts of the *true* label of every bar fed forward (IS+OOS), reset alongside the
|
|
//--- predicted counts above. Used both to display the actual class distribution in the status
|
|
//--- label text and, more importantly, to drive IS-sample oversampling in Train() (see backProp calls
|
|
//--- below) - rarer classes get replayed more times so gradient descent doesn't just learn to
|
|
//--- always call the majority (usually Neutral) class.
|
|
int m_trueBuyCount;
|
|
int m_trueSellCount;
|
|
int m_trueNeutralCount;
|
|
//--- snapshot of the class totals above, taken at the end of the PREVIOUS era (see Train()'s
|
|
//--- era-reset block) and held fixed for the whole of the current era. The oversampling ratio
|
|
//--- below is computed from these frozen counts rather than the live, still-accumulating
|
|
//--- m_true*Count values - using the live counts made the ratio order-dependent within a single
|
|
//--- era (chronological, oldest-to-newest bar processing means whichever class happens to be
|
|
//--- numerically behind at any given moment gets amplified up to 5x, even if that's just a
|
|
//--- transient artifact of which regime the era's early bars came from, not the class's true
|
|
//--- overall rarity) - a real source of run-to-run oscillation in the predicted class mix. All
|
|
//--- zero on era 0, when oversampling simply falls back to 1x (see the reps calc in Train()).
|
|
int m_prevEraTrueBuyCount;
|
|
int m_prevEraTrueSellCount;
|
|
int m_prevEraTrueNeutralCount;
|
|
//--- per-era OOS confusion counts, reset each era; used to compute per-class OOS recall (hits/total)
|
|
//--- for the status label text and, more importantly, as an additional convergence gate alongside the
|
|
//--- blended dOosForecast accuracy - a model that "wins" only by calling everything Neutral will
|
|
//--- have high dOosForecast but near-zero Buy/Sell recall, and should NOT be allowed to converge.
|
|
int m_oosBuyHits, m_oosBuyTotal;
|
|
int m_oosSellHits, m_oosSellTotal;
|
|
int m_oosNeutralHits, m_oosNeutralTotal;
|
|
//--- same per-era OOS confusion counts as above but keyed by PREDICTED class instead of true class,
|
|
//--- i.e. per-class precision (of the bars this era where the model called Sell, how many actually
|
|
//--- were Sell?) rather than recall (of the bars that actually were Sell, how many did it catch?).
|
|
//--- Recall alone can't distinguish "the model over-fires Sell and happens to also catch enough real
|
|
//--- Buys/Neutrals to clear their recall floors" from "the model is genuinely well-calibrated" - a
|
|
//--- skewed Predicted-this-era count (see m_countSellSignals) with recall still passing the gate is
|
|
//--- exactly that failure mode, and precision is what would expose it.
|
|
int m_oosBuyPredicted, m_oosBuyPredictedHits;
|
|
int m_oosSellPredicted, m_oosSellPredictedHits;
|
|
int m_oosNeutralPredicted, m_oosNeutralPredictedHits;
|
|
//--- Confidence calibration: the classification head's forward pass still uses SIGMOID per-neuron
|
|
//--- (see BuildFreshTopology()'s SIGMOID comment - bounded activation, avoids logit runaway), but
|
|
//--- the BACKWARD pass (CNet::backProp/backPropOCL in AI\Network.mqh) now trains all 3 neurons
|
|
//--- jointly against a true softmax+categorical-cross-entropy gradient (softmax_i - target_i), not
|
|
//--- 3 independent binary-cross-entropy targets. ApplyClassificationSoftmax() at READ time
|
|
//--- reproduces the same normalization the loss was actually trained against, so the value it
|
|
//--- returns is now a genuinely loss-consistent probability (still not literature-perfect
|
|
//--- calibration - no temperature scaling/Platt scaling has been applied - but no longer
|
|
//--- structurally decoupled from what was optimized). SignedAIConfidence()/g_AISignedConfidence
|
|
//--- exposes it, and Money\MoneyIntelligent.mqh:AdjustRiskAmount() scales position-sizing risk%
|
|
//--- directly off it.
|
|
//--- m_oosConfidenceSum accumulates MathAbs(dPrevSignal) (the claimed confidence) over every
|
|
//--- classified OOS bar this era; compared against this era's actual OOS accuracy
|
|
//--- ((m_oosBuyHits+m_oosSellHits+m_oosNeutralHits)/m_oosSamples) at era-end to derive
|
|
//--- m_confidenceCalScale - a single empirical multiplier ("the model claims 80% on average but is
|
|
//--- only right 60% of the time -> scale reported confidence by 0.75") applied to the MAGNITUDE
|
|
//--- only, never the sign/class decision, in SignedAIConfidence(). EMA-blended across eras (same
|
|
//--- smoothing factor as dOosForecast) so one noisy era can't swing it, and clamped to [0.3, 1.5]
|
|
//--- so a thin/degenerate OOS window can't drive it to something absurd. Starts at 1.0 (no
|
|
//--- correction) until the first era with OOS samples computes a real value.
|
|
double m_oosConfidenceSum;
|
|
double m_confidenceCalScale;
|
|
//--- minimum acceptable OOS recall (%) for the Buy and Sell classes individually before Train() is
|
|
//--- allowed to declare convergence; a class with zero OOS samples this era doesn't block (avoids a
|
|
//--- deadlock when a given era's OOS window happens to contain no examples of that class)
|
|
int m_minDirectionalRecallPct;
|
|
//--- There is deliberately NO minimum-confidence input here any more, and no member holding one.
|
|
//--- Confidence is expressed through the VOTE WEIGHT (ConfidenceTier() -> PatternWeightForTier()),
|
|
//--- not through a separate entry floor, so a weak call is not blocked at the AI's own boundary - it
|
|
//--- votes weakly and is then filtered by the same Min vote to open threshold that filters a weak
|
|
//--- classic vote. That is what makes ONE input genuinely govern both engines.
|
|
//--- The floor it replaced was a scale mismatch: Min_Vote_Open fed it directly as a 0..1 probability
|
|
//--- while ALSO being the 0..100 averaged-vote threshold, and a 3-class argmax winner is
|
|
//--- arithmetically >= 1/3, so every setting from 0 to 33 gated precisely nothing while every setting
|
|
//--- above that silently re-quartiled the confidence tiers as a side effect. See ConfidenceTier().
|
|
//--- Stops the per-era EMA update of the measured class priors after the first real measurement, so
|
|
//--- the tau*log(prior) offsets stay pinned to the distribution the run started from.
|
|
bool m_freezePriorCalibration;
|
|
//--- THE class-imbalance correction: tau in Menon et al. 2021's logit adjustment. tau*log(prior_c) is
|
|
//--- added to each class logit in the TRAINING gradient only, so the network absorbs the offset and
|
|
//--- its RAW argmax is already balanced-error-optimal at inference - no second correction at read
|
|
//--- time. 0 disables it. This is the sole survivor of the nine-input imbalance section audited away
|
|
//--- on 2026-07-31 (see Variables\Inputs.mqh); it is the only one of them with a consistency
|
|
//--- guarantee, and it is consistent for exactly the balanced-error metric checkpoint selection
|
|
//--- already ranks on. m_logitAdjustLogged keeps the once-per-run tau/cap report to one line.
|
|
double m_logitAdjustTau;
|
|
bool m_logitAdjustLogged;
|
|
//--- Latch for the counterpart warning: the correction DECLINING to install. See ApplyLogitAdjustment().
|
|
bool m_logitAdjustSkipWarned;
|
|
|
|
//--- True class base rates (natural, un-oversampled), measured from the label distribution each era
|
|
//--- (UpdateClassPriors, EMA-blended for stability) and PERSISTED alongside the weights (.stats
|
|
//--- sidecar) so live inference - including after a restart, when no training re-runs - calibrates
|
|
//--- exactly as training did. 0 = not yet measured => AdjustedSignalFromSoftmax falls back to raw.
|
|
double m_priorBuy, m_priorSell, m_priorNeutral;
|
|
//--- Per-era OOS "fired" confusion counts under the LIVE decision rule: a directional call is counted
|
|
//--- whenever the prior-corrected posterior is non-Neutral, i.e. exactly the bars on which the
|
|
//--- deployed EA would cast a directional vote. m_oosBuyFiredHits/m_oosBuyFired = live Buy precision;
|
|
//--- likewise Sell. This is the metric that predicts forward-trading performance (recall/argmax-
|
|
//--- precision above score the RAW argmax, before the base-rate correction the live decision applies).
|
|
//--- Reset each era. Whether a counted vote also clears Min_Vote_Open against the other filters'
|
|
//--- average is an aggregate question this per-bar scorer cannot see - see its use site.
|
|
int m_oosBuyFired, m_oosBuyFiredHits;
|
|
//--- The same live-fired population as above, but BUCKETED BY CONFIDENCE TIER (ConfidenceTier(), 4
|
|
//--- buckets quartiled from the head's structural floor). Exists to answer the one question the
|
|
//--- aggregate precision cannot: whether raising Min_Vote_Open would actually buy precision, and how
|
|
//--- much coverage it would cost. Tier weights are 25/50/75/100, and for an AI-only configuration the
|
|
//--- averaged vote IS the tier weight, so these four rows map directly onto the input: a floor of 50
|
|
//--- keeps tiers 1-3, 75 keeps 2-3, 100 keeps tier 3 alone. Measuring it beats guessing at it - the
|
|
//--- floor is only worth raising if precision actually rises monotonically across the tiers, and if it
|
|
//--- does not, that is itself the finding (the model's confidence is not calibrated to correctness).
|
|
int m_oosTierFired[4], m_oosTierHits[4];
|
|
int m_oosSellFired, m_oosSellFiredHits;
|
|
//--- Cumulative (compounded, persistent) DIRECTIONAL accuracy = the win-rate of the model's Buy/Sell
|
|
//--- calls: of the bars it actually called Buy or Sell, how many matched the true label. Neutral ("no
|
|
//--- trade") calls are deliberately EXCLUDED - counting them inflates the rate to ~80%+ (Neutral is the
|
|
//--- ~94% majority the model gets right for free) and tells you nothing about trade quality. Summed
|
|
//--- across every era AND carried across restarts via the .stats sidecar (WST5); never reset era-to-era,
|
|
//--- so the panel shows a stable win-rate that only firms up as more pivots confirm. Incremented at the
|
|
//--- IS (pass 2) and OOS (pass 3) hit sites, only when the PREDICTION is directional, skipped for
|
|
//--- Reset only by ResetWeights (a fresh model).
|
|
long m_cumIsCorrect, m_cumIsTotal;
|
|
long m_cumOosCorrect, m_cumOosTotal;
|
|
//--- Latest live-fired precision (%) and fire count per direction (-1 = n/a), cached at era end for
|
|
//--- the status panel/log the same way m_lastBuyRecallPct is (see its comment).
|
|
int m_lastBuyFiredPrecPct, m_lastSellFiredPrecPct;
|
|
int m_lastBuyFired, m_lastSellFired;
|
|
//--- Ceiling on the class-balance sampleWeight multiplier (see its computation in Train(), keyed off
|
|
//--- m_prevEraTrueBuyCount/Sell/Neutral). MAX_WEIGHT_DELTA (AI\Network.mqh/.cl, DirectML\WarriorCPU.cpp)
|
|
//--- bounds how far a SINGLE weight can move in one step, but it does nothing to stop many small
|
|
//--- steps in the same direction from accumulating into a large net swing across an era - with real
|
|
//--- label counts around Buy 1340 / Sell 1312 / Neutral 7013 the uncapped ratio is ~5.3x on every
|
|
//--- single Buy/Sell example, every era. At the old hardcoded 3.0x ceiling that was STILL strong
|
|
//--- enough in practice to let one era's Buy gradients overwrite the separability the previous era
|
|
//--- had just built for Sell (and vice versa) - the observed anti-correlated Buy/Sell recall whipsaw
|
|
//--- (e.g. era 8 Buy:0%/Sell:26% -> era 9 Buy:10%/Sell:57% -> era 10 Buy:1%/Sell:37%), never both
|
|
//--- classes improving together. Was tunable (ClassSampleWeight input, CSW_15/1.5x default) as a
|
|
//--- loss-level multiplier, but that mechanism proved structurally too weak against Adam's near-
|
|
//--- invariance to constant gradient rescaling (Kingma & Ba 2015) - see the m_isTrainQueue queueing
|
|
//--- block's oversampling comment for the full history. Class-balance correction now happens
|
|
//--- entirely via data-level oversampling (repCount in that queueing block); this member is
|
|
//--- currently unread by that path. The ClassSampleWeight INPUT that used to set it was removed
|
|
//--- (it had no effect); the member is kept at its constructor default in case a smaller, additive
|
|
//--- loss-level nudge is ever reintroduced on top of oversampling.
|
|
double m_maxClassSampleWeight;
|
|
//--- FOCAL LOSS REMOVED 2026-07-31 (was m_focalGamma / the FocalLossGamma input). Lin et al. 2017's
|
|
//--- (1-pt)^gamma is sound, but it corrects the SAME axis as the logit-adjusted loss, and stacking
|
|
//--- two corrections on one axis is the failure Buda et al. 2018 warns about (cited in the queueing
|
|
//--- block). It was also running at gamma*0.125, damped by a replay flag whose path was already dead.
|
|
//--- The ladder's escape is the learning-rate warm restart; the gamma anneal was a monotone step to
|
|
//--- zero, so nothing went with it. Full nine-input audit: the class-imbalance block in Inputs.mqh.
|
|
//--- ZigZag repainting embargo, in bars. The real MQL5 ZigZag (CustomIndicators\ADZigZag.mq5 - a
|
|
//--- renamed, logic-untouched copy of the stock ZigZag.mq5 at its stock defaults: Depth=12,
|
|
//--- Deviation=5 points, Backstep=3) revises its most recent 1-3 legs as new bars arrive (see
|
|
//--- ADZigZag.mq5's own ExtRecalc), so a bar's ADZigZagBuffer value is only trusted once at least
|
|
//--- this many MORE bars have closed after it.
|
|
//--- SCOPE NARROWED 2026-08-01: this used to gate the training LABELS as well, back when the target
|
|
//--- was the exact confirmed pivot. The target is now the triple barrier, whose lookahead is its own
|
|
//--- horizon (m_barrierHorizonBars), so this constant survives for exactly one job - keeping the
|
|
//--- swing-context INPUT FEATURES (m_useSwingContext) from reading a leg that can still change.
|
|
int m_swingConfirmationBars;
|
|
//--- Vertical barrier of the triple-barrier label, in bars - see BARRIER_HORIZON_LADDER_COUNT. Derived
|
|
//--- once by ComputeBarrierHorizonBars() at the start of the label prebuild and then held for the run.
|
|
//--- It is ALSO the label's lookahead depth, so it is what the IS/OOS embargo and the online-learning
|
|
//--- confirmation delay must both wait out: a bar's barrier label is not knowable until this many more
|
|
//--- bars have closed after it. That job used to belong to m_swingConfirmationBars, which answered the
|
|
//--- ZigZag question ("has this leg stopped repainting") and no longer answers the label's.
|
|
int m_barrierHorizonBars;
|
|
//--- Latch so the invalid-TP fallback in BarrierMultiples() shouts once, not once per labelled bar.
|
|
bool m_barrierFallbackWarned;
|
|
//--- Did the LAST TripleBarrierLabel() call run out of horizon without either barrier being touched?
|
|
//--- Neutral conflates two very different outcomes - "the trade timed out" and "the stop was hit
|
|
//--- before the target" - and only the first one indicts the horizon. Counting them apart is what
|
|
//--- lets the m*k horizon scaling in ComputeBarrierHorizonBars() be VERIFIED against a real run
|
|
//--- rather than trusted: a high timeout share means the horizon is too short for the configured
|
|
//--- barrier, a high stop-out share just means the barrier is hard.
|
|
bool m_lastBarrierTimedOut;
|
|
//--- Excursions of the bar TripleBarrierLabel() just resolved, in ATR units, published the same way
|
|
//--- m_lastBarrierTimedOut is: the walk that finds them is the walk the label already does, so they
|
|
//--- cost one max and one min per bar rather than a second pass over history.
|
|
double m_lastExcUp; // (maxHigh - entry)/ATR over the horizon, >= 0
|
|
double m_lastExcDown; // (entry - minLow)/ATR, >= 0
|
|
//--- DERIVED barrier multiples, in ATR units, taken from the measured excursion distribution rather
|
|
//--- than from an enum. Zero means "not derived yet" and BarrierMultiples() falls back to the mode
|
|
//--- constants. Continuous on purpose: the whole point is to stop snapping the geometry to {2,3} x
|
|
//--- {2,3,4,6,8,10}, a grid whose members were guesses.
|
|
double m_derivedSlMult;
|
|
double m_derivedTpMult;
|
|
bool m_geometryDerived;
|
|
int m_geometryDerivePasses; // fixed-point iteration counter, capped
|
|
//--- Median confirmed ZigZag leg in bars, UNSCALED by the barrier. The window excursions are measured
|
|
//--- over, kept independent of the geometry so sizing the geometry from them cannot feed back.
|
|
int m_swingMedianBars;
|
|
int m_labelPrebuildTimeoutCount;
|
|
//--- safety valve: Train()'s do-while loop has no other bound on how many eras it will run
|
|
//--- before giving up, so a config that can't reach the convergence objective (e.g. too few
|
|
//--- swing-confirmed examples for the min recall bar to be reachable) would otherwise loop
|
|
//--- forever, permanently keeping the era-progress status label up instead of the normal per-tick
|
|
//--- info line and burning CPU nonstop. When the cap is hit, the operator is prompted (see
|
|
//--- PromptContinuePastEraCap): CONTINUE resets the era counter and keeps training; STOP deploys
|
|
//--- the best checkpoint found so far (FinalizeTrainRun) and terminates training. Headless
|
|
//--- (tester/optimizer) runs can't prompt, so they take the STOP branch automatically.
|
|
int m_maxErasPerRun;
|
|
//--- Train() runs its per-bar loop synchronously, and MQL5 is single-threaded per chart - a
|
|
//--- multi-minute era would otherwise starve the terminal's chart-event queue for that whole
|
|
//--- stretch, including the control panel's own click/drag hit-testing (Panel\ControlPanel.mqh),
|
|
//--- which depends entirely on CHARTEVENT_MOUSE_MOVE being delivered promptly. Train() is
|
|
//--- therefore chunked: each call does at most ~TRAIN_TIME_BUDGET_MS of work, then returns and
|
|
//--- picks back up exactly where it left off on the next call (re-triggered via the same "New
|
|
//--- Bar" custom-event scheduling ScheduleTrainingIfNeeded() already used) - the members below
|
|
//--- are what makes that resumable across calls.
|
|
bool m_trainRunActive; // true: a run (schedule -> convergence/stop) is in progress, possibly spanning many Train() calls
|
|
bool m_eraResumePending; // true: yielded mid-bar-loop last call - resume the SAME era, don't start a new one
|
|
int m_resumeBars;
|
|
int m_resumeTotalIter;
|
|
int m_resumeOosCutoff;
|
|
int m_resumeBarIndex;
|
|
bool m_resumeAddLoop;
|
|
//--- Pass 2 of the era loop: bar indices pass 1 (sequential feedForward/scoring) queued as
|
|
//--- IS-eligible for backProp, trained on in a freshly shuffled order instead of pass 1's own
|
|
//--- fixed chronological (oldest-to-newest) visitation order. Root cause this targets: Adam's
|
|
//--- momentum (b1, AI\Network.mqh) has an effective memory window of ~1/(1-b1) steps, which lands
|
|
//--- close to the length of the contiguous same-class label runs the target produces (triple-barrier
|
|
//--- labels make those runs LONGER than the exact-pivot ones this was first measured against, since
|
|
//--- adjacent bars share most of their forward window and usually resolve the same way, so the
|
|
//--- argument holds a fortiori) - replaying those runs in the SAME order every single era let momentum lock onto
|
|
//--- whichever run it was currently passing through, with the network's end-of-era state
|
|
//--- disproportionately reflecting whatever it last walked through (recency), not a globally
|
|
//--- balanced fit. Symptom observed in practice: IS error climbing era-over-era (0.16->0.55+ over
|
|
//--- 25 eras on the same fixed dataset) instead of settling, and OOS Buy/Sell recall whipsawing
|
|
//--- between near-0% and 70-100% despite near-equal Buy/Sell label counts. Shuffling is the
|
|
//--- standard SGD fix - see the m_isPass2Active declaration below for how it's sequenced against
|
|
//--- pass 1 within Train()'s existing resumable-chunk machinery.
|
|
int m_isTrainQueue[];
|
|
int m_isTrainQueueCount;
|
|
//--- Parallel to m_isTrainQueue (same index, kept in lockstep through the Fisher-Yates shuffle
|
|
//--- below) - the per-occurrence weight to apply when this queued slot is trained on in pass 2,
|
|
//--- decided once at queue time (see the queueing block's oversampling comment) rather than
|
|
//--- recomputed in pass 2. Currently always 1.0: class-balance correction is carried entirely by
|
|
//--- repCount (how many times a bar was duplicated into the queue), not by any per-occurrence
|
|
//--- weight scaling - see the queueing block's comment for why stacking a second correction here
|
|
//--- caused two separate training collapses. Kept as a real per-slot array rather than a literal
|
|
//--- 1.0 so a future, smaller/safer supplemental weight could be reintroduced without re-touching
|
|
//--- the queueing or shuffle code.
|
|
double m_isTrainQueueWeightScale[];
|
|
//--- Parallel to m_isTrainQueue too (same index, same lockstep swap in the shuffle): true on exactly
|
|
//--- ONE occurrence per duplicated bar - the rep-0 slot written at queue time. Its only job is to keep
|
|
//--- the reported IS accuracy (m_cumIsCorrect/m_cumIsTotal) measured on the NATURAL class distribution
|
|
//--- while backProp still trains on the oversampled one.
|
|
//--- Why: the queue duplicates each Buy/Sell bar up to repCount times (~21x at the observed 30.7:1
|
|
//--- imbalance), so counting every occurrence measured IS accuracy over a set that is ~58% directional,
|
|
//--- while the OOS counter measures the real ~6% directional distribution. Both excluded Neutral and
|
|
//--- both used the identical formula, so they LOOKED directly comparable - and reported things like
|
|
//--- "IS 77% / OOS 12%", which reads as catastrophic overfitting when the two numbers were simply
|
|
//--- scored against base rates an order of magnitude apart (77% vs a 58% baseline is a 1.33x lift;
|
|
//--- 12% vs a 6.1% baseline is 1.97x - the OOS side was actually the STRONGER one).
|
|
//--- Counting primaries only puts both metrics on the same footing, so the IS/OOS gap once again means
|
|
//--- what everyone reads it as meaning: generalisation. Training itself is completely unaffected -
|
|
//--- every occurrence still backprops exactly as before; this flag is read only by the counter.
|
|
bool m_isTrainQueuePrimary[];
|
|
//--- Pass 2's cursor into the (already shuffled) m_isTrainQueue - lets pass 2 itself yield/resume
|
|
//--- mid-queue under the same TRAIN_TIME_BUDGET_MS chunk budget pass 1 already yields under.
|
|
int m_isTrainCursor;
|
|
//--- true: pass 1 (sequential) has finished for this era and pass 2 (shuffled backProp) is either
|
|
//--- running or has yielded mid-queue - Train() skips straight past pass 1's loop on resume when
|
|
//--- this is set. Reset to false only at a fresh era's start (never mid-run).
|
|
bool m_isPass2Active;
|
|
//--- true: pass 2 has already run to natural completion for this era (m_isPass2Active's own
|
|
//--- false state is ambiguous between "not started yet" and "already finished" - both look
|
|
//--- identical to a plain `if(!m_isPass2Active)` check). Needed because pass 3 (OOS scoring) can
|
|
//--- itself yield/resume across multiple Train() calls same as passes 1/2 do; without this flag,
|
|
//--- every resume into an unfinished pass 3 fell through the `if(!m_isPass2Active)` guards on both
|
|
//--- pass 1 and pass 2 and re-ran the ENTIRE shuffled queue again (pass 1 itself was a no-op on
|
|
//--- resume since its own loop cursor `i` was already exhausted, but pass 2 re-shuffled and replayed
|
|
//--- from scratch every single time) - silently multiplying the predicted-class counts (and the
|
|
//--- extra, unintended backProp() calls) once per resume, for as long as pass 3 kept needing more
|
|
//--- than one chunk to finish. Reset to false only at a fresh era's start, alongside m_isPass2Active.
|
|
bool m_isPass2Done;
|
|
//--- Pass 3: chronological, OOS-region-only re-walk that happens AFTER pass 2 has actually trained
|
|
//--- on this era's IS data - see m_isTrainQueue's declaration comment for why OOS scoring can no
|
|
//--- longer just happen inline during pass 1 (that would score every era's OOS window against
|
|
//--- weights from BEFORE this era's training, one full era stale - and for era 0 specifically,
|
|
//--- against the still-untrained cold-start network, which is why era 0's OOS recall used to show
|
|
//--- a meaningless 100% Neutral / 0% Buy / 0% Sell every time). Cursor walks i downward from
|
|
//--- m_oosScoreStartIndex to 0, mirroring pass 1's own iteration bounds/order for whichever bars
|
|
//--- satisfy isOOS - order matters here (unlike pass 2) since dOosForecast/dOosError are recursive
|
|
//--- EMAs over the visitation sequence, not order-independent.
|
|
bool m_isPass3Active;
|
|
int m_oosScoreIndex;
|
|
int m_oosScoreStartIndex;
|
|
datetime m_lastBarTime;
|
|
//--- This model's own learning-rate trajectory. `eta` (AI\Network.mqh) is a single file-scope
|
|
//--- global shared by every CNet in the process - PAI/CONV/LSTM each train independently and
|
|
//--- asynchronously (their own Train() calls are interleaved via separate chart-event/timer
|
|
//--- scheduling, not synchronized), but all three previously read AND wrote that one global, so
|
|
//--- one model's era-end regression/recovery decay/recovery of `eta` (see Train()'s era-end
|
|
//--- block) silently changed the learning rate the OTHER two models' very next backProp() call
|
|
//--- used too - an unintended coupling between what's supposed to be three independent training
|
|
//--- trajectories. Train() now restores `eta` from this member right before touching it, and
|
|
//--- saves it back before returning (both at the mid-chunk yield and at the natural end of the
|
|
//--- call) - MQL5 gives one chart's EA a single execution thread and each Train() call runs to
|
|
//--- completion or to its own yield point before another event is dispatched, so this save/
|
|
//--- restore is enough to isolate each model's schedule with no change needed to the neuron-level
|
|
//--- or OpenCL/DirectML call signatures that read the global directly.
|
|
double m_modelEta;
|
|
//--- Ceiling the era-end recovery bump (Train()'s isBetterEra block) restores `eta` toward - used
|
|
//--- to be the raw `lr` constant unconditionally, which is only correct for ADAM. SGD's own starting
|
|
//--- rate is the separate SgdLearningRate input (AI\Network.mqh) - clamping SGD's recovery bump to
|
|
//--- plain `lr` (AdamLearningRate) would silently cap it back down to Adam's ceiling after the
|
|
//--- first regression+recovery cycle, undoing the whole point of giving SGD its own configured
|
|
//--- rate. Computed once at construction from this model's own m_optimizationAlgo, matching
|
|
//--- m_modelEta's per-instance isolation (see that member's comment).
|
|
double m_etaCeiling;
|
|
int m_erasSinceCooldown; // eras completed since the last cooldown reset - replaces the old per-call-only "erasThisCall"
|
|
CArrayDouble m_oosWindow; // run-scoped OOS stability window (used to be a Train()-local CArrayDouble)
|
|
double m_bestOosForecast;
|
|
//--- Balanced accuracy (macro-recall: mean of Buy/Sell/Neutral OOS recall) of the era the current
|
|
//--- checkpoint was taken from. This is the metric the checkpoint SELECTION ranks on now, in place
|
|
//--- of the blended dOosForecast - blended accuracy is dominated by Neutral (~96% of bars), so among
|
|
//--- otherwise-acceptable eras it silently preferred the MOST Neutral-leaning weights, deploying a
|
|
//--- model that scores well on paper but under-calls Buy/Sell. Balanced accuracy weights every class
|
|
//--- equally, so "best" now means "most accurate across all three classes" - exactly what a 3-class
|
|
//--- signal product wants. Kept SEPARATE from m_bestOosForecast (which still snapshots the blended
|
|
//--- value at the same checkpoint) because FinalizeTrainRun()/the restore-on-regression branch reset
|
|
//--- dOosForecast to m_bestOosForecast, and that must stay the blended EMA the rest of the machinery
|
|
//--- expects. The directional recall FLOOR (directionalRecallOK) still gates convergence unchanged;
|
|
//--- this only fixes which era gets deployed among the candidates. -1 until the first ranked era.
|
|
double m_bestBalancedOos;
|
|
//--- whether the era m_bestOosForecast/the checkpoint was taken from also cleared the per-class
|
|
//--- directional recall floor (see directionalRecallOK below) - part of the "best" ranking itself,
|
|
//--- not just a side note, so blended accuracy alone can never outrank a directionally-useful era
|
|
//--- (see the checkpoint/eta-decay comment in Train()'s era-end block for why that matters).
|
|
bool m_bestPassedRecall;
|
|
bool m_haveOosCheckpoint;
|
|
bool m_oosStable;
|
|
bool m_objectiveMet;
|
|
//--- Plateau ladder state (see the PLATEAU_* constants). m_erasSinceBestBalanced counts eras since
|
|
//--- the last NEW BEST balanced accuracy; m_plateauStage is how far up the escalation it has climbed.
|
|
//--- Both are run-scoped and deliberately NOT persisted, matching m_modelEta/m_bestBalancedOos, which
|
|
//--- also restart fresh - a resumed run re-earns its patience rather than resuming mid-escalation.
|
|
int m_erasSinceBestBalanced;
|
|
int m_plateauStage;
|
|
//--- m_focalGammaRuntime removed 2026-07-31 with focal loss itself - see the removal note at the
|
|
//--- former m_focalGamma above. The plateau ladder keeps its learning-rate warm restart, which was
|
|
//--- always the actual escape; the gamma anneal beside it stepped monotonically to zero anyway.
|
|
uint m_syncWaitStartTick; // 0 = not waiting on history sync; else GetTickCount() when the wait began
|
|
//--- 3 no-op passes on a fresh start (see InitNeuralNetwork()/ResetWeights()), each its own separately-
|
|
//--- scheduled Train() call (not a tight in-process loop), so the broker/terminal's history sync gets
|
|
//--- several real, wall-clock-separated chances to finish before the era loop commits to a bar count.
|
|
int m_warmupPassesRemaining;
|
|
//--- fractal/swing-confirmation/trend-context Buy/Sell label cache: the label at a given now-relative
|
|
//--- bar index only depends on price/ATR history, never on model state, so recomputing it every era
|
|
//--- (as opposed to once per real bar close) is pure waste. Rebuilt fresh whenever `bars` changes
|
|
//--- (see the invalidation check in Train()) rather than incrementally appended, since MQL5 timeseries
|
|
//--- indices are relative to "now" and shift by one on every new candle - a full rebuild on change
|
|
//--- sidesteps needing datetime-keyed/incremental bookkeeping entirely.
|
|
//--- Filled in the same pass as the label caches below and gated by the SAME m_labelCacheHasValue, so
|
|
//--- a bar either has both or neither and no third validity flag can drift out of step with them.
|
|
double m_excUpCache[];
|
|
double m_excDownCache[];
|
|
bool m_labelCacheBuy[];
|
|
bool m_labelCacheSell[];
|
|
bool m_labelCacheHasValue[];
|
|
int m_labelCacheBars; // 0 = no cache built yet
|
|
datetime m_labelCacheAnchorTime; // m_Time.GetData(0) at last (re)build - 2nd invalidation key
|
|
void ComputeLabelForBar(int i, int bars, bool &buy, bool &sell);
|
|
void AdvanceBarrierLabelState(int i, int bars);
|
|
//--- The triple-barrier verdict for one bar - the training TARGET. See BARRIER_TIE_GOES_TO_STOP.
|
|
//--- `idx` is a now-relative index; the scan walks FORWARD in time, i.e. toward index 0, and needs
|
|
//--- m_barrierHorizonBars of them to exist, so callers must keep idx >= m_barrierHorizonBars.
|
|
//--- Returns Neutral for any bar it cannot resolve (no ATR, ran out of history), which is the same
|
|
//--- answer as "no setup" and keeps the caller free of a third outcome to handle.
|
|
ENUM_SIGNAL TripleBarrierLabel(int idx);
|
|
//--- Resolves the SL/TP ATR multiples the label uses from the EA's live SL_Mode/TP_Mode. Split out
|
|
//--- because the INTELLIGENT modes scale with AI confidence, which does not exist at label time -
|
|
//--- see the definition for why the label uses their zero-confidence base instead.
|
|
void BarrierMultiples(double &slMult, double &tpMult);
|
|
//--- Median confirmed-ZigZag-leg length over the training window, snapped to the horizon ladder.
|
|
int ComputeBarrierHorizonBars(int bars);
|
|
//--- Resolves m_barrierHorizonBars exactly once per process, from live buffers. Needed on BOTH paths,
|
|
//--- which is the whole reason it is not simply inlined in the prebuild: a DEPLOYED model never enters
|
|
//--- Train(), so it never reaches StartLabelCachePrebuild() - yet OnlineLearnStep() reads the horizon
|
|
//--- as its confirmation delay. Left unresolved there it would sit at BARRIER_HORIZON_FALLBACK and, on
|
|
//--- any symbol whose real horizon is longer, backprop bars whose barriers had not actually resolved -
|
|
//--- silent lookahead in the one place that writes to a live, trading model.
|
|
void EnsureBarrierHorizon(int bars);
|
|
bool m_barrierHorizonResolved;
|
|
//--- full per-bar INPUT feature vector cache (everything BufferTempData() computes: ATR-normalized
|
|
//--- OHLC, time-of-day encoding, volume delta, AD indicator buffers, ...). Same rationale as the
|
|
//--- label cache above - a given now-relative bar index's feature vector only depends on price/
|
|
//--- indicator history, never on model state, so recomputing it is pure waste: BufferTempData()
|
|
//--- used to rebuild every bar from scratch once per historyBars-wide window it appears in, AND
|
|
//--- again on every subsequent era on top of that. Flat array (idx*m_neuronsCount + feature),
|
|
//--- not a true 2D array - MQL5 can't dynamically resize an inner dimension. Shares the label
|
|
//--- cache's invalidation trigger (see EnsureBarCachesCapacity()) since both are keyed on the exact
|
|
//--- same now-relative index frame.
|
|
double m_featureCache[];
|
|
bool m_featureCacheHasValue[]; // true once idx has been resolved (valid OR invalid)
|
|
bool m_featureCacheValid[]; // false: BufferTempDataCompute(idx) returned false (e.g.
|
|
// no ATR yet) - cached as a miss so it isn't retried forever
|
|
bool BufferTempDataCompute(int idx);
|
|
//--- Nearest confirmed (non-repainting) ZigZag pivot at or after fromIdx - see this method's
|
|
//--- definition comment and m_useSwingContext's declaration comment for the repainting-embargo
|
|
//--- rationale callers must apply to fromIdx before calling this.
|
|
bool FindConfirmedZigZagPivot(int fromIdx, int &pivotIdx, double &pivotPrice, bool &pivotIsLow);
|
|
bool EnsureBarCachesCapacity(int bars);
|
|
//--- Eager label-cache pre-build + true-label tally, run once per fresh start (see
|
|
//--- m_warmupPassesRemaining) BEFORE era 0's real training loop begins. Without it, era 0 trains with
|
|
//--- m_prevEraTrueBuyCount/Sell/Neutral all still 0, so UpdateClassPriors() has no measured
|
|
//--- distribution and era 0 alone gets ZERO correction for label imbalance. Pre-scanning the whole IS
|
|
//--- window upfront (reusing the same cache/ComputeLabelForBar() the era loop uses) lets era 0 start
|
|
//--- from real measured class base rates.
|
|
bool m_labelCachePrebuilt; // true once the one-time pre-scan has completed
|
|
bool m_labelPrebuildActive; // true while a chunked pre-scan is in progress
|
|
bool m_prebuildSeedPending; // true: era 0's era-start reset must NOT stomp the
|
|
// prebuild-seeded m_prevEraTrue* counts with the
|
|
// still-empty live tally (see Train()'s era-start block)
|
|
int m_labelPrebuildBars;
|
|
int m_labelPrebuildOosCutoff;
|
|
int m_labelPrebuildIndex;
|
|
int m_labelPrebuildBuyCount;
|
|
int m_labelPrebuildSellCount;
|
|
int m_labelPrebuildNeutralCount;
|
|
void StartLabelCachePrebuild(void);
|
|
void AdvanceLabelCachePrebuild(void);
|
|
//--- Evaluation-only continual-learning OOS simulation: once the core model converges, a CLONE of its
|
|
//--- weights (never the production Net itself) walks forward through the OOS window bar-by-bar,
|
|
//--- scoring each bar with its current weights THEN learning from it - simulating how the model would
|
|
//--- adapt in live/forward trading. This must never feed back into Net or the real OOS convergence
|
|
//--- metric (dOosForecast/m_oosSamples), so it's tracked in entirely separate members and the clone is
|
|
//--- discarded (never Save()'d) once each walk completes.
|
|
CNet *m_simOosNet; // NULL when no simulation is active
|
|
bool m_simOosRunActive;
|
|
int m_simOosCutoff; // oosCutoff snapshot from the run that converged
|
|
int m_simOosBarIndex; // resume point, m_simOosCutoff-1 down to 0
|
|
double m_simOosForecast; // smoothed accuracy - separate from dOosForecast
|
|
int m_simOosSamples;
|
|
void StartOosContinualSimulation(int bars, int oosCutoff);
|
|
void AdvanceOosSimulationChunk(void);
|
|
//--- same resumability problem one level up: TuneIndicatorsAndTrain()'s own trial loop calls
|
|
//--- Train() per trial and used to assume each call ran an entire trial to completion synchronously
|
|
int m_tuneTrialIndex; // -1 = no multi-trial tuning run in progress
|
|
double m_tuneBestOosForecast;
|
|
bool m_tuneLastTrialWasWin;
|
|
bool m_tuneHaveBestCheckpoint;
|
|
datetime m_tuneStartTrainBar;
|
|
//=== Filter-based indicator auto-tuner (see TuneIndicatorsByFilter) =============================
|
|
//--- Replaced a genetic + successive-halving search on 2026-08-01. That search scored every candidate
|
|
//--- by TRAINING a throwaway network on it, which cost 1152 eras (9-48 h depending on topology) before
|
|
//--- the real model started, and its short screening rungs could not separate candidates at all. The
|
|
//--- filter scores candidates by the mutual information between the resulting FEATURES and the LABELS -
|
|
//--- arithmetic over the feature cache, no training - so it finishes in seconds and its cost is
|
|
//--- independent of topology. See TuneIndicatorsByFilter() for the measurements and the honest limit.
|
|
bool m_tuneFilterDone; // the one-shot filter pass has run for this model
|
|
//--- Mutual information between one feature column and the 3-class label, and the whole-vector score.
|
|
double FeatureColumnMI(const double &vals[], const int &labels[], int n);
|
|
//--- Returns the MEAN per-feature marginal MI. Side-effects two more numbers that the mean alone
|
|
//--- cannot express, both read by TuneIndicatorsByFilter's report - MQL5 forbids a default value on a
|
|
//--- reference parameter, so members rather than out-params:
|
|
//--- m_miBestColumn - the STRONGEST single feature's MI. A mean over 26 columns hides one good
|
|
//--- column among 25 useless ones, which is exactly the case worth catching.
|
|
//--- m_miLabelEntropy - H(Y) in nats for the sampled labels, so MI can be quoted as a FRACTION of
|
|
//--- what there is to know. "0.001 nats" means nothing on its own; "0.1% of the
|
|
//--- label's entropy" is a magnitude anyone can act on.
|
|
double ScoreCurrentParamsByMI(bool shuffleLabels = false);
|
|
//--- The same work split in two, so the permutation test can extract the sample ONCE and reuse it for
|
|
//--- every null draw. BuildMiSample returns the sample count (or -1); ScoreMiSample shuffles `labels`
|
|
//--- in place when asked, so the observed statistic must always be taken before the first draw.
|
|
//--- labelBarOffset != 0 takes the LABEL from bar i+offset while the features still come from bar i,
|
|
//--- which is what the alignment scan needs. The sampled range is trimmed by MiShiftPad() at both
|
|
//--- ends - a FIXED amount, never by |offset| - so every build enumerates the same bars in the same
|
|
//--- order and two builds can be compared row by row. Returns -1 if the offset exceeds the pad.
|
|
//--- `target` selects WHICH outcome the features are scored against (MI_TARGET_*). Anything other
|
|
//--- than the barrier class is continuous and is discretised into 3 equal-frequency bins at the end
|
|
//--- of the build, so every downstream consumer sees the same 3-class shape it already handles.
|
|
int BuildMiSample(double &cols[], int &labels[], int labelBarOffset = 0,
|
|
int featureBarOffset = 0, int target = MI_TARGET_BARRIER);
|
|
//--- Is an "optimal SL/TP" head learnable? Scores the features against excursion magnitude and
|
|
//--- asymmetry instead of the barrier class - a different question, see the definition.
|
|
void ReportExcursionInformation(void);
|
|
//--- Sets the ATR multiples from the measured MFE/MAE quantiles instead of the mode enums. Returns
|
|
//--- false (and leaves the configured pair standing) when there are too few resolved excursions.
|
|
bool DeriveBarrierGeometry(void);
|
|
//--- LAG PROFILE: how far back the features still say anything about the entry they precede. Prints
|
|
//--- MI at feature lag k = 0..m_historyBars against the same block-permutation null, and returns the
|
|
//--- deepest lag that clears it - i.e. the lookback the data actually supports, rather than the 20 that
|
|
//--- was picked by hand and never measured.
|
|
//--- WHY THIS WAS MISSING AND WHY IT MATTERS: BuildMiSample samples ONE bar. Every "MI is at the noise
|
|
//--- floor" verdict this codebase has produced therefore described the ENTRY BAR's features only, while
|
|
//--- the network is fed m_historyBars of them. If information lived at lag 7 and not lag 0 the report
|
|
//--- would have said "no signal" while the model could still learn - so the diagnostic we have been
|
|
//--- deciding on had a blind spot exactly the width of the input vector.
|
|
int ReportFeatureLagProfile(void);
|
|
//--- Bars trimmed from each end of every MI sample. Must cover the largest offset any caller asks
|
|
//--- for: the alignment scan's MI_ALIGN_MAX_SHIFT and the positive control's horizon/4. Expressed
|
|
//--- once here so the control cannot drift out of agreement with the range it is sampled over -
|
|
//--- which is precisely the failure this replaced.
|
|
int MiShiftPad(void) const
|
|
{
|
|
//--- Also covers m_historyBars, because the lag profile shifts the FEATURES that far back and every
|
|
//--- build must still enumerate the identical bar set (see BuildMiSample's fixed-pad note - padding
|
|
//--- by the requested offset instead is what voided the positive control on 2026-08-02).
|
|
return MathMax((int)MathMax(m_historyBars, 0),
|
|
MathMax(MI_ALIGN_MAX_SHIFT, MathMax(m_barrierHorizonBars, 1) / 4));
|
|
}
|
|
double ScoreMiSample(const double &cols[], int &labels[], int n, bool shuffleLabels);
|
|
//--- The permutation test + verdict, split out of the tuner so it is NOT gated on era 0 with it - see
|
|
//--- the definition. Read-only; runs once per attach, whether or not the sweep did.
|
|
void ReportFeatureLabelInformation(void);
|
|
//--- Smallest BarsCalculated() across the ENABLED tunable indicators, or -1 when none is on. A handle
|
|
//--- created by IndicatorCreate() calculates asynchronously, so a candidate scored before its handle
|
|
//--- has caught up is scored on an empty or partial buffer. The tuner reports this so "the parameter
|
|
//--- change did not reach the features" can be told apart from "it reached them but they weren't ready".
|
|
int TunableBarsCalculated(void);
|
|
bool m_miReportDone;
|
|
//--- Eras the MI report has waited for the cross-asset panel to exist, so it describes the SAME
|
|
//--- feature vector training uses. Bounded, so a terminal that never syncs the reference symbols
|
|
//--- still gets its diagnostics rather than silently getting none.
|
|
int m_miReportDeferrals;
|
|
//--- Ranks every selectable SL/TP pairing by how much the SAME features say about THAT barrier
|
|
//--- outcome at ENTRY time - see the definition. Read-only: it relabels a sampled copy, never the
|
|
//--- label cache, and restores the barrier state it borrowed.
|
|
void ReportBarrierGeometryScan(void);
|
|
//--- Scan overrides consulted by BarrierMultiples(). Both > 0 or neither applies; 0 = off. Live only
|
|
//--- for the duration of ReportBarrierGeometryScan, and nothing persisted is keyed on them.
|
|
double m_barrierScanSlMult;
|
|
double m_barrierScanTpMult;
|
|
//--- true => BuildMiSample computes each label with TripleBarrierLabel() instead of reading the cache,
|
|
//--- because a hypothetical geometry's labels are by definition not cached.
|
|
bool m_barrierScanLiveLabels;
|
|
//--- timed-out labels seen during one geometry's live relabel - see the scan's dir/to columns.
|
|
int m_barrierScanTimeouts;
|
|
//--- set by ComputeBarrierHorizonBars: this geometry needs MORE time than BARRIER_HORIZON_MAX allows,
|
|
//--- so its label truncates a trade the EA would hold to SL/TP. Disqualifies it from the scan.
|
|
bool m_barrierHorizonClamped;
|
|
double m_miBestColumn;
|
|
double m_miLabelEntropy;
|
|
//--- bars between two consecutive MI sample rows, set by BuildMiSample - see its note.
|
|
int m_miStrideBars;
|
|
//--- independent label blocks the permutation null was built from (rows within one barrier horizon
|
|
//--- move together, so THIS - not the row count - is the sample size the p-value really rests on).
|
|
int m_miNullBlocks;
|
|
void TuneIndicatorsByFilter(void);
|
|
//================================================================================================
|
|
void FinalizeTrainRun(void);
|
|
//--- The "this is now THE model" persistence sequence, shared by every deploy path so they can't
|
|
//--- drift apart: weights (carrying the current m_trainingComplete flag), the pure-MQL5 inference
|
|
//--- self-check, the calibration sidecar, and the EMA shadow. Callers set m_trainingComplete first -
|
|
//--- the flag is written INTO the .nnw here. Used by FinalizeTrainRun() (ladder/era-cap/stop deploys)
|
|
//--- and by DeployNow()/RetrainDeployed() (the panel buttons).
|
|
void PersistDeployedModel(void);
|
|
//--- On hitting the per-run era cap: asks the operator whether to keep training (true) or deploy
|
|
//--- the best checkpoint and stop (false). Headless (tester/optimizer) can't show a dialog, so it
|
|
//--- returns false. See m_maxErasPerRun's declaration comment.
|
|
bool PromptContinuePastEraCap(double bestOos);
|
|
//--- variables
|
|
//--- training control, driven by the control panel (Warrior_EA.mq5); Train()/OnTickHandler
|
|
//--- poll these rather than being torn down/rebuilt, so pausing/stopping never loses in-memory state
|
|
bool m_trainingPaused; // true: Train() blocks between eras until unpaused
|
|
bool m_trainingStopRequested; // true: OnTickHandler stops scheduling new training passes
|
|
//--- true for ANY Strategy Tester run - a single backtest AND every optimization pass (MQL_TESTER):
|
|
//--- the run must NEVER train. It seeds the agent-local cache from the deployed production model (see
|
|
//--- InitNeuralNetwork) and runs pure inference over the window - what a buyer expects from "backtest
|
|
//--- my deployed EA", and what makes optimizing TRADING parameters (SL/TP, filters, MM) fast and
|
|
//--- comparable (the AI is held fixed, not retrained per config). Training + online continual learning
|
|
//--- happen only on a live chart, where this is false.
|
|
bool m_inferenceOnly;
|
|
//--- true only when the current Net weights came from a saved .nnw on disk, not from a freshly-built
|
|
//--- random topology. Used to let an inference-only tester replay a seeded model even if that model's
|
|
//--- persisted trainingComplete flag is still false, while still blocking the "no model found, built a
|
|
//--- fresh topology" path from placing random-weight trades.
|
|
bool m_modelLoadedFromDisk;
|
|
//--- Set by EnforceTopologyContract() when a just-loaded .nnw was built by a superseded architecture
|
|
//--- that cannot be repaired in place (currently: a different conv receptive field, whose weight
|
|
//--- tensor is a different SHAPE). InitNeuralNetwork discards the load and retrains. A .nnw persists
|
|
//--- the architecture, not just the weights - see AI\Network.mqh CNet::FirstConvWindow.
|
|
bool m_topologySuperseded;
|
|
//--- true once ValidateCpuInference() has confirmed this model's pure-MQL5 forward pass matches the
|
|
//--- compute backend's within tolerance (see CNet::SetCpuInference). Persisted in the .stats sidecar
|
|
//--- so an inference-only backtest can run DLL-free; if false (e.g. a conv/LSTM topology not yet
|
|
//--- ported, or a validation miss) the backtest falls back to the DLL. Measured at deploy on the
|
|
//--- chart (where a backend exists to compare against), never in the tester itself.
|
|
bool m_mqlInferenceValidated;
|
|
string m_fileName;
|
|
string m_folderPath;
|
|
//--- which file Train()'s Net.Save() calls (and this method's own Net.Load()) actually target:
|
|
//--- the shared FILE_COMMON production weights normally, or a LOCAL per-agent cache file when
|
|
//--- running inside the Strategy Tester/optimizer (see InitNeuralNetwork) so that repeated
|
|
//--- optimization passes with an unchanged topology can reuse an already-trained model instead of
|
|
//--- re-running every era from scratch, without ever touching the live production .nnw/.cfg.
|
|
string m_activeFileName;
|
|
bool m_activeFileCommon;
|
|
//--- Name of the terminal-wide global variable this instance holds as an exclusive claim on
|
|
//--- m_activeFileName, or "" when it holds none. See AcquireConfigLock().
|
|
string m_configLockName;
|
|
//--- user-settable via Inputs.mqh's TrainingOptimizer (SGD or ADAM), read into this member at
|
|
//--- construction. Honored by all three signal types - PAI/CONV via BuildFreshTopology(), and
|
|
//--- LSTM via CSignalLSTM::AddCustomLayers - since CNeuronLSTMOCL (AI\Network.mqh) now has an
|
|
//--- accelerated SGD+momentum path (LSTM_UpdateWeightsMomentum) alongside its original Adam-only
|
|
//--- one, and CNet::CNet's GPU/DirectML init gate no longer restricts LSTM topologies to ADAM.
|
|
int m_optimizationAlgo;
|
|
int m_historyBars;
|
|
int m_outputNeuronsCount;
|
|
int m_minNeuronsCount;
|
|
int m_initialNeuronsCount;
|
|
int m_neuronsCount;
|
|
double m_neuronsReduction;
|
|
int m_hiddenLayersCount;
|
|
//--- LSTM-only recurrent hidden-unit count - see LstmHiddenSize's declaration comment
|
|
//--- (Variables\Inputs.mqh). Harmless, unused constant contribution to m_fingerprint for MLP/CONV.
|
|
int m_lstmHiddenSize;
|
|
//--- CONV-only convolutional output-filter count - see ConvFilterCount's declaration comment
|
|
//--- (Variables\Inputs.mqh). Harmless, unused constant contribution to m_fingerprint for MLP/LSTM.
|
|
int m_convFilterCount;
|
|
int m_minTrainYear;
|
|
bool m_isInitialized;
|
|
//--- true once OnDeinit has begun - see MarkShutdown()/FinalizeTrainRun().
|
|
bool m_shutdownInProgress;
|
|
int m_fractalPeriods;
|
|
//--- "weights" of the 4 CONFIDENCE TIERS a live AI fire can land in (0-100 each), the AI equivalent
|
|
//--- of classic indicators' several geometric m_pattern_N members. ConfidenceTier() buckets the
|
|
//--- live confidence magnitude (CalibratedConfidenceMagnitude()) into 4 equal bands between the
|
|
//--- HEAD'S OWN structural floor and 1.0 - 1/3 for the 3-class softmax, 0.5 for the regression head -
|
|
//--- tier 0 = weakest possible directional call, tier 3 = near-certain. UpdateSignalsWeights()
|
|
//--- (Expert\ExpertSignalCustom.mqh) then calibrates each tier's weight independently from its OWN
|
|
//--- realized win rate, same as any classic pattern.
|
|
//---
|
|
//--- The defaults span the SAME 0-100 conviction scale the classic patterns use, and that is the
|
|
//--- whole mechanism by which one Min vote to open governs both engines. Confidence is expressed as
|
|
//--- vote strength rather than as a separate entry floor, so Min_Vote_Open reads directly as "which
|
|
//--- tier is good enough", with no second scale to reason about:
|
|
//--- 25 tier 0 conf 0.333-0.500 (3-class) / 0.500-0.625 (regression) - a bare plurality
|
|
//--- 50 tier 1 conf 0.500-0.667 / 0.625-0.750
|
|
//--- 75 tier 2 conf 0.667-0.833 / 0.750-0.875
|
|
//--- 100 tier 3 conf 0.833-1.000 / 0.875-1.000 - near-certain
|
|
//--- So Min_Vote_Open = 50 lets tier 1 and up trade when the AI votes alone, 75 lets tier 2 and up,
|
|
//--- and so on. A near-coin-flip 0.34 argmax is NOT blocked at the AI boundary - it votes 25 and is
|
|
//--- filtered by the vote threshold, exactly as a weight-10 classic confirmation would be. Note these
|
|
//--- are only the DEFAULTS: with UseDatabaseRanking on, each tier's weight moves to its own measured
|
|
//--- win rate, which is precisely why the tiers must be separate patterns rather than one continuous
|
|
//--- confidence-to-weight formula - a formula would leave the ranking system nothing to calibrate.
|
|
//---
|
|
//--- This used to be a single m_pattern_0 covering every fire regardless of confidence. That was a
|
|
//--- real bug, not just a missed opportunity: with only one pattern, UpdateSignalsWeights()'
|
|
//--- averaging step collapses (average of one value is that value), so the SAME win rate got written
|
|
//--- into both the pattern weight AND the module weight (filter.Weight()) - and Direction() multiplies
|
|
//--- them together. A genuine 70%-win-rate model was therefore scored as (0.70*70)=49, not 70 - a
|
|
//--- quadratic, not linear, derating that got harsher the further from 100% the model's real win rate
|
|
//--- was. Classic indicators never hit this because their module weight blends across SEVERAL
|
|
//--- differently-performing patterns, diluting any one pattern's own score instead of squaring it.
|
|
//--- Splitting into 4 genuinely-different tiers restores that same blending for AI signals.
|
|
//---
|
|
//--- Defaults (constructor) are graduated 80/87/93/100 - all within the 80-100 band classic
|
|
//--- indicators reserve for their most rigorous patterns, since even tier 0 has already passed the
|
|
//--- training-completion gate, the confidence floor, and the alternation gate (LongCondition's
|
|
//--- comments) before ever reaching here. Hybrid mode (AIType==HYBRID) uses the same shared
|
|
//--- weighted-vote path as the other models; no quorum gate is needed anymore.
|
|
int m_pattern_0, m_pattern_1, m_pattern_2, m_pattern_3;
|
|
//--- functions
|
|
virtual bool InitIndicators(CIndicators *indicators);
|
|
//--- sets ID/m_id/m_folderPath/m_fileName/m_pattern_count from the subclass constructor - defaults
|
|
//--- to 4 (the confidence tiers - see m_pattern_0's declaration comment), not 1
|
|
void SetIdentity(string id, string shortId, int patternCount = 4);
|
|
//--- hook for neuron-type-specific layers (Conv+Pool, LSTM, ...); default is a plain perceptron (no-op)
|
|
virtual bool AddCustomLayers(CArrayObj *topology) { return true; }
|
|
//--- Reusable front-end stages, composed by the AddCustomLayers() overrides. Each subclass names the
|
|
//--- stages it wants instead of re-declaring the CLayerDescription fields, so the topologies cannot
|
|
//--- silently drift apart: HYBRID is *defined* as AddConvStage + AddLstmStage, which makes its
|
|
//--- "matches the standalone CONV front-end exactly, then adds LSTM" contract structural rather than
|
|
//--- a comment. (They had already drifted - HYBRID guarded the LSTM step with MathMax(1,...) and
|
|
//--- CSignalLSTM did not, so a historyBars of 1 gave the two a different step.)
|
|
bool AddConvStage(CArrayObj *topology);
|
|
bool AddLstmStage(CArrayObj *topology);
|
|
//--- Which front-end stages this subclass's AddCustomLayers() actually appends. Declared once per
|
|
//--- subclass and consumed by everything that has to reason about the built shape (the LSTM capacity
|
|
//--- budget, the startup config line), so those can never disagree with what was constructed. A
|
|
//--- virtual rather than an AIType check, so a future composition cannot silently get the wrong answer.
|
|
virtual bool UsesConvStage(void) const { return false; }
|
|
virtual bool UsesLstmStage(void) const { return false; }
|
|
//--- AddConvStage runs BEFORE AddLstmStage wherever both are present (HYBRID), so the LSTM is fed the
|
|
//--- conv feature map rather than the raw flattened input.
|
|
bool HasConvBeforeLstm(void) const { return UsesConvStage() && UsesLstmStage(); }
|
|
//--- Conv chain shape - see the definitions above AddConvStage. Every consumer reads these rather
|
|
//--- than re-deriving the arithmetic, so the built topology and the logged shape cannot disagree.
|
|
int ConvReceptiveFieldBars(void) const;
|
|
int ConvFirstStagePositions(void) const;
|
|
bool HasSecondConvStage(void) const;
|
|
int ConvOutputPositions(void) const;
|
|
int ConvOutputWidth(void) const;
|
|
//--- Actual input width the LSTM block sees, which is NOT always the flattened input.
|
|
int LstmFanIn(void) const;
|
|
//--- " | conv 21->8 x20 bars | lstm 160->32" for the startup config line; "" when neither applies.
|
|
string FrontEndConfigSummary(void) const;
|
|
//--- Appends a batch-normalization layer, or does nothing (returning success) when EnableBatchNorm is
|
|
//--- off. `units` is advisory only - CNet sizes the layer from whatever sits below it, because a conv
|
|
//--- or pool stage's output width is derived inside the CNet constructor and is not knowable here.
|
|
//--- See AI\NeuronBatchNorm.mqh for what the layer does and why it exists.
|
|
bool AddBatchNormStage(CArrayObj *topology, int units);
|
|
//--- hardcoded activation for the common tapering Dense hidden-layer stack built by
|
|
//--- BuildFreshTopology() (below AddCustomLayers, above the output layer). PRELU (leaky ReLU,
|
|
//--- 0.01 slope) is the default: it doesn't saturate/vanish the way TANH does across a deep
|
|
//--- taper, and is what the Conv layer itself already uses (see CSignalCONV::AddCustomLayers).
|
|
//--- CSignalLSTM overrides this to TANH instead - these Dense layers sit directly on top of the
|
|
//--- LSTM layer's own TANH-bounded ([-1,1]) output, so keeping them bounded too avoids feeding an
|
|
//--- unbounded activation straight off a bounded recurrent output, which is untested territory
|
|
//--- here and not what the user asked for ("Tanh for LSTM gates").
|
|
virtual ENUM_ACTIVATION HiddenLayerActivation(void) { return PRELU; }
|
|
//--- Single source of truth for the output head's activation. BuildFreshTopology() stamps it into a
|
|
//--- NEW topology; EnforceTopologyContract() re-asserts it after every Load(), because a .nnw
|
|
//--- persists the activation and would otherwise pin a superseded architecture forever (see
|
|
//--- CNet::EnforceOutputActivation's declaration comment in AI\Network.mqh for the incident this
|
|
//--- comes from). Deliberately one expression called from both places - when these were two separate
|
|
//--- literals, changing the head in BuildFreshTopology() silently did nothing to any existing model.
|
|
//--- Regression (1 output): TANH - its native [-1,1] range maps directly onto the -1/0/1
|
|
//--- Sell/Neutral/Buy target convention. Classification (3 outputs): SIGMOID - see the long
|
|
//--- rationale at BuildFreshTopology()'s use of this method for why the head must stay BOUNDED.
|
|
ENUM_ACTIVATION OutputLayerActivation(void) const { return (m_outputNeuronsCount == 1) ? TANH : SIGMOID; }
|
|
//--- Width of the first dense layer, DERIVED rather than configured. It used to be an input
|
|
//--- (FIRST_LAYER_NEURONS, default 500) whose only sensible value depends entirely on two things the
|
|
//--- user cannot see: how wide the input vector ended up after feature selection, and how much
|
|
//--- in-sample data the study period actually yields. Left to a hand-picked constant it was badly
|
|
//--- wrong - 500 units against a 420-wide input is 210,500 weights, 72% of a 292,583-weight model,
|
|
//--- against ~36,500 training bars of which only ~2,236 are directional. That is ~8 parameters per
|
|
//--- sample, and it EXPANDS a set of highly correlated inputs instead of compressing them. The
|
|
//--- symptom is already in the logs: the shallowest topology consistently beat the deepest, which is
|
|
//--- what over-parameterization looks like from the outside.
|
|
//--- Computed from the STUDY PERIOD rather than from bars currently downloaded, so the answer is a
|
|
//--- deterministic function of the inputs and cannot drift as history fills in - and is then snapped
|
|
//--- to a coarse power-of-two ladder so even a large error in the estimate lands on the same rung.
|
|
//--- Every field it reads is already part of the weights-filename fingerprint, so the derived value
|
|
//--- needs no fingerprint entry of its own. MUST be called before the fingerprint is built and never
|
|
//--- again (see the note on fingerprint-feeding members at the top of this file).
|
|
int ComputeFirstLayerWidth(void) const;
|
|
//--- Expected in-sample training rows for the configured study period, split and timeframe. Factored
|
|
//--- out of ComputeFirstLayerWidth so every derived capacity decision spends the SAME budget - three
|
|
//--- stages each guessing at the training-set size independently is how they drift apart.
|
|
double EstimatedInSampleBars(void) const;
|
|
//--- Conv output-filter count and LSTM hidden width, DERIVED for the same reason the first-layer width
|
|
//--- is. Both were inputs whose defaults (16 filters, 32 units) were fixed constants picked without
|
|
//--- reference to the input width they sit on or the data available to fit them - so on a minimal
|
|
//--- feature set the conv stage EXPANDED the input, and the LSTM block quietly carried more weights
|
|
//--- than the entire dense taper below it. Both MUST be called before the fingerprint is built and
|
|
//--- never again: they assign fingerprint-feeding members (see the note at the top of this file).
|
|
int ComputeConvFilterCount(void) const;
|
|
int ComputeLstmHiddenSize(void) const;
|
|
//--- Dense-taper DEPTH, derived 2026-07-30 from the two endpoints the taper connects. It was the depth
|
|
//--- suffix on each AI_CHOICE entry (MLP_3L/MLP_4L/..._2L); asking a user to pick a layer count while
|
|
//--- the code derives the widths those layers taper between is asking for half a decision. Reads
|
|
//--- m_initialNeuronsCount, so it MUST be called after ComputeFirstLayerWidth and before the
|
|
//--- fingerprint - see the note on fingerprint-feeding members at the top of this file.
|
|
int ComputeHiddenLayerCount(void) const;
|
|
//--- Re-assert everything about a just-loaded net that lives in the FILE but is owned by the CODE.
|
|
//--- Call after every successful Net.Load(); no-ops (and stays silent) when the file already agrees.
|
|
void EnforceTopologyContract(void);
|
|
//--- common network bootstrap: indicators, topology build/load, training-file bookkeeping
|
|
bool InitNeuralNetwork(CIndicators *indicators);
|
|
//--- Exclusive per-config claim, so two charts can never train into one set of model files. Every
|
|
//--- retrain-affecting input is already hashed into m_activeFileName, so "same file" IS "same
|
|
//--- config" - which makes the filename the only correct lock identity. Live charts only: each
|
|
//--- tester/optimizer agent is a separate process with its own sandboxed _optcache copy, and they
|
|
//--- are *meant* to run the same config in parallel.
|
|
bool AcquireConfigLock(void);
|
|
void ReleaseConfigLock(void);
|
|
void DrawObject(datetime time, double signal, double high, double low);
|
|
void DeleteObject(datetime time);
|
|
//--- Time-ordered NMS sweep over m_arrowSignalCache: prunes each same-direction run down to its
|
|
//--- earliest bar (deleting redundant neighbors within m_signalClusterWindow). Run once per era end.
|
|
void PruneDirectionalClusters(int bars);
|
|
//--- Live newest-bar NMS accept test (time-keyed, idempotent per bar time - see m_signalClusterWindow).
|
|
bool NmsLiveAccept(datetime barTime, ENUM_SIGNAL dir, double conf)
|
|
{
|
|
if(m_signalClusterWindow <= 0)
|
|
return true;
|
|
if(dir != Buy && dir != Sell)
|
|
return true;
|
|
// Idempotent re-eval of the same bar (RefreshLatestSignal can run more than once per bar).
|
|
if(dir == Buy && m_nmsLiveBuyTime == barTime)
|
|
return m_nmsLiveBuyAccept;
|
|
if(dir == Sell && m_nmsLiveSellTime == barTime)
|
|
return m_nmsLiveSellAccept;
|
|
long minGap = (long)m_signalClusterWindow * PeriodSeconds();
|
|
datetime lastSame = (dir == Buy) ? m_nmsLiveBuyTime : m_nmsLiveSellTime;
|
|
bool accept;
|
|
// 1) Same-direction contiguous collapse: suppress if within the window of the previous SEEN
|
|
// same-direction bar (advance last-seen below either way, so a whole run collapses to one).
|
|
if(lastSame != 0 && (long)(barTime - lastSame) <= minGap)
|
|
accept = false;
|
|
else
|
|
{
|
|
// 2) Cross-direction resolution vs the last KEPT opposite signal: keep the stronger side.
|
|
accept = true;
|
|
if(m_nmsLiveKeptTime != 0 && m_nmsLiveKeptDir != dir &&
|
|
(long)(barTime - m_nmsLiveKeptTime) <= minGap)
|
|
{
|
|
if(conf > m_nmsLiveKeptConf)
|
|
DeleteObject(m_nmsLiveKeptTime); // this bar is stronger: remove the weaker opposite arrow
|
|
else
|
|
accept = false; // the kept opposite is stronger: suppress this bar
|
|
}
|
|
}
|
|
if(dir == Buy)
|
|
{
|
|
m_nmsLiveBuyTime = barTime;
|
|
m_nmsLiveBuyAccept = accept;
|
|
}
|
|
else
|
|
{
|
|
m_nmsLiveSellTime = barTime;
|
|
m_nmsLiveSellAccept = accept;
|
|
}
|
|
if(accept)
|
|
{
|
|
m_nmsLiveKeptTime = barTime;
|
|
m_nmsLiveKeptDir = dir;
|
|
m_nmsLiveKeptConf = conf;
|
|
}
|
|
return accept;
|
|
}
|
|
int PurgeChart(void);
|
|
ENUM_SIGNAL DoubleToSignal(double value);
|
|
//--- Shared status-label formatting for all three of Train()'s era passes (pass 1 sequential scan/
|
|
//--- display, pass 2 shuffled backProp, pass 3 post-training OOS scoring) - see m_isTrainQueue's and
|
|
//--- m_isPass2Active's declaration comments for why the era loop is now three passes instead of one.
|
|
//--- Extracted so the panel keeps updating every bar throughout ALL three passes instead of freezing
|
|
//--- during pass 2/3 the way it did when this was pass 1-only inline code - a stalled-looking panel
|
|
//--- during a still-running era was reported as looking "stuck". Throttled internally (see
|
|
//--- m_lastStatusLabelUpdateTick) - SetStatusLabel() (System\StatusLabel.mqh) does real text-layout
|
|
//--- work (TextGetSize/WrapLineInto per line) AND an unconditional ChartRedraw(0) every call, which
|
|
//--- gets slower as more chart objects accumulate over a long backtest. Calling it unconditionally
|
|
//--- once per bar was already the existing (pass-1-only) behavior; doing that again independently in
|
|
//--- pass 2 AND pass 3 roughly TRIPLED total ChartRedraw() calls per era and was the actual cause of
|
|
//--- a training run observed taking 1.5+ hours without completing era 0 - not the shuffle itself.
|
|
//--- forceRefresh=true bypasses the throttle below - used exactly once per era, right after the
|
|
//--- era-end block (Train()) finalizes m_eraCount/dOosForecast, so the panel's "Era %d"/"OOS Acc"
|
|
//--- fields update in the SAME moment the console's "training in progress - era N" line does.
|
|
//--- Without it, the panel's last real redraw during a normal (throttled) bar-scan call happened
|
|
//--- DURING pass 3, before m_eraCount++ - so it kept showing era N-1's number (with what was, by
|
|
//--- then, already era N's near-final accuracy) for this era's entire duration, only catching up to
|
|
//--- the correct era number once the NEXT era's own bar-scan calls started - a full one-era-behind
|
|
//--- display lag relative to the console log, reported in practice.
|
|
void UpdateTrainingStatusLabel(const string &progressLine, double neuron0, double neuron1, double neuron2, double signalValue, bool forceRefresh = false);
|
|
//--- Per-instance (NOT a function-local static - see CExpertSignalCustom::Direction()'s declaration
|
|
//--- comment for why that distinction matters for a method shared across PAI/CONV/LSTM instances)
|
|
//--- wall-clock throttle gate for UpdateTrainingStatusLabel()'s ChartRedraw().
|
|
uint m_lastStatusLabelUpdateTick;
|
|
//--- Last values passed to UpdateTrainingStatusLabel() - cached (updated on EVERY call, throttled
|
|
//--- or not) so the forced era-end refresh above has something real to redraw with instead of a
|
|
//--- stale/zeroed placeholder, since no "current bar" exists once an era's own three passes are done.
|
|
double m_lastDisplayNeuron0, m_lastDisplayNeuron1, m_lastDisplayNeuron2, m_lastDisplaySignal;
|
|
//--- Latest OOS Buy/Sell recall (-1 = n/a, same convention as logBuyRecallPct etc.), cached the same
|
|
//--- way as m_lastDisplayNeuron0 above so UpdateTrainingStatusLabel() can show it on every call, not
|
|
//--- just the era-end one that actually just computed it. Surfaced on-chart (not just the Experts
|
|
//--- log) because a Buy/Sell-diluted-by-Neutral headline accuracy number is what a trader watching
|
|
//--- the panel sees by default, but Buy/Sell recall is what actually predicts trading performance -
|
|
//--- Neutral is "don't trade," so a model can look good on blended accuracy purely by calling Neutral
|
|
//--- often, while its actual Buy/Sell calls are unreliable.
|
|
int m_lastBuyRecallPct, m_lastSellRecallPct;
|
|
//--- turns the classification output layer's 3 values (TempData[0..2], SIGMOID activation - see
|
|
//--- BuildFreshTopology() - each already in [0,1]) into a softmax probability distribution in
|
|
//--- place, and returns the signed dPrevSignal convention (+P(buy), -P(sell), 0.0 exactly for
|
|
//--- neutral) used by both Train()'s live-forecast branch and RefreshLatestSignal(). Softmax is
|
|
//--- monotonic per-element so it can't change which of the 3 wins - it only exists to turn the 3
|
|
//--- independently-trained values into a normalized confidence. Max-subtracted before exp() for
|
|
//--- numerical stability regardless (safe since softmax(x) == softmax(x - max(x))).
|
|
double ApplyClassificationSoftmax(void);
|
|
//--- Post-hoc logit adjustment / prior correction: reads the raw softmax probabilities
|
|
//--- ApplyClassificationSoftmax() just left in TempData[0..2] and returns the PRIOR-CORRECTED signed
|
|
//--- decision (same +P'(buy)/-P'(sell)/0-neutral convention). This is the exact rule live trading
|
|
//--- fires on and the rule the live-fired precision metric scores. See the definition for the math.
|
|
double AdjustedSignalFromSoftmax(void);
|
|
//--- EMA-updates the persisted true class base rates (m_priorBuy/Sell/Neutral) from a just-finished
|
|
//--- era's true class counts. No-op on an empty/degenerate tally.
|
|
void UpdateClassPriors(long buyCnt, long sellCnt, long neutralCnt);
|
|
//--- Installs tau*log(prior_c) on Net from the freshly measured priors. Called once per era
|
|
//--- start, straight after UpdateClassPriors, so the offsets track the same distribution the
|
|
//--- era is scored against. No-op (and actively clears stale offsets) when the input is off.
|
|
void ApplyLogitAdjustment(void);
|
|
//--- Small binary sidecar (fileName + ".stats") persisting the calibration state that must survive a
|
|
//--- restart for live trading to behave like training: the true class priors and m_confidenceCalScale.
|
|
bool SaveModelStats(string fileName, bool common);
|
|
bool LoadModelStats(string fileName, bool common);
|
|
//--- Deploy-time (chart, backend present) self-check: runs the just-saved deployed model through both
|
|
//--- the backend and a temporary pure-MQL5 (CNet::SetCpuInference) clone on the same input window and
|
|
//--- returns true only if the outputs match within CPU_INFERENCE_MAX_DIFF. Gates whether an
|
|
//--- inference-only backtest may run DLL-free. Fails safe (returns false) on any error/mismatch or an
|
|
//--- architecture whose CPU path isn't ported yet - the caller then keeps the model on the DLL path.
|
|
bool ValidateCpuInference(void);
|
|
//--- Build the panel's "Buy/Sell accuracy: IS x% OOS y%" line (directional win-rate, Neutral excluded)
|
|
//--- from the cumulative counts (m_cumIsCorrect etc.); returns "...: measuring..." until at least one
|
|
//--- directional call has been validated. Shared by the training and live/complete simple panels.
|
|
string ComputeCompoundedAccuracyLine(void);
|
|
//--- Persist/restore the drawn directional arrows (the "WarSig_" objects) to a sidecar file so they
|
|
//--- survive an EA remove/re-add, recompile, or restart WITHOUT a retrain - the chart objects are
|
|
//--- destroyed on unload (destructor PurgeChart) and OnInit has no other way to bring them back.
|
|
//--- Stores each arrow's time/code/price and its hide state (OBJPROP_TIMEFRAMES), so the show/hide
|
|
//--- toggle is preserved too. Chart-only (a backtest has no persistent chart to restore to).
|
|
bool SaveChartSignals(bool pruneChartObjects = true);
|
|
void LoadChartSignals(void);
|
|
//--- The shutdown half of that pair: persist, THEN clear the chart, and report both counts. See the
|
|
//--- definition for why the order is fixed and why the clear is conditional on the write.
|
|
void PersistAndClearChartSignals(void);
|
|
//--- How many arrows the last successful SaveChartSignals() wrote - reporting only.
|
|
int m_lastArrowsSaved;
|
|
//--- Wipe this model's drawn arrows AND their .arrows sidecar, plus any deferred restore still in
|
|
//--- flight. Call from every path that discards or replaces the trained weights - see the definition
|
|
//--- for why leaving them behind resurrects a dead model's calls through SaveChartSignals.
|
|
void ClearPersistedChartSignals(const string reason);
|
|
//--- Deferred ("async") half of LoadChartSignals: LoadChartSignals only PARSES the sidecar into the
|
|
//--- m_arrowRestore* buffers (an ~80KB read - instant) and returns, so OnInit never blocks; this then
|
|
//--- creates the chart objects in ARROW_RESTORE_BUDGET_MS slices, driven by the same 500ms timer that
|
|
//--- already paces training. MQL5 has no threads - a chart runs one thread - so blocking OnInit is what
|
|
//--- made the terminal look frozen (no panel, no status label, no journal) while ~2900 arrows were
|
|
//--- rebuilt. Time-boxed slices give the terminal room to paint the UI between them instead.
|
|
void AdvanceChartSignalRestore(void);
|
|
//--- parsed-but-not-yet-drawn arrows, consumed by AdvanceChartSignalRestore (see above)
|
|
datetime m_arrowRestoreTime[];
|
|
int m_arrowRestoreCode[];
|
|
double m_arrowRestorePrice[];
|
|
long m_arrowRestoreTf[];
|
|
int m_arrowRestoreIndex;
|
|
bool m_arrowRestorePending;
|
|
uint m_arrowRestoreStartMs;
|
|
//--- Deferred ("async") half of StartChartSignalRescan (public, defined inline further down): drains
|
|
//--- the per-bar inference loop in ARROW_RESTORE_BUDGET_MS slices off PollTraining's timer instead of
|
|
//--- blocking the button-click handler for however long a full lookback scan takes. Internal-only -
|
|
//--- called from PollTraining(), never from outside the class - so this stays protected while
|
|
//--- StartChartSignalRescan()/RescanPending() (which the panel button needs) are public.
|
|
void AdvanceChartSignalRescan(void);
|
|
int m_rescanIndex;
|
|
int m_rescanHi;
|
|
int m_rescanBarsNow;
|
|
bool m_rescanPending;
|
|
uint m_rescanStartMs;
|
|
//--- Raw (PRE prior-correction) argmax tally, accumulated per-bar across AdvanceChartSignalRescan's
|
|
//--- slices - lets the completion log distinguish "the network itself calls Neutral almost everywhere"
|
|
//--- from "the network still discriminates, but AdjustedSignalFromSoftmax's logit-prior correction is
|
|
//--- suppressing it down to Neutral" - both produce an identical all-Neutral m_arrowSignalCache/empty
|
|
//--- chart otherwise.
|
|
int m_rescanRawBuy;
|
|
int m_rescanRawSell;
|
|
int m_rescanRawNeutral;
|
|
bool ResizeBuffers(int barIndex);
|
|
bool RefreshData();
|
|
#ifdef WARRIOR_EXPORT_FEATURES
|
|
//--- RESEARCH BUILD ONLY, never compiled into a shipped binary. Dumps exactly what the network sees -
|
|
//--- one row per bar: index, time, OHLC, ATR, then the m_neuronsCount feature values - to a CSV under
|
|
//--- Common\Files\Warrior_EA\Research\. Exporting the RAW BARS alongside the features is the point:
|
|
//--- with OHLC+ATR in hand every barrier geometry, horizon and in-trade target can be recomputed
|
|
//--- offline, so a research question costs seconds in Python instead of a compile/attach/read cycle.
|
|
void ExportFeatureMatrix(void);
|
|
//--- Raw OHLCV for a grid of symbols/timeframes - see the definition for why the grid is worth more
|
|
//--- than the engineered features on their own.
|
|
void ExportRawRates(void);
|
|
#endif
|
|
bool BufferTempData(int idx);
|
|
//--- Assembles the full m_historyBars-wide input window ending AT bar r into TempData, OLDEST BAR
|
|
//--- FIRST. Use this everywhere instead of hand-rolling the loop: the chronological order is load-
|
|
//--- bearing for the LSTM/HYBRID stacks and cannot be enforced by convention across eight call
|
|
//--- sites. See the definition comment in AIBase\Features.mqh for the measurement behind that.
|
|
bool BuildFeatureWindow(int r);
|
|
//--- shared by OnTickHandler() and the timer-driven PollTraining() - see definition
|
|
void ScheduleTrainingIfNeeded(void);
|
|
void Train(datetime StartTrainBar = 0);
|
|
//--- outer loop around Train(): when AutoTuneIndicators is on, tries randomized AD indicator
|
|
//--- input variations across m_indicatorTuneTrials calls to Train(), keeping the best-OOS one
|
|
void TuneIndicatorsAndTrain(datetime StartTrainBar = 0);
|
|
//--- recomputes dPrevSignal/chart arrow for the most recent bar; used after restoring a
|
|
//--- checkpointed model at the end of Train() so the live signal matches the deployed weights
|
|
void RefreshLatestSignal();
|
|
//--- inference-only "new bar" handler used once m_trainingComplete is true - see
|
|
//--- ScheduleTrainingIfNeeded()'s declaration comment for why this must NOT call Net.backProp()
|
|
void RefreshConvergedSignal(void);
|
|
//--- Online continual-learning step (live chart only) - see its implementation comment and the
|
|
//--- ONLINE_LEARN_* tunables. Backprops the deployed Net on bars whose ZigZag label has just become
|
|
//--- CONFIRMED (m_swingConfirmationBars matured), then blends the shadow under a rolling-accuracy
|
|
//--- guardrail. No-op in the tester/optimizer (m_inferenceOnly) and while training is active.
|
|
void OnlineLearnStep(void);
|
|
//--- Alpha-balanced focal sample weight (Lin et al. 2017 eq. 5) for ONE streamed bar - see the
|
|
//--- ONLINE_LEARN_* block's CLASS IMBALANCE comment for the derivation. Shared deliberately by
|
|
//--- OnlineLearnStep() (the live path) and AdvanceOosSimulationChunk() (the simulation of that
|
|
//--- path): the simulation's reported accuracy is only a valid forecast of live continual-learning
|
|
//--- behaviour if it optimises the IDENTICAL objective, so the formula must exist in exactly one
|
|
//--- place. p* are this bar's pre-update softmax probabilities, already normalised in place by
|
|
//--- ApplyClassificationSoftmax(). Returns 1.0 for the regression head (no class structure).
|
|
double OnlineSampleWeight(ENUM_SIGNAL trueSignal, double pBuy, double pSell, double pNeutral);
|
|
//--- lazily bootstraps m_shadowNet if it's still NULL: tries loading a persisted shadow file
|
|
//--- first (continuity across EA restarts), falling back to cloning Net's current weights (via
|
|
//--- the same Save()/Load() pattern StartOosContinualSimulation() uses for m_simOosNet) if no
|
|
//--- compatible shadow file exists yet. No-op if m_shadowNet is already valid. See m_shadowNet's
|
|
//--- declaration comment for the full EMA shadow-weight deployment rationale.
|
|
void EnsureShadowNet(void);
|
|
//--- persists m_shadowNet alongside every Net.Save() call, using the same run metadata (error/
|
|
//--- undefine/forecast/era/trainingComplete/indicator params) the caller already computed for
|
|
//--- Net.Save() itself - see m_shadowNet's declaration comment. No-op if the shadow isn't
|
|
//--- bootstrapped yet.
|
|
void SaveShadowNet(const double &indicatorParams[]);
|
|
//--- method of initialization of the indicators
|
|
bool InitOpen(CIndicators *indicators);
|
|
bool InitClose(CIndicators *indicators);
|
|
bool InitHigh(CIndicators *indicators);
|
|
bool InitLow(CIndicators *indicators);
|
|
bool InitVolumes(CIndicators *indicators);
|
|
bool InitTime(CIndicators *indicators);
|
|
//--- addToCollection=false is used by ReInitADIndicators() to rebuild an already-collected
|
|
//--- handle's params (here: a re-tuned period) without re-adding the (same) pointer into
|
|
//--- indicators a second time
|
|
bool InitMA(CIndicators *indicators, bool addToCollection = true);
|
|
bool InitRSI(CIndicators *indicators, bool addToCollection = true);
|
|
bool InitMACDFeature(CIndicators *indicators, bool addToCollection = true);
|
|
bool InitIchimoku(CIndicators *indicators, bool addToCollection = true);
|
|
bool InitADCumulativeDelta(CIndicators *indicators, bool addToCollection = true);
|
|
bool InitADShorteningOfThrust(CIndicators *indicators, bool addToCollection = true);
|
|
bool InitADWyckoffEventStream(CIndicators *indicators, bool addToCollection = true);
|
|
bool InitADWyckoffFailedStructure(CIndicators *indicators, bool addToCollection = true);
|
|
bool InitADWyckoffSignificantBarInversion(CIndicators *indicators, bool addToCollection = true);
|
|
bool InitADZigZag(CIndicators *indicators, bool addToCollection = true);
|
|
//--- common=false targets a LOCAL (non-shared) file - used by the tester/optimizer per-agent
|
|
//--- weight cache so cross-pass reuse never touches the production FILE_COMMON config/weights.
|
|
bool SaveTopologyConfiguration(string fileName, int initialNeuronsCount, int hiddenLayersCount, double neuronsReduction, int minNeuronsCount, int optimizationAlgo, int historyBars, int outputNeuronsCount, int neuronsCount, int studyPeriod, int minTrainYear, bool isInitialized, int stopTrainWR, int fractalPeriods, int convFilterCount, int lstmHiddenSize, bool common = true);
|
|
//--- The four DERIVED shape fields are by REFERENCE and are ADOPTED from the .cfg, not compared
|
|
//--- against it. See the block in the definition for why a derived value must never be able to
|
|
//--- mismatch: it is measured from data that legitimately changes, and a mismatch here discards
|
|
//--- a trained model. studyPeriod left the parameter list entirely - the input is gone; its
|
|
//--- on-disk slot is still read positionally and ignored, like the retired MinWR slot.
|
|
bool LoadAndCompareTopologyConfiguration(string fileName, int &initialNeuronsCount, int &hiddenLayersCount, double neuronsReduction, int minNeuronsCount, int optimizationAlgo, int historyBars, int outputNeuronsCount, int neuronsCount, int minTrainYear, bool isInitialized, int stopTrainWR, int fractalPeriods, int &convFilterCount, int &lstmHiddenSize, bool common = true);
|
|
//--- Retry helpers for the tester/opt seed-copy race: a live chart's own atomic Save() (write .savetmp,
|
|
//--- then FileMove() over the real file) can hold the source or destination file for a moment, and a
|
|
//--- concurrent FileCopy/FileOpen from a Strategy Tester agent reading the SAME production file can hit
|
|
//--- a transient Windows sharing violation in that narrow window. Both retry a handful of times with a
|
|
//--- short pause rather than silently treating a transient lock as "no model"/"corrupt file" - see their
|
|
//--- call sites in InitNeuralNetwork.
|
|
bool CopyFileWithRetry(string srcFileName, string dstFileName);
|
|
bool CopySharedFile(string srcFileName, string dstFileName, bool quiet);
|
|
bool LoadNetWithRetry(double &indicatorParams[]);
|
|
//--- input data
|
|
bool m_useVolumes;
|
|
bool m_useTime;
|
|
bool m_useATR;
|
|
//--- Uses its own period (m_indicatorTuner.maPeriod), fed as ATR-normalized OHLC distance-from-MA
|
|
//--- (4 values, same convention as the base close-open/high-open/low-open features) plus the MA's
|
|
//--- own bar-over-bar change (1 value, ATR-normalized like every other price-domain feature here -
|
|
//--- not volume's previous-bar-ratio scheme, since a moving average lives in price units and
|
|
//--- already has ATR as its natural scale reference). See BufferTempDataCompute()'s m_useMA block
|
|
//--- for the exact 5 values. maPeriod starts equal to the Classic Signals PeriodMA input (see
|
|
//--- CADIndicatorTuner's constructor) but may diverge from it once AutoTuneIndicators searches a
|
|
//--- trial - the Classic Signals MA vote itself is untouched by that search, since it needs no
|
|
//--- training/warm-up and there is nothing for a tuning trial to validate it against.
|
|
bool m_useMA;
|
|
//--- RSI is already a 0-100 oscillator, so the only transform needed is /100 to match every other
|
|
//--- feature's roughly [-1,1]/[0,1] scale - no ATR or distance normalization applies. See
|
|
//--- BufferTempDataCompute()'s m_useRSI block for the exact value. Same maPeriod/rsiPeriod
|
|
//--- divergence-from-the-Classic-Signals-input note as m_useMA above applies to rsiPeriod.
|
|
bool m_useRSI;
|
|
//--- MACD as 3 ATR-normalized values (main line, signal line, histogram) - see
|
|
//--- BufferTempDataCompute()'s m_useMACD block. Deliberately kept to 3 despite MACD being cheap: the
|
|
//--- point of adding it next to m_useMA is the SECOND timescale (m_useMA supplies exactly one moving
|
|
//--- average) and the histogram, which is the only acceleration term anywhere in the feature vector -
|
|
//--- the level information itself is already covered by the MA distances. ATR-normalized rather than
|
|
//--- left raw because the MACD lines live in price units, exactly like the MA feature.
|
|
bool m_useMACD;
|
|
//--- Ichimoku as 8 values - see BufferTempDataCompute()'s m_useIchimoku block for each. This is the
|
|
//--- widest single classic-indicator feature here and it earns that width by carrying multi-timescale
|
|
//--- support/resistance GEOMETRY (three lookbacks plus a forward-projected cloud) that nothing else in
|
|
//--- the vector encodes: the swing-context block's confirmed pivots are >=m_swingConfirmationBars bars
|
|
//--- stale by construction, and its Donchian/SMA values are single-scale.
|
|
//--- LOOKAHEAD - MT5's iIchimoku stores raw per-bar values and shifts only the DRAWING, so the cloud
|
|
//--- sitting under bar idx is SenkouSpan*(idx + ichiKijun), and the Chikou value plotted at bar idx
|
|
//--- would be Close(idx - ichiKijun) - a FUTURE bar. The feature block applies the +Kijun offset and
|
|
//--- never calls ChinkouSpan(); Signals\SignalIchimoku.mqh's class comment documents the buffer
|
|
//--- convention in full, and the same reasoning governs both.
|
|
bool m_useIchimoku;
|
|
//--- Normalized ZigZag swing-context features: 5 confirmed-pivot values (direction/magnitude/age of
|
|
//--- the last CONFIRMED swing) plus 4 recent-price-action values (Donchian range position at 20/50
|
|
//--- bars, 20-bar return, 20-bar SMA extension) that give fresh, non-repainting trend/position
|
|
//--- context the >=100-bar-stale pivot anchor can't - see BufferTempDataCompute()'s m_useSwingContext
|
|
//--- block for the exact 9 values. Reads the same
|
|
//--- m_ADZigZag the training labels already come from (see that member's declaration comment) rather
|
|
//--- than a separate indicator instance - NOT gated behind AutoTuneIndicators/ReInitADIndicators like
|
|
//--- the AD* indicators below, since m_ADZigZag itself is deliberately never tuned (same reason as
|
|
//--- the label side: tuning the ground truth alongside the model scored against it would let a trial
|
|
//--- cherry-pick an easier target). Critical correctness constraint, not just a style choice: ZigZag's
|
|
//--- most recent 1-3 legs repaint (see m_swingConfirmationBars' declaration comment), so every read of
|
|
//--- m_ADZigZag for THIS feature - exactly like the label side - must only trust a pivot that is
|
|
//--- already at least m_swingConfirmationBars bars old relative to the bar the feature is being
|
|
//--- computed for. Skipping that embargo would leak future information a live bar couldn't actually
|
|
//--- have had yet - silent lookahead bias inflating backtest/training performance without being real.
|
|
bool m_useSwingContext;
|
|
//--- see System\NewsRelevance.mqh's declaration comment for what this feature actually encodes
|
|
//--- (event proximity + impact, not actual-vs-forecast deviation) and why the forward-looking half
|
|
//--- of it isn't lookahead bias.
|
|
bool m_useNews;
|
|
int m_newsFeatureWindowMinutes;
|
|
//--- Cross-asset panel: the only feature block here whose inputs are NOT a transform of this
|
|
//--- symbol's own OHLCV series. See System\CrossAsset.mqh for the reasoning; in short, every other
|
|
//--- feature the network sees is a function of one price series, and that whole family measured at
|
|
//--- the noise floor, so the panel exists to give it information a single series cannot contain.
|
|
//--- Built ONCE per training run (BuildCrossAssetPanel) rather than per bar - a per-bar cross-symbol
|
|
//--- lookup would be pairs x bars iBarShift calls.
|
|
bool m_useCrossAsset;
|
|
CCrossAssetPanel m_crossAsset;
|
|
bool BuildCrossAssetPanel(int bars);
|
|
//--- Spread as a feature. The only microstructure channel that is BOTH available on FX and
|
|
//--- genuinely historical in the Strategy Tester ("during testing, the spread is not modeled but
|
|
//--- is taken from historical data") - unlike swap (no history at all), signed tick flow
|
|
//--- (TICK_FLAG_BUY/SELL are empty on FX) or depth of market (absent on retail FX, never replayed).
|
|
//--- Measured as the strongest single feature in research/test_spread.py, though see the feature
|
|
//--- block for what it actually encodes and why that is less than it first appears.
|
|
//--- Series is copied ONCE per bar grid, not per bar: CopySpread is a range call, not a lookup.
|
|
bool m_useSpreadFeature;
|
|
int m_spreadSeries[];
|
|
int m_spreadSeriesBars;
|
|
//--- Newest bar the copy was anchored to. MQL5 series indices are relative to NOW, so a single
|
|
//--- new closed candle shifts every index by one: a cache keyed only on length would keep serving
|
|
//--- index 0 as a bar that is no longer the newest, silently misaligning the spread series against
|
|
//--- the price buffers it must line up with. Same invalidation key the label/feature bar caches
|
|
//--- use (see EnsureBarCachesCapacity) and the same failure the zero-direction hunt traced.
|
|
datetime m_spreadSeriesAnchor;
|
|
datetime m_crossAssetAnchor;
|
|
bool EnsureSpreadSeries(int bars);
|
|
bool m_useADCumulativeDelta;
|
|
bool m_useADShorteningOfThrust;
|
|
bool m_useADWyckoffEventStream;
|
|
bool m_useADWyckoffFailedStructure;
|
|
bool m_useADWyckoffSignificantBarInversion;
|
|
|
|
public:
|
|
CExpertSignalAIBase(void);
|
|
~CExpertSignalAIBase(void);
|
|
//--- "voting" that price will grow/fall, common to every AI signal (single market model)
|
|
virtual int LongCondition(void);
|
|
virtual int ShortCondition(void);
|
|
// |dPrevSignal| is already a 0..1 confidence for classification output (softmax
|
|
// probability of the winning class) and typically bounded for regression output
|
|
// (tanh-activated network); OpenParams() clamps regardless. Scaled by m_confidenceCalScale
|
|
// (classification head only - 1.0/no-op for regression) so callers get an empirically
|
|
// calibrated magnitude instead of the raw, uncalibrated softmax value - see
|
|
// m_confidenceCalScale's declaration comment.
|
|
double CalibratedConfidenceMagnitude(void) const
|
|
{
|
|
double mag = MathAbs(dPrevSignal);
|
|
if(!MathIsValidNumber(mag))
|
|
return 0.0;
|
|
if(m_outputNeuronsCount == 3)
|
|
mag = MathMin(1.0, mag * m_confidenceCalScale);
|
|
if(!MathIsValidNumber(mag))
|
|
return 0.0;
|
|
return mag;
|
|
}
|
|
virtual double AIConfidence(void) override { return CalibratedConfidenceMagnitude(); }
|
|
// Signed for direction-aware use (AI-driven early exit): sign matches dPrevSignal's
|
|
// convention (+ buy, - sell, 0 neutral/no signal yet). dPrevSignal == -2 is the
|
|
// "not yet studied" sentinel, not a real sell signal - treat it as no confidence.
|
|
virtual double SignedAIConfidence(void) override
|
|
{
|
|
if(dPrevSignal == -2)
|
|
return 0.0;
|
|
double sign = (dPrevSignal > 0.0) ? 1.0 : (dPrevSignal < 0.0) ? -1.0 : 0.0;
|
|
if(sign == 0.0)
|
|
return 0.0;
|
|
return sign * CalibratedConfidenceMagnitude();
|
|
}
|
|
//--- event handlers, common to every AI signal
|
|
virtual void OnTickHandler(void);
|
|
//--- drives the same training-scheduling check as OnTickHandler(), but callable from a timer so
|
|
//--- it isn't dependent on ticks (which don't arrive while the market is closed)
|
|
void PollTraining(void);
|
|
virtual void OnChartEventHandler(const int id,
|
|
const long &lparam,
|
|
const double &dparam,
|
|
const string &sparam);
|
|
//--- methods of adjusting "weights" of the 4 confidence-tier market models - see m_pattern_0's
|
|
//--- declaration comment
|
|
void Pattern_0(int value) { m_pattern_0 = value; }
|
|
void Pattern_1(int value) { m_pattern_1 = value; }
|
|
void Pattern_2(int value) { m_pattern_2 = value; }
|
|
void Pattern_3(int value) { m_pattern_3 = value; }
|
|
virtual void ApplyPatternWeight(int patternNumber, int weight);
|
|
//--- buckets the live confidence magnitude into one of the 4 tiers above - see m_pattern_0's
|
|
//--- declaration comment. Public so PollTraining()/status-display code could surface which tier is
|
|
//--- currently active if ever useful, though LongCondition/ShortCondition are the only callers today.
|
|
int ConfidenceTier(void);
|
|
int PatternWeightForTier(int tier);
|
|
//--- methods of setting adjustable parameters
|
|
//--- No public setter for m_initialNeuronsCount. It feeds the weights-filename fingerprint, and it is
|
|
//--- now derived exactly once, inside InitNeuralNetwork(), before that fingerprint is built - see
|
|
//--- ComputeFirstLayerWidth(). An external setter could only ever be called after construction and
|
|
//--- would either be ignored (if before init) or silently re-key the model mid-run (if after).
|
|
void OutputNeuronsCount(int value) { m_outputNeuronsCount = value; }
|
|
//--- No setter: the taper's endpoints are derived, not configured. See BuildFreshTopology()'s taper
|
|
//--- block and the note in Variables\Inputs.mqh. Kept as members only because the .cfg topology
|
|
//--- sidecar's field layout is positional and rewriting it would invalidate every model on disk.
|
|
void HiddenLayersCount(int value) { m_hiddenLayersCount = value; }
|
|
void LstmHiddenSize(int value) { m_lstmHiddenSize = value; }
|
|
void ConvFilterCount(int value) { m_convFilterCount = value; }
|
|
//--- StopTrainWR(int) removed with the MinWR input - there is no absolute accuracy target any more;
|
|
//--- see LEGACY_CONVERGE_WR_SLOT and the plateau ladder.
|
|
void MinDirectionalRecall(int value) { m_minDirectionalRecallPct = value; }
|
|
//--- MinSignalConfidence(double) removed with the AI entry floor - confidence now reaches the trade
|
|
//--- decision as vote weight (ConfidenceTier), gated by the one Min vote to open threshold that the
|
|
//--- classic votes already answer to. See m_pattern_0's declaration comment.
|
|
//--- The SINGLE class-imbalance control - tau in Menon et al.'s logit adjustment. 0 disables the
|
|
//--- correction entirely. OversampleParity/EnableMinorityReplay/ConstrainReplay/LogitPriorStrength/
|
|
//--- UseLogitAdjustedLoss/FocalLossGamma/UseStaticPrior were removed 2026-07-31; see the
|
|
//--- class-imbalance block in Variables\Inputs.mqh for the audit that found five of them inert.
|
|
void LogitAdjustTau(double value) { m_logitAdjustTau = MathMax(0.0, value); }
|
|
void FreezePriorCalibration(bool value) { m_freezePriorCalibration = value; }
|
|
void SignalClusterWindow(int value) { m_signalClusterWindow = value; }
|
|
void SwingConfirmationBars(int value) { m_swingConfirmationBars = value; }
|
|
void EnableOnlineLearning(bool value) { m_enableOnlineLearning = value; }
|
|
void MaxErasPerRun(int value) { m_maxErasPerRun = value; }
|
|
void OOSSplit(int value) { m_oosSplitPct = value; }
|
|
void HistoryBars(int value) { m_historyBars = value; }
|
|
void MinTrainYear(int value) { m_minTrainYear = value; }
|
|
void UseVolumes(bool value) { m_useVolumes = value; }
|
|
void UseTime(bool value) { m_useTime = value; }
|
|
void UseATR(bool value) { m_useATR = value; }
|
|
void UseMA(bool value) { m_useMA = value; }
|
|
void UseRSI(bool value) { m_useRSI = value; }
|
|
void UseMACD(bool value) { m_useMACD = value; }
|
|
void UseIchimoku(bool value) { m_useIchimoku = value; }
|
|
void UseSwingContext(bool value) { m_useSwingContext = value; }
|
|
void UseNews(bool value) { m_useNews = value; }
|
|
void NewsFeatureWindowMinutes(int value) { m_newsFeatureWindowMinutes = value; }
|
|
void UseCrossAsset(bool value) { m_useCrossAsset = value; }
|
|
void UseSpreadFeature(bool value) { m_useSpreadFeature = value; }
|
|
void UseADCumulativeDelta(bool value) { m_useADCumulativeDelta = value; }
|
|
void UseADShorteningOfThrust(bool value) { m_useADShorteningOfThrust = value; }
|
|
void UseADWyckoffEventStream(bool value) { m_useADWyckoffEventStream = value; }
|
|
void UseADWyckoffFailedStructure(bool value) { m_useADWyckoffFailedStructure = value; }
|
|
void UseADWyckoffSignificantBarInversion(bool value) { m_useADWyckoffSignificantBarInversion = value; }
|
|
void AutoTuneIndicators(bool value) { m_autoTuneIndicators = value; }
|
|
//--- control-panel API (Warrior_EA.mq5): current-config-only training/weights control.
|
|
//--- "current config" == this signal instance's own m_fileName (symbol+period+id+topology),
|
|
//--- never touches another signal type's or another symbol/timeframe's saved files.
|
|
void PauseTraining(void) { m_trainingPaused = true; PrintVerbose(ID + ": training paused by user (era " + IntegerToString(m_eraCount) + ")"); }
|
|
void ResumeTraining(void) { m_trainingPaused = false; PrintVerbose(ID + ": training resumed by user (era " + IntegerToString(m_eraCount) + ")"); }
|
|
bool IsTrainingPaused(void) const { return m_trainingPaused; }
|
|
bool IsTrainingStopped(void) const { return m_trainingStopRequested; }
|
|
bool TrainingComplete(void) const { return m_trainingComplete; }
|
|
//--- Set by OnDeinit before it calls StopTraining(), so FinalizeTrainRun() can tell a user-pressed Stop
|
|
//--- (persist the deployed model now - nothing else will) from a shutdown (PersistWeightsOnShutdown is
|
|
//--- moments away and writes the same bytes). See the guard in FinalizeTrainRun.
|
|
void MarkShutdown(void) { m_shutdownInProgress = true; }
|
|
void StopTraining(void)
|
|
{
|
|
m_trainingStopRequested = true;
|
|
m_trainingPaused = false;
|
|
//--- ScheduleTrainingIfNeeded() refuses to schedule another "New Bar" event while
|
|
//--- m_trainingStopRequested is set, so a run interrupted mid-chunk would otherwise never get
|
|
//--- called again to finalize (restore the best checkpoint, persist state) - do it synchronously
|
|
//--- here instead. Safe to block briefly: this is just a checkpoint file restore, not the
|
|
//--- multi-minute bar loop.
|
|
if(m_trainRunActive)
|
|
FinalizeTrainRun();
|
|
Print(ID + ": training stopped by user (era " + IntegerToString(m_eraCount) + ", weights as of last completed era retained)");
|
|
PrintInferenceTally();
|
|
}
|
|
//--- Inference-path census, printed at shutdown. WHY: a 2026-07-31 backtest of a CONVERGED CONV model
|
|
//--- produced "Final directional result: 0.00000000" on every one of 1744 bars and therefore ZERO
|
|
//--- trades, and nothing in the log could distinguish the three candidate causes - RefreshLatestSignal
|
|
//--- never running, running but bailing at one of its two early returns, or running fine and the model
|
|
//--- genuinely answering Neutral every time. Each implies a completely different fix. Counting is the
|
|
//--- cheapest way to tell them apart and it costs nothing per bar.
|
|
//--- Records one directional decision's passage through the readiness gate in LongCondition()/
|
|
//--- ShortCondition(). Called with `directional` = "this condition's own class is what the model
|
|
//--- actually answered", so a Neutral bar counts as neither blocked nor passed - the question being
|
|
//--- measured is what happened to the calls that HAD something to say.
|
|
void NoteVoteGate(bool directional)
|
|
{
|
|
if(!directional)
|
|
return;
|
|
bool open = m_trainingComplete || (m_inferenceOnly && m_modelLoadedFromDisk);
|
|
if(m_voteGateCompleteAtFirst < 0)
|
|
{
|
|
m_voteGateCompleteAtFirst = (int)m_trainingComplete;
|
|
m_voteGateLoadedAtFirst = (int)m_modelLoadedFromDisk;
|
|
}
|
|
if(open)
|
|
m_voteGatePassed++;
|
|
else
|
|
m_voteGateBlocked++;
|
|
}
|
|
void PrintInferenceTally(void)
|
|
{
|
|
long attempts = m_refreshOk + m_refreshFailFeatures + m_refreshFailShort;
|
|
if(attempts <= 0)
|
|
{
|
|
Print(ID + ": inference census - RefreshLatestSignal was NEVER CALLED (0 attempts). The new-bar gate never fired.");
|
|
return;
|
|
}
|
|
Print(ID + ": inference census - ", attempts, " refresh attempts: ", m_refreshOk, " completed, ",
|
|
m_refreshFailFeatures, " bailed in BufferTempData, ", m_refreshFailShort, " bailed on a short feature window",
|
|
" | decisions Buy:", m_refreshBuy, " Sell:", m_refreshSell, " Neutral:", m_refreshNeutral);
|
|
//--- Second half of the census, and the half that separates "the model said nothing" from "the model
|
|
//--- spoke and was not allowed to vote" - see m_voteGateBlocked for why that distinction is the whole
|
|
//--- point. A run with directional decisions above and voteGate passed:0 below is the readiness-gate
|
|
//--- failure, NOT a Neutral model, and the fix is on the model-load path.
|
|
if(m_voteGateCompleteAtFirst < 0)
|
|
Print(ID + ": inference census - vote gate was NEVER REACHED (no directional decision ever hit "
|
|
"LongCondition/ShortCondition). Either every decision was Neutral, or this filter was never polled.");
|
|
else
|
|
Print(ID + ": inference census - vote gate passed:", m_voteGatePassed, " blocked:", m_voteGateBlocked,
|
|
" | at first vote trainingComplete=", (m_voteGateCompleteAtFirst != 0 ? "true" : "false"),
|
|
" modelLoadedFromDisk=", (m_voteGateLoadedAtFirst != 0 ? "true" : "false"),
|
|
" inferenceOnly=", (m_inferenceOnly ? "true" : "false"),
|
|
(m_voteGateBlocked > 0 && m_voteGatePassed == 0
|
|
? " <-- EVERY directional call was discarded here. This is the zero-direction cause."
|
|
: ""));
|
|
}
|
|
void StartTraining(void)
|
|
{
|
|
if(!m_trainingStopRequested && !m_trainingPaused)
|
|
return;
|
|
m_trainingStopRequested = false;
|
|
m_trainingPaused = false;
|
|
if(!bEventStudy)
|
|
bEventStudy = EventChartCustom(ChartID(), 1, (long)dtStudied, 0, "Resume");
|
|
Print(ID + ": training (re)started by user (era " + IntegerToString(m_eraCount) + ")");
|
|
}
|
|
//--- Has an era ever cleared the per-class recall floor and been checkpointed this run? This is the
|
|
//--- same quality bar the plateau ladder's auto-deploy requires (see PLATEAU_STAGE_DEPLOY), exposed so
|
|
//--- the panel can warn before a MANUAL deploy ships a model that ignores Buy or Sell.
|
|
bool HasRecallPassingCheckpoint(void) const { return m_bestPassedRecall; }
|
|
//--- MANUAL deploy (panel "Deploy Model"): finalise whatever the run has found so far as THE model -
|
|
//--- exactly what the plateau ladder does on its own at stage 3, just triggered early by the operator.
|
|
//--- Restores the best checkpoint (not whatever era the run happened to be mid-way through), persists
|
|
//--- weights + calibration + shadow, and flips to live inference.
|
|
//--- Deliberately does NOT set m_trainingStopRequested: like every other deploy path, a deployed model
|
|
//--- still runs live inference AND online continual learning. A panel Stop is the thing that halts
|
|
//--- everything - see StopTraining()/OnlineLearnStep()'s gate. Reversible via RetrainDeployed().
|
|
bool DeployNow(void)
|
|
{
|
|
if(CheckPointer(Net) == POINTER_INVALID || !m_isInitialized)
|
|
return false;
|
|
if(m_trainingComplete)
|
|
return true; // already deployed - nothing to do
|
|
//--- Set BEFORE any save below: the flag is written INTO the .nnw, so persisting first would store
|
|
//--- "still training" and a restart would resume the era loop instead of running the deployed model.
|
|
m_trainingComplete = true;
|
|
m_trainingPaused = false;
|
|
m_trainingStopRequested = false;
|
|
if(m_trainRunActive || m_haveOosCheckpoint)
|
|
FinalizeTrainRun(); // restores the best checkpoint, persists, ends the run
|
|
else
|
|
{
|
|
//--- Nothing trained this session (e.g. deploying a model that was just loaded from disk), so
|
|
//--- there is no in-memory checkpoint to restore - persist exactly what is loaded right now.
|
|
PersistDeployedModel();
|
|
SaveChartSignals();
|
|
}
|
|
RefreshLatestSignal();
|
|
Print(ID + ": model DEPLOYED by user at era " + IntegerToString(m_eraCount) +
|
|
" (balanced accuracy " + (m_bestBalancedOos < 0 ? "n/a" : DoubleToString(m_bestBalancedOos, 1) + "%") +
|
|
", blended OOS " + DoubleToString(dOosForecast, 1) + "%) - training stopped, now running live inference" +
|
|
(m_enableOnlineLearning ? " with online continual learning" : "") +
|
|
". Use the panel's \"Retrain Model\" to resume training from here.");
|
|
return true;
|
|
}
|
|
//--- The inverse of DeployNow(), and the ONLY way back: while m_trainingComplete is set,
|
|
//--- ScheduleTrainingIfNeeded() routes every tick to the converged/inference branch, so StartTraining()
|
|
//--- alone can never revive a deployed model (it clears the stop flag, but the complete flag still wins
|
|
//--- that branch). Clearing it here re-arms the normal training path, continuing from the DEPLOYED
|
|
//--- weights rather than from scratch - "reset weights" is the separate, destructive button for that.
|
|
//--- The plateau ladder, best-checkpoint tracking and annealed gamma all reset themselves when Train()
|
|
//--- starts its next fresh run (see the !m_trainRunActive block), so a retrain does not inherit the
|
|
//--- exhausted stage that deployed the model and immediately re-deploy it.
|
|
void RetrainDeployed(void)
|
|
{
|
|
if(!m_trainingComplete)
|
|
return;
|
|
m_trainingComplete = false;
|
|
m_trainingStopRequested = false;
|
|
m_trainingPaused = false;
|
|
//--- Persist the cleared flag immediately. Otherwise a terminal restart before the first era
|
|
//--- completes would reload the .nnw still marked complete and silently go back to inference-only,
|
|
//--- looking like the button did nothing.
|
|
PersistDeployedModel();
|
|
if(!bEventStudy)
|
|
bEventStudy = EventChartCustom(ChartID(), 1, (long)dtStudied, 0, "Retrain");
|
|
Print(ID + ": RETRAINING the deployed model from era " + IntegerToString(m_eraCount) +
|
|
" - keeping its current weights as the starting point (use \"Delete & Reset Weights\" to start from scratch instead).");
|
|
}
|
|
//--- Manual "rescan" of the drawn signal arrows: purges every arrow currently on the chart (namespaced
|
|
//--- delete - user drawings untouched) and re-infers the last SIGNAL_RESCAN_LOOKBACK_BARS bars from the
|
|
//--- CURRENTLY deployed weights, then re-runs the same end-of-era NMS declutter (PruneDirectionalClusters)
|
|
//--- used during training so the fresh set matches what a live re-render would have produced. Wired to
|
|
//--- the panel's Hide->Show Signals sequence: without this, "restore" only ever replays whatever was
|
|
//--- last saved to the .arrows sidecar, which for a long-deployed model can be a stale historical render
|
|
//--- from whenever it was last actually trained - years-old arrows crowding out anything recent. Chart-only
|
|
//--- (no persistent chart in the tester/optimizer) and a no-op until a model has something to infer with.
|
|
//--- This only does the cheap setup (buffer resize, arrow purge, cache alloc) and QUEUES the per-bar
|
|
//--- inference loop for AdvanceChartSignalRescan() to drain in time-boxed slices off the timer - see
|
|
//--- that method's comment for why the loop itself must never run in one blocking pass. Returns true
|
|
//--- once a rescan has been queued (check RescanPending() for completion), false if there was nothing
|
|
//--- to rescan (no deployed model, tester/optimizer context, etc).
|
|
bool StartChartSignalRescan(void)
|
|
{
|
|
if(MQLInfoInteger(MQL_TESTER) || MQLInfoInteger(MQL_OPTIMIZATION) || MQLInfoInteger(MQL_FORWARD))
|
|
return false;
|
|
if(CheckPointer(Net) == POINTER_INVALID || !m_isInitialized || !m_trainingComplete)
|
|
return false;
|
|
if(m_outputNeuronsCount != 1 && m_outputNeuronsCount != 3)
|
|
return false;
|
|
int barsAvail = Bars(m_symbol.Name(), PERIOD_CURRENT);
|
|
int barsNow = MathMin(SIGNAL_RESCAN_LOOKBACK_BARS, barsAvail);
|
|
if(barsNow <= m_historyBars)
|
|
return false;
|
|
if(!ResizeBuffers(barsNow) || !RefreshData())
|
|
return false;
|
|
EnsureShadowNet();
|
|
//--- Drop only the arrows THIS rescan is about to re-judge - i.e. those within [now, oldest bar of
|
|
//--- the barsNow window] - not every namespaced arrow on the chart. PruneDirectionalClusters below
|
|
//--- only touches indices inside [0, barsNow), so anything older survives untouched either way; the
|
|
//--- previous version called ObjectsDeleteAll(0, SIG_ARROW_PREFIX) unconditionally, which wiped
|
|
//--- EVERY arrow ever drawn (years of history, going back to whenever this EA was first attached)
|
|
//--- and then only ever redrew the last SIGNAL_RESCAN_LOOKBACK_BARS (~7 months on H1) - anything
|
|
//--- older was gone permanently the moment Show Signals was clicked, with no way back short of the
|
|
//--- .arrows sidecar (itself only ever a snapshot from whatever the LAST save happened to catch).
|
|
//--- Reported 2026-07-26: a user's multi-year arrow history vanished down to whatever a stale
|
|
//--- .arrows file held, immediately after their first-ever successful Show Signals click. This scoped
|
|
//--- delete is the fix - older arrows are never in scope to be wiped in the first place.
|
|
datetime rescanCutoffTime = m_Time.GetData(barsNow - 1);
|
|
for(int oi = ObjectsTotal(0, 0, OBJ_ARROW) - 1; oi >= 0; oi--)
|
|
{
|
|
string onm = ObjectName(0, oi, 0, OBJ_ARROW);
|
|
if(StringFind(onm, SIG_ARROW_PREFIX) != 0)
|
|
continue;
|
|
if((datetime)ObjectGetInteger(0, onm, OBJPROP_TIME) >= rescanCutoffTime)
|
|
ObjectDelete(0, onm);
|
|
}
|
|
ArrayResize(m_arrowSignalCache, barsNow);
|
|
ArrayInitialize(m_arrowSignalCache, -2.0);
|
|
m_rescanBarsNow = barsNow;
|
|
m_rescanHi = barsNow - m_historyBars;
|
|
m_rescanIndex = 0;
|
|
m_rescanRawBuy = 0;
|
|
m_rescanRawSell = 0;
|
|
m_rescanRawNeutral = 0;
|
|
m_rescanPending = (m_rescanHi > 0);
|
|
m_rescanStartMs = GetTickCount();
|
|
if(m_rescanPending)
|
|
Print(ID + ": rescanning last " + IntegerToString(m_rescanHi) + " bars against the deployed model (progressive, non-blocking)...");
|
|
return m_rescanPending;
|
|
}
|
|
//--- true while a queued rescan (StartChartSignalRescan above) still has slices left for
|
|
//--- AdvanceChartSignalRescan to drain - polled by Warrior_EA.mq5's FinalizeSignalsRescanIfDone() to
|
|
//--- know when it's safe to (re)apply arrow visibility and report the Show Signals click as complete.
|
|
bool RescanPending(void) const { return m_rescanPending; }
|
|
//--- forces a save of the network's current in-memory weights/state regardless of era-completion
|
|
//--- state; called from OnDeinit() so shutdown/chart-removal never loses more than the current tick
|
|
//--- of learning, and a subsequent restart's Train() resumes from m_eraCount rather than the last
|
|
//--- fully-completed era only.
|
|
//--- Heavy weight/state persistence ONLY (net weights + calibration sidecar + shadow net). Split out
|
|
//--- from PersistOnShutdown() so OnDeinit() can run the cheap chart save+purge FIRST: on the CPU-DLL
|
|
//--- box this recursive save (two full nets) is the slow/fragile step, and if it ever stalls past MT5's
|
|
//--- deinit budget or faults, the arrows and status panel must already be gone, not stranded on the
|
|
//--- chart (the reported "cleanup not going well - signals/panel stay after Abnormal termination").
|
|
bool PersistWeightsOnShutdown(void)
|
|
{
|
|
if(CheckPointer(Net) == POINTER_INVALID || !m_isInitialized)
|
|
return false;
|
|
//--- An inference-only run (any Strategy Tester pass - see m_inferenceOnly) trains NOTHING, so there
|
|
//--- is no new state to persist and this save can only do harm. It is exactly what corrupted the
|
|
//--- tester cache on 2026-07-26: when the seeded load failed, the !netLoaded path reset the in-memory
|
|
//--- state to a fresh untrained net (era 0), and this unconditional shutdown save then wrote THAT
|
|
//--- over the good seeded copy - which, because seeding only re-runs when the cache file is absent,
|
|
//--- silently poisoned every subsequent backtest. Skipping it makes the tester cache strictly
|
|
//--- read-only for single backtests: it can only ever be (re)written by the seed copy from
|
|
//--- production, never by a run's own in-memory state.
|
|
if(m_inferenceOnly)
|
|
{
|
|
PrintVerbose(ID + ": inference-only run - skipping the shutdown weight save (nothing was trained; the cached model is left exactly as seeded).");
|
|
return true;
|
|
}
|
|
//--- Nothing trained and nothing loaded => there is no state to persist, and writing anyway is
|
|
//--- actively harmful. This is what defeated the panel's reset-weights button: ResetWeights
|
|
//--- correctly deletes the .nnw/.cfg/.stats/_ckpt/_shadow set and the .arrows sidecar, but if the
|
|
//--- EA is then detached before a single era completes, THIS save immediately re-created a .nnw
|
|
//--- from the freshly-built, never-run net - so the next attach loaded an era-0 stub instead of
|
|
//--- starting clean. For LSTM/HYBRID that stub is worse than useless: a layer that has never run a
|
|
//--- forward pass has m_iInputs<=0, so CNeuronLSTMOCL::Save omits every LSTM buffer (see the note
|
|
//--- in its Load()). m_eraCount==0 && !m_modelLoadedFromDisk is exactly the state ResetWeights
|
|
//--- leaves behind, and also the first-ever-attach state - both cases have nothing worth writing.
|
|
if(m_eraCount == 0 && !m_modelLoadedFromDisk)
|
|
{
|
|
PrintVerbose(ID + ": no era completed and no model loaded - skipping the shutdown weight save (leaving the model files absent so the next attach starts genuinely clean).");
|
|
return true;
|
|
}
|
|
double currentIndicatorParams[];
|
|
m_indicatorTuner.Flatten(currentIndicatorParams);
|
|
bool ok = Net.Save(m_activeFileName + ".nnw", dError, dUndefine, dForecast, dtStudied, m_activeFileCommon, m_eraCount, m_trainingComplete, currentIndicatorParams);
|
|
//--- calibration state (class priors + confidence scale) must travel with the weights so live
|
|
//--- trading behaves like training after a restart - see SaveModelStats().
|
|
if(!SaveModelStats(m_activeFileName, m_activeFileCommon))
|
|
Print(ID + ": ERROR - shutdown SaveModelStats failed for " + m_activeFileName + ". Calibration state not persisted.");
|
|
//--- Deliberately do NOT save the shadow net here. On the CPU-DLL box a second full-net write
|
|
//--- (~18MB) roughly DOUBLES the shutdown cost, and OnDeinit has a limited budget before MT5 reports
|
|
//--- "Abnormal termination" and skips the rest of the teardown. (An earlier revision also blamed that
|
|
//--- overrun for the "failed to allocate layer 0" reloads; that was a misdiagnosis - the real cause
|
|
//--- was a lost virtual override in the read path, see AI\Network.mqh's CLayer::CreateElement note.
|
|
//--- Halving the shutdown cost is still worth it on its own.) The shadow is NOT lost: it is saved
|
|
//--- every era (Train()), at FinalizeTrainRun(), and every ONLINE_LEARN_PERSIST_EVERY bars during
|
|
//--- online learning, and it self-heals (EnsureShadowNet re-blends from the main Net when
|
|
//--- missing/stale). Worst case a
|
|
//--- shutdown loses only the shadow's in-progress-era drift, which re-converges - a far better
|
|
//--- trade than risking the whole model to an over-budget shutdown.
|
|
if(!ok)
|
|
Print(ID + ": ERROR - failed to persist weights on shutdown for " + m_activeFileName + ", error " + IntegerToString(GetLastError()));
|
|
else
|
|
PrintVerbose(ID + ": weights persisted on shutdown (era " + IntegerToString(m_eraCount) + ", trainingComplete=" + (string)m_trainingComplete + ")");
|
|
return ok;
|
|
}
|
|
//--- Persist the drawn arrows to disk, then remove THIS EA's chart visuals (arrows + status label).
|
|
//--- Called early in OnDeinit(), before the heavy weight save, so a later stall/fault in the save can
|
|
//--- never leave the chart littered. Idempotent - the destructor's PurgeChart() then simply no-ops.
|
|
//--- Deliberately NOT part of SaveWeightsNow(): a mid-session manual save must not wipe the chart.
|
|
//---
|
|
//--- 2026-08-01: there used to be a SECOND behaviour here, selected by a `preserveChartArrows` flag the
|
|
//--- caller derived from the deinit reason - on a recompile / input change / template swap / symbol
|
|
//--- change the arrows were deliberately LEFT on the chart, on the theory that an in-process reload
|
|
//--- should not flicker. It is gone, and the flag with it. Three reasons, in order of weight:
|
|
//--- 1. It is the reported defect. "The EA removes its panel and label but its signals stay" is what
|
|
//--- that branch does by design, and an operator has no way to tell a deliberate warm-reload
|
|
//--- preserve from a cleanup that failed.
|
|
//--- 2. It was only correct when the reload keeps the SAME config. Change an input that feeds the
|
|
//--- weights fingerprint - which is exactly what REASON_PARAMETERS means - and the preserved
|
|
//--- arrows belong to a model this chart is no longer running, with nothing to mark them stale.
|
|
//--- 3. The save/restore mechanism it was avoiding already handles this case, progressively and
|
|
//--- without blocking OnInit (LoadChartSignals + AdvanceChartSignalRestore). Keeping a second,
|
|
//--- subtly different route to the same outcome bought a few hundred milliseconds of flicker.
|
|
//--- One path now: persist, clear, restore on the next attach if a model for this config exists.
|
|
void ShutdownChartCleanup(void)
|
|
{
|
|
PersistAndClearChartSignals();
|
|
}
|
|
//--- Full shutdown persistence (weights + arrows), preserved for the panel's manual "save weights"
|
|
//--- button (SaveWeightsNow) - does NOT purge the chart. OnDeinit no longer calls this; it runs
|
|
//--- ShutdownChartCleanup() then PersistWeightsOnShutdown() so cleanup can't be starved by the save.
|
|
bool PersistOnShutdown(void)
|
|
{
|
|
bool ok = PersistWeightsOnShutdown();
|
|
//--- Persist the drawn arrows too so a re-add/recompile restores them without a retrain.
|
|
SaveChartSignals();
|
|
return ok;
|
|
}
|
|
//--- explicit manual save, identical persistence to PersistOnShutdown() but user-triggered from the panel
|
|
bool SaveWeightsNow(void) { return PersistOnShutdown(); }
|
|
//--- reloads this signal's current-config weights file from disk, discarding any unsaved in-memory
|
|
//--- training progress since the last successful save
|
|
bool LoadWeightsNow(void)
|
|
{
|
|
if(CheckPointer(Net) == POINTER_INVALID)
|
|
return false;
|
|
double loadedIndicatorParams[];
|
|
bool netLoaded = Net.Load(m_activeFileName + ".nnw", dError, dUndefine, dForecast, dtStudied, m_activeFileCommon, m_eraCount, m_trainingComplete, loadedIndicatorParams);
|
|
if(!netLoaded)
|
|
{
|
|
Print(ID + ": ERROR - failed to load weights from " + m_activeFileName + ".nnw, error " + IntegerToString(GetLastError()));
|
|
return false;
|
|
}
|
|
m_modelLoadedFromDisk = true;
|
|
//--- the file may carry a superseded architecture - correct it before anything reads the net
|
|
EnforceTopologyContract();
|
|
//--- restore the calibration state that pairs with these weights (priors + confidence scale) so a
|
|
//--- manual reload keeps live decisions calibrated exactly as the saved model was - see LoadModelStats().
|
|
LoadModelStats(m_activeFileName, m_activeFileCommon);
|
|
if(ArraySize(loadedIndicatorParams) == AD_TUNE_PARAM_COUNT)
|
|
{
|
|
m_indicatorTuner.Unflatten(loadedIndicatorParams);
|
|
if(m_indicatorsPtr != NULL)
|
|
ReInitADIndicators(m_indicatorsPtr);
|
|
}
|
|
//--- this just swapped dtStudied/m_eraCount/Net's weights out from under whatever a chunked
|
|
//--- Train() run (see its declaration comment) had cached for the era/trial it was mid-way
|
|
//--- through - discard that resumable state so the next Train() call starts a fresh era
|
|
//--- against the just-loaded dtStudied instead of resuming bar-loop bookkeeping computed
|
|
//--- against a now-stale one.
|
|
m_trainRunActive = false;
|
|
m_eraResumePending = false;
|
|
m_haveOosCheckpoint = false;
|
|
m_oosWindow.Clear();
|
|
m_tuneTrialIndex = -1;
|
|
RefreshLatestSignal();
|
|
Print(ID + ": weights reloaded from disk (era " + IntegerToString(m_eraCount) + ", trainingComplete=" + (string)m_trainingComplete + ")");
|
|
return true;
|
|
}
|
|
//--- deletes this signal's current-config saved files only (weights, topology config, in-progress
|
|
//--- checkpoint) and rebuilds a fresh untrained topology in memory so training restarts from era 0.
|
|
//--- Never touches another signal type's or another symbol/timeframe's files - m_fileName already
|
|
//--- embeds symbol+period+id+output-count+opt-algo.
|
|
bool ResetWeights(void)
|
|
{
|
|
bool stopped = m_trainingStopRequested;
|
|
m_trainingStopRequested = true; // hold off any in-flight Train() scheduling while we reset
|
|
//--- targets whichever file this run is actually training against (see InitNeuralNetwork): the
|
|
//--- shared production weights normally, or the local tester/optimizer cache during a backtest -
|
|
//--- so resetting from the panel during a visual-mode backtest can never wipe the live model.
|
|
int flags = m_activeFileCommon ? FILE_COMMON : 0;
|
|
string nnw = m_activeFileName + ".nnw";
|
|
string cfg = m_activeFileName + ".cfg";
|
|
string ckpt = m_activeFileName + "_ckpt.tmp";
|
|
//--- The calibration/online-learning sidecar (.stats: priors, confidence scale, CPU-inference marker,
|
|
//--- and the online watermark/guardrail - see SaveModelStats) and the deployed EMA shadow
|
|
//--- (_shadow.nnw - see SaveShadowNet) both pair with the weights being erased. Delete them too, or
|
|
//--- a fresh retrain would silently inherit the OLD model's calibration and blend into a stale shadow
|
|
//--- (EnsureShadowNet loads _shadow.nnw from disk before it ever clones the new Net).
|
|
string stats = m_activeFileName + ".stats";
|
|
string shadow = m_activeFileName + "_shadow.nnw";
|
|
//--- EnsureShadowNet clones the live net through this temp file (see CloneNetInto). A crash or a
|
|
//--- reset mid-clone leaves it on disk shaped for the model being erased; sweep it with the rest.
|
|
string shadowClone = m_activeFileName + "_shadowclone.tmp";
|
|
ResetLastError();
|
|
if(FileIsExist(nnw, flags) && !FileDelete(nnw, flags))
|
|
Print(ID + ": ERROR - failed to delete " + nnw + ", error " + IntegerToString(GetLastError()));
|
|
ResetLastError();
|
|
if(FileIsExist(cfg, flags) && !FileDelete(cfg, flags))
|
|
Print(ID + ": ERROR - failed to delete " + cfg + ", error " + IntegerToString(GetLastError()));
|
|
ResetLastError();
|
|
if(FileIsExist(ckpt, flags) && !FileDelete(ckpt, flags))
|
|
Print(ID + ": ERROR - failed to delete " + ckpt + ", error " + IntegerToString(GetLastError()));
|
|
ResetLastError();
|
|
if(FileIsExist(stats, flags) && !FileDelete(stats, flags))
|
|
Print(ID + ": ERROR - failed to delete " + stats + ", error " + IntegerToString(GetLastError()));
|
|
ResetLastError();
|
|
if(FileIsExist(shadow, flags) && !FileDelete(shadow, flags))
|
|
Print(ID + ": ERROR - failed to delete " + shadow + ", error " + IntegerToString(GetLastError()));
|
|
ResetLastError();
|
|
if(FileIsExist(shadowClone, flags) && !FileDelete(shadowClone, flags))
|
|
Print(ID + ": ERROR - failed to delete " + shadowClone + ", error " + IntegerToString(GetLastError()));
|
|
//--- The drawn signal arrows and their .arrows sidecar belong to the model being erased, exactly
|
|
//--- like the .stats/_shadow sidecars above - see ClearPersistedChartSignals().
|
|
ClearPersistedChartSignals("weights reset from the panel");
|
|
m_eraCount = 0;
|
|
m_trainingComplete = false;
|
|
m_modelLoadedFromDisk = false;
|
|
dtStudied = 0;
|
|
dError = -1;
|
|
dUndefine = 0;
|
|
dForecast = 0;
|
|
dPrevSignal = 0;
|
|
m_nmsLiveBuyTime = 0;
|
|
m_nmsLiveSellTime = 0;
|
|
m_nmsLiveBuyAccept = false;
|
|
m_nmsLiveSellAccept = false;
|
|
m_nmsLiveKeptTime = 0;
|
|
m_nmsLiveKeptDir = Neutral;
|
|
m_nmsLiveKeptConf = 0;
|
|
dOosError = -1;
|
|
dOosForecast = 0;
|
|
m_oosSamples = 0;
|
|
//--- fresh model => wipe the compounded/persistent accuracy history too (it is only reset here; a
|
|
//--- normal restart restores it from .stats, and it survives era-to-era). See m_cumIsCorrect.
|
|
m_cumIsCorrect = 0;
|
|
m_cumIsTotal = 0;
|
|
m_cumOosCorrect = 0;
|
|
m_cumOosTotal = 0;
|
|
//--- discard any in-progress chunked run/tuning state - it references buffers/checkpoints from
|
|
//--- before this reset and must never be resumed into the freshly rebuilt topology below
|
|
m_trainRunActive = false;
|
|
m_eraResumePending = false;
|
|
m_haveOosCheckpoint = false;
|
|
m_oosWindow.Clear();
|
|
m_syncWaitStartTick = 0;
|
|
m_tuneTrialIndex = -1;
|
|
//--- an explicit reset restarts from era 0 against a fresh topology - re-verify history sync and
|
|
//--- rebuild the label cache too, and drop any in-flight continual-learning OOS simulation, since
|
|
//--- both would otherwise reference bars/weights from before this reset.
|
|
m_warmupPassesRemaining = 3;
|
|
m_labelCacheBars = 0;
|
|
m_labelCacheAnchorTime = 0;
|
|
m_labelCachePrebuilt = false;
|
|
m_labelPrebuildActive = false;
|
|
m_prebuildSeedPending = false;
|
|
if(m_simOosRunActive)
|
|
{
|
|
delete m_simOosNet;
|
|
m_simOosNet = NULL;
|
|
m_simOosRunActive = false;
|
|
}
|
|
//--- Re-seed before building a fresh topology so weight init is genuinely random, not dominated
|
|
//--- by whatever fixed/deterministic seed the genetic tuner's last candidate evaluation left in
|
|
//--- the MQL5 RNG state. Matches the tuner's own post-tune rebuild path
|
|
//--- (line ~5071) and Warrior_EA.mq5's OnInit.
|
|
MathSrand(GetTickCount());
|
|
bool rebuilt = BuildFreshTopology();
|
|
if(!rebuilt)
|
|
Print(ID + ": ERROR - failed to rebuild fresh topology after weights reset");
|
|
else
|
|
{
|
|
//--- Stamp the config fingerprint with isInitialized=FALSE, NOT m_isInitialized. The compare in
|
|
//--- InitNeuralNetwork (and its own fresh-start .cfg write) always run before m_isInitialized is
|
|
//--- set true at the end of init, so they always use false. ResetWeights is invoked from the panel
|
|
//--- AFTER init, where m_isInitialized is true - passing it here would make this the ONLY .cfg on
|
|
//--- disk with isInitialized=true, so the very next attach spuriously fails the compare
|
|
//--- ("Configuration mismatch. Deleting file") and needlessly discards the weights we just reset.
|
|
//--- This flag is runtime lifecycle state, not a topology/input parameter, so it must never gate reuse.
|
|
SaveTopologyConfiguration(m_activeFileName, m_initialNeuronsCount, m_hiddenLayersCount, m_neuronsReduction, m_minNeuronsCount, m_optimizationAlgo, m_historyBars, m_outputNeuronsCount, m_neuronsCount, LEGACY_STUDY_PERIOD_SLOT, m_minTrainYear, false, LEGACY_CONVERGE_WR_SLOT, m_fractalPeriods, m_convFilterCount, m_lstmHiddenSize, m_activeFileCommon);
|
|
Print(ID + ": weights reset - training will restart from era 0 (current config only: " + m_activeFileName + ")");
|
|
}
|
|
m_trainingStopRequested = stopped;
|
|
if(!stopped && !bEventStudy)
|
|
bEventStudy = EventChartCustom(ChartID(), 1, 0, 0, "Reset");
|
|
return rebuilt;
|
|
}
|
|
};
|
|
//+------------------------------------------------------------------+
|
|
//+------------------------------------------------------------------+
|
|
//| IMPLEMENTATION |
|
|
//| |
|
|
//| CExpertSignalAIBase's method bodies live in these partial files. |
|
|
//| They MUST be included here, after the class declaration above, |
|
|
//| and nowhere else. Order between them does not matter - they are |
|
|
//| all out-of-class definitions of an already-declared class. |
|
|
//| |
|
|
//| The class was ~8200 lines in one file; this splits the bodies by |
|
|
//| responsibility so a change to, say, chart drawing no longer means |
|
|
//| scrolling past the era loop. Nothing was rewritten in the move. |
|
|
//+------------------------------------------------------------------+
|
|
#include "AIBase\Training.mqh"
|
|
#include "AIBase\Lifecycle.mqh"
|
|
#include "AIBase\Topology.mqh"
|
|
#include "AIBase\Labels.mqh"
|
|
#include "AIBase\OnlineLearning.mqh"
|
|
#include "AIBase\AutoTune.mqh"
|
|
#include "AIBase\Inference.mqh"
|
|
#include "AIBase\Persistence.mqh"
|
|
#include "AIBase\ChartUI.mqh"
|
|
#include "AIBase\Features.mqh"
|
|
//+------------------------------------------------------------------+
|