refactor: split CExpertSignalAIBase implementation by responsibility
ExpertSignalAIBase.mqh was 8216 lines: the class declaration followed by 87
method bodies covering training, labelling, feature extraction, persistence,
chart drawing, online learning, the GA auto-tuner and inference, all in one
file. Train() alone is 1492 lines; a change to arrow drawing meant scrolling
past the era loop.
Moved the bodies into Expert\AIBase\, included at the bottom of the original
after the class declaration:
Training.mqh 1607 era loop, plateau ladder, checkpoint select, deploy
Features.mqh 1093 indicator creation + per-bar input feature vector
ChartUI.mqh 634 arrows, arrow persistence, status panel, cleanup
Persistence.mqh 492 .stats/.cfg sidecars, CPU-inference validation, copy
OnlineLearning.mqh 461 live continual learning, EMA shadow, OOS simulator
Labels.mqh 309 ZigZag pivot labels, async label-cache prebuild
AutoTune.mqh 275 genetic tuner (population, crossover, halving)
Inference.mqh 235 softmax, prior calibration, class priors
ExpertSignalAIBase.mqh 8216 -> 3131 (declaration + topology build only)
This is a pure relocation - verified mechanically, not by eye: HEAD's file
reconstructed from the eight partials plus the surviving remainder is
byte-identical to HEAD, span for span (scratchpad verify_split.py). No
declaration moved, no signature changed, no code rewritten, so behaviour is
unchanged by construction.
Compiles 0 errors, 0 warnings.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-29 00:42:45 -04:00
//+------------------------------------------------------------------+
//| Warrior_EA |
//| AnimateDread |
//| |
//| Live continual learning, the EMA shadow net, and the OOS continu|
//| |
//| PARTIAL IMPLEMENTATION FILE - not standalone. |
//| This holds CExpertSignalAIBase method BODIES only. The class |
//| declaration lives in Expert\ExpertSignalAIBase.mqh, which |
//| #includes this file at the bottom, after the declaration. Do not |
//| include it anywhere else and do not compile it on its own. |
//| |
//| Split out purely to make the 8216-line original navigable; the |
//| code inside was moved verbatim, not rewritten. |
//+------------------------------------------------------------------+
# ifndef WARRIOR_AIBASE_ONLINELEARNING_MQH
# define WARRIOR_AIBASE_ONLINELEARNING_MQH
//+------------------------------------------------------------------+
//| Clones the just-converged Net into a separate CNet (m_simOosNet) |
//| and arms a chunked bar-by-bar walk through the OOS window - see |
//| AdvanceOosSimulationChunk(). Evaluation-only: the clone's learned |
//| weights are never written back to Net or any persisted file. |
//+------------------------------------------------------------------+
void CExpertSignalAIBase : : StartOosContinualSimulation ( int bars , int oosCutoff )
{
if ( m_simOosRunActive )
{
delete m_simOosNet ;
m_simOosNet = NULL ;
m_simOosRunActive = false ;
}
if ( oosCutoff < = 0 )
return ; // nothing to walk this run
//--- Clone via the full Save()/Load() pair. Load() calls InitOpenCL()/InitDirectML() before
//--- reconstructing layers, so a bare "new CNet(NULL)" ends up with a GPU/DirectML backend matching
//--- production. Any lighter-weight restore that reused the CALLER's opencl/directml pointers would
//--- be wrong here: on a fresh CNet(NULL) (whose constructor no-ops for a NULL description) those are
//--- unset, and the clone would come out degenerate. (A file-based checkpoint pair used to sit beside
//--- Save/Load and had exactly that flaw; it has been removed - the in-run snapshot is now the
//--- in-memory CNet::CaptureWeights/RestoreWeights.)
//--- Co-locate this ephemeral clone temp with the active model (COMMON on a live chart, LOCAL in the
//--- tester sandbox) instead of always LOCAL. Uses m_activeFileName for the same reason (the active
//--- model's base name, whichever context we're in).
string simFile = m_activeFileName + " _simoos.tmp " ;
int simFlags = m_activeFileCommon ? FILE_COMMON : 0 ;
double ip [ ] ;
if ( ! Net . Save ( simFile , 0.0 , 0.0 , 0.0 , dtStudied , m_activeFileCommon , m_eraCount , m_trainingComplete , ip ) )
return ;
m_simOosNet = new CNet ( NULL ) ;
double loadE , loadU , loadF ;
datetime loadTime ;
long loadEra ;
bool loadComplete ;
double loadIp [ ] ;
bool loaded = m_simOosNet . Load ( simFile , loadE , loadU , loadF , loadTime , m_activeFileCommon , loadEra , loadComplete , loadIp , true /*quiet: this evaluation-only sim is optional - on a miss it simply doesn't run*/ ) ;
FileDelete ( simFile , simFlags ) ;
if ( ! loaded )
{
delete m_simOosNet ;
m_simOosNet = NULL ;
return ;
}
m_simOosCutoff = oosCutoff ;
m_simOosBarIndex = oosCutoff - 1 ;
m_simOosForecast = 0 ;
m_simOosSamples = 0 ;
m_simOosRunActive = true ;
}
//+------------------------------------------------------------------+
//| Advances the evaluation-only continual-learning OOS walk by up |
//| to TRAIN_TIME_BUDGET_MS of work, then yields (same chunking |
//| pattern as the real era loop's m_eraResumePending). For each bar, |
//| oldest-OOS to newest: predict with the clone's CURRENT weights, |
//| score against the cached true label, THEN let the clone learn |
//| from it (single pass, no oversampling replay) - simulating how |
//| the model would adapt bar-by-bar in real forward trading. Never |
//| touches Net, never Saves the clone - purely an evaluation metric.|
//+------------------------------------------------------------------+
void CExpertSignalAIBase : : AdvanceOosSimulationChunk ( void )
{
const uint SIM_TIME_BUDGET_MS = 80 ;
uint chunkStartTick = GetTickCount ( ) ;
//--- Mirror OnlineLearnStep()'s pinned rate for the duration of this chunk, and hand the shared
//--- global back on BOTH exit paths - this simulation is only a valid forecast of live continual
//--- learning if it steps at the same size, and `eta` is shared by every signal instance.
double savedEta = eta ;
eta = m_modelEta * ONLINE_LEARN_ETA_SCALE ;
int i ;
for ( i = m_simOosBarIndex ; i > = 0 ; i - - )
{
if ( GetTickCount ( ) - chunkStartTick > = SIM_TIME_BUDGET_MS )
{
m_simOosBarIndex = i ;
eta = savedEta ;
return ;
}
if ( i > = ArraySize ( m_labelCacheHasValue ) | | ! m_labelCacheHasValue [ i ] )
continue ; // no cached label for this bar (e.g. right at a window edge) - nothing to learn from
//--- Window ends AT (includes) bar i - see Train()'s matching r declaration comment for why.
int r = i ;
fix: the sequence models were reading the window backwards
BuildFeatureWindow() replaces eight hand-rolled copies of the same loop
and feeds the window OLDEST BAR FIRST. Every copy fed it newest-first,
because MQL5 timeseries indices run backwards and `r + b` with b ascending
walks into the past.
Harmless for PAI and CONV - a dense layer learns a weight per position
either way, a conv learns time-mirrored kernels. Not harmless for the
recurrent stacks:
- LSTM_SeqStepForward reads `inputs + t*Iw`, so step t is block t.
- It writes output[] only when t == steps-1: the visible output IS the
last hidden state.
- c_t = f*c_{t-1} + i*g decays toward the start of the sequence.
lstm_seq_flowcheck.cpp measured block 0's influence on the output at
1.2e-2 of block T-1's, at the shipped forget bias of 1.0.
So the bar being PREDICTED sat at the far end of the decay and the output
was handed to the OLDEST bar in the window - the exact inverse of what the
window is for. ~80x backwards on LSTM and HYBRID, on all three tiers
(OpenCL kernel, CPU DLL, pure-MQL5 inference), which is why it never
surfaced as a backend discrepancy.
This does not create edge - the MI diagnostics read at the noise floor
(p=0.4975) with a working positive control. It makes the one hypothesis
those diagnostics explicitly do NOT cover testable: they are marginal and
per-bar, and state they "cannot rule out one that only exists in
combination or across time". The sequence model is the instrument for
across-time structure and it has been crippled, so that hypothesis has
never been honestly tested.
Fingerprint gets an unconditional |WIN:2 - the vector keeps its shape and
its features, so a stale .nnw would load cleanly and run a model fitted to
one ordering against the other, silently. Re-keying every config is the
point, not collateral damage. FORCES A FULL RETRAIN.
Also: the now-relative bar caches are re-keyed on the two live paths.
EnsureBarCachesCapacity() was only ever called from training paths, but
once m_trainingComplete is set ScheduleTrainingIfNeeded() routes every bar
to RefreshConvergedSignal() and Train() is never re-entered - so nothing
cleared the feature cache again for the life of the process. A chart that
trained to convergence kept replaying the rows computed for the last
training era's bar grid: the live signal froze at its convergence-time
value, and OnlineLearnStep() backpropped those stale features against
freshly resolved labels. Backtests were never affected (an inference-only
process never allocates the arrays, so every read recomputes).
Compiles clean: 0 errors, 0 warnings.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-07 18:28:44 -04:00
if ( ! BuildFeatureWindow ( r ) )
refactor: split CExpertSignalAIBase implementation by responsibility
ExpertSignalAIBase.mqh was 8216 lines: the class declaration followed by 87
method bodies covering training, labelling, feature extraction, persistence,
chart drawing, online learning, the GA auto-tuner and inference, all in one
file. Train() alone is 1492 lines; a change to arrow drawing meant scrolling
past the era loop.
Moved the bodies into Expert\AIBase\, included at the bottom of the original
after the class declaration:
Training.mqh 1607 era loop, plateau ladder, checkpoint select, deploy
Features.mqh 1093 indicator creation + per-bar input feature vector
ChartUI.mqh 634 arrows, arrow persistence, status panel, cleanup
Persistence.mqh 492 .stats/.cfg sidecars, CPU-inference validation, copy
OnlineLearning.mqh 461 live continual learning, EMA shadow, OOS simulator
Labels.mqh 309 ZigZag pivot labels, async label-cache prebuild
AutoTune.mqh 275 genetic tuner (population, crossover, halving)
Inference.mqh 235 softmax, prior calibration, class priors
ExpertSignalAIBase.mqh 8216 -> 3131 (declaration + topology build only)
This is a pure relocation - verified mechanically, not by eye: HEAD's file
reconstructed from the eight partials plus the surviving remainder is
byte-identical to HEAD, span for span (scratchpad verify_split.py). No
declaration moved, no signature changed, no code rewritten, so behaviour is
unchanged by construction.
Compiles 0 errors, 0 warnings.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-29 00:42:45 -04:00
continue ;
m_simOosNet . feedForward ( TempData ) ;
m_simOosNet . getResults ( TempData ) ;
double simSignal = ( m_outputNeuronsCount = = 3 ) ? ApplyClassificationSoftmax ( ) : TempData [ 0 ] ;
//--- Pre-update softmax probabilities, read before TempData is rebuilt as the target vector -
//--- feeds the same alpha-balanced focal weight the live path applies (OnlineSampleWeight).
double sBuy = ( TempData . Total ( ) > 0 ) ? TempData . At ( 0 ) : 0.0 ;
double sSell = ( TempData . Total ( ) > 1 ) ? TempData . At ( 1 ) : 0.0 ;
double sNeutral = ( TempData . Total ( ) > 2 ) ? TempData . At ( 2 ) : 0.0 ;
bool buy = m_labelCacheBuy [ i ] ;
bool sell = m_labelCacheSell [ i ] ;
ENUM_SIGNAL trueSignal = buy ? Buy : ( sell ? Sell : Neutral ) ;
bool hit = ( DoubleToSignal ( simSignal ) = = trueSignal ) ;
m_simOosSamples + + ;
if ( hit )
m_simOosForecast + = ( 100 - m_simOosForecast ) / Net . recentAverageSmoothingFactor ;
else
m_simOosForecast - = m_simOosForecast / Net . recentAverageSmoothingFactor ;
TempData . Clear ( ) ;
if ( m_outputNeuronsCount = = 1 )
TempData . Add ( buy & & ! sell ? 1 : ! buy & & sell ? -1 : 0 ) ;
else
if ( m_outputNeuronsCount = = 3 )
{
TempData . Add ( buy ? LABEL_SMOOTH_HIGH : LABEL_SMOOTH_LOW ) ;
TempData . Add ( sell ? LABEL_SMOOTH_HIGH : LABEL_SMOOTH_LOW ) ;
TempData . Add ( ( ! buy & & ! sell ) ? LABEL_SMOOTH_HIGH : LABEL_SMOOTH_LOW ) ;
}
m_simOosNet . backProp ( TempData , OnlineSampleWeight ( trueSignal , sBuy , sSell , sNeutral ) ) ;
}
eta = savedEta ;
delete m_simOosNet ;
m_simOosNet = NULL ;
m_simOosRunActive = false ;
Print ( ID + " : continual-learning OOS simulation complete - " + IntegerToString ( m_simOosSamples ) + " samples, accuracy " + DoubleToString ( m_simOosForecast , 1 ) + " % " ) ;
}
//+------------------------------------------------------------------+
//| Alpha-balanced focal weight for one streamed bar - the cost-level |
//| imbalance correction used by BOTH the live continual-learning path |
//| and its OOS simulation. See the declaration comment and the |
//| ONLINE_LEARN_* block's CLASS IMBALANCE note for the derivation. |
//+------------------------------------------------------------------+
double CExpertSignalAIBase : : OnlineSampleWeight ( ENUM_SIGNAL trueSignal , double pBuy , double pSell , double pNeutral )
{
//--- Regression head has no class structure to balance.
if ( m_outputNeuronsCount ! = 3 )
return 1.0 ;
double weight = 1.0 ;
//--- alpha_c: inverse class frequency from the measured, persisted priors, normalised so the
//--- MAJORITY class is exactly 1.0 (a majority bar is never down-weighted below parity) and only
//--- minority bars are ever up-weighted. Unmeasured priors - a model deployed before any prior was
//--- recorded - skip the alpha term rather than divide by zero; focal's (1-p_t)^gamma still applies.
double priorMax = MathMax ( m_priorNeutral , MathMax ( m_priorBuy , m_priorSell ) ) ;
bool priorsUsable = ( priorMax > 0.0 & & m_priorBuy > 0.0 & & m_priorSell > 0.0 & & m_priorNeutral > 0.0 ) ;
if ( priorsUsable & & trueSignal ! = Neutral )
{
double truePrior = ( trueSignal = = Buy ) ? m_priorBuy : m_priorSell ;
refactor(ai): nine class-imbalance inputs down to two
The imbalance section offered nine controls for one job. Audited against the
code, five of them did not do what their names said at the shipped defaults:
AILogitPriorStrength DEAD - Inference.mqh's post-hoc prior early-returns
whenever the adjusted loss is on, which is default.
OversampleParity DEAD in training - Training.mqh gated the replay loop
on !useLogitAdjustedLoss (correctly, citing Buda et
al. 2018). Live only in the online-learning path.
EnableMinorityReplay DEAD as replay. It survived ONLY as a focal-gamma
damper - "replay minority bars through pass-2
oversampling" was a focal-loss switch.
ConstrainReplay DEAD as a cap; it only chose damper 0.125 vs 0.25.
UseStaticPrior An exact duplicate of FreezePriorCalibration - the two
were OR'd together in the single place either is read.
So they were not five mechanisms fighting; they were one mechanism plus eight
knobs that mostly described machinery that no longer ran. That is worse than
a real conflict, because the log agreed with the names: the label-cache line
printed "reps up to 28x (90% parity) (seeding era 0's class-balance
oversampling)" on every run, describing an oversampling pass that had been
switched off. It is fixed here too - it cost this session a wrong diagnosis.
The one genuine redundancy was focal loss, running at gamma*0.125 alongside
the adjusted loss: two corrections on the same axis, the exact stacking
failure this file already cited Buda et al. for in two other places, damped
by a replay flag whose replay path was itself dead. Removed rather than
re-tuned. The plateau ladder is unaffected - its escape is the learning-rate
warm restart; the gamma anneal beside it only ever stepped toward zero.
WHAT REMAINS is logit-adjusted loss (Menon et al. 2021) plus a prior freeze:
LogitAdjustTau 0 = off; replaces the separate EnableLogitAdjusted-
Loss boolean, since a strength dial where 0 already
means off does not need an on/off switch beside it.
FreezePriorCalibration unchanged.
It is the only one of the six corrections with a consistency guarantee, and
it is consistent for exactly the balanced-error metric checkpoint selection
already ranks on - so the loss and the deploy decision optimize one thing.
The online continual-learning path keeps its own alpha-balanced focal weight,
now as constants pinned to the removed inputs' shipped defaults, so its
behaviour is unchanged. It legitimately needs its own correction:
ApplyLogitAdjustment() only runs inside a training run, so a deployed model
that was reloaded carries no logit offsets and would otherwise stream 31:1
data into itself uncorrected.
The weights-filename fingerprint is BYTE-IDENTICAL. The focal slot was a
double fed to a %d conversion and had always emitted a literal 0; the |MR:
segment is written as the constant its shipped defaults produced. Dropping
either would have re-keyed every model and forced a from-scratch retrain of
the one topology currently converged and trading.
Also removed as orphans: FOCAL_GAMMA_PRESET, MAX_OVERSAMPLE_REPLICAS,
OVERSAMPLE_PARITY_FRACTION, PLATEAU_GAMMA_STEP, and the now-unreachable
"neutralized by prior correction" diagnostic.
Both builds compile 0 errors, 0 warnings. No retrain forced.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-31 11:46:57 -04:00
//--- Measured ratio scaled by ONLINE_LEARN_PARITY, then capped so a single rare bar can never
//--- deliver an outsized kick to an already-validated deployed model. Both were shared inputs
//--- until 2026-07-31; see the CLASS IMBALANCE note above ONLINE_LEARN_MAX_CLASS_WEIGHT for why
//--- this engine keeps its own cost-level correction now that Train() corrects in the gradient.
weight = MathMin ( MathMax ( 1.0 , ( priorMax / truePrior ) * ONLINE_LEARN_PARITY ) , ONLINE_LEARN_ALPHA_CAP ) ;
refactor: split CExpertSignalAIBase implementation by responsibility
ExpertSignalAIBase.mqh was 8216 lines: the class declaration followed by 87
method bodies covering training, labelling, feature extraction, persistence,
chart drawing, online learning, the GA auto-tuner and inference, all in one
file. Train() alone is 1492 lines; a change to arrow drawing meant scrolling
past the era loop.
Moved the bodies into Expert\AIBase\, included at the bottom of the original
after the class declaration:
Training.mqh 1607 era loop, plateau ladder, checkpoint select, deploy
Features.mqh 1093 indicator creation + per-bar input feature vector
ChartUI.mqh 634 arrows, arrow persistence, status panel, cleanup
Persistence.mqh 492 .stats/.cfg sidecars, CPU-inference validation, copy
OnlineLearning.mqh 461 live continual learning, EMA shadow, OOS simulator
Labels.mqh 309 ZigZag pivot labels, async label-cache prebuild
AutoTune.mqh 275 genetic tuner (population, crossover, halving)
Inference.mqh 235 softmax, prior calibration, class priors
ExpertSignalAIBase.mqh 8216 -> 3131 (declaration + topology build only)
This is a pure relocation - verified mechanically, not by eye: HEAD's file
reconstructed from the eight partials plus the surviving remainder is
byte-identical to HEAD, span for span (scratchpad verify_split.py). No
declaration moved, no signature changed, no code rewritten, so behaviour is
unchanged by construction.
Compiles 0 errors, 0 warnings.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-29 00:42:45 -04:00
}
refactor(ai): nine class-imbalance inputs down to two
The imbalance section offered nine controls for one job. Audited against the
code, five of them did not do what their names said at the shipped defaults:
AILogitPriorStrength DEAD - Inference.mqh's post-hoc prior early-returns
whenever the adjusted loss is on, which is default.
OversampleParity DEAD in training - Training.mqh gated the replay loop
on !useLogitAdjustedLoss (correctly, citing Buda et
al. 2018). Live only in the online-learning path.
EnableMinorityReplay DEAD as replay. It survived ONLY as a focal-gamma
damper - "replay minority bars through pass-2
oversampling" was a focal-loss switch.
ConstrainReplay DEAD as a cap; it only chose damper 0.125 vs 0.25.
UseStaticPrior An exact duplicate of FreezePriorCalibration - the two
were OR'd together in the single place either is read.
So they were not five mechanisms fighting; they were one mechanism plus eight
knobs that mostly described machinery that no longer ran. That is worse than
a real conflict, because the log agreed with the names: the label-cache line
printed "reps up to 28x (90% parity) (seeding era 0's class-balance
oversampling)" on every run, describing an oversampling pass that had been
switched off. It is fixed here too - it cost this session a wrong diagnosis.
The one genuine redundancy was focal loss, running at gamma*0.125 alongside
the adjusted loss: two corrections on the same axis, the exact stacking
failure this file already cited Buda et al. for in two other places, damped
by a replay flag whose replay path was itself dead. Removed rather than
re-tuned. The plateau ladder is unaffected - its escape is the learning-rate
warm restart; the gamma anneal beside it only ever stepped toward zero.
WHAT REMAINS is logit-adjusted loss (Menon et al. 2021) plus a prior freeze:
LogitAdjustTau 0 = off; replaces the separate EnableLogitAdjusted-
Loss boolean, since a strength dial where 0 already
means off does not need an on/off switch beside it.
FreezePriorCalibration unchanged.
It is the only one of the six corrections with a consistency guarantee, and
it is consistent for exactly the balanced-error metric checkpoint selection
already ranks on - so the loss and the deploy decision optimize one thing.
The online continual-learning path keeps its own alpha-balanced focal weight,
now as constants pinned to the removed inputs' shipped defaults, so its
behaviour is unchanged. It legitimately needs its own correction:
ApplyLogitAdjustment() only runs inside a training run, so a deployed model
that was reloaded carries no logit offsets and would otherwise stream 31:1
data into itself uncorrected.
The weights-filename fingerprint is BYTE-IDENTICAL. The focal slot was a
double fed to a %d conversion and had always emitted a literal 0; the |MR:
segment is written as the constant its shipped defaults produced. Dropping
either would have re-keyed every model and forced a from-scratch retrain of
the one topology currently converged and trading.
Also removed as orphans: FOCAL_GAMMA_PRESET, MAX_OVERSAMPLE_REPLICAS,
OVERSAMPLE_PARITY_FRACTION, PLATEAU_GAMMA_STEP, and the now-unreachable
"neutralized by prior correction" diagnostic.
Both builds compile 0 errors, 0 warnings. No retrain forced.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-31 11:46:57 -04:00
//--- gamma: down-weights bars the model already gets right (the overwhelming Neutral majority), so
//--- the update concentrates on genuinely informative confirmations. Constant since 2026-07-31.
if ( ONLINE_LEARN_FOCAL_GAMMA > 0.0 )
refactor: split CExpertSignalAIBase implementation by responsibility
ExpertSignalAIBase.mqh was 8216 lines: the class declaration followed by 87
method bodies covering training, labelling, feature extraction, persistence,
chart drawing, online learning, the GA auto-tuner and inference, all in one
file. Train() alone is 1492 lines; a change to arrow drawing meant scrolling
past the era loop.
Moved the bodies into Expert\AIBase\, included at the bottom of the original
after the class declaration:
Training.mqh 1607 era loop, plateau ladder, checkpoint select, deploy
Features.mqh 1093 indicator creation + per-bar input feature vector
ChartUI.mqh 634 arrows, arrow persistence, status panel, cleanup
Persistence.mqh 492 .stats/.cfg sidecars, CPU-inference validation, copy
OnlineLearning.mqh 461 live continual learning, EMA shadow, OOS simulator
Labels.mqh 309 ZigZag pivot labels, async label-cache prebuild
AutoTune.mqh 275 genetic tuner (population, crossover, halving)
Inference.mqh 235 softmax, prior calibration, class priors
ExpertSignalAIBase.mqh 8216 -> 3131 (declaration + topology build only)
This is a pure relocation - verified mechanically, not by eye: HEAD's file
reconstructed from the eight partials plus the surviving remainder is
byte-identical to HEAD, span for span (scratchpad verify_split.py). No
declaration moved, no signature changed, no code rewritten, so behaviour is
unchanged by construction.
Compiles 0 errors, 0 warnings.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-29 00:42:45 -04:00
{
double pt = ( trueSignal = = Buy ) ? pBuy : ( trueSignal = = Sell ) ? pSell : pNeutral ;
pt = MathMax ( 0.0 , MathMin ( 1.0 , pt ) ) ;
refactor(ai): nine class-imbalance inputs down to two
The imbalance section offered nine controls for one job. Audited against the
code, five of them did not do what their names said at the shipped defaults:
AILogitPriorStrength DEAD - Inference.mqh's post-hoc prior early-returns
whenever the adjusted loss is on, which is default.
OversampleParity DEAD in training - Training.mqh gated the replay loop
on !useLogitAdjustedLoss (correctly, citing Buda et
al. 2018). Live only in the online-learning path.
EnableMinorityReplay DEAD as replay. It survived ONLY as a focal-gamma
damper - "replay minority bars through pass-2
oversampling" was a focal-loss switch.
ConstrainReplay DEAD as a cap; it only chose damper 0.125 vs 0.25.
UseStaticPrior An exact duplicate of FreezePriorCalibration - the two
were OR'd together in the single place either is read.
So they were not five mechanisms fighting; they were one mechanism plus eight
knobs that mostly described machinery that no longer ran. That is worse than
a real conflict, because the log agreed with the names: the label-cache line
printed "reps up to 28x (90% parity) (seeding era 0's class-balance
oversampling)" on every run, describing an oversampling pass that had been
switched off. It is fixed here too - it cost this session a wrong diagnosis.
The one genuine redundancy was focal loss, running at gamma*0.125 alongside
the adjusted loss: two corrections on the same axis, the exact stacking
failure this file already cited Buda et al. for in two other places, damped
by a replay flag whose replay path was itself dead. Removed rather than
re-tuned. The plateau ladder is unaffected - its escape is the learning-rate
warm restart; the gamma anneal beside it only ever stepped toward zero.
WHAT REMAINS is logit-adjusted loss (Menon et al. 2021) plus a prior freeze:
LogitAdjustTau 0 = off; replaces the separate EnableLogitAdjusted-
Loss boolean, since a strength dial where 0 already
means off does not need an on/off switch beside it.
FreezePriorCalibration unchanged.
It is the only one of the six corrections with a consistency guarantee, and
it is consistent for exactly the balanced-error metric checkpoint selection
already ranks on - so the loss and the deploy decision optimize one thing.
The online continual-learning path keeps its own alpha-balanced focal weight,
now as constants pinned to the removed inputs' shipped defaults, so its
behaviour is unchanged. It legitimately needs its own correction:
ApplyLogitAdjustment() only runs inside a training run, so a deployed model
that was reloaded carries no logit offsets and would otherwise stream 31:1
data into itself uncorrected.
The weights-filename fingerprint is BYTE-IDENTICAL. The focal slot was a
double fed to a %d conversion and had always emitted a literal 0; the |MR:
segment is written as the constant its shipped defaults produced. Dropping
either would have re-keyed every model and forced a from-scratch retrain of
the one topology currently converged and trading.
Also removed as orphans: FOCAL_GAMMA_PRESET, MAX_OVERSAMPLE_REPLICAS,
OVERSAMPLE_PARITY_FRACTION, PLATEAU_GAMMA_STEP, and the now-unreachable
"neutralized by prior correction" diagnostic.
Both builds compile 0 errors, 0 warnings. No retrain forced.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-31 11:46:57 -04:00
weight * = MathPow ( 1.0 - pt , ONLINE_LEARN_FOCAL_GAMMA ) ;
refactor: split CExpertSignalAIBase implementation by responsibility
ExpertSignalAIBase.mqh was 8216 lines: the class declaration followed by 87
method bodies covering training, labelling, feature extraction, persistence,
chart drawing, online learning, the GA auto-tuner and inference, all in one
file. Train() alone is 1492 lines; a change to arrow drawing meant scrolling
past the era loop.
Moved the bodies into Expert\AIBase\, included at the bottom of the original
after the class declaration:
Training.mqh 1607 era loop, plateau ladder, checkpoint select, deploy
Features.mqh 1093 indicator creation + per-bar input feature vector
ChartUI.mqh 634 arrows, arrow persistence, status panel, cleanup
Persistence.mqh 492 .stats/.cfg sidecars, CPU-inference validation, copy
OnlineLearning.mqh 461 live continual learning, EMA shadow, OOS simulator
Labels.mqh 309 ZigZag pivot labels, async label-cache prebuild
AutoTune.mqh 275 genetic tuner (population, crossover, halving)
Inference.mqh 235 softmax, prior calibration, class priors
ExpertSignalAIBase.mqh 8216 -> 3131 (declaration + topology build only)
This is a pure relocation - verified mechanically, not by eye: HEAD's file
reconstructed from the eight partials plus the surviving remainder is
byte-identical to HEAD, span for span (scratchpad verify_split.py). No
declaration moved, no signature changed, no code rewritten, so behaviour is
unchanged by construction.
Compiles 0 errors, 0 warnings.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-29 00:42:45 -04:00
}
return weight ;
}
//+------------------------------------------------------------------+
//| Online continual-learning step - LIVE CHART ONLY. Once a model is |
//| deployed (m_trainingComplete) it keeps learning from real market |
//| structure the same supervised way it was trained: predicting the |
feat(ai): triple-barrier labels replace exact-pivot ZigZag targets
The 31:1 class imbalance was self-inflicted by the TARGET, not a property
of the market. Labelling only the exact bar where a ZigZag pivot confirms
gave Buy 1164 / Sell 1164 / Neutral 35841, and every correction mechanism
this codebase accumulated sits downstream of that one choice: the
logit-adjusted loss and its range cap, the prior EMA, the +-3.0 output-bias
seed, balanced-accuracy-then-precision selection with its coverage floor,
the recall floor and its catch-22, the alternation gate, NMS, and the four
oversampling designs that collapsed before them.
The reference this engine is built on (references/neuronetworksbook.pdf
ch. 3.1/3.3) also uses ZigZag, but targets the DIRECTION TO THE NEXT
EXTREMUM on every bar - ~50/50 by construction, with no imbalance to
correct at all. It never had this problem because it never asked "is this
the pivot bar".
Labels are now the triple barrier (Lopez de Prado ch. 3), using the EA's
OWN SL_Mode/TP_Mode: does a trade opened at this bar's close reach its
target before its stop, within a horizon. Buy = long resolves, Sell =
short resolves, Neutral = neither. Consequences:
- dir-precision in the era line stops being a proxy and becomes the win
rate of the strategy under its own exit rules.
- Expected balance ~25/25/50 at the shipped 1:3 (gambler's ruin), i.e.
~2:1 instead of 31:1. Measured and logged at the end of the prebuild.
- Spread is charged on both legs, so it is a NET win rate.
- Intrabar ambiguity resolves to the STOP. OHLC cannot order two touches
inside one bar and the optimistic reading is how a backtested edge
becomes a live loss.
ZigZag stays as input features (EnableSwingContext) and now also supplies
the vertical barrier: the horizon is the median confirmed leg length,
snapped to a coarse ladder. Derived, not configured, and deliberately kept
out of the filename fingerprint - a filename keyed on a measured quantity
orphans a trained model the moment the measurement moves.
Removed, because the premise died with the old target:
- the alternation gate. Correct for pivot labels (a ZigZag cannot emit two
same-type pivots in a row, so a repeat was provably a false fire), and
wrong for barrier labels, which answer each bar independently. It also
took its worst consequence with it: a one-sided model previously got ONE
trade per backtest, a hard blocker on marketplace validation.
- SignalClusterWindow now defaults off - it de-duplicated repeats that are
now real trades. Kept as an opt-in display control.
- LABEL_WINDOW_BARS, the pivot-widening pass, ConfirmedZigZagLabel.
- the era-0 output-bias seed now needs a genuinely dominant class (0.70)
rather than 0.40; at ~50% Neutral a +-3.0 seed is a distortion, not a
correction.
Also fixed, both found while wiring the above:
1. RefreshConvergedSignal sized its buffers from a date delta
(Bars(sym, period, dtStudied, TimeCurrent())). dtStudied is a training
watermark; in the tester it is loaded from a live-chart save AHEAD of
the simulated date, so the interval inverted, Bars() returned ~0, and
the buffer came out at exactly m_historyBars - deep enough for the OHLC
window and far too shallow for the Donchian-50 / 20-bar-return / SMA
extension behind it. Inference silently computed DIFFERENT features
from the ones training learned on, live as well as in the tester. Now
sized from what the feature builder actually needs.
2. The barrier horizon is resolved on the deployed path too. A deployed
model never enters Train(), so it never reached the prebuild, and
OnlineLearnStep reads the horizon as its confirmation delay - left at
the fallback it would have backpropped bars whose barriers had not
resolved. Silent lookahead in the one place that writes to a live model.
SL_Mode/TP_Mode join the weights fingerprint: they define the labels now,
so a model trained at 1:3 must never be silently reused at 1:1. This
re-keys every pre-existing model by design - none were trained on this task.
Inference census extended with the vote gate. LongCondition/ShortCondition
open with a readiness check the refresh counters never see; in the tester it
reduces to "the seeded _optcache.nnw must have LOADED", and if it did not,
every vote is hard-zeroed while the model still answers Buy. The old three
counters would have read that as "the model says Neutral" - false, and a
completely different fix. This is the leading candidate for the
zero-direction backtest and the census can now name it in one run.
Both builds compile 0 errors / 0 warnings. Forces a full retrain.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-31 20:39:49 -04:00
//| TRIPLE-BARRIER outcome for each bar. The critical rule the user |
//| asked for is the confirmation delay - a bar's barrier label is not |
//| knowable until m_barrierHorizonBars more bars have closed after it |
//| (that is the vertical barrier itself), so the model must NEVER |
//| backprop the newest bars against an unresolved outcome, even |
//| though it happily EMITS a live signal on them. This method |
//| therefore only ever learns from the "confirmable frontier" and |
//| older: the newest bar whose now-relative index is |
//| >= m_barrierHorizonBars. Everything newer than that is inference- |
refactor: split CExpertSignalAIBase implementation by responsibility
ExpertSignalAIBase.mqh was 8216 lines: the class declaration followed by 87
method bodies covering training, labelling, feature extraction, persistence,
chart drawing, online learning, the GA auto-tuner and inference, all in one
file. Train() alone is 1492 lines; a change to arrow drawing meant scrolling
past the era loop.
Moved the bodies into Expert\AIBase\, included at the bottom of the original
after the class declaration:
Training.mqh 1607 era loop, plateau ladder, checkpoint select, deploy
Features.mqh 1093 indicator creation + per-bar input feature vector
ChartUI.mqh 634 arrows, arrow persistence, status panel, cleanup
Persistence.mqh 492 .stats/.cfg sidecars, CPU-inference validation, copy
OnlineLearning.mqh 461 live continual learning, EMA shadow, OOS simulator
Labels.mqh 309 ZigZag pivot labels, async label-cache prebuild
AutoTune.mqh 275 genetic tuner (population, crossover, halving)
Inference.mqh 235 softmax, prior calibration, class priors
ExpertSignalAIBase.mqh 8216 -> 3131 (declaration + topology build only)
This is a pure relocation - verified mechanically, not by eye: HEAD's file
reconstructed from the eight partials plus the surviving remainder is
byte-identical to HEAD, span for span (scratchpad verify_split.py). No
declaration moved, no signature changed, no code rewritten, so behaviour is
unchanged by construction.
Compiles 0 errors, 0 warnings.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-29 00:42:45 -04:00
//| only until it, too, matures - identical to how training holds its |
//| recent bars in the OOS holdout and embargoes the boundary band. |
feat(ai): triple-barrier labels replace exact-pivot ZigZag targets
The 31:1 class imbalance was self-inflicted by the TARGET, not a property
of the market. Labelling only the exact bar where a ZigZag pivot confirms
gave Buy 1164 / Sell 1164 / Neutral 35841, and every correction mechanism
this codebase accumulated sits downstream of that one choice: the
logit-adjusted loss and its range cap, the prior EMA, the +-3.0 output-bias
seed, balanced-accuracy-then-precision selection with its coverage floor,
the recall floor and its catch-22, the alternation gate, NMS, and the four
oversampling designs that collapsed before them.
The reference this engine is built on (references/neuronetworksbook.pdf
ch. 3.1/3.3) also uses ZigZag, but targets the DIRECTION TO THE NEXT
EXTREMUM on every bar - ~50/50 by construction, with no imbalance to
correct at all. It never had this problem because it never asked "is this
the pivot bar".
Labels are now the triple barrier (Lopez de Prado ch. 3), using the EA's
OWN SL_Mode/TP_Mode: does a trade opened at this bar's close reach its
target before its stop, within a horizon. Buy = long resolves, Sell =
short resolves, Neutral = neither. Consequences:
- dir-precision in the era line stops being a proxy and becomes the win
rate of the strategy under its own exit rules.
- Expected balance ~25/25/50 at the shipped 1:3 (gambler's ruin), i.e.
~2:1 instead of 31:1. Measured and logged at the end of the prebuild.
- Spread is charged on both legs, so it is a NET win rate.
- Intrabar ambiguity resolves to the STOP. OHLC cannot order two touches
inside one bar and the optimistic reading is how a backtested edge
becomes a live loss.
ZigZag stays as input features (EnableSwingContext) and now also supplies
the vertical barrier: the horizon is the median confirmed leg length,
snapped to a coarse ladder. Derived, not configured, and deliberately kept
out of the filename fingerprint - a filename keyed on a measured quantity
orphans a trained model the moment the measurement moves.
Removed, because the premise died with the old target:
- the alternation gate. Correct for pivot labels (a ZigZag cannot emit two
same-type pivots in a row, so a repeat was provably a false fire), and
wrong for barrier labels, which answer each bar independently. It also
took its worst consequence with it: a one-sided model previously got ONE
trade per backtest, a hard blocker on marketplace validation.
- SignalClusterWindow now defaults off - it de-duplicated repeats that are
now real trades. Kept as an opt-in display control.
- LABEL_WINDOW_BARS, the pivot-widening pass, ConfirmedZigZagLabel.
- the era-0 output-bias seed now needs a genuinely dominant class (0.70)
rather than 0.40; at ~50% Neutral a +-3.0 seed is a distortion, not a
correction.
Also fixed, both found while wiring the above:
1. RefreshConvergedSignal sized its buffers from a date delta
(Bars(sym, period, dtStudied, TimeCurrent())). dtStudied is a training
watermark; in the tester it is loaded from a live-chart save AHEAD of
the simulated date, so the interval inverted, Bars() returned ~0, and
the buffer came out at exactly m_historyBars - deep enough for the OHLC
window and far too shallow for the Donchian-50 / 20-bar-return / SMA
extension behind it. Inference silently computed DIFFERENT features
from the ones training learned on, live as well as in the tester. Now
sized from what the feature builder actually needs.
2. The barrier horizon is resolved on the deployed path too. A deployed
model never enters Train(), so it never reached the prebuild, and
OnlineLearnStep reads the horizon as its confirmation delay - left at
the fallback it would have backpropped bars whose barriers had not
resolved. Silent lookahead in the one place that writes to a live model.
SL_Mode/TP_Mode join the weights fingerprint: they define the labels now,
so a model trained at 1:3 must never be silently reused at 1:1. This
re-keys every pre-existing model by design - none were trained on this task.
Inference census extended with the vote gate. LongCondition/ShortCondition
open with a readiness check the refresh counters never see; in the tester it
reduces to "the seeded _optcache.nnw must have LOADED", and if it did not,
every vote is hard-zeroed while the model still answers Buy. The old three
counters would have read that as "the model says Neutral" - false, and a
completely different fix. This is the leading candidate for the
zero-direction backtest and the census can now name it in one run.
Both builds compile 0 errors / 0 warnings. Forces a full retrain.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-31 20:39:49 -04:00
//| (Was m_swingConfirmationBars, which answered the ZigZag repainting |
//| question. That is no longer the label's lookahead - see |
//| m_barrierHorizonBars.) |
refactor: split CExpertSignalAIBase implementation by responsibility
ExpertSignalAIBase.mqh was 8216 lines: the class declaration followed by 87
method bodies covering training, labelling, feature extraction, persistence,
chart drawing, online learning, the GA auto-tuner and inference, all in one
file. Train() alone is 1492 lines; a change to arrow drawing meant scrolling
past the era loop.
Moved the bodies into Expert\AIBase\, included at the bottom of the original
after the class declaration:
Training.mqh 1607 era loop, plateau ladder, checkpoint select, deploy
Features.mqh 1093 indicator creation + per-bar input feature vector
ChartUI.mqh 634 arrows, arrow persistence, status panel, cleanup
Persistence.mqh 492 .stats/.cfg sidecars, CPU-inference validation, copy
OnlineLearning.mqh 461 live continual learning, EMA shadow, OOS simulator
Labels.mqh 309 ZigZag pivot labels, async label-cache prebuild
AutoTune.mqh 275 genetic tuner (population, crossover, halving)
Inference.mqh 235 softmax, prior calibration, class priors
ExpertSignalAIBase.mqh 8216 -> 3131 (declaration + topology build only)
This is a pure relocation - verified mechanically, not by eye: HEAD's file
reconstructed from the eight partials plus the surviving remainder is
byte-identical to HEAD, span for span (scratchpad verify_split.py). No
declaration moved, no signature changed, no code rewritten, so behaviour is
unchanged by construction.
Compiles 0 errors, 0 warnings.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-29 00:42:45 -04:00
//| |
//| Mechanism per newly-matured bar (oldest->newest, exactly |
//| AdvanceOosSimulationChunk()'s predict-score-then-learn step, but |
//| on the REAL deployed Net): build the same feature window training |
//| used, feedForward, score the prediction against the confirmed |
//| label (guardrail EMA), then backProp that label. The deployed |
//| SHADOW is nudged toward Net by SHADOW_WEIGHT_TAU only while the |
//| rolling accuracy holds up; if it decays the blend FREEZES (live |
//| keeps trading the last-good shadow, Net keeps adapting so it can |
//| recover) - drift can never reach the account. State persists in the |
//| .stats sidecar so a restart neither re-learns old bars nor skips. |
//+------------------------------------------------------------------+
void CExpertSignalAIBase : : OnlineLearnStep ( void )
{
//--- Hard gates. m_inferenceOnly covers BOTH the single backtest and every optimization pass (see
//--- its declaration comment): in the tester the model is held FIXED, so continual learning is a
//--- live-chart-only behaviour (forward-test it on a demo account, not the Strategy Tester).
perf(autotune): replace the genetic search with a filter score - hours to seconds
MEASURED COST OF THE GA, which is what retired it. Per generation:
rung 0: 8 cand x 3 seeds x 3 eras = 72 eras
rung 1: 4 cand x 3 seeds x 8 eras = 96
rung 2: 2 cand x 3 seeds x 20 eras = 120
= 288 eras/generation x 4 generations = 1152 eras BEFORE the winner's
real training began. Against the observed era times on SP500 H1:
PAI 29.1 s/era -> 9.3 h (matches the observed 00:37 -> 09:22)
CONV 41.3 s/era -> 13.2 h
LSTM 150.4 s/era -> 48.1 h
HYBRID 154.6 s/era -> 49.5 h
Two days to tune is not a first-run experience, and it is the phase in
which the panel goes quiet, which is what made it look like a hang.
It also bought nothing. The space is 90 points (10 MA periods x 9 MA
types), so 1152 evaluations revisited each point ~13 times; and rungs of
3 and 8 eras cannot separate two MA periods at all. The 2026-08-01 run
proves it: every finalist scored 25.0-25.9% balanced accuracy - below the
33.3% one-class floor, i.e. indistinguishable noise - and the search then
"deployed the winner" of that.
THE ERROR WAS THE SCORING FUNCTION, not its constants. Using a full
training run to choose a feature's period is a wrapper method paying
wrapper prices for a decision that does not need one. The reference book
does not do this: ch. 3.3 selects inputs by measuring each candidate
indicator's CORRELATION with the target and dropping the ones with none,
with no network involved.
So: rank candidates by the MUTUAL INFORMATION between the resulting
feature vector and the triple-barrier label. MI rather than correlation
because the label is 3-class categorical and the features are not
monotonically related to it. Equal-FREQUENCY binning (rank-based),
because these features are ATR-normalised and heavy-tailed - fixed-width
bins put nearly everything in one bucket and report ~0 information for a
genuinely useful feature.
Scoring is arithmetic over the feature cache, so it costs seconds and its
cost is independent of topology: LSTM now tunes as fast as the MLP.
Coordinate sweep, not product sweep - cost is the SUM of per-parameter
candidate counts, so enabling every indicator stays affordable - with a
second pass that breaks early once nothing moves.
Sampling is IS-ONLY. Letting the OOS window influence which indicator
settings ship would mean the holdout had been used for selection and had
stopped being a holdout.
HONEST LIMIT, recorded because it is the price: MI is marginal, so a
parameter that only pays off in combination with another can be missed
(Guyon & Elisseeff 2003, filter vs wrapper). Given the wrapper it
replaces was ranking pure noise at 48 h a run, this is strictly better.
Deleted with it: GaRungEras/GaExtract/GaStore/GaMutate/GaRandomCandidate/
GaBlockCrossover/GaSortAliveByScoreDesc/GaBreedNextGeneration, 14 m_ga*
members, the GA_*/TUNE_POP_* constants, and ComputeTuneTrialBudget.
AND m_evalMode/m_evalEraBudget, because nothing set them any more - 28
read sites all permanently inert. That is not a tidy-up: the `if
(!m_evalMode)` guard on UpdateClassPriors is exactly what silently
disabled the imbalance correction for entire runs two commits ago. Dead
machinery that still reads like live machinery is this codebase's most
expensive recurring bug, and leaving 28 more instances of it would have
been indefensible.
The panel's tuning-progress state goes too - tuning no longer takes long
enough to need one.
Both builds compile 0 errors / 0 warnings.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-01 10:29:31 -04:00
if ( ! m_enableOnlineLearning | | m_inferenceOnly | | m_trainRunActive )
refactor: split CExpertSignalAIBase implementation by responsibility
ExpertSignalAIBase.mqh was 8216 lines: the class declaration followed by 87
method bodies covering training, labelling, feature extraction, persistence,
chart drawing, online learning, the GA auto-tuner and inference, all in one
file. Train() alone is 1492 lines; a change to arrow drawing meant scrolling
past the era loop.
Moved the bodies into Expert\AIBase\, included at the bottom of the original
after the class declaration:
Training.mqh 1607 era loop, plateau ladder, checkpoint select, deploy
Features.mqh 1093 indicator creation + per-bar input feature vector
ChartUI.mqh 634 arrows, arrow persistence, status panel, cleanup
Persistence.mqh 492 .stats/.cfg sidecars, CPU-inference validation, copy
OnlineLearning.mqh 461 live continual learning, EMA shadow, OOS simulator
Labels.mqh 309 ZigZag pivot labels, async label-cache prebuild
AutoTune.mqh 275 genetic tuner (population, crossover, halving)
Inference.mqh 235 softmax, prior calibration, class priors
ExpertSignalAIBase.mqh 8216 -> 3131 (declaration + topology build only)
This is a pure relocation - verified mechanically, not by eye: HEAD's file
reconstructed from the eight partials plus the surviving remainder is
byte-identical to HEAD, span for span (scratchpad verify_split.py). No
declaration moved, no signature changed, no code rewritten, so behaviour is
unchanged by construction.
Compiles 0 errors, 0 warnings.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-29 00:42:45 -04:00
return ;
if ( ! m_trainingComplete | | m_trainingStopRequested | | m_trainingPaused )
return ;
if ( MQLInfoInteger ( MQL_TESTER ) | | MQLInfoInteger ( MQL_OPTIMIZATION ) | | MQLInfoInteger ( MQL_FORWARD ) )
return ; // belt-and-braces: never adapt weights inside any tester context
if ( CheckPointer ( Net ) = = POINTER_INVALID | | Net . CpuInference ( ) )
return ; // DLL-free inference build has no backend to backprop through
if ( CheckPointer ( m_shadowNet ) = = POINTER_INVALID )
return ; // nothing deployed to blend into yet (RefreshLatestSignal bootstraps it first)
if ( m_outputNeuronsCount ! = 1 & & m_outputNeuronsCount ! = 3 )
return ;
feat(ai): triple-barrier labels replace exact-pivot ZigZag targets
The 31:1 class imbalance was self-inflicted by the TARGET, not a property
of the market. Labelling only the exact bar where a ZigZag pivot confirms
gave Buy 1164 / Sell 1164 / Neutral 35841, and every correction mechanism
this codebase accumulated sits downstream of that one choice: the
logit-adjusted loss and its range cap, the prior EMA, the +-3.0 output-bias
seed, balanced-accuracy-then-precision selection with its coverage floor,
the recall floor and its catch-22, the alternation gate, NMS, and the four
oversampling designs that collapsed before them.
The reference this engine is built on (references/neuronetworksbook.pdf
ch. 3.1/3.3) also uses ZigZag, but targets the DIRECTION TO THE NEXT
EXTREMUM on every bar - ~50/50 by construction, with no imbalance to
correct at all. It never had this problem because it never asked "is this
the pivot bar".
Labels are now the triple barrier (Lopez de Prado ch. 3), using the EA's
OWN SL_Mode/TP_Mode: does a trade opened at this bar's close reach its
target before its stop, within a horizon. Buy = long resolves, Sell =
short resolves, Neutral = neither. Consequences:
- dir-precision in the era line stops being a proxy and becomes the win
rate of the strategy under its own exit rules.
- Expected balance ~25/25/50 at the shipped 1:3 (gambler's ruin), i.e.
~2:1 instead of 31:1. Measured and logged at the end of the prebuild.
- Spread is charged on both legs, so it is a NET win rate.
- Intrabar ambiguity resolves to the STOP. OHLC cannot order two touches
inside one bar and the optimistic reading is how a backtested edge
becomes a live loss.
ZigZag stays as input features (EnableSwingContext) and now also supplies
the vertical barrier: the horizon is the median confirmed leg length,
snapped to a coarse ladder. Derived, not configured, and deliberately kept
out of the filename fingerprint - a filename keyed on a measured quantity
orphans a trained model the moment the measurement moves.
Removed, because the premise died with the old target:
- the alternation gate. Correct for pivot labels (a ZigZag cannot emit two
same-type pivots in a row, so a repeat was provably a false fire), and
wrong for barrier labels, which answer each bar independently. It also
took its worst consequence with it: a one-sided model previously got ONE
trade per backtest, a hard blocker on marketplace validation.
- SignalClusterWindow now defaults off - it de-duplicated repeats that are
now real trades. Kept as an opt-in display control.
- LABEL_WINDOW_BARS, the pivot-widening pass, ConfirmedZigZagLabel.
- the era-0 output-bias seed now needs a genuinely dominant class (0.70)
rather than 0.40; at ~50% Neutral a +-3.0 seed is a distortion, not a
correction.
Also fixed, both found while wiring the above:
1. RefreshConvergedSignal sized its buffers from a date delta
(Bars(sym, period, dtStudied, TimeCurrent())). dtStudied is a training
watermark; in the tester it is loaded from a live-chart save AHEAD of
the simulated date, so the interval inverted, Bars() returned ~0, and
the buffer came out at exactly m_historyBars - deep enough for the OHLC
window and far too shallow for the Donchian-50 / 20-bar-return / SMA
extension behind it. Inference silently computed DIFFERENT features
from the ones training learned on, live as well as in the tester. Now
sized from what the feature builder actually needs.
2. The barrier horizon is resolved on the deployed path too. A deployed
model never enters Train(), so it never reached the prebuild, and
OnlineLearnStep reads the horizon as its confirmation delay - left at
the fallback it would have backpropped bars whose barriers had not
resolved. Silent lookahead in the one place that writes to a live model.
SL_Mode/TP_Mode join the weights fingerprint: they define the labels now,
so a model trained at 1:3 must never be silently reused at 1:1. This
re-keys every pre-existing model by design - none were trained on this task.
Inference census extended with the vote gate. LongCondition/ShortCondition
open with a readiness check the refresh counters never see; in the tester it
reduces to "the seeded _optcache.nnw must have LOADED", and if it did not,
every vote is hard-zeroed while the model still answers Buy. The old three
counters would have read that as "the model says Neutral" - false, and a
completely different fix. This is the leading candidate for the
zero-direction backtest and the census can now name it in one run.
Both builds compile 0 errors / 0 warnings. Forces a full retrain.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-31 20:39:49 -04:00
int conf = MathMax ( m_barrierHorizonBars , 1 ) ;
refactor: split CExpertSignalAIBase implementation by responsibility
ExpertSignalAIBase.mqh was 8216 lines: the class declaration followed by 87
method bodies covering training, labelling, feature extraction, persistence,
chart drawing, online learning, the GA auto-tuner and inference, all in one
file. Train() alone is 1492 lines; a change to arrow drawing meant scrolling
past the era loop.
Moved the bodies into Expert\AIBase\, included at the bottom of the original
after the class declaration:
Training.mqh 1607 era loop, plateau ladder, checkpoint select, deploy
Features.mqh 1093 indicator creation + per-bar input feature vector
ChartUI.mqh 634 arrows, arrow persistence, status panel, cleanup
Persistence.mqh 492 .stats/.cfg sidecars, CPU-inference validation, copy
OnlineLearning.mqh 461 live continual learning, EMA shadow, OOS simulator
Labels.mqh 309 ZigZag pivot labels, async label-cache prebuild
AutoTune.mqh 275 genetic tuner (population, crossover, halving)
Inference.mqh 235 softmax, prior calibration, class priors
ExpertSignalAIBase.mqh 8216 -> 3131 (declaration + topology build only)
This is a pure relocation - verified mechanically, not by eye: HEAD's file
reconstructed from the eight partials plus the surviving remainder is
byte-identical to HEAD, span for span (scratchpad verify_split.py). No
declaration moved, no signature changed, no code rewritten, so behaviour is
unchanged by construction.
Compiles 0 errors, 0 warnings.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-29 00:42:45 -04:00
int barsAvail = Bars ( m_symbol . Name ( ) , PERIOD_CURRENT ) ;
//--- Need the frontier bar (now-relative index conf) plus a full feature window BEHIND it, plus a
//--- little slack so a short catch-up walk stays in-bounds.
int need = conf + ( int ) m_historyBars + 2 ;
if ( barsAvail < need )
return ;
//--- Load enough history for the frontier window and a bounded catch-up; RefreshConvergedSignal()
//--- only sized buffers relative to dtStudied (newest bars), which is too shallow to reach the
//--- confirmation frontier.
int wantBars = MathMin ( need + ONLINE_LEARN_MAX_CATCHUP , barsAvail ) ;
if ( ! ResizeBuffers ( wantBars ) | | ! RefreshData ( ) )
return ;
fix: the sequence models were reading the window backwards
BuildFeatureWindow() replaces eight hand-rolled copies of the same loop
and feeds the window OLDEST BAR FIRST. Every copy fed it newest-first,
because MQL5 timeseries indices run backwards and `r + b` with b ascending
walks into the past.
Harmless for PAI and CONV - a dense layer learns a weight per position
either way, a conv learns time-mirrored kernels. Not harmless for the
recurrent stacks:
- LSTM_SeqStepForward reads `inputs + t*Iw`, so step t is block t.
- It writes output[] only when t == steps-1: the visible output IS the
last hidden state.
- c_t = f*c_{t-1} + i*g decays toward the start of the sequence.
lstm_seq_flowcheck.cpp measured block 0's influence on the output at
1.2e-2 of block T-1's, at the shipped forget bias of 1.0.
So the bar being PREDICTED sat at the far end of the decay and the output
was handed to the OLDEST bar in the window - the exact inverse of what the
window is for. ~80x backwards on LSTM and HYBRID, on all three tiers
(OpenCL kernel, CPU DLL, pure-MQL5 inference), which is why it never
surfaced as a backend discrepancy.
This does not create edge - the MI diagnostics read at the noise floor
(p=0.4975) with a working positive control. It makes the one hypothesis
those diagnostics explicitly do NOT cover testable: they are marginal and
per-bar, and state they "cannot rule out one that only exists in
combination or across time". The sequence model is the instrument for
across-time structure and it has been crippled, so that hypothesis has
never been honestly tested.
Fingerprint gets an unconditional |WIN:2 - the vector keeps its shape and
its features, so a stale .nnw would load cleanly and run a model fitted to
one ordering against the other, silently. Re-keying every config is the
point, not collateral damage. FORCES A FULL RETRAIN.
Also: the now-relative bar caches are re-keyed on the two live paths.
EnsureBarCachesCapacity() was only ever called from training paths, but
once m_trainingComplete is set ScheduleTrainingIfNeeded() routes every bar
to RefreshConvergedSignal() and Train() is never re-entered - so nothing
cleared the feature cache again for the life of the process. A chart that
trained to convergence kept replaying the rows computed for the last
training era's bar grid: the live signal froze at its convergence-time
value, and OnlineLearnStep() backpropped those stale features against
freshly resolved labels. Backtests were never affected (an inference-only
process never allocates the arrays, so every read recomputes).
Compiles clean: 0 errors, 0 warnings.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-07 18:28:44 -04:00
//--- Same now-relative invalidation RefreshConvergedSignal() does, and needed independently of it: this
//--- runs on a DEEPER bar grid (wantBars reaches the confirmation frontier, that one only reaches the
//--- newest feature window), so the two legitimately disagree about `bars` and each must re-key the
//--- cache for the grid it is about to read. Stale rows matter more here than anywhere else - this is
//--- the one path that WRITES to a live, trading model, so a mismatched (features, label) pair is not a
//--- wrong arrow, it is a wrong weight update. See RefreshConvergedSignal()'s comment for why nothing
//--- else clears this once training has completed.
//--- Note the catch-up walk below is unaffected in cost: this fires once per call, before the loop, so
//--- the overlapping windows inside the loop still share cached rows.
EnsureBarCachesCapacity ( wantBars ) ;
refactor: split CExpertSignalAIBase implementation by responsibility
ExpertSignalAIBase.mqh was 8216 lines: the class declaration followed by 87
method bodies covering training, labelling, feature extraction, persistence,
chart drawing, online learning, the GA auto-tuner and inference, all in one
file. Train() alone is 1492 lines; a change to arrow drawing meant scrolling
past the era loop.
Moved the bodies into Expert\AIBase\, included at the bottom of the original
after the class declaration:
Training.mqh 1607 era loop, plateau ladder, checkpoint select, deploy
Features.mqh 1093 indicator creation + per-bar input feature vector
ChartUI.mqh 634 arrows, arrow persistence, status panel, cleanup
Persistence.mqh 492 .stats/.cfg sidecars, CPU-inference validation, copy
OnlineLearning.mqh 461 live continual learning, EMA shadow, OOS simulator
Labels.mqh 309 ZigZag pivot labels, async label-cache prebuild
AutoTune.mqh 275 genetic tuner (population, crossover, halving)
Inference.mqh 235 softmax, prior calibration, class priors
ExpertSignalAIBase.mqh 8216 -> 3131 (declaration + topology build only)
This is a pure relocation - verified mechanically, not by eye: HEAD's file
reconstructed from the eight partials plus the surviving remainder is
byte-identical to HEAD, span for span (scratchpad verify_split.py). No
declaration moved, no signature changed, no code rewritten, so behaviour is
unchanged by construction.
Compiles 0 errors, 0 warnings.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-29 00:42:45 -04:00
datetime frontierTime = m_Time . GetData ( conf ) ;
if ( frontierTime < = 0 )
return ;
//--- First step of this deployment (or a model that never online-learned): DON'T retroactively
//--- backfill the whole history through backprop in one shot - that could shift the just-validated
//--- deployed model materially before any live confirmation. Anchor the watermark at the current
//--- frontier and begin learning from genuinely new confirmations forward.
if ( m_onlineLearnedUpToTime < = 0 )
{
m_onlineLearnedUpToTime = frontierTime ;
return ;
}
if ( frontierTime < = m_onlineLearnedUpToTime )
return ; // no bar has matured past the watermark since last time
//--- Seed the guardrail EMA from the model's deploy-time OOS accuracy the first time we actually
//--- learn, so the floor is meaningful from the very first update (not a cold 0 that would trip it).
if ( m_onlineRollingAcc < 0.0 )
m_onlineRollingAcc = ( dForecast > 0.0 & & dForecast < = 100.0 ) ? dForecast : 100.0 ;
//--- Guardrail floor: deploy baseline minus a margin, never below the absolute minimum.
double baseline = ( dForecast > 0.0 & & dForecast < = 100.0 ) ? dForecast : 100.0 ;
double accFloor = MathMax ( ONLINE_LEARN_MIN_ACC , baseline - ONLINE_LEARN_ACC_MARGIN ) ;
//--- Find the oldest not-yet-learned confirmed bar: walk from the frontier (index conf) toward older
//--- bars (increasing index) until we pass the watermark or hit the catch-up cap, then learn newest-
//--- ward from there so bars are consumed in strict chronological (oldest->newest) order.
int oldestIdx = conf ;
while ( oldestIdx < barsAvail - 1
& & oldestIdx < conf + ONLINE_LEARN_MAX_CATCHUP
& & m_Time . GetData ( oldestIdx ) > m_onlineLearnedUpToTime )
oldestIdx + + ;
//--- oldestIdx now points at the first bar whose time is <= watermark (already learned) or the cap;
//--- the newest UNLEARNED bar is one step newer (idx-1). Learn from idx = oldestIdx-1 down to conf.
//--- Pin the learning rate for the duration of this walk and restore it after: `eta` is a GLOBAL
//--- shared by every signal instance, so leaving it modified would corrupt another model's training
//--- chunk - see ONLINE_LEARN_ETA_SCALE's comment.
double savedEta = eta ;
eta = m_modelEta * ONLINE_LEARN_ETA_SCALE ;
int learned = 0 ;
for ( int idx = oldestIdx - 1 ; idx > = conf ; idx - - )
{
datetime bt = m_Time . GetData ( idx ) ;
if ( bt < = m_onlineLearnedUpToTime )
continue ; // already learned (defensive; the walk above should exclude it)
//--- Build this bar's feature window - IDENTICAL to Train()/RefreshLatestSignal(): ends AT bar idx
//--- and extends m_historyBars into the past. No lookahead (all bars are older than idx).
fix: the sequence models were reading the window backwards
BuildFeatureWindow() replaces eight hand-rolled copies of the same loop
and feeds the window OLDEST BAR FIRST. Every copy fed it newest-first,
because MQL5 timeseries indices run backwards and `r + b` with b ascending
walks into the past.
Harmless for PAI and CONV - a dense layer learns a weight per position
either way, a conv learns time-mirrored kernels. Not harmless for the
recurrent stacks:
- LSTM_SeqStepForward reads `inputs + t*Iw`, so step t is block t.
- It writes output[] only when t == steps-1: the visible output IS the
last hidden state.
- c_t = f*c_{t-1} + i*g decays toward the start of the sequence.
lstm_seq_flowcheck.cpp measured block 0's influence on the output at
1.2e-2 of block T-1's, at the shipped forget bias of 1.0.
So the bar being PREDICTED sat at the far end of the decay and the output
was handed to the OLDEST bar in the window - the exact inverse of what the
window is for. ~80x backwards on LSTM and HYBRID, on all three tiers
(OpenCL kernel, CPU DLL, pure-MQL5 inference), which is why it never
surfaced as a backend discrepancy.
This does not create edge - the MI diagnostics read at the noise floor
(p=0.4975) with a working positive control. It makes the one hypothesis
those diagnostics explicitly do NOT cover testable: they are marginal and
per-bar, and state they "cannot rule out one that only exists in
combination or across time". The sequence model is the instrument for
across-time structure and it has been crippled, so that hypothesis has
never been honestly tested.
Fingerprint gets an unconditional |WIN:2 - the vector keeps its shape and
its features, so a stale .nnw would load cleanly and run a model fitted to
one ordering against the other, silently. Re-keying every config is the
point, not collateral damage. FORCES A FULL RETRAIN.
Also: the now-relative bar caches are re-keyed on the two live paths.
EnsureBarCachesCapacity() was only ever called from training paths, but
once m_trainingComplete is set ScheduleTrainingIfNeeded() routes every bar
to RefreshConvergedSignal() and Train() is never re-entered - so nothing
cleared the feature cache again for the life of the process. A chart that
trained to convergence kept replaying the rows computed for the last
training era's bar grid: the live signal froze at its convergence-time
value, and OnlineLearnStep() backpropped those stale features against
freshly resolved labels. Backtests were never affected (an inference-only
process never allocates the arrays, so every read recomputes).
Compiles clean: 0 errors, 0 warnings.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-07 18:28:44 -04:00
//--- "Identical" is now enforced rather than asserted - all three go through BuildFeatureWindow().
if ( ! BuildFeatureWindow ( idx ) )
refactor: split CExpertSignalAIBase implementation by responsibility
ExpertSignalAIBase.mqh was 8216 lines: the class declaration followed by 87
method bodies covering training, labelling, feature extraction, persistence,
chart drawing, online learning, the GA auto-tuner and inference, all in one
file. Train() alone is 1492 lines; a change to arrow drawing meant scrolling
past the era loop.
Moved the bodies into Expert\AIBase\, included at the bottom of the original
after the class declaration:
Training.mqh 1607 era loop, plateau ladder, checkpoint select, deploy
Features.mqh 1093 indicator creation + per-bar input feature vector
ChartUI.mqh 634 arrows, arrow persistence, status panel, cleanup
Persistence.mqh 492 .stats/.cfg sidecars, CPU-inference validation, copy
OnlineLearning.mqh 461 live continual learning, EMA shadow, OOS simulator
Labels.mqh 309 ZigZag pivot labels, async label-cache prebuild
AutoTune.mqh 275 genetic tuner (population, crossover, halving)
Inference.mqh 235 softmax, prior calibration, class priors
ExpertSignalAIBase.mqh 8216 -> 3131 (declaration + topology build only)
This is a pure relocation - verified mechanically, not by eye: HEAD's file
reconstructed from the eight partials plus the surviving remainder is
byte-identical to HEAD, span for span (scratchpad verify_split.py). No
declaration moved, no signature changed, no code rewritten, so behaviour is
unchanged by construction.
Compiles 0 errors, 0 warnings.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-29 00:42:45 -04:00
{
//--- window not buildable this bar (e.g. an indicator hole) - advance the watermark past it so
//--- we don't wedge re-trying the same bar forever, but learn nothing from it.
m_onlineLearnedUpToTime = bt ;
continue ;
}
//--- Predict with the CURRENT (pre-update) weights, then score against the confirmed label for the
//--- rolling guardrail - exactly AdvanceOosSimulationChunk()'s predict-before-learn measurement.
Net . feedForward ( TempData ) ;
Net . getResults ( TempData ) ;
double predSignal = ( m_outputNeuronsCount = = 3 ) ? ApplyClassificationSoftmax ( ) : TempData [ 0 ] ;
feat(ai): triple-barrier labels replace exact-pivot ZigZag targets
The 31:1 class imbalance was self-inflicted by the TARGET, not a property
of the market. Labelling only the exact bar where a ZigZag pivot confirms
gave Buy 1164 / Sell 1164 / Neutral 35841, and every correction mechanism
this codebase accumulated sits downstream of that one choice: the
logit-adjusted loss and its range cap, the prior EMA, the +-3.0 output-bias
seed, balanced-accuracy-then-precision selection with its coverage floor,
the recall floor and its catch-22, the alternation gate, NMS, and the four
oversampling designs that collapsed before them.
The reference this engine is built on (references/neuronetworksbook.pdf
ch. 3.1/3.3) also uses ZigZag, but targets the DIRECTION TO THE NEXT
EXTREMUM on every bar - ~50/50 by construction, with no imbalance to
correct at all. It never had this problem because it never asked "is this
the pivot bar".
Labels are now the triple barrier (Lopez de Prado ch. 3), using the EA's
OWN SL_Mode/TP_Mode: does a trade opened at this bar's close reach its
target before its stop, within a horizon. Buy = long resolves, Sell =
short resolves, Neutral = neither. Consequences:
- dir-precision in the era line stops being a proxy and becomes the win
rate of the strategy under its own exit rules.
- Expected balance ~25/25/50 at the shipped 1:3 (gambler's ruin), i.e.
~2:1 instead of 31:1. Measured and logged at the end of the prebuild.
- Spread is charged on both legs, so it is a NET win rate.
- Intrabar ambiguity resolves to the STOP. OHLC cannot order two touches
inside one bar and the optimistic reading is how a backtested edge
becomes a live loss.
ZigZag stays as input features (EnableSwingContext) and now also supplies
the vertical barrier: the horizon is the median confirmed leg length,
snapped to a coarse ladder. Derived, not configured, and deliberately kept
out of the filename fingerprint - a filename keyed on a measured quantity
orphans a trained model the moment the measurement moves.
Removed, because the premise died with the old target:
- the alternation gate. Correct for pivot labels (a ZigZag cannot emit two
same-type pivots in a row, so a repeat was provably a false fire), and
wrong for barrier labels, which answer each bar independently. It also
took its worst consequence with it: a one-sided model previously got ONE
trade per backtest, a hard blocker on marketplace validation.
- SignalClusterWindow now defaults off - it de-duplicated repeats that are
now real trades. Kept as an opt-in display control.
- LABEL_WINDOW_BARS, the pivot-widening pass, ConfirmedZigZagLabel.
- the era-0 output-bias seed now needs a genuinely dominant class (0.70)
rather than 0.40; at ~50% Neutral a +-3.0 seed is a distortion, not a
correction.
Also fixed, both found while wiring the above:
1. RefreshConvergedSignal sized its buffers from a date delta
(Bars(sym, period, dtStudied, TimeCurrent())). dtStudied is a training
watermark; in the tester it is loaded from a live-chart save AHEAD of
the simulated date, so the interval inverted, Bars() returned ~0, and
the buffer came out at exactly m_historyBars - deep enough for the OHLC
window and far too shallow for the Donchian-50 / 20-bar-return / SMA
extension behind it. Inference silently computed DIFFERENT features
from the ones training learned on, live as well as in the tester. Now
sized from what the feature builder actually needs.
2. The barrier horizon is resolved on the deployed path too. A deployed
model never enters Train(), so it never reached the prebuild, and
OnlineLearnStep reads the horizon as its confirmation delay - left at
the fallback it would have backpropped bars whose barriers had not
resolved. Silent lookahead in the one place that writes to a live model.
SL_Mode/TP_Mode join the weights fingerprint: they define the labels now,
so a model trained at 1:3 must never be silently reused at 1:1. This
re-keys every pre-existing model by design - none were trained on this task.
Inference census extended with the vote gate. LongCondition/ShortCondition
open with a readiness check the refresh counters never see; in the tester it
reduces to "the seeded _optcache.nnw must have LOADED", and if it did not,
every vote is hard-zeroed while the model still answers Buy. The old three
counters would have read that as "the model says Neutral" - false, and a
completely different fix. This is the leading candidate for the
zero-direction backtest and the census can now name it in one run.
Both builds compile 0 errors / 0 warnings. Forces a full retrain.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-31 20:39:49 -04:00
//--- Same target rule training used. `idx` is at or beyond the confirmation frontier (conf ==
//--- m_barrierHorizonBars, enforced above), so the forward window this reads is fully closed.
ENUM_SIGNAL trueSignal = TripleBarrierLabel ( idx ) ;
refactor: split CExpertSignalAIBase implementation by responsibility
ExpertSignalAIBase.mqh was 8216 lines: the class declaration followed by 87
method bodies covering training, labelling, feature extraction, persistence,
chart drawing, online learning, the GA auto-tuner and inference, all in one
file. Train() alone is 1492 lines; a change to arrow drawing meant scrolling
past the era loop.
Moved the bodies into Expert\AIBase\, included at the bottom of the original
after the class declaration:
Training.mqh 1607 era loop, plateau ladder, checkpoint select, deploy
Features.mqh 1093 indicator creation + per-bar input feature vector
ChartUI.mqh 634 arrows, arrow persistence, status panel, cleanup
Persistence.mqh 492 .stats/.cfg sidecars, CPU-inference validation, copy
OnlineLearning.mqh 461 live continual learning, EMA shadow, OOS simulator
Labels.mqh 309 ZigZag pivot labels, async label-cache prebuild
AutoTune.mqh 275 genetic tuner (population, crossover, halving)
Inference.mqh 235 softmax, prior calibration, class priors
ExpertSignalAIBase.mqh 8216 -> 3131 (declaration + topology build only)
This is a pure relocation - verified mechanically, not by eye: HEAD's file
reconstructed from the eight partials plus the surviving remainder is
byte-identical to HEAD, span for span (scratchpad verify_split.py). No
declaration moved, no signature changed, no code rewritten, so behaviour is
unchanged by construction.
Compiles 0 errors, 0 warnings.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-29 00:42:45 -04:00
bool hit = ( DoubleToSignal ( predSignal ) = = trueSignal ) ;
m_onlineRollingAcc + = ( 100.0 * ( hit ? 1.0 : 0.0 ) - m_onlineRollingAcc ) / ONLINE_ACC_SMOOTH ;
//--- Per-class softmax probabilities as of THIS bar's pre-update prediction. Must be read here,
//--- before TempData is rebuilt as the target vector below - ApplyClassificationSoftmax() has
//--- already normalised TempData[0..2] in place into a genuine distribution (same contract pass 2
//--- relies on for its own focal term).
double pBuy = ( TempData . Total ( ) > 0 ) ? TempData . At ( 0 ) : 0.0 ;
double pSell = ( TempData . Total ( ) > 1 ) ? TempData . At ( 1 ) : 0.0 ;
double pNeutral = ( TempData . Total ( ) > 2 ) ? TempData . At ( 2 ) : 0.0 ;
//--- Build the target vector - identical encoding to Train()/AdvanceOosSimulationChunk().
bool buy = ( trueSignal = = Buy ) ;
bool sell = ( trueSignal = = Sell ) ;
TempData . Clear ( ) ;
if ( m_outputNeuronsCount = = 1 )
TempData . Add ( buy & & ! sell ? 1 : ( ! buy & & sell ? -1 : 0 ) ) ;
else
{
TempData . Add ( buy ? LABEL_SMOOTH_HIGH : LABEL_SMOOTH_LOW ) ;
TempData . Add ( sell ? LABEL_SMOOTH_HIGH : LABEL_SMOOTH_LOW ) ;
TempData . Add ( ( ! buy & & ! sell ) ? LABEL_SMOOTH_HIGH : LABEL_SMOOTH_LOW ) ;
}
Net . backProp ( TempData , OnlineSampleWeight ( trueSignal , pBuy , pSell , pNeutral ) ) ;
m_onlineSamples + + ;
//--- Deploy the improvement ONLY while accuracy holds. Warmup: allow the first few blends (the
//--- model was just validated at deploy, steps are tiny) until the EMA has enough samples to judge.
bool blendOk = ( m_onlineSamples < = ONLINE_LEARN_WARMUP ) | | ( m_onlineRollingAcc > = accFloor ) ;
if ( blendOk )
{
m_shadowNet . BlendWeightsFrom ( Net , SHADOW_WEIGHT_TAU ) ;
if ( m_onlineBlendFrozen )
{
m_onlineBlendFrozen = false ;
Print ( ID + " : online-learning deployment RESUMED - rolling accuracy recovered to "
+ DoubleToString ( m_onlineRollingAcc , 1 ) + " % (floor " + DoubleToString ( accFloor , 1 ) + " %) " ) ;
}
}
else if ( ! m_onlineBlendFrozen )
{
m_onlineBlendFrozen = true ;
Print ( ID + " : online-learning deployment FROZEN - rolling accuracy " + DoubleToString ( m_onlineRollingAcc , 1 )
+ " % fell below floor " + DoubleToString ( accFloor , 1 ) + " %; live keeps trading the last-good model while it adapts " ) ;
}
m_onlineLearnedUpToTime = bt ;
learned + + ;
m_onlineBarsSincePersist + + ;
}
//--- Hand the shared global back exactly as found, on BOTH exit paths below - see the matching
//--- savedEta assignment above for why this must not leak out of this function.
eta = savedEta ;
if ( learned < = 0 )
return ;
//--- Periodic durable persistence so a crash loses at most ONLINE_LEARN_PERSIST_EVERY bars of
//--- adaptation (shutdown also persists via PersistOnShutdown()).
if ( m_onlineBarsSincePersist > = ONLINE_LEARN_PERSIST_EVERY )
{
double ip [ ] ;
m_indicatorTuner . Flatten ( ip ) ;
bool saveOk = Net . Save ( m_activeFileName + " .nnw " , dError , dUndefine , dForecast , dtStudied , m_activeFileCommon , m_eraCount , m_trainingComplete , ip ) ;
if ( ! saveOk )
Print ( ID + " : ERROR - online-learning Net.Save failed for " + m_activeFileName + " .nnw. Retrying next persist interval instead of resetting the bars-since-persist counter. " ) ;
SaveShadowNet ( ip ) ;
if ( ! SaveModelStats ( m_activeFileName , m_activeFileCommon ) )
Print ( ID + " : ERROR - online-learning SaveModelStats failed for " + m_activeFileName + " . " ) ;
// Only reset the counter on a successful weight save - resetting unconditionally on a
// transient failure would silently double the effective data-loss window on the NEXT failure too.
if ( saveOk )
{
m_onlineBarsSincePersist = 0 ;
PrintVerbose ( ID + " : online-learning checkpoint saved ( " + IntegerToString ( ( int ) m_onlineSamples )
+ " total updates, rolling acc " + DoubleToString ( m_onlineRollingAcc , 1 ) + " %) " ) ;
}
}
}
//+------------------------------------------------------------------+
//| |
//+------------------------------------------------------------------+
void CExpertSignalAIBase : : SaveShadowNet ( const double & indicatorParams [ ] )
{
if ( CheckPointer ( m_shadowNet ) = = POINTER_INVALID )
return ;
m_shadowNet . Save ( m_activeFileName + " _shadow.nnw " , dError , dUndefine , dForecast , dtStudied , m_activeFileCommon , m_eraCount , m_trainingComplete , indicatorParams ) ;
}
//+------------------------------------------------------------------+
//| |
//+------------------------------------------------------------------+
void CExpertSignalAIBase : : EnsureShadowNet ( void )
{
if ( CheckPointer ( m_shadowNet ) ! = POINTER_INVALID )
return ;
//--- Pure-MQL5 inference (DLL-free backtest): no era blending happens, so the shadow would just be a
//--- copy of Net - and bootstrapping one via Save/Load would spin a compute backend up on the clone,
//--- defeating the DLL-free goal. Skip it; RefreshLatestSignal() falls back to Net directly.
if ( CheckPointer ( Net ) ! = POINTER_INVALID & & Net . CpuInference ( ) )
return ;
string shadowFile = m_activeFileName + " _shadow.nnw " ;
if ( FileIsExist ( shadowFile , m_activeFileCommon ? FILE_COMMON : 0 ) )
{
CNet * loaded = new CNet ( NULL ) ;
if ( CheckPointer ( loaded ) ! = POINTER_INVALID )
{
double loadE , loadU , loadF ;
datetime loadTime ;
long loadEra ;
bool loadComplete ;
double loadIp [ ] ;
if ( loaded . Load ( shadowFile , loadE , loadU , loadF , loadTime , m_activeFileCommon , loadEra , loadComplete , loadIp , true /*quiet: a miss just falls through to the clone bootstrap below*/ ) )
{
m_shadowNet = loaded ;
//--- This second CNet spins up its OWN compute backend, so on a fresh attach the log shows a
//--- second backend-init block right after the main model's. Name it here (verbose) so it
//--- reads as "the shadow net came up" rather than "the EA started twice".
PrintVerbose ( ID + " : EMA shadow net restored from " + shadowFile + " (its own network instance - hence a second compute-backend init) " ) ;
return ;
}
delete loaded ;
}
}
//--- No compatible persisted shadow - bootstrap from Net's current weights. Clone via the full
//--- Save()/Load() pair - see StartOosContinualSimulation()'s matching comment for why a lighter
//--- restore is not safe here (it would need opencl/directml already initialized on the target
//--- CNet, which a bare "new CNet(NULL)" does not have).
if ( CheckPointer ( Net ) = = POINTER_INVALID )
return ;
//--- Attempt the clone bootstrap at most once per topology (see m_shadowBootstrapAttempted). On the
//--- tester's CPU-DLL fallback a second full-net clone can fail to load; retrying every bar would
//--- rebuild the compute backend each tick and crawl. Falling back to Net is correct and lossless here.
if ( m_shadowBootstrapAttempted )
return ;
m_shadowBootstrapAttempted = true ;
//--- Co-locate the ephemeral clone temp with the active model (COMMON on a live chart, LOCAL in the
//--- tester sandbox) instead of always LOCAL.
string cloneFile = m_activeFileName + " _shadowclone.tmp " ;
int cloneFlags = m_activeFileCommon ? FILE_COMMON : 0 ;
double ip [ ] ;
if ( ! Net . Save ( cloneFile , 0.0 , 0.0 , 0.0 , dtStudied , m_activeFileCommon , m_eraCount , m_trainingComplete , ip ) )
return ;
CNet * clone = new CNet ( NULL ) ;
if ( CheckPointer ( clone ) = = POINTER_INVALID )
{
FileDelete ( cloneFile , cloneFlags ) ;
return ;
}
double loadE , loadU , loadF ;
datetime loadTime ;
long loadEra ;
bool loadComplete ;
double loadIp [ ] ;
bool loaded = clone . Load ( cloneFile , loadE , loadU , loadF , loadTime , m_activeFileCommon , loadEra , loadComplete , loadIp , true /*quiet: best-effort clone, the miss is handled gracefully below*/ ) ;
FileDelete ( cloneFile , cloneFlags ) ;
if ( ! loaded )
{
delete clone ;
//--- Best-effort: without a shadow, live signals read the main Net directly (RefreshLatestSignal's
//--- deployNet fallback), which is correct and lossless - so one calm line, not an error.
//--- This used to be described as EXPECTED on a CPU-DLL box ("can't allocate a 2nd net"). It is not:
//--- the clone load was failing for the same reason the MAIN model load was - CLayer::CreateElement
//--- had stopped overriding CArrayObj::CreateElement, so every CNet::Load failed at layer 0
//--- regardless of backend (see AI\Network.mqh). With that fixed this path should be rare; if it
//--- shows up repeatedly, investigate rather than assume a hardware limit.
PrintVerbose ( ID + " : EMA shadow net could not be bootstrapped - live signals use the main model directly (lossless fallback). " ) ;
return ;
}
m_shadowNet = clone ;
//--- See the matching note on the restore path above: a second CNet means a second compute-backend init
//--- in the log, which is expected, not a duplicated EA.
PrintVerbose ( ID + " : EMA shadow net bootstrapped from the main model's current weights (its own network instance - hence a second compute-backend init) " ) ;
}
# endif // WARRIOR_AIBASE_ONLINELEARNING_MQH