Warrior_EA/Expert/AIBase/Labels.mqh

500 lines
30 KiB
MQL5
Raw Permalink Normal View History

refactor: split CExpertSignalAIBase implementation by responsibility ExpertSignalAIBase.mqh was 8216 lines: the class declaration followed by 87 method bodies covering training, labelling, feature extraction, persistence, chart drawing, online learning, the GA auto-tuner and inference, all in one file. Train() alone is 1492 lines; a change to arrow drawing meant scrolling past the era loop. Moved the bodies into Expert\AIBase\, included at the bottom of the original after the class declaration: Training.mqh 1607 era loop, plateau ladder, checkpoint select, deploy Features.mqh 1093 indicator creation + per-bar input feature vector ChartUI.mqh 634 arrows, arrow persistence, status panel, cleanup Persistence.mqh 492 .stats/.cfg sidecars, CPU-inference validation, copy OnlineLearning.mqh 461 live continual learning, EMA shadow, OOS simulator Labels.mqh 309 ZigZag pivot labels, async label-cache prebuild AutoTune.mqh 275 genetic tuner (population, crossover, halving) Inference.mqh 235 softmax, prior calibration, class priors ExpertSignalAIBase.mqh 8216 -> 3131 (declaration + topology build only) This is a pure relocation - verified mechanically, not by eye: HEAD's file reconstructed from the eight partials plus the surviving remainder is byte-identical to HEAD, span for span (scratchpad verify_split.py). No declaration moved, no signature changed, no code rewritten, so behaviour is unchanged by construction. Compiles 0 errors, 0 warnings. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-29 00:42:45 -04:00
//+------------------------------------------------------------------+
//| Warrior_EA |
//| AnimateDread |
//| |
feat(target): delete the barrier/geometry stack - the label is the verdict Step 3 of the swing-pivot plan, whole-hog. The swing label is now the ONE target and the era verdict is precision + recall per class against the label's own base rate - no win rate, no break-even, no expectancy, no geometry anywhere in training. DELETED - Expert/Excursion/ (4), Expert/BarrierHorizon/ (4), GeometrySweep, FirstPassageLadder, Labeling/TripleBarrier.mqh (CLabelOverlap survives in Labeling/LabelOverlap.mqh), 3 test EAs. - TripleBarrierLabel + walk, fractal label, geometry derivation/scan/ adoption, exit-policy replay, excursion MI targets, the drift verdict (DIRECTION_INTELLIGENT), the recall floor, balanced-accuracy telemetry, the barrier defines, the .cfg geometry adopt (slots kept as zeros for the positional layout), the derived-geometry live-order override. - TRAINING_TARGET input/enum: direction models are always swing; META2 re-keys the meta head onto label agreement (descriptor loses its two geometry slots). REWORKED - Labels.mqh (1795 -> ~370 lines): AdvanceSwingLabelState with FINALITY-GATED CACHING - an unresolved bar (pivot pair uncommitted) is never cached, so it can never freeze as a false Neutral; training, calibration, OOS scoring and online learning all skip unresolved bars. - SDeployVerdict: significance-only; SOosTally chance = larger directional class share; pooled gate poolability = timeframe (record v2). - Purge/embargo/declustering gaps: the measured mean label resolution lag (LabelResolutionBars), not a barrier horizon. - Pool purge key + backfill DB rows: marked at the bar the label resolved on (m_labelResolveAge), not a fabricated barrier touch. - Online learning frontier: finality, not a horizon delay. - m_bestBalancedOos -> m_bestSelectionScore, m_erasSinceBestBalanced -> m_erasSinceBest, ensemble vote outcome arrays -> label arrays. STEP 4 folded in: Entry_Multiplier / SL_Mode / TP_Mode / tradingdirection are inputs again - trade management is the tester GA's search space. Fingerprints: every direction model re-keys (TGT:SWG1 now unconditional, CUT token gone); META1 -> META2. Full retrain, as planned. Compile-verified in _claude_stage: Warrior_EA + both surviving test EAs, 0 errors, 0 warnings each. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-24 20:42:31 -04:00
//| Swing-pivot labelling and the async label-cache prebuild. |
refactor: split CExpertSignalAIBase implementation by responsibility ExpertSignalAIBase.mqh was 8216 lines: the class declaration followed by 87 method bodies covering training, labelling, feature extraction, persistence, chart drawing, online learning, the GA auto-tuner and inference, all in one file. Train() alone is 1492 lines; a change to arrow drawing meant scrolling past the era loop. Moved the bodies into Expert\AIBase\, included at the bottom of the original after the class declaration: Training.mqh 1607 era loop, plateau ladder, checkpoint select, deploy Features.mqh 1093 indicator creation + per-bar input feature vector ChartUI.mqh 634 arrows, arrow persistence, status panel, cleanup Persistence.mqh 492 .stats/.cfg sidecars, CPU-inference validation, copy OnlineLearning.mqh 461 live continual learning, EMA shadow, OOS simulator Labels.mqh 309 ZigZag pivot labels, async label-cache prebuild AutoTune.mqh 275 genetic tuner (population, crossover, halving) Inference.mqh 235 softmax, prior calibration, class priors ExpertSignalAIBase.mqh 8216 -> 3131 (declaration + topology build only) This is a pure relocation - verified mechanically, not by eye: HEAD's file reconstructed from the eight partials plus the surviving remainder is byte-identical to HEAD, span for span (scratchpad verify_split.py). No declaration moved, no signature changed, no code rewritten, so behaviour is unchanged by construction. Compiles 0 errors, 0 warnings. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-29 00:42:45 -04:00
//+------------------------------------------------------------------+
#ifndef WARRIOR_AIBASE_LABELS_MQH
#define WARRIOR_AIBASE_LABELS_MQH
//+------------------------------------------------------------------+
//| A CLOSED CANDLE IS NOT A REASON TO RELABEL ANYTHING. |
//| |
//| Series indices are relative to now, so one new bar moves every |
//| cached bar's index by one. That used to invalidate the whole |
//| prebuild, which then rebuilt from scratch - on a timeframe where |
//| a bar closes faster than a run finishes, the labels were being |
//| recomputed continuously and the training set never held still. |
//| The labels themselves do not change: shift them and walk only the |
//| newest `delta` bars. |
//| |
//| Refuses (-> full rebuild) when a prebuild is mid-flight, since |
//| its cursor is an index into the array being moved. |
//+------------------------------------------------------------------+
bool CExpertSignalAIBase::ShiftBarCaches(const int bars, const int delta)
{
if(delta <= 0 || bars <= delta || m_labelCacheBars <= 0 || m_labelPrebuildActive)
return false;
if(ArrayResize(m_labelCacheBuy, bars) < 0 || ArrayResize(m_labelCacheSell, bars) < 0
feat(target): delete the barrier/geometry stack - the label is the verdict Step 3 of the swing-pivot plan, whole-hog. The swing label is now the ONE target and the era verdict is precision + recall per class against the label's own base rate - no win rate, no break-even, no expectancy, no geometry anywhere in training. DELETED - Expert/Excursion/ (4), Expert/BarrierHorizon/ (4), GeometrySweep, FirstPassageLadder, Labeling/TripleBarrier.mqh (CLabelOverlap survives in Labeling/LabelOverlap.mqh), 3 test EAs. - TripleBarrierLabel + walk, fractal label, geometry derivation/scan/ adoption, exit-policy replay, excursion MI targets, the drift verdict (DIRECTION_INTELLIGENT), the recall floor, balanced-accuracy telemetry, the barrier defines, the .cfg geometry adopt (slots kept as zeros for the positional layout), the derived-geometry live-order override. - TRAINING_TARGET input/enum: direction models are always swing; META2 re-keys the meta head onto label agreement (descriptor loses its two geometry slots). REWORKED - Labels.mqh (1795 -> ~370 lines): AdvanceSwingLabelState with FINALITY-GATED CACHING - an unresolved bar (pivot pair uncommitted) is never cached, so it can never freeze as a false Neutral; training, calibration, OOS scoring and online learning all skip unresolved bars. - SDeployVerdict: significance-only; SOosTally chance = larger directional class share; pooled gate poolability = timeframe (record v2). - Purge/embargo/declustering gaps: the measured mean label resolution lag (LabelResolutionBars), not a barrier horizon. - Pool purge key + backfill DB rows: marked at the bar the label resolved on (m_labelResolveAge), not a fabricated barrier touch. - Online learning frontier: finality, not a horizon delay. - m_bestBalancedOos -> m_bestSelectionScore, m_erasSinceBestBalanced -> m_erasSinceBest, ensemble vote outcome arrays -> label arrays. STEP 4 folded in: Entry_Multiplier / SL_Mode / TP_Mode / tradingdirection are inputs again - trade management is the tester GA's search space. Fingerprints: every direction model re-keys (TGT:SWG1 now unconditional, CUT token gone); META1 -> META2. Full retrain, as planned. Compile-verified in _claude_stage: Warrior_EA + both surviving test EAs, 0 errors, 0 warnings each. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-24 20:42:31 -04:00
|| ArrayResize(m_labelResolveAge, bars) < 0 || ArrayResize(m_labelCacheHasValue, bars) < 0
|| ArrayResize(m_featureCacheHasValue, bars) < 0 || ArrayResize(m_featureCacheValid, bars) < 0
|| ArrayResize(m_featureCache, bars * m_neuronsCount) < 0)
return false;
//--- Backwards, so a source element is never overwritten before it is read.
for(int i = bars - 1; i >= delta; i--)
{
int j = i - delta;
m_labelCacheBuy[i] = m_labelCacheBuy[j];
m_labelCacheSell[i] = m_labelCacheSell[j];
feat(target): delete the barrier/geometry stack - the label is the verdict Step 3 of the swing-pivot plan, whole-hog. The swing label is now the ONE target and the era verdict is precision + recall per class against the label's own base rate - no win rate, no break-even, no expectancy, no geometry anywhere in training. DELETED - Expert/Excursion/ (4), Expert/BarrierHorizon/ (4), GeometrySweep, FirstPassageLadder, Labeling/TripleBarrier.mqh (CLabelOverlap survives in Labeling/LabelOverlap.mqh), 3 test EAs. - TripleBarrierLabel + walk, fractal label, geometry derivation/scan/ adoption, exit-policy replay, excursion MI targets, the drift verdict (DIRECTION_INTELLIGENT), the recall floor, balanced-accuracy telemetry, the barrier defines, the .cfg geometry adopt (slots kept as zeros for the positional layout), the derived-geometry live-order override. - TRAINING_TARGET input/enum: direction models are always swing; META2 re-keys the meta head onto label agreement (descriptor loses its two geometry slots). REWORKED - Labels.mqh (1795 -> ~370 lines): AdvanceSwingLabelState with FINALITY-GATED CACHING - an unresolved bar (pivot pair uncommitted) is never cached, so it can never freeze as a false Neutral; training, calibration, OOS scoring and online learning all skip unresolved bars. - SDeployVerdict: significance-only; SOosTally chance = larger directional class share; pooled gate poolability = timeframe (record v2). - Purge/embargo/declustering gaps: the measured mean label resolution lag (LabelResolutionBars), not a barrier horizon. - Pool purge key + backfill DB rows: marked at the bar the label resolved on (m_labelResolveAge), not a fabricated barrier touch. - Online learning frontier: finality, not a horizon delay. - m_bestBalancedOos -> m_bestSelectionScore, m_erasSinceBestBalanced -> m_erasSinceBest, ensemble vote outcome arrays -> label arrays. STEP 4 folded in: Entry_Multiplier / SL_Mode / TP_Mode / tradingdirection are inputs again - trade management is the tester GA's search space. Fingerprints: every direction model re-keys (TGT:SWG1 now unconditional, CUT token gone); META1 -> META2. Full retrain, as planned. Compile-verified in _claude_stage: Warrior_EA + both surviving test EAs, 0 errors, 0 warnings each. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-24 20:42:31 -04:00
m_labelResolveAge[i] = m_labelResolveAge[j];
m_labelCacheHasValue[i] = m_labelCacheHasValue[j];
m_featureCacheHasValue[i] = m_featureCacheHasValue[j];
m_featureCacheValid[i] = m_featureCacheValid[j];
int to = i * m_neuronsCount, from = j * m_neuronsCount;
for(int k = 0; k < m_neuronsCount; k++)
m_featureCache[to + k] = m_featureCache[from + k];
}
for(int i = 0; i < delta; i++)
{
m_labelCacheHasValue[i] = false;
m_featureCacheHasValue[i] = false;
}
m_labelCacheBars = bars;
m_labelCacheAnchorTime = m_Time.GetData(0);
return true;
}
//+------------------------------------------------------------------+
refactor: split CExpertSignalAIBase implementation by responsibility ExpertSignalAIBase.mqh was 8216 lines: the class declaration followed by 87 method bodies covering training, labelling, feature extraction, persistence, chart drawing, online learning, the GA auto-tuner and inference, all in one file. Train() alone is 1492 lines; a change to arrow drawing meant scrolling past the era loop. Moved the bodies into Expert\AIBase\, included at the bottom of the original after the class declaration: Training.mqh 1607 era loop, plateau ladder, checkpoint select, deploy Features.mqh 1093 indicator creation + per-bar input feature vector ChartUI.mqh 634 arrows, arrow persistence, status panel, cleanup Persistence.mqh 492 .stats/.cfg sidecars, CPU-inference validation, copy OnlineLearning.mqh 461 live continual learning, EMA shadow, OOS simulator Labels.mqh 309 ZigZag pivot labels, async label-cache prebuild AutoTune.mqh 275 genetic tuner (population, crossover, halving) Inference.mqh 235 softmax, prior calibration, class priors ExpertSignalAIBase.mqh 8216 -> 3131 (declaration + topology build only) This is a pure relocation - verified mechanically, not by eye: HEAD's file reconstructed from the eight partials plus the surviving remainder is byte-identical to HEAD, span for span (scratchpad verify_split.py). No declaration moved, no signature changed, no code rewritten, so behaviour is unchanged by construction. Compiles 0 errors, 0 warnings. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-29 00:42:45 -04:00
bool CExpertSignalAIBase::EnsureBarCachesCapacity(int bars)
{
if(bars == m_labelCacheBars && m_Time.GetData(0) == m_labelCacheAnchorTime)
return false;
//--- New candles only: shift instead of wiping. Returns false = "nothing to rebuild", which is
//--- exactly what the caller does with an unchanged cache.
if(m_labelCacheAnchorTime > 0 && m_Time.GetData(0) > m_labelCacheAnchorTime
&& ShiftBarCaches(bars, bars - m_labelCacheBars))
return false;
refactor: split CExpertSignalAIBase implementation by responsibility ExpertSignalAIBase.mqh was 8216 lines: the class declaration followed by 87 method bodies covering training, labelling, feature extraction, persistence, chart drawing, online learning, the GA auto-tuner and inference, all in one file. Train() alone is 1492 lines; a change to arrow drawing meant scrolling past the era loop. Moved the bodies into Expert\AIBase\, included at the bottom of the original after the class declaration: Training.mqh 1607 era loop, plateau ladder, checkpoint select, deploy Features.mqh 1093 indicator creation + per-bar input feature vector ChartUI.mqh 634 arrows, arrow persistence, status panel, cleanup Persistence.mqh 492 .stats/.cfg sidecars, CPU-inference validation, copy OnlineLearning.mqh 461 live continual learning, EMA shadow, OOS simulator Labels.mqh 309 ZigZag pivot labels, async label-cache prebuild AutoTune.mqh 275 genetic tuner (population, crossover, halving) Inference.mqh 235 softmax, prior calibration, class priors ExpertSignalAIBase.mqh 8216 -> 3131 (declaration + topology build only) This is a pure relocation - verified mechanically, not by eye: HEAD's file reconstructed from the eight partials plus the surviving remainder is byte-identical to HEAD, span for span (scratchpad verify_split.py). No declaration moved, no signature changed, no code rewritten, so behaviour is unchanged by construction. Compiles 0 errors, 0 warnings. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-29 00:42:45 -04:00
ArrayResize(m_labelCacheBuy, bars);
ArrayResize(m_labelCacheSell, bars);
feat(target): delete the barrier/geometry stack - the label is the verdict Step 3 of the swing-pivot plan, whole-hog. The swing label is now the ONE target and the era verdict is precision + recall per class against the label's own base rate - no win rate, no break-even, no expectancy, no geometry anywhere in training. DELETED - Expert/Excursion/ (4), Expert/BarrierHorizon/ (4), GeometrySweep, FirstPassageLadder, Labeling/TripleBarrier.mqh (CLabelOverlap survives in Labeling/LabelOverlap.mqh), 3 test EAs. - TripleBarrierLabel + walk, fractal label, geometry derivation/scan/ adoption, exit-policy replay, excursion MI targets, the drift verdict (DIRECTION_INTELLIGENT), the recall floor, balanced-accuracy telemetry, the barrier defines, the .cfg geometry adopt (slots kept as zeros for the positional layout), the derived-geometry live-order override. - TRAINING_TARGET input/enum: direction models are always swing; META2 re-keys the meta head onto label agreement (descriptor loses its two geometry slots). REWORKED - Labels.mqh (1795 -> ~370 lines): AdvanceSwingLabelState with FINALITY-GATED CACHING - an unresolved bar (pivot pair uncommitted) is never cached, so it can never freeze as a false Neutral; training, calibration, OOS scoring and online learning all skip unresolved bars. - SDeployVerdict: significance-only; SOosTally chance = larger directional class share; pooled gate poolability = timeframe (record v2). - Purge/embargo/declustering gaps: the measured mean label resolution lag (LabelResolutionBars), not a barrier horizon. - Pool purge key + backfill DB rows: marked at the bar the label resolved on (m_labelResolveAge), not a fabricated barrier touch. - Online learning frontier: finality, not a horizon delay. - m_bestBalancedOos -> m_bestSelectionScore, m_erasSinceBestBalanced -> m_erasSinceBest, ensemble vote outcome arrays -> label arrays. STEP 4 folded in: Entry_Multiplier / SL_Mode / TP_Mode / tradingdirection are inputs again - trade management is the tester GA's search space. Fingerprints: every direction model re-keys (TGT:SWG1 now unconditional, CUT token gone); META1 -> META2. Full retrain, as planned. Compile-verified in _claude_stage: Warrior_EA + both surviving test EAs, 0 errors, 0 warnings each. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-24 20:42:31 -04:00
//--- Sized with the label caches they share a validity flag with, so they can never disagree
diag: is "optimal SL/TP" learnable? Score the features against excursions Proposed direction: train the net to predict entry/SL/TP that maximise return and minimise drawdown, rather than to classify direction. Before rebuilding a head, measure whether the target is learnable at all. That question splits into two that behave nothing alike: HOW FAR price travels (MFE/MAE) - essentially volatility, and volatility clustering is about the most robust regularity in markets. WHICH WAY it goes first (the asymmetry) - direction, which is what every noise-floor verdict in this project has been about. Expectancy comes ONLY from the second. The first buys position sizing and drawdown control - worth having under prop-firm limits, but not an edge: exit management on RANDOM entries already moved the payoff ratio 0.92 -> 5.72 with expectancy FLAT. Crucially this is NOT already answered. Every MI figure here scored the triple-barrier label, i.e. one specific question at one fixed geometry. A noise-floor result there says nothing about whether excursion MAGNITUDE is learnable - different target, different answer. Four targets, and the verdict is the CONTRAST, printed explicitly because the dangerous misreading of "UP clears" is "we can predict profitable trades": RANGE (up+dn) - realised volatility, included as a POSITIVE CONTROL that SHOULD clear. Every prior verdict here lacked a control expected to pass; a range target at the floor indicts the measurement, not the market. UP / DOWN - MFE / MAE. ASYMMETRY - up-dn, the only one that can pay. Collected inside the walk the label already does (one max, one min per bar). The early-out when both barriers resolved is GONE: it would have truncated the excursions at whichever bar tripped the last barrier, making the measurement a function of the CURRENT SL/TP - the circularity this is trying to escape. The loop was already bounded by the horizon, so only the average cost moves. Discretised into 3 EQUAL-FREQUENCY bins, so every downstream piece (block permutation, null, p-value) is reused unchanged. Equal-frequency because MFE is fat-tailed and fixed-width bins would put nearly every row in bin 0; it also pins H(Y) at ln(3)=1.099 for all four, making them comparable to each other and to the barrier label's ~1.02 instead of confounded by class balance. Two bugs fixed in this code before it ever ran, both of which would have produced a plausible quiet wrong answer rather than an error: - TripleBarrierLabel early-returns on invalid ATR/close BEFORE the point the accumulators were reset, so one bar's excursions would be cached under another bar's index. Cleared at the top now, ahead of every return. - An unresolvable bar is still flagged as labelled but carries excursions of exactly 0. Under equal-frequency binning a block of identical zeros drags the lowest cut onto zero and a third of the sample lands in one uninformative bin - a depressed score that reads as "not predictable", a false negative in the direction that would wrongly kill the idea. Rows where both excursions are zero are dropped; price cannot travel zero both ways over a whole horizon. Read-only diagnostic. No topology or label change: no retrain of its own. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-07 10:22:41 -04:00
//--- about how many bars they cover.
feat(target): delete the barrier/geometry stack - the label is the verdict Step 3 of the swing-pivot plan, whole-hog. The swing label is now the ONE target and the era verdict is precision + recall per class against the label's own base rate - no win rate, no break-even, no expectancy, no geometry anywhere in training. DELETED - Expert/Excursion/ (4), Expert/BarrierHorizon/ (4), GeometrySweep, FirstPassageLadder, Labeling/TripleBarrier.mqh (CLabelOverlap survives in Labeling/LabelOverlap.mqh), 3 test EAs. - TripleBarrierLabel + walk, fractal label, geometry derivation/scan/ adoption, exit-policy replay, excursion MI targets, the drift verdict (DIRECTION_INTELLIGENT), the recall floor, balanced-accuracy telemetry, the barrier defines, the .cfg geometry adopt (slots kept as zeros for the positional layout), the derived-geometry live-order override. - TRAINING_TARGET input/enum: direction models are always swing; META2 re-keys the meta head onto label agreement (descriptor loses its two geometry slots). REWORKED - Labels.mqh (1795 -> ~370 lines): AdvanceSwingLabelState with FINALITY-GATED CACHING - an unresolved bar (pivot pair uncommitted) is never cached, so it can never freeze as a false Neutral; training, calibration, OOS scoring and online learning all skip unresolved bars. - SDeployVerdict: significance-only; SOosTally chance = larger directional class share; pooled gate poolability = timeframe (record v2). - Purge/embargo/declustering gaps: the measured mean label resolution lag (LabelResolutionBars), not a barrier horizon. - Pool purge key + backfill DB rows: marked at the bar the label resolved on (m_labelResolveAge), not a fabricated barrier touch. - Online learning frontier: finality, not a horizon delay. - m_bestBalancedOos -> m_bestSelectionScore, m_erasSinceBestBalanced -> m_erasSinceBest, ensemble vote outcome arrays -> label arrays. STEP 4 folded in: Entry_Multiplier / SL_Mode / TP_Mode / tradingdirection are inputs again - trade management is the tester GA's search space. Fingerprints: every direction model re-keys (TGT:SWG1 now unconditional, CUT token gone); META1 -> META2. Full retrain, as planned. Compile-verified in _claude_stage: Warrior_EA + both surviving test EAs, 0 errors, 0 warnings each. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-24 20:42:31 -04:00
ArrayResize(m_labelResolveAge, bars);
refactor: split CExpertSignalAIBase implementation by responsibility ExpertSignalAIBase.mqh was 8216 lines: the class declaration followed by 87 method bodies covering training, labelling, feature extraction, persistence, chart drawing, online learning, the GA auto-tuner and inference, all in one file. Train() alone is 1492 lines; a change to arrow drawing meant scrolling past the era loop. Moved the bodies into Expert\AIBase\, included at the bottom of the original after the class declaration: Training.mqh 1607 era loop, plateau ladder, checkpoint select, deploy Features.mqh 1093 indicator creation + per-bar input feature vector ChartUI.mqh 634 arrows, arrow persistence, status panel, cleanup Persistence.mqh 492 .stats/.cfg sidecars, CPU-inference validation, copy OnlineLearning.mqh 461 live continual learning, EMA shadow, OOS simulator Labels.mqh 309 ZigZag pivot labels, async label-cache prebuild AutoTune.mqh 275 genetic tuner (population, crossover, halving) Inference.mqh 235 softmax, prior calibration, class priors ExpertSignalAIBase.mqh 8216 -> 3131 (declaration + topology build only) This is a pure relocation - verified mechanically, not by eye: HEAD's file reconstructed from the eight partials plus the surviving remainder is byte-identical to HEAD, span for span (scratchpad verify_split.py). No declaration moved, no signature changed, no code rewritten, so behaviour is unchanged by construction. Compiles 0 errors, 0 warnings. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-29 00:42:45 -04:00
ArrayResize(m_labelCacheHasValue, bars);
ArrayInitialize(m_labelCacheHasValue, false);
ArrayResize(m_featureCache, bars * m_neuronsCount);
ArrayResize(m_featureCacheHasValue, bars);
ArrayResize(m_featureCacheValid, bars);
ArrayInitialize(m_featureCacheHasValue, false);
m_labelCacheBars = bars;
m_labelCacheAnchorTime = m_Time.GetData(0);
return true;
}
//+------------------------------------------------------------------+
fix(labels): overlapping-label sample correction + horizon cap on the scale ladder Three defects, all surfaced by the 2026-08-17 SP500 H4 run that shipped stop 4.86 / target 9.71 (width 14.57*ATR, horizon 384). 1. EVERY STANDARD ERROR ASSUMED INDEPENDENT SAMPLES. Triple-barrier labels started one per bar overlap by the label's lifespan, so n calls are worth ~n/L independent observations (Lopez de Prado, AFML ch. 4 - sample uniqueness). All three sqrt(p(1-p)/n) sites divided by the RAW count. The tell: the operating point's null-of-the-maximum gate is family-wise and should fire on ~5% of eras under the null. Measured fire rates - PAI 47/73 (64%), ConvLSTM 9/24, LSTM 8/21 (38%), CONV 4/62 (6%). CONV, the only model whose margin distribution admits few bins, sat on the null; the rest cleared a bar that was too low by ~sqrt(L). PAI's deployed threshold consequently alternated between the ENDS of its own range era to era (0.10 -> 0.88 -> 0.86 -> 0.66; coverage 16% <-> 73%). TripleBarrierLabel now records when each label became KNOWABLE - the first winning touch, or both stops, or the timeout - and the prebuild accumulates the mean. EffectiveSampleSize() feeds the operating point, the member deploy gate and the ensemble vote gate. Conservative by construction (n/L is an upper bound on the damage); gates get harder, never easier. 2. THE SCALE LADDER RAN AWAY, again. Horizon scales as swingMedian*sl*tp, and since 4d8cb08 reachability is measured OVER that horizon - so a wider rung buys itself the time that makes it look reachable. Same target -> horizon -> reach -> target loop the excursion window is kept short to avoid; fixing the window confusion reopened it through the other door. It walked 128 -> 256 -> 384 bars and stopped at q90, the widest rung there is, with every rung reading 39-48% against a 20% floor. A floor nothing fails selects nothing. Rungs whose required horizon exceeds BARRIER_HORIZON_MAX are now rejected - the same rule ReportGeometryExpectancyScan already applied. It was printing the shipped pair as CLAMPED and disqualified ('h384!') two lines under the deriver that chose it: two subsystems, one geometry, opposite verdicts. 3. THE RUNG SNAP DESTROYED THE RATIO IT WAS COMPARING. Both legs snapped independently to the coarse first-passage grid, re-rating each candidate: q90 4.86/9.71 -> 5.00/10.00 (2.00), q85 4.07/8.14 -> 5.00/10.00 (IDENTICAL measurement), q75 3.07/6.13 -> 4.00/6.50 (1.63 - a nearer target). So the ladder compared win shares taken at ratios from 1.63 to 2.17 and read the differences as scale. It is why the reach column came out non-monotone in width (q75 48.5% above q90 42.9%). The stop now snaps to its nearest rung in log space and the target follows the ratio off it; the pair actually measured is returned and logged, so a collision reads as a collision. Also: LadderWinShare guarded against the conditional (fractal) geometry path, which fills n from m_fracLegCount while leaving idxList empty - a latent out-of-bounds on a currently-dead path. New log lines: mean label lifespan and effective n on the label-cache line, the required-vs-available horizon per rung, and the grid pair the reconciliation actually measured (its tolerance now scales with the grid skew instead of a flat 5pp). Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-17 01:12:05 -04:00
//| Mean bars-to-resolution over the label cache. 1.0 until something |
//| has been measured, which makes EffectiveSampleSize() the identity |
//| - the pre-2026-08-17 behaviour. That default is deliberate: an |
//| UNMEASURED overlap must not silently shrink anyone's sample, so |
//| the correction switches itself on only once it has evidence. |
//+------------------------------------------------------------------+
fix(topology): the capacity budget counted overlapping bars as independent examples EstimatedInSampleBars() returned raw bars (11372 on SP500 H4) and every derived capacity decision spent that: first-layer width, conv filters, LSTM hidden size. But triple-barrier labels overlap - mean lifespan 9.4 bars - so the label cache line on the same run already reports those bars are worth ~1210 independent observations. Sizing a network against RAW bars while grading it against EFFECTIVE ones is two subsystems disagreeing about one sample, and it disagreed in the dangerous direction because the capacity side was the optimistic one: the warning's "roughly 1.1 weights per training bar" is nearer 11 per independent observation. EffectiveSampleSize() has existed since 2026-08-17 and is applied at eight sites, all of them statistics. This adds the ninth, in the one place that decides how many parameters get fitted. Applied inside EstimatedInSampleBars() rather than at the call sites, because that function exists precisely so the three stages spend one budget. SELF-ENABLING AND THEREFORE INERT WHERE IT MATTERS MOST, which is why this is two changes and not one. MeanLabelLifespan() is 1.0 until a label cache has measured something, so on a model's first build - before any label exists - the deflation is correctly the identity: an unmeasured overlap must not invent a shrink. A fresh attach constructs a fresh object, so its counters are zero too; only a mid-session weights reset carries real evidence into a rebuild. That is deliberately safe (no attach can now re-derive a narrower topology and discard trained weights) but it would have left the first build - the case you most want the truth for - quoting the flattering figure. So ReportDetectability now restates capacity against the effective sample at the first moment L is real, for the topology already pinned. It re-sizes nothing; it reports what was bought. Placed ABOVE that function's break-even guard on purpose - a degenerate geometry is exactly when you want to know the net is over-parameterised, and "it only fires for sane configs" is how the 2026-08-18 IS-error stop managed never to fire at all. The warning also names its basis now (independent observations and L, or an explicit "overlap NOT YET MEASURED, this is an UPPER BOUND"), so a flattering number can never again read as a measured one. Also factors FirstLayerFanIn() out of ComputeFirstLayerWidth so the capacity REPORT charges for exactly what the capacity DECISION charged for - same reason RequiredHorizonBars was factored out after the 2026-08-17 divergence - and makes MeanLabelLifespan()/EffectiveSampleSize() const so the const budget path can call them. Verified: no recursion (EstimatedInSampleBars -> EffectiveSampleSize -> EstimatedInSampleBarsRaw, which computes from Bars() alone); both new StringFormat sites hand-counted (basis 3/3 and 1/1, CAPACITY 10 specifiers / 10 arguments). NOT COMPILED - user compiles in MetaEditor.
2026-08-19 18:43:48 -04:00
double CExpertSignalAIBase::MeanLabelLifespan(void) const
fix(labels): overlapping-label sample correction + horizon cap on the scale ladder Three defects, all surfaced by the 2026-08-17 SP500 H4 run that shipped stop 4.86 / target 9.71 (width 14.57*ATR, horizon 384). 1. EVERY STANDARD ERROR ASSUMED INDEPENDENT SAMPLES. Triple-barrier labels started one per bar overlap by the label's lifespan, so n calls are worth ~n/L independent observations (Lopez de Prado, AFML ch. 4 - sample uniqueness). All three sqrt(p(1-p)/n) sites divided by the RAW count. The tell: the operating point's null-of-the-maximum gate is family-wise and should fire on ~5% of eras under the null. Measured fire rates - PAI 47/73 (64%), ConvLSTM 9/24, LSTM 8/21 (38%), CONV 4/62 (6%). CONV, the only model whose margin distribution admits few bins, sat on the null; the rest cleared a bar that was too low by ~sqrt(L). PAI's deployed threshold consequently alternated between the ENDS of its own range era to era (0.10 -> 0.88 -> 0.86 -> 0.66; coverage 16% <-> 73%). TripleBarrierLabel now records when each label became KNOWABLE - the first winning touch, or both stops, or the timeout - and the prebuild accumulates the mean. EffectiveSampleSize() feeds the operating point, the member deploy gate and the ensemble vote gate. Conservative by construction (n/L is an upper bound on the damage); gates get harder, never easier. 2. THE SCALE LADDER RAN AWAY, again. Horizon scales as swingMedian*sl*tp, and since 4d8cb08 reachability is measured OVER that horizon - so a wider rung buys itself the time that makes it look reachable. Same target -> horizon -> reach -> target loop the excursion window is kept short to avoid; fixing the window confusion reopened it through the other door. It walked 128 -> 256 -> 384 bars and stopped at q90, the widest rung there is, with every rung reading 39-48% against a 20% floor. A floor nothing fails selects nothing. Rungs whose required horizon exceeds BARRIER_HORIZON_MAX are now rejected - the same rule ReportGeometryExpectancyScan already applied. It was printing the shipped pair as CLAMPED and disqualified ('h384!') two lines under the deriver that chose it: two subsystems, one geometry, opposite verdicts. 3. THE RUNG SNAP DESTROYED THE RATIO IT WAS COMPARING. Both legs snapped independently to the coarse first-passage grid, re-rating each candidate: q90 4.86/9.71 -> 5.00/10.00 (2.00), q85 4.07/8.14 -> 5.00/10.00 (IDENTICAL measurement), q75 3.07/6.13 -> 4.00/6.50 (1.63 - a nearer target). So the ladder compared win shares taken at ratios from 1.63 to 2.17 and read the differences as scale. It is why the reach column came out non-monotone in width (q75 48.5% above q90 42.9%). The stop now snaps to its nearest rung in log space and the target follows the ratio off it; the pair actually measured is returned and logged, so a collision reads as a collision. Also: LadderWinShare guarded against the conditional (fractal) geometry path, which fills n from m_fracLegCount while leaving idxList empty - a latent out-of-bounds on a currently-dead path. New log lines: mean label lifespan and effective n on the label-cache line, the required-vs-available horizon per rung, and the grid pair the reconciliation actually measured (its tolerance now scales with the grid skew instead of a flat 5pp). Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-17 01:12:05 -04:00
{
feat(target): delete the barrier/geometry stack - the label is the verdict Step 3 of the swing-pivot plan, whole-hog. The swing label is now the ONE target and the era verdict is precision + recall per class against the label's own base rate - no win rate, no break-even, no expectancy, no geometry anywhere in training. DELETED - Expert/Excursion/ (4), Expert/BarrierHorizon/ (4), GeometrySweep, FirstPassageLadder, Labeling/TripleBarrier.mqh (CLabelOverlap survives in Labeling/LabelOverlap.mqh), 3 test EAs. - TripleBarrierLabel + walk, fractal label, geometry derivation/scan/ adoption, exit-policy replay, excursion MI targets, the drift verdict (DIRECTION_INTELLIGENT), the recall floor, balanced-accuracy telemetry, the barrier defines, the .cfg geometry adopt (slots kept as zeros for the positional layout), the derived-geometry live-order override. - TRAINING_TARGET input/enum: direction models are always swing; META2 re-keys the meta head onto label agreement (descriptor loses its two geometry slots). REWORKED - Labels.mqh (1795 -> ~370 lines): AdvanceSwingLabelState with FINALITY-GATED CACHING - an unresolved bar (pivot pair uncommitted) is never cached, so it can never freeze as a false Neutral; training, calibration, OOS scoring and online learning all skip unresolved bars. - SDeployVerdict: significance-only; SOosTally chance = larger directional class share; pooled gate poolability = timeframe (record v2). - Purge/embargo/declustering gaps: the measured mean label resolution lag (LabelResolutionBars), not a barrier horizon. - Pool purge key + backfill DB rows: marked at the bar the label resolved on (m_labelResolveAge), not a fabricated barrier touch. - Online learning frontier: finality, not a horizon delay. - m_bestBalancedOos -> m_bestSelectionScore, m_erasSinceBestBalanced -> m_erasSinceBest, ensemble vote outcome arrays -> label arrays. STEP 4 folded in: Entry_Multiplier / SL_Mode / TP_Mode / tradingdirection are inputs again - trade management is the tester GA's search space. Fingerprints: every direction model re-keys (TGT:SWG1 now unconditional, CUT token gone); META1 -> META2. Full retrain, as planned. Compile-verified in _claude_stage: Warrior_EA + both surviving test EAs, 0 errors, 0 warnings each. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-24 20:42:31 -04:00
//--- The cap is the label's structural bound: the scan window a resolution lag can never exceed.
return m_labelOverlap.MeanLifespan(SWING_SCAN_CAP_BARS);
fix(labels): overlapping-label sample correction + horizon cap on the scale ladder Three defects, all surfaced by the 2026-08-17 SP500 H4 run that shipped stop 4.86 / target 9.71 (width 14.57*ATR, horizon 384). 1. EVERY STANDARD ERROR ASSUMED INDEPENDENT SAMPLES. Triple-barrier labels started one per bar overlap by the label's lifespan, so n calls are worth ~n/L independent observations (Lopez de Prado, AFML ch. 4 - sample uniqueness). All three sqrt(p(1-p)/n) sites divided by the RAW count. The tell: the operating point's null-of-the-maximum gate is family-wise and should fire on ~5% of eras under the null. Measured fire rates - PAI 47/73 (64%), ConvLSTM 9/24, LSTM 8/21 (38%), CONV 4/62 (6%). CONV, the only model whose margin distribution admits few bins, sat on the null; the rest cleared a bar that was too low by ~sqrt(L). PAI's deployed threshold consequently alternated between the ENDS of its own range era to era (0.10 -> 0.88 -> 0.86 -> 0.66; coverage 16% <-> 73%). TripleBarrierLabel now records when each label became KNOWABLE - the first winning touch, or both stops, or the timeout - and the prebuild accumulates the mean. EffectiveSampleSize() feeds the operating point, the member deploy gate and the ensemble vote gate. Conservative by construction (n/L is an upper bound on the damage); gates get harder, never easier. 2. THE SCALE LADDER RAN AWAY, again. Horizon scales as swingMedian*sl*tp, and since 4d8cb08 reachability is measured OVER that horizon - so a wider rung buys itself the time that makes it look reachable. Same target -> horizon -> reach -> target loop the excursion window is kept short to avoid; fixing the window confusion reopened it through the other door. It walked 128 -> 256 -> 384 bars and stopped at q90, the widest rung there is, with every rung reading 39-48% against a 20% floor. A floor nothing fails selects nothing. Rungs whose required horizon exceeds BARRIER_HORIZON_MAX are now rejected - the same rule ReportGeometryExpectancyScan already applied. It was printing the shipped pair as CLAMPED and disqualified ('h384!') two lines under the deriver that chose it: two subsystems, one geometry, opposite verdicts. 3. THE RUNG SNAP DESTROYED THE RATIO IT WAS COMPARING. Both legs snapped independently to the coarse first-passage grid, re-rating each candidate: q90 4.86/9.71 -> 5.00/10.00 (2.00), q85 4.07/8.14 -> 5.00/10.00 (IDENTICAL measurement), q75 3.07/6.13 -> 4.00/6.50 (1.63 - a nearer target). So the ladder compared win shares taken at ratios from 1.63 to 2.17 and read the differences as scale. It is why the reach column came out non-monotone in width (q75 48.5% above q90 42.9%). The stop now snaps to its nearest rung in log space and the target follows the ratio off it; the pair actually measured is returned and logged, so a collision reads as a collision. Also: LadderWinShare guarded against the conditional (fractal) geometry path, which fills n from m_fracLegCount while leaving idxList empty - a latent out-of-bounds on a currently-dead path. New log lines: mean label lifespan and effective n on the label-cache line, the required-vs-available horizon per rung, and the grid pair the reconciliation actually measured (its tolerance now scales with the grid skew instead of a flat 5pp). Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-17 01:12:05 -04:00
}
//+------------------------------------------------------------------+
//| Independent observations behind `rawN` overlapping labels. |
fix(labels): overlapping-label sample correction + horizon cap on the scale ladder Three defects, all surfaced by the 2026-08-17 SP500 H4 run that shipped stop 4.86 / target 9.71 (width 14.57*ATR, horizon 384). 1. EVERY STANDARD ERROR ASSUMED INDEPENDENT SAMPLES. Triple-barrier labels started one per bar overlap by the label's lifespan, so n calls are worth ~n/L independent observations (Lopez de Prado, AFML ch. 4 - sample uniqueness). All three sqrt(p(1-p)/n) sites divided by the RAW count. The tell: the operating point's null-of-the-maximum gate is family-wise and should fire on ~5% of eras under the null. Measured fire rates - PAI 47/73 (64%), ConvLSTM 9/24, LSTM 8/21 (38%), CONV 4/62 (6%). CONV, the only model whose margin distribution admits few bins, sat on the null; the rest cleared a bar that was too low by ~sqrt(L). PAI's deployed threshold consequently alternated between the ENDS of its own range era to era (0.10 -> 0.88 -> 0.86 -> 0.66; coverage 16% <-> 73%). TripleBarrierLabel now records when each label became KNOWABLE - the first winning touch, or both stops, or the timeout - and the prebuild accumulates the mean. EffectiveSampleSize() feeds the operating point, the member deploy gate and the ensemble vote gate. Conservative by construction (n/L is an upper bound on the damage); gates get harder, never easier. 2. THE SCALE LADDER RAN AWAY, again. Horizon scales as swingMedian*sl*tp, and since 4d8cb08 reachability is measured OVER that horizon - so a wider rung buys itself the time that makes it look reachable. Same target -> horizon -> reach -> target loop the excursion window is kept short to avoid; fixing the window confusion reopened it through the other door. It walked 128 -> 256 -> 384 bars and stopped at q90, the widest rung there is, with every rung reading 39-48% against a 20% floor. A floor nothing fails selects nothing. Rungs whose required horizon exceeds BARRIER_HORIZON_MAX are now rejected - the same rule ReportGeometryExpectancyScan already applied. It was printing the shipped pair as CLAMPED and disqualified ('h384!') two lines under the deriver that chose it: two subsystems, one geometry, opposite verdicts. 3. THE RUNG SNAP DESTROYED THE RATIO IT WAS COMPARING. Both legs snapped independently to the coarse first-passage grid, re-rating each candidate: q90 4.86/9.71 -> 5.00/10.00 (2.00), q85 4.07/8.14 -> 5.00/10.00 (IDENTICAL measurement), q75 3.07/6.13 -> 4.00/6.50 (1.63 - a nearer target). So the ladder compared win shares taken at ratios from 1.63 to 2.17 and read the differences as scale. It is why the reach column came out non-monotone in width (q75 48.5% above q90 42.9%). The stop now snaps to its nearest rung in log space and the target follows the ratio off it; the pair actually measured is returned and logged, so a collision reads as a collision. Also: LadderWinShare guarded against the conditional (fractal) geometry path, which fills n from m_fracLegCount while leaving idxList empty - a latent out-of-bounds on a currently-dead path. New log lines: mean label lifespan and effective n on the label-cache line, the required-vs-available horizon per rung, and the grid pair the reconciliation actually measured (its tolerance now scales with the grid skew instead of a flat 5pp). Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-17 01:12:05 -04:00
//+------------------------------------------------------------------+
fix(topology): the capacity budget counted overlapping bars as independent examples EstimatedInSampleBars() returned raw bars (11372 on SP500 H4) and every derived capacity decision spent that: first-layer width, conv filters, LSTM hidden size. But triple-barrier labels overlap - mean lifespan 9.4 bars - so the label cache line on the same run already reports those bars are worth ~1210 independent observations. Sizing a network against RAW bars while grading it against EFFECTIVE ones is two subsystems disagreeing about one sample, and it disagreed in the dangerous direction because the capacity side was the optimistic one: the warning's "roughly 1.1 weights per training bar" is nearer 11 per independent observation. EffectiveSampleSize() has existed since 2026-08-17 and is applied at eight sites, all of them statistics. This adds the ninth, in the one place that decides how many parameters get fitted. Applied inside EstimatedInSampleBars() rather than at the call sites, because that function exists precisely so the three stages spend one budget. SELF-ENABLING AND THEREFORE INERT WHERE IT MATTERS MOST, which is why this is two changes and not one. MeanLabelLifespan() is 1.0 until a label cache has measured something, so on a model's first build - before any label exists - the deflation is correctly the identity: an unmeasured overlap must not invent a shrink. A fresh attach constructs a fresh object, so its counters are zero too; only a mid-session weights reset carries real evidence into a rebuild. That is deliberately safe (no attach can now re-derive a narrower topology and discard trained weights) but it would have left the first build - the case you most want the truth for - quoting the flattering figure. So ReportDetectability now restates capacity against the effective sample at the first moment L is real, for the topology already pinned. It re-sizes nothing; it reports what was bought. Placed ABOVE that function's break-even guard on purpose - a degenerate geometry is exactly when you want to know the net is over-parameterised, and "it only fires for sane configs" is how the 2026-08-18 IS-error stop managed never to fire at all. The warning also names its basis now (independent observations and L, or an explicit "overlap NOT YET MEASURED, this is an UPPER BOUND"), so a flattering number can never again read as a measured one. Also factors FirstLayerFanIn() out of ComputeFirstLayerWidth so the capacity REPORT charges for exactly what the capacity DECISION charged for - same reason RequiredHorizonBars was factored out after the 2026-08-17 divergence - and makes MeanLabelLifespan()/EffectiveSampleSize() const so the const budget path can call them. Verified: no recursion (EstimatedInSampleBars -> EffectiveSampleSize -> EstimatedInSampleBarsRaw, which computes from Bars() alone); both new StringFormat sites hand-counted (basis 3/3 and 1/1, CAPACITY 10 specifiers / 10 arguments). NOT COMPILED - user compiles in MetaEditor.
2026-08-19 18:43:48 -04:00
double CExpertSignalAIBase::EffectiveSampleSize(double rawN) const
fix(labels): overlapping-label sample correction + horizon cap on the scale ladder Three defects, all surfaced by the 2026-08-17 SP500 H4 run that shipped stop 4.86 / target 9.71 (width 14.57*ATR, horizon 384). 1. EVERY STANDARD ERROR ASSUMED INDEPENDENT SAMPLES. Triple-barrier labels started one per bar overlap by the label's lifespan, so n calls are worth ~n/L independent observations (Lopez de Prado, AFML ch. 4 - sample uniqueness). All three sqrt(p(1-p)/n) sites divided by the RAW count. The tell: the operating point's null-of-the-maximum gate is family-wise and should fire on ~5% of eras under the null. Measured fire rates - PAI 47/73 (64%), ConvLSTM 9/24, LSTM 8/21 (38%), CONV 4/62 (6%). CONV, the only model whose margin distribution admits few bins, sat on the null; the rest cleared a bar that was too low by ~sqrt(L). PAI's deployed threshold consequently alternated between the ENDS of its own range era to era (0.10 -> 0.88 -> 0.86 -> 0.66; coverage 16% <-> 73%). TripleBarrierLabel now records when each label became KNOWABLE - the first winning touch, or both stops, or the timeout - and the prebuild accumulates the mean. EffectiveSampleSize() feeds the operating point, the member deploy gate and the ensemble vote gate. Conservative by construction (n/L is an upper bound on the damage); gates get harder, never easier. 2. THE SCALE LADDER RAN AWAY, again. Horizon scales as swingMedian*sl*tp, and since 4d8cb08 reachability is measured OVER that horizon - so a wider rung buys itself the time that makes it look reachable. Same target -> horizon -> reach -> target loop the excursion window is kept short to avoid; fixing the window confusion reopened it through the other door. It walked 128 -> 256 -> 384 bars and stopped at q90, the widest rung there is, with every rung reading 39-48% against a 20% floor. A floor nothing fails selects nothing. Rungs whose required horizon exceeds BARRIER_HORIZON_MAX are now rejected - the same rule ReportGeometryExpectancyScan already applied. It was printing the shipped pair as CLAMPED and disqualified ('h384!') two lines under the deriver that chose it: two subsystems, one geometry, opposite verdicts. 3. THE RUNG SNAP DESTROYED THE RATIO IT WAS COMPARING. Both legs snapped independently to the coarse first-passage grid, re-rating each candidate: q90 4.86/9.71 -> 5.00/10.00 (2.00), q85 4.07/8.14 -> 5.00/10.00 (IDENTICAL measurement), q75 3.07/6.13 -> 4.00/6.50 (1.63 - a nearer target). So the ladder compared win shares taken at ratios from 1.63 to 2.17 and read the differences as scale. It is why the reach column came out non-monotone in width (q75 48.5% above q90 42.9%). The stop now snaps to its nearest rung in log space and the target follows the ratio off it; the pair actually measured is returned and logged, so a collision reads as a collision. Also: LadderWinShare guarded against the conditional (fractal) geometry path, which fills n from m_fracLegCount while leaving idxList empty - a latent out-of-bounds on a currently-dead path. New log lines: mean label lifespan and effective n on the label-cache line, the required-vs-available horizon per rung, and the grid pair the reconciliation actually measured (its tolerance now scales with the grid skew instead of a flat 5pp). Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-17 01:12:05 -04:00
{
feat(target): delete the barrier/geometry stack - the label is the verdict Step 3 of the swing-pivot plan, whole-hog. The swing label is now the ONE target and the era verdict is precision + recall per class against the label's own base rate - no win rate, no break-even, no expectancy, no geometry anywhere in training. DELETED - Expert/Excursion/ (4), Expert/BarrierHorizon/ (4), GeometrySweep, FirstPassageLadder, Labeling/TripleBarrier.mqh (CLabelOverlap survives in Labeling/LabelOverlap.mqh), 3 test EAs. - TripleBarrierLabel + walk, fractal label, geometry derivation/scan/ adoption, exit-policy replay, excursion MI targets, the drift verdict (DIRECTION_INTELLIGENT), the recall floor, balanced-accuracy telemetry, the barrier defines, the .cfg geometry adopt (slots kept as zeros for the positional layout), the derived-geometry live-order override. - TRAINING_TARGET input/enum: direction models are always swing; META2 re-keys the meta head onto label agreement (descriptor loses its two geometry slots). REWORKED - Labels.mqh (1795 -> ~370 lines): AdvanceSwingLabelState with FINALITY-GATED CACHING - an unresolved bar (pivot pair uncommitted) is never cached, so it can never freeze as a false Neutral; training, calibration, OOS scoring and online learning all skip unresolved bars. - SDeployVerdict: significance-only; SOosTally chance = larger directional class share; pooled gate poolability = timeframe (record v2). - Purge/embargo/declustering gaps: the measured mean label resolution lag (LabelResolutionBars), not a barrier horizon. - Pool purge key + backfill DB rows: marked at the bar the label resolved on (m_labelResolveAge), not a fabricated barrier touch. - Online learning frontier: finality, not a horizon delay. - m_bestBalancedOos -> m_bestSelectionScore, m_erasSinceBestBalanced -> m_erasSinceBest, ensemble vote outcome arrays -> label arrays. STEP 4 folded in: Entry_Multiplier / SL_Mode / TP_Mode / tradingdirection are inputs again - trade management is the tester GA's search space. Fingerprints: every direction model re-keys (TGT:SWG1 now unconditional, CUT token gone); META1 -> META2. Full retrain, as planned. Compile-verified in _claude_stage: Warrior_EA + both surviving test EAs, 0 errors, 0 warnings each. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-24 20:42:31 -04:00
return m_labelOverlap.EffectiveSampleSize(rawN, SWING_SCAN_CAP_BARS);
refactor: split CExpertSignalAIBase implementation by responsibility ExpertSignalAIBase.mqh was 8216 lines: the class declaration followed by 87 method bodies covering training, labelling, feature extraction, persistence, chart drawing, online learning, the GA auto-tuner and inference, all in one file. Train() alone is 1492 lines; a change to arrow drawing meant scrolling past the era loop. Moved the bodies into Expert\AIBase\, included at the bottom of the original after the class declaration: Training.mqh 1607 era loop, plateau ladder, checkpoint select, deploy Features.mqh 1093 indicator creation + per-bar input feature vector ChartUI.mqh 634 arrows, arrow persistence, status panel, cleanup Persistence.mqh 492 .stats/.cfg sidecars, CPU-inference validation, copy OnlineLearning.mqh 461 live continual learning, EMA shadow, OOS simulator Labels.mqh 309 ZigZag pivot labels, async label-cache prebuild AutoTune.mqh 275 genetic tuner (population, crossover, halving) Inference.mqh 235 softmax, prior calibration, class priors ExpertSignalAIBase.mqh 8216 -> 3131 (declaration + topology build only) This is a pure relocation - verified mechanically, not by eye: HEAD's file reconstructed from the eight partials plus the surviving remainder is byte-identical to HEAD, span for span (scratchpad verify_split.py). No declaration moved, no signature changed, no code rewritten, so behaviour is unchanged by construction. Compiles 0 errors, 0 warnings. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-29 00:42:45 -04:00
}
//+------------------------------------------------------------------+
//| Nearest confirmed ZigZag pivot at fromIdx or older (now-relative |
//| index, so "older" means scanning with INCREASING p - see this |
feat(target): delete the barrier/geometry stack - the label is the verdict Step 3 of the swing-pivot plan, whole-hog. The swing label is now the ONE target and the era verdict is precision + recall per class against the label's own base rate - no win rate, no break-even, no expectancy, no geometry anywhere in training. DELETED - Expert/Excursion/ (4), Expert/BarrierHorizon/ (4), GeometrySweep, FirstPassageLadder, Labeling/TripleBarrier.mqh (CLabelOverlap survives in Labeling/LabelOverlap.mqh), 3 test EAs. - TripleBarrierLabel + walk, fractal label, geometry derivation/scan/ adoption, exit-policy replay, excursion MI targets, the drift verdict (DIRECTION_INTELLIGENT), the recall floor, balanced-accuracy telemetry, the barrier defines, the .cfg geometry adopt (slots kept as zeros for the positional layout), the derived-geometry live-order override. - TRAINING_TARGET input/enum: direction models are always swing; META2 re-keys the meta head onto label agreement (descriptor loses its two geometry slots). REWORKED - Labels.mqh (1795 -> ~370 lines): AdvanceSwingLabelState with FINALITY-GATED CACHING - an unresolved bar (pivot pair uncommitted) is never cached, so it can never freeze as a false Neutral; training, calibration, OOS scoring and online learning all skip unresolved bars. - SDeployVerdict: significance-only; SOosTally chance = larger directional class share; pooled gate poolability = timeframe (record v2). - Purge/embargo/declustering gaps: the measured mean label resolution lag (LabelResolutionBars), not a barrier horizon. - Pool purge key + backfill DB rows: marked at the bar the label resolved on (m_labelResolveAge), not a fabricated barrier touch. - Online learning frontier: finality, not a horizon delay. - m_bestBalancedOos -> m_bestSelectionScore, m_erasSinceBestBalanced -> m_erasSinceBest, ensemble vote outcome arrays -> label arrays. STEP 4 folded in: Entry_Multiplier / SL_Mode / TP_Mode / tradingdirection are inputs again - trade management is the tester GA's search space. Fingerprints: every direction model re-keys (TGT:SWG1 now unconditional, CUT token gone); META1 -> META2. Full retrain, as planned. Compile-verified in _claude_stage: Warrior_EA + both surviving test EAs, 0 errors, 0 warnings each. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-24 20:42:31 -04:00
//| file's now-relative-index convention). |
refactor: split CExpertSignalAIBase implementation by responsibility ExpertSignalAIBase.mqh was 8216 lines: the class declaration followed by 87 method bodies covering training, labelling, feature extraction, persistence, chart drawing, online learning, the GA auto-tuner and inference, all in one file. Train() alone is 1492 lines; a change to arrow drawing meant scrolling past the era loop. Moved the bodies into Expert\AIBase\, included at the bottom of the original after the class declaration: Training.mqh 1607 era loop, plateau ladder, checkpoint select, deploy Features.mqh 1093 indicator creation + per-bar input feature vector ChartUI.mqh 634 arrows, arrow persistence, status panel, cleanup Persistence.mqh 492 .stats/.cfg sidecars, CPU-inference validation, copy OnlineLearning.mqh 461 live continual learning, EMA shadow, OOS simulator Labels.mqh 309 ZigZag pivot labels, async label-cache prebuild AutoTune.mqh 275 genetic tuner (population, crossover, halving) Inference.mqh 235 softmax, prior calibration, class priors ExpertSignalAIBase.mqh 8216 -> 3131 (declaration + topology build only) This is a pure relocation - verified mechanically, not by eye: HEAD's file reconstructed from the eight partials plus the surviving remainder is byte-identical to HEAD, span for span (scratchpad verify_split.py). No declaration moved, no signature changed, no code rewritten, so behaviour is unchanged by construction. Compiles 0 errors, 0 warnings. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-29 00:42:45 -04:00
//+------------------------------------------------------------------+
bool CExpertSignalAIBase::FindConfirmedZigZagPivot(int fromIdx, int &pivotIdx, double &pivotPrice, bool &pivotIsLow)
{
for(int p = MathMax(fromIdx, 0); p < fromIdx + SWING_SCAN_CAP_BARS; p++)
{
if(m_Open.GetData(p) == EMPTY_VALUE)
return false; // ran off the end of available history
double zz = m_zigZag.GetData(0, p);
refactor: split CExpertSignalAIBase implementation by responsibility ExpertSignalAIBase.mqh was 8216 lines: the class declaration followed by 87 method bodies covering training, labelling, feature extraction, persistence, chart drawing, online learning, the GA auto-tuner and inference, all in one file. Train() alone is 1492 lines; a change to arrow drawing meant scrolling past the era loop. Moved the bodies into Expert\AIBase\, included at the bottom of the original after the class declaration: Training.mqh 1607 era loop, plateau ladder, checkpoint select, deploy Features.mqh 1093 indicator creation + per-bar input feature vector ChartUI.mqh 634 arrows, arrow persistence, status panel, cleanup Persistence.mqh 492 .stats/.cfg sidecars, CPU-inference validation, copy OnlineLearning.mqh 461 live continual learning, EMA shadow, OOS simulator Labels.mqh 309 ZigZag pivot labels, async label-cache prebuild AutoTune.mqh 275 genetic tuner (population, crossover, halving) Inference.mqh 235 softmax, prior calibration, class priors ExpertSignalAIBase.mqh 8216 -> 3131 (declaration + topology build only) This is a pure relocation - verified mechanically, not by eye: HEAD's file reconstructed from the eight partials plus the surviving remainder is byte-identical to HEAD, span for span (scratchpad verify_split.py). No declaration moved, no signature changed, no code rewritten, so behaviour is unchanged by construction. Compiles 0 errors, 0 warnings. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-29 00:42:45 -04:00
if(zz == 0.0)
continue;
pivotIdx = p;
pivotPrice = zz;
pivotIsLow = (zz <= m_Low.GetData(p) + _Point);
return true;
}
return false;
}
//+------------------------------------------------------------------+
feat(label): pivot-EVENT target replaces direction-to-next-pivot The old target asked "which way is the next pivot", which every bar of a ~13-20 bar leg answers identically - so the net could not tell a fresh turn from mid-trend and learned the prevailing direction instead. Its own zero-skill reference showed it: chance sat at 56/44, i.e. the label WAS the drift, and the gate's standing warning ("a model that only reproduces it has found the drift, not an edge") applied to the target itself. Buy now means a swing LOW commits within PIVOT_LABEL_TOLERANCE_BARS bars, Sell a swing HIGH, Neutral no turn that close. Pivot type is read from ZigZagBuffer[p] == Low[p], exact by construction in ZigZag.mq5. The existing P1-final-once-P2-commits rule is kept and now also settles the NEGATIVE verdict, so the Neutral majority is permanent rather than provisional. Measured on a full fresh run, all 6 charts: class balance 56/44/~0 -> 13.7/13.7/72.6 (imbalance 5.3:1) label overlap ~31 bars -> 5 bars independent obs 368-1086 -> 2331-7032 weights/obs 9.2-26.2 -> 1.1-4.2 coverage 100% of bars -> 17-48% 23 of 24 models fire all three classes at precision 18-32% vs 13-15% chance; SP500's ensemble reaches DEPLOYABLE (32.3% vs a 24.0% bar). Two bindings had to move with the label: - The capacity deflator. m_swingLifespan fed EstimatedInSampleBars() as raw/31, measured from the legs. Overlap is now a property of the LABEL - one turn is callable by exactly the tolerance window - so it is the window, not a leg measurement. Missing this would have kept every model sized for a sixth of its real evidence. - A dormant cold-start seed. Labels.mqh seeds the output bias toward the dominant class above COLD_START_SEED_MIN_DOMINANCE (0.70); at 56/44 it never armed, at 72.6% Neutral it does - writing a fixed +-3.0 against a true prior spread of ~1.75, which would start every net predicting Neutral ~95% of the time. Now seeds the measured log-prior, zero-centred and capped by the same guard rail the logit adjustment uses (Lin et al. 2017). TGT:SWG1 -> TGT:PVT1:<tolerance>, with the window in the token because it is part of the label: every .nnw is invalidated and the fleet retrains. Depth is still gated, and now for a precise reason: the first dense layer stays at FIRST_LAYER_MIN_WIDTH because budget = effN/(inputWidth+1) is 11.2 at input 624. Reaching the next rung needs inputWidth <= ~218, i.e. feature pruning - not architecture. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-26 00:16:16 -04:00
//| IS A SWING PIVOT ABOUT TO COMMIT, AND WHICH WAY DOES IT TURN? |
//| |
//| Buy = a swing LOW lands within PIVOT_LABEL_TOLERANCE_BARS |
//| bars of here - the turn up is at hand, buy it. |
//| Sell = a swing HIGH lands in that window - the turn down is. |
//| Neutral = no pivot that close. Most bars. Mid-leg is not a call. |
//| |
//| THIS REPLACED A DIRECTION-TO-NEXT-PIVOT LABEL (2026-08-25), and |
//| the distinction is the whole point. The old target asked "which |
//| side of the next pivot am I on", which every bar in a ~20-bar leg |
//| answers identically - so the net could not tell a fresh turn from |
//| mid-trend and simply learned the prevailing direction. Its own |
//| zero-skill reference showed it: chance sat at 56/44, i.e. the |
//| label WAS the drift, and the deploy gate's standing warning - |
//| "a model that only reproduces it has found the drift, not an |
//| edge" - applied to the target itself. This one fires only at the |
//| decision point, so a correct call is worth something. |
//| |
//| SECOND-ORDER, AND LARGE: label overlap collapses. Under the old |
//| target ~31 consecutive bars shared one pivot, so EffectiveSample- |
//| Size deflated 15,045 OOS calls to 440 independent ones and the |
//| deploy gate could not certify ANY edge for want of observations. |
//| Here a pivot marks only the PIVOT_LABEL_TOLERANCE_BARS bars that |
//| can call it - see m_lastLabelLifespan below. |
//| |
//| Geometry-free: the label owes nothing to a stop, a target or a |
//| horizon, which is what lets trade management be tuned separately |
//| instead of being baked into what the net learns. |
//| |
feat(target): delete the barrier/geometry stack - the label is the verdict Step 3 of the swing-pivot plan, whole-hog. The swing label is now the ONE target and the era verdict is precision + recall per class against the label's own base rate - no win rate, no break-even, no expectancy, no geometry anywhere in training. DELETED - Expert/Excursion/ (4), Expert/BarrierHorizon/ (4), GeometrySweep, FirstPassageLadder, Labeling/TripleBarrier.mqh (CLabelOverlap survives in Labeling/LabelOverlap.mqh), 3 test EAs. - TripleBarrierLabel + walk, fractal label, geometry derivation/scan/ adoption, exit-policy replay, excursion MI targets, the drift verdict (DIRECTION_INTELLIGENT), the recall floor, balanced-accuracy telemetry, the barrier defines, the .cfg geometry adopt (slots kept as zeros for the positional layout), the derived-geometry live-order override. - TRAINING_TARGET input/enum: direction models are always swing; META2 re-keys the meta head onto label agreement (descriptor loses its two geometry slots). REWORKED - Labels.mqh (1795 -> ~370 lines): AdvanceSwingLabelState with FINALITY-GATED CACHING - an unresolved bar (pivot pair uncommitted) is never cached, so it can never freeze as a false Neutral; training, calibration, OOS scoring and online learning all skip unresolved bars. - SDeployVerdict: significance-only; SOosTally chance = larger directional class share; pooled gate poolability = timeframe (record v2). - Purge/embargo/declustering gaps: the measured mean label resolution lag (LabelResolutionBars), not a barrier horizon. - Pool purge key + backfill DB rows: marked at the bar the label resolved on (m_labelResolveAge), not a fabricated barrier touch. - Online learning frontier: finality, not a horizon delay. - m_bestBalancedOos -> m_bestSelectionScore, m_erasSinceBestBalanced -> m_erasSinceBest, ensemble vote outcome arrays -> label arrays. STEP 4 folded in: Entry_Multiplier / SL_Mode / TP_Mode / tradingdirection are inputs again - trade management is the tester GA's search space. Fingerprints: every direction model re-keys (TGT:SWG1 now unconditional, CUT token gone); META1 -> META2. Full retrain, as planned. Compile-verified in _claude_stage: Warrior_EA + both surviving test EAs, 0 errors, 0 warnings each. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-24 20:42:31 -04:00
//| The pivot is m_zigZag's, the SAME definition the swing-context |
//| features already walk - one notion of "pivot" in the codebase, |
//| not two that can drift apart. |
//+------------------------------------------------------------------+
ENUM_SIGNAL CExpertSignalAIBase::SwingPivotDirectionLabel(int idx)
{
feat(label): pivot-EVENT target replaces direction-to-next-pivot The old target asked "which way is the next pivot", which every bar of a ~13-20 bar leg answers identically - so the net could not tell a fresh turn from mid-trend and learned the prevailing direction instead. Its own zero-skill reference showed it: chance sat at 56/44, i.e. the label WAS the drift, and the gate's standing warning ("a model that only reproduces it has found the drift, not an edge") applied to the target itself. Buy now means a swing LOW commits within PIVOT_LABEL_TOLERANCE_BARS bars, Sell a swing HIGH, Neutral no turn that close. Pivot type is read from ZigZagBuffer[p] == Low[p], exact by construction in ZigZag.mq5. The existing P1-final-once-P2-commits rule is kept and now also settles the NEGATIVE verdict, so the Neutral majority is permanent rather than provisional. Measured on a full fresh run, all 6 charts: class balance 56/44/~0 -> 13.7/13.7/72.6 (imbalance 5.3:1) label overlap ~31 bars -> 5 bars independent obs 368-1086 -> 2331-7032 weights/obs 9.2-26.2 -> 1.1-4.2 coverage 100% of bars -> 17-48% 23 of 24 models fire all three classes at precision 18-32% vs 13-15% chance; SP500's ensemble reaches DEPLOYABLE (32.3% vs a 24.0% bar). Two bindings had to move with the label: - The capacity deflator. m_swingLifespan fed EstimatedInSampleBars() as raw/31, measured from the legs. Overlap is now a property of the LABEL - one turn is callable by exactly the tolerance window - so it is the window, not a leg measurement. Missing this would have kept every model sized for a sixth of its real evidence. - A dormant cold-start seed. Labels.mqh seeds the output bias toward the dominant class above COLD_START_SEED_MIN_DOMINANCE (0.70); at 56/44 it never armed, at 72.6% Neutral it does - writing a fixed +-3.0 against a true prior spread of ~1.75, which would start every net predicting Neutral ~95% of the time. Now seeds the measured log-prior, zero-centred and capped by the same guard rail the logit adjustment uses (Lin et al. 2017). TGT:SWG1 -> TGT:PVT1:<tolerance>, with the window in the token because it is part of the label: every .nnw is invalidated and the fleet retrains. Depth is still gated, and now for a precise reason: the first dense layer stays at FIRST_LAYER_MIN_WIDTH because budget = effN/(inputWidth+1) is 11.2 at input 624. Reaching the next rung needs inputWidth <= ~218, i.e. feature pruning - not architecture. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-26 00:16:16 -04:00
//--- LABEL-OVERLAP SPAN for the effective-sample machinery, NOT bars-to-resolution any more. What
//--- the SE correction needs is how many labelled bars share one underlying event, and one pivot
//--- can be called by exactly the PIVOT_LABEL_TOLERANCE_BARS bars that precede it. Resolution lag
//--- (how long until the pivot is CONFIRMED) is a different quantity and no longer belongs here:
//--- it says when a label may be trusted, not how much independent evidence it carries.
//--- 0 still means "unresolved", which is what gates the caching in AdvanceSwingLabelState.
m_lastLabelLifespan = 0;
double entry = m_Close.GetData(idx);
double atr = m_ATR.Main(idx);
//--- EMPTY_VALUE (a cold/short indicator read) IS DBL_MAX, and MathIsValidNumber(DBL_MAX) is
//--- true - it is a real finite number, just not one this indicator ever meant to report. Without
//--- the explicit == EMPTY_VALUE check, a cold ATR passes both tests, minMove below becomes
//--- ~1.8e307, and every bar in the sweep labels Neutral and is cached as resolved - permanently,
//--- since nothing currently invalidates the label cache when the indicator later warms up (see
//--- LabelCacheInvalidateAll()). FeatureBuilder.mqh's equivalent ATR guard already does this.
if(!MathIsValidNumber(entry) || entry <= 0.0 || entry == EMPTY_VALUE ||
!MathIsValidNumber(atr) || atr <= 0.0 || atr == EMPTY_VALUE)
return Neutral;
double spread = (double)m_symbol.Spread() * m_symbol.Point();
if(!MathIsValidNumber(spread) || spread < 0.0)
spread = 0.0;
double minMove = MathMax(2.0 * spread, 0.10 * atr);
//--- FINALITY IS AN EVENT, NOT A WAITING PERIOD. ZigZag.mq5's selection loop can only ever erase
//--- ZigZagBuffer[last_high_pos] while hunting a bottom, or [last_low_pos] while hunting a peak -
//--- so a pivot leaves the erasable slot for good the moment the OPPOSITE pivot is committed, and
feat(label): pivot-EVENT target replaces direction-to-next-pivot The old target asked "which way is the next pivot", which every bar of a ~13-20 bar leg answers identically - so the net could not tell a fresh turn from mid-trend and learned the prevailing direction instead. Its own zero-skill reference showed it: chance sat at 56/44, i.e. the label WAS the drift, and the gate's standing warning ("a model that only reproduces it has found the drift, not an edge") applied to the target itself. Buy now means a swing LOW commits within PIVOT_LABEL_TOLERANCE_BARS bars, Sell a swing HIGH, Neutral no turn that close. Pivot type is read from ZigZagBuffer[p] == Low[p], exact by construction in ZigZag.mq5. The existing P1-final-once-P2-commits rule is kept and now also settles the NEGATIVE verdict, so the Neutral majority is permanent rather than provisional. Measured on a full fresh run, all 6 charts: class balance 56/44/~0 -> 13.7/13.7/72.6 (imbalance 5.3:1) label overlap ~31 bars -> 5 bars independent obs 368-1086 -> 2331-7032 weights/obs 9.2-26.2 -> 1.1-4.2 coverage 100% of bars -> 17-48% 23 of 24 models fire all three classes at precision 18-32% vs 13-15% chance; SP500's ensemble reaches DEPLOYABLE (32.3% vs a 24.0% bar). Two bindings had to move with the label: - The capacity deflator. m_swingLifespan fed EstimatedInSampleBars() as raw/31, measured from the legs. Overlap is now a property of the LABEL - one turn is callable by exactly the tolerance window - so it is the window, not a leg measurement. Missing this would have kept every model sized for a sixth of its real evidence. - A dormant cold-start seed. Labels.mqh seeds the output bias toward the dominant class above COLD_START_SEED_MIN_DOMINANCE (0.70); at 56/44 it never armed, at 72.6% Neutral it does - writing a fixed +-3.0 against a true prior spread of ~1.75, which would start every net predicting Neutral ~95% of the time. Now seeds the measured log-prior, zero-centred and capped by the same guard rail the logit adjustment uses (Lin et al. 2017). TGT:SWG1 -> TGT:PVT1:<tolerance>, with the window in the token because it is part of the label: every .nnw is invalidated and the fleet retrains. Depth is still gated, and now for a precise reason: the first dense layer stays at FIRST_LAYER_MIN_WIDTH because budget = effN/(inputWidth+1) is 11.2 at input 624. Reaching the next rung needs inputWidth <= ~218, i.e. feature pruning - not architecture. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-26 00:16:16 -04:00
//--- can never move again. P1 (the first pivot ahead of this bar) becomes final when P2 exists;
//--- pivots alternate by construction, so P2 is just the next non-zero bar and needs no type test.
//--- Until then the label is not knowable and the bar stays unresolved - that is the entire
//--- lookahead control for this target, exact rather than a confirmation-bar guess.
//---
//--- P1's finality also settles a NEGATIVE verdict, which is what makes the Neutral majority
//--- (~75% at the shipped tolerance) trustworthy rather than merely current: nothing can appear
//--- between here and P1, so if P1 is final and sits beyond the tolerance window, "no turn here"
//--- is permanent, not provisional. Without that, three quarters of the training set would be a
//--- label that could still change.
double p1Price = 0.0;
int p1Idx = -1;
for(int p = idx - 1; p >= MathMax(idx - SWING_SCAN_CAP_BARS, 1); p--)
{
if(m_Open.GetData(p) == EMPTY_VALUE)
return Neutral; // ran off loaded history before P2 confirmed
double pivot = m_zigZag.GetData(0, p);
//--- == EMPTY_VALUE, same reason as the atr/entry guard above: a cold ZigZag (buffer not yet
//--- filled) reads EMPTY_VALUE == DBL_MAX at every index, which is a real, positive,
//--- MathIsValidNumber()-passing number - so without this it reads as a pivot ABOVE every
//--- close, and every bar in the sweep labels Buy and is cached as resolved.
if(pivot == 0.0 || pivot == EMPTY_VALUE || !MathIsValidNumber(pivot))
continue;
feat(label): pivot-EVENT target replaces direction-to-next-pivot The old target asked "which way is the next pivot", which every bar of a ~13-20 bar leg answers identically - so the net could not tell a fresh turn from mid-trend and learned the prevailing direction instead. Its own zero-skill reference showed it: chance sat at 56/44, i.e. the label WAS the drift, and the gate's standing warning ("a model that only reproduces it has found the drift, not an edge") applied to the target itself. Buy now means a swing LOW commits within PIVOT_LABEL_TOLERANCE_BARS bars, Sell a swing HIGH, Neutral no turn that close. Pivot type is read from ZigZagBuffer[p] == Low[p], exact by construction in ZigZag.mq5. The existing P1-final-once-P2-commits rule is kept and now also settles the NEGATIVE verdict, so the Neutral majority is permanent rather than provisional. Measured on a full fresh run, all 6 charts: class balance 56/44/~0 -> 13.7/13.7/72.6 (imbalance 5.3:1) label overlap ~31 bars -> 5 bars independent obs 368-1086 -> 2331-7032 weights/obs 9.2-26.2 -> 1.1-4.2 coverage 100% of bars -> 17-48% 23 of 24 models fire all three classes at precision 18-32% vs 13-15% chance; SP500's ensemble reaches DEPLOYABLE (32.3% vs a 24.0% bar). Two bindings had to move with the label: - The capacity deflator. m_swingLifespan fed EstimatedInSampleBars() as raw/31, measured from the legs. Overlap is now a property of the LABEL - one turn is callable by exactly the tolerance window - so it is the window, not a leg measurement. Missing this would have kept every model sized for a sixth of its real evidence. - A dormant cold-start seed. Labels.mqh seeds the output bias toward the dominant class above COLD_START_SEED_MIN_DOMINANCE (0.70); at 56/44 it never armed, at 72.6% Neutral it does - writing a fixed +-3.0 against a true prior spread of ~1.75, which would start every net predicting Neutral ~95% of the time. Now seeds the measured log-prior, zero-centred and capped by the same guard rail the logit adjustment uses (Lin et al. 2017). TGT:SWG1 -> TGT:PVT1:<tolerance>, with the window in the token because it is part of the label: every .nnw is invalidated and the fleet retrains. Depth is still gated, and now for a precise reason: the first dense layer stays at FIRST_LAYER_MIN_WIDTH because budget = effN/(inputWidth+1) is 11.2 at input 624. Reaching the next rung needs inputWidth <= ~218, i.e. feature pruning - not architecture. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-26 00:16:16 -04:00
if(p1Idx < 0)
{
feat(label): pivot-EVENT target replaces direction-to-next-pivot The old target asked "which way is the next pivot", which every bar of a ~13-20 bar leg answers identically - so the net could not tell a fresh turn from mid-trend and learned the prevailing direction instead. Its own zero-skill reference showed it: chance sat at 56/44, i.e. the label WAS the drift, and the gate's standing warning ("a model that only reproduces it has found the drift, not an edge") applied to the target itself. Buy now means a swing LOW commits within PIVOT_LABEL_TOLERANCE_BARS bars, Sell a swing HIGH, Neutral no turn that close. Pivot type is read from ZigZagBuffer[p] == Low[p], exact by construction in ZigZag.mq5. The existing P1-final-once-P2-commits rule is kept and now also settles the NEGATIVE verdict, so the Neutral majority is permanent rather than provisional. Measured on a full fresh run, all 6 charts: class balance 56/44/~0 -> 13.7/13.7/72.6 (imbalance 5.3:1) label overlap ~31 bars -> 5 bars independent obs 368-1086 -> 2331-7032 weights/obs 9.2-26.2 -> 1.1-4.2 coverage 100% of bars -> 17-48% 23 of 24 models fire all three classes at precision 18-32% vs 13-15% chance; SP500's ensemble reaches DEPLOYABLE (32.3% vs a 24.0% bar). Two bindings had to move with the label: - The capacity deflator. m_swingLifespan fed EstimatedInSampleBars() as raw/31, measured from the legs. Overlap is now a property of the LABEL - one turn is callable by exactly the tolerance window - so it is the window, not a leg measurement. Missing this would have kept every model sized for a sixth of its real evidence. - A dormant cold-start seed. Labels.mqh seeds the output bias toward the dominant class above COLD_START_SEED_MIN_DOMINANCE (0.70); at 56/44 it never armed, at 72.6% Neutral it does - writing a fixed +-3.0 against a true prior spread of ~1.75, which would start every net predicting Neutral ~95% of the time. Now seeds the measured log-prior, zero-centred and capped by the same guard rail the logit adjustment uses (Lin et al. 2017). TGT:SWG1 -> TGT:PVT1:<tolerance>, with the window in the token because it is part of the label: every .nnw is invalidated and the fleet retrains. Depth is still gated, and now for a precise reason: the first dense layer stays at FIRST_LAYER_MIN_WIDTH because budget = effN/(inputWidth+1) is 11.2 at input 624. Reaching the next rung needs inputWidth <= ~218, i.e. feature pruning - not architecture. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-26 00:16:16 -04:00
p1Price = pivot;
p1Idx = p;
continue;
}
feat(label): pivot-EVENT target replaces direction-to-next-pivot The old target asked "which way is the next pivot", which every bar of a ~13-20 bar leg answers identically - so the net could not tell a fresh turn from mid-trend and learned the prevailing direction instead. Its own zero-skill reference showed it: chance sat at 56/44, i.e. the label WAS the drift, and the gate's standing warning ("a model that only reproduces it has found the drift, not an edge") applied to the target itself. Buy now means a swing LOW commits within PIVOT_LABEL_TOLERANCE_BARS bars, Sell a swing HIGH, Neutral no turn that close. Pivot type is read from ZigZagBuffer[p] == Low[p], exact by construction in ZigZag.mq5. The existing P1-final-once-P2-commits rule is kept and now also settles the NEGATIVE verdict, so the Neutral majority is permanent rather than provisional. Measured on a full fresh run, all 6 charts: class balance 56/44/~0 -> 13.7/13.7/72.6 (imbalance 5.3:1) label overlap ~31 bars -> 5 bars independent obs 368-1086 -> 2331-7032 weights/obs 9.2-26.2 -> 1.1-4.2 coverage 100% of bars -> 17-48% 23 of 24 models fire all three classes at precision 18-32% vs 13-15% chance; SP500's ensemble reaches DEPLOYABLE (32.3% vs a 24.0% bar). Two bindings had to move with the label: - The capacity deflator. m_swingLifespan fed EstimatedInSampleBars() as raw/31, measured from the legs. Overlap is now a property of the LABEL - one turn is callable by exactly the tolerance window - so it is the window, not a leg measurement. Missing this would have kept every model sized for a sixth of its real evidence. - A dormant cold-start seed. Labels.mqh seeds the output bias toward the dominant class above COLD_START_SEED_MIN_DOMINANCE (0.70); at 56/44 it never armed, at 72.6% Neutral it does - writing a fixed +-3.0 against a true prior spread of ~1.75, which would start every net predicting Neutral ~95% of the time. Now seeds the measured log-prior, zero-centred and capped by the same guard rail the logit adjustment uses (Lin et al. 2017). TGT:SWG1 -> TGT:PVT1:<tolerance>, with the window in the token because it is part of the label: every .nnw is invalidated and the fleet retrains. Depth is still gated, and now for a precise reason: the first dense layer stays at FIRST_LAYER_MIN_WIDTH because budget = effN/(inputWidth+1) is 11.2 at input 624. Reaching the next rung needs inputWidth <= ~218, i.e. feature pruning - not architecture. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-26 00:16:16 -04:00
//--- P2 IS COMMITTED, SO P1 IS FINAL AND THIS BAR IS TRAINABLE. Everything below is decided.
//--- One pivot can be called by the PIVOT_LABEL_TOLERANCE_BARS bars in front of it, and that
//--- - not the distance to P2 - is what the effective-sample correction must divide by.
m_lastLabelLifespan = PIVOT_LABEL_TOLERANCE_BARS;
//--- OUT OF REACH: P1 is real and final but too far ahead to be this bar's call. Mid-leg.
if((idx - p1Idx) > PIVOT_LABEL_TOLERANCE_BARS)
return Neutral;
//--- IS THE LEG WORTH TAKING? The move that pays is the one AFTER the turn - P1 to P2 - not
//--- the approach to it. A ZigZag wiggle smaller than the spread is a pivot the indicator is
//--- entitled to draw and no one can trade, and labelling it Buy teaches the net to call
//--- turns that cost money to act on. Same minMove the old target charged, same reasoning.
if(MathAbs(p1Price - pivot) < minMove)
return Neutral;
//--- WHICH WAY IT TURNS. ZigZag.mq5 stores exactly High[p] at a peak or Low[p] at a bottom
//--- (ZigZag.mq5:140-165), so the comparison is a type test, not an approximation. Same
//--- idiom as FindConfirmedZigZagPivot() above, tolerance included.
bool p1IsLow = (p1Price <= m_Low.GetData(p1Idx) + _Point);
//--- A bottom ahead is a turn UP to be bought; a peak ahead is a turn DOWN to be sold. The
//--- PIVOT TYPE IS THE WHOLE SIGNAL - no comparison against this bar's close. Gating on
//--- whether the pivot sits above or below `entry` would drop exactly the bars where the turn
//--- has not finished coming to us, which is most of the early ones, and would bias the two
//--- classes asymmetrically the moment the leg is not symmetric around the close. How much
//--- adverse move is left before the turn is a trade-management question, and this label is
//--- deliberately geometry-free (see the header) so that stays tunable separately.
return (p1IsLow ? Buy : Sell);
}
return Neutral; // P1 still repainting, or no pivot pair inside the cap
}
//+------------------------------------------------------------------+
feat(target): delete the barrier/geometry stack - the label is the verdict Step 3 of the swing-pivot plan, whole-hog. The swing label is now the ONE target and the era verdict is precision + recall per class against the label's own base rate - no win rate, no break-even, no expectancy, no geometry anywhere in training. DELETED - Expert/Excursion/ (4), Expert/BarrierHorizon/ (4), GeometrySweep, FirstPassageLadder, Labeling/TripleBarrier.mqh (CLabelOverlap survives in Labeling/LabelOverlap.mqh), 3 test EAs. - TripleBarrierLabel + walk, fractal label, geometry derivation/scan/ adoption, exit-policy replay, excursion MI targets, the drift verdict (DIRECTION_INTELLIGENT), the recall floor, balanced-accuracy telemetry, the barrier defines, the .cfg geometry adopt (slots kept as zeros for the positional layout), the derived-geometry live-order override. - TRAINING_TARGET input/enum: direction models are always swing; META2 re-keys the meta head onto label agreement (descriptor loses its two geometry slots). REWORKED - Labels.mqh (1795 -> ~370 lines): AdvanceSwingLabelState with FINALITY-GATED CACHING - an unresolved bar (pivot pair uncommitted) is never cached, so it can never freeze as a false Neutral; training, calibration, OOS scoring and online learning all skip unresolved bars. - SDeployVerdict: significance-only; SOosTally chance = larger directional class share; pooled gate poolability = timeframe (record v2). - Purge/embargo/declustering gaps: the measured mean label resolution lag (LabelResolutionBars), not a barrier horizon. - Pool purge key + backfill DB rows: marked at the bar the label resolved on (m_labelResolveAge), not a fabricated barrier touch. - Online learning frontier: finality, not a horizon delay. - m_bestBalancedOos -> m_bestSelectionScore, m_erasSinceBestBalanced -> m_erasSinceBest, ensemble vote outcome arrays -> label arrays. STEP 4 folded in: Entry_Multiplier / SL_Mode / TP_Mode / tradingdirection are inputs again - trade management is the tester GA's search space. Fingerprints: every direction model re-keys (TGT:SWG1 now unconditional, CUT token gone); META1 -> META2. Full retrain, as planned. Compile-verified in _claude_stage: Warrior_EA + both surviving test EAs, 0 errors, 0 warnings each. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-24 20:42:31 -04:00
//| Resolves and caches the swing label for one bar. |
//| |
//| FINALITY-GATED CACHING: only a resolved label (P2 committed, so |
//| m_lastLabelLifespan > 0) may enter the cache. An unresolved bar |
//| is left uncached and revisited on a later pass - caching its |
//| provisional Neutral would freeze a label that is still unknowable |
//| and never update it once the pivot pair commits. |
//+------------------------------------------------------------------+
void CExpertSignalAIBase::AdvanceSwingLabelState(int idx, int bars)
{
if(idx < 2 || idx >= bars || m_labelCacheHasValue[idx])
return;
ENUM_SIGNAL verdict = SwingPivotDirectionLabel(idx);
if(m_lastLabelLifespan <= 0)
return;
//--- MEAN LABEL LIFESPAN, accumulated on the IS population only, matching the final tally pass:
//--- it deflates standard errors computed on that population, and a diagnostic that mixes two
//--- populations is worse than no diagnostic.
bool countable = (idx >= MathMax(2, m_labelPrebuildOosCutoff)
&& idx <= bars - MathMax(m_historyBars, 0) - 1);
if(countable)
m_labelOverlap.Accumulate(m_lastLabelLifespan);
m_labelCacheBuy[idx] = (verdict == Buy);
m_labelCacheSell[idx] = (verdict == Sell);
//--- Stored under the SAME validity flag as the label: the pool purge key reads it back as the
//--- earliest bar this label could have been known on.
if(idx < ArraySize(m_labelResolveAge))
m_labelResolveAge[idx] = m_lastLabelLifespan;
m_labelCacheHasValue[idx] = true;
}
//+------------------------------------------------------------------+
refactor: split CExpertSignalAIBase implementation by responsibility ExpertSignalAIBase.mqh was 8216 lines: the class declaration followed by 87 method bodies covering training, labelling, feature extraction, persistence, chart drawing, online learning, the GA auto-tuner and inference, all in one file. Train() alone is 1492 lines; a change to arrow drawing meant scrolling past the era loop. Moved the bodies into Expert\AIBase\, included at the bottom of the original after the class declaration: Training.mqh 1607 era loop, plateau ladder, checkpoint select, deploy Features.mqh 1093 indicator creation + per-bar input feature vector ChartUI.mqh 634 arrows, arrow persistence, status panel, cleanup Persistence.mqh 492 .stats/.cfg sidecars, CPU-inference validation, copy OnlineLearning.mqh 461 live continual learning, EMA shadow, OOS simulator Labels.mqh 309 ZigZag pivot labels, async label-cache prebuild AutoTune.mqh 275 genetic tuner (population, crossover, halving) Inference.mqh 235 softmax, prior calibration, class priors ExpertSignalAIBase.mqh 8216 -> 3131 (declaration + topology build only) This is a pure relocation - verified mechanically, not by eye: HEAD's file reconstructed from the eight partials plus the surviving remainder is byte-identical to HEAD, span for span (scratchpad verify_split.py). No declaration moved, no signature changed, no code rewritten, so behaviour is unchanged by construction. Compiles 0 errors, 0 warnings. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-29 00:42:45 -04:00
//| Kicks off the one-time eager label-cache pre-build for a fresh |
//| start (see m_labelCachePrebuilt's declaration comment). Computes |
//| the bar count/OOS split exactly as Train()'s era-start block |
//| would, then arms AdvanceLabelCachePrebuild() to do the actual |
//| chunked scan on this and subsequent Train() calls. |
//+------------------------------------------------------------------+
void CExpertSignalAIBase::StartLabelCachePrebuild(void)
{
fix: a restart no longer loses the measured geometry or the training window Terminal restart, 22:25: all four resumed models sat on empty windows with enum 2:6 barriers. Three interlocking causes, all visible in one log excerpt: 1) THE PRE-SCAN WINDOW WAS SIZED BY THE SAVED WATERMARK. A resumed model's dtStudied sits at its last studied bar, so Bars(dtStudied, now) ~ 0 and the resumed-model MI pre-scan built a zero-bar "complete" label cache - logged as "Buy: 0 | Sell: 0 | Neutral: 0". Train()'s own era start RESETS dtStudied to the training-window rule before computing its window; the pre-scan did not. The rule is now factored into TrainWindowStart() and both use it. The scan also refuses to arm before SERIES_SYNCHRONIZED (it ran in the same second as OnInit), and deployed models keep their watermark - for them it gates inference recency, not a training window. 2) THE HORIZON LATCHED ON AN INDICATOR WARM-UP. ComputeBarrierHorizonBars ran against a ZigZag with 0 calculated legs, fell back, and EnsureBarrierHorizon latched fallback(32) x slMult x tpMult = 384 for the process lifetime. A leg-starved horizon is now PROVISIONAL: re-resolved on the next rebuild, the label cache wiped if it moved (labels from two horizons answer different questions), and the geometry deriver refuses to run from it - a pair derived over a warm-up window would get PINNED. 3) THE DERIVED GEOMETRY WAS NEVER PERSISTED. The .cfg is written at model creation and at weights-reset - both BEFORE era 0 derives - so the measured pair lived only in memory: every restart read back zeros, adopted nothing, fell back to the enum barriers, and the era-0-only gate meant a resumed model could NEVER re-derive. A full day of training on 3.33/1.62 resumed as 2:6. Now: the settled pair is pinned to the .cfg the moment derivation completes (one-shot, atomic write), and the derive gate accepts any model with no pinned pair, not just era 0 - mid-run stability is carried by m_geometryDerived itself, which never allows a second derivation. Both build variants compile 0 errors, 0 warnings. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-09 22:40:43 -04:00
//--- Not armed until the history is synced - the caller retries on its next scheduled call. Without
//--- this, a terminal restart ran the resumed-model pre-scan in the same second as OnInit, against
//--- whatever the terminal had loaded so far.
if(!SeriesInfoInteger(m_symbol.Name(), PERIOD_CURRENT, SERIES_SYNCHRONIZED))
return;
//--- ZIGZAG READINESS. A cold custom indicator (buffer not yet filled after Create()) reads
//--- EMPTY_VALUE at every index, including the newest bar - same signature ADIndicatorCold checks
//--- for feature reads (Expert\Features\FeatureBuilder.mqh). SwingPivotDirectionLabel() now refuses
//--- an EMPTY_VALUE pivot outright, but without this gate a cold sweep would just retry forever
//--- indistinguishably from "no pivot yet", and every already-scanned bar in that window still gets
//--- cached as resolved-Neutral by the ATR guard right above it in the same function. Checked
//--- directly rather than through ADIndicatorCold(): that helper also stamps the feature-builder's
//--- own transient-fail diagnostic (m_featureFailTransient/SetFailBlock), which belongs to a
//--- different call cycle (BufferTempDataCompute) and must not be touched from here.
if(m_zigZag.GetData(0, 0) == EMPTY_VALUE)
return; // retried on the next scheduled call, same contract as every guard in this function
//--- THE GAP IN THE TEARDOWN GUARDS (ad80e0b), found by the 2026-08-17 21:58 shutdown. Normally
//--- that is a once-per-run cost and it does not matter.
fix(chart): a purge that reports "zero leftovers" was only ever checking its own list 2026-08-17 21:58: all three charts hit "Abnormal termination" ~5.3 s into OnDeinit with NO cleanup-timings line - the teardown was starved again. The 22:00 init purge then removed 993 / 1373 / 1557 stranded objects and reported ZERO by-name leftovers on every chart, and the charts still came up with duplicated panels. "Nothing matching our prefixes remains" and "the chart is clean" are different statements and only the first was being made. Three changes, in the order they matter: 1. WHY the teardown starved, and it is a gap in ad80e0b. StartLabelCachePrebuild runs ResizeBuffers + RefreshData over the FULL study window (33,984 bars on XAUUSD), unchunked, and OnDeinit cannot begin until it returns. Normally a once-per-run cost. That night SP500 and XAUUSD LSTM were wedged in the "cache invalidated at era start" loop, which calls it on EVERY Train() call - two members re-preparing tens of thousands of bars indefinitely. The terminal closed into that. Guarded now, plus a resumable guard in the prebuild chunk loop (the tally pass after it is not chunked). 2. Catch-all "Warrior" prefix in WarriorChartPrefixes. Every family this EA creates is named Warrior* except the arrows (WarSig_), so one bare prefix covers the three named entries AND anything a rename or a stale .ex5 left under a name nobody remembers. Still a prefix delete, never ObjectsDeleteAll(chart) - the user's own drawings are not ours to remove. Does not defeat skipArrows: "WarSig_" does not start with "Warrior". 3. The init purge now REPORTS the residue it did not claim, by name (up to 12). Not deleted - an unmatched object may belong to the user or another indicator. If a Warrior panel is visible and appears in neither the removed count nor this list, the prefix list has drifted a third time and the name is in the journal instead of being inferred from a screenshot. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-17 22:07:09 -04:00
if(ShutdownRequested())
return;
fix: a restart no longer loses the measured geometry or the training window Terminal restart, 22:25: all four resumed models sat on empty windows with enum 2:6 barriers. Three interlocking causes, all visible in one log excerpt: 1) THE PRE-SCAN WINDOW WAS SIZED BY THE SAVED WATERMARK. A resumed model's dtStudied sits at its last studied bar, so Bars(dtStudied, now) ~ 0 and the resumed-model MI pre-scan built a zero-bar "complete" label cache - logged as "Buy: 0 | Sell: 0 | Neutral: 0". Train()'s own era start RESETS dtStudied to the training-window rule before computing its window; the pre-scan did not. The rule is now factored into TrainWindowStart() and both use it. The scan also refuses to arm before SERIES_SYNCHRONIZED (it ran in the same second as OnInit), and deployed models keep their watermark - for them it gates inference recency, not a training window. 2) THE HORIZON LATCHED ON AN INDICATOR WARM-UP. ComputeBarrierHorizonBars ran against a ZigZag with 0 calculated legs, fell back, and EnsureBarrierHorizon latched fallback(32) x slMult x tpMult = 384 for the process lifetime. A leg-starved horizon is now PROVISIONAL: re-resolved on the next rebuild, the label cache wiped if it moved (labels from two horizons answer different questions), and the geometry deriver refuses to run from it - a pair derived over a warm-up window would get PINNED. 3) THE DERIVED GEOMETRY WAS NEVER PERSISTED. The .cfg is written at model creation and at weights-reset - both BEFORE era 0 derives - so the measured pair lived only in memory: every restart read back zeros, adopted nothing, fell back to the enum barriers, and the era-0-only gate meant a resumed model could NEVER re-derive. A full day of training on 3.33/1.62 resumed as 2:6. Now: the settled pair is pinned to the .cfg the moment derivation completes (one-shot, atomic write), and the derive gate accepts any model with no pinned pair, not just era 0 - mid-run stability is carried by m_geometryDerived itself, which never allows a second derivation. Both build variants compile 0 errors, 0 warnings. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-09 22:40:43 -04:00
//--- A model that is still TRAINING sizes its window by the training rule, not by the saved study
//--- watermark. Train()'s own era start applies this exact reset (TrainWindowStart) - this makes
//--- the pre-scan and the era loop agree. Deployed (complete) models keep their watermark: for them
//--- dtStudied gates INFERENCE recency, and this scan must not touch it.
fix: a restart no longer loses the measured geometry or the training window Terminal restart, 22:25: all four resumed models sat on empty windows with enum 2:6 barriers. Three interlocking causes, all visible in one log excerpt: 1) THE PRE-SCAN WINDOW WAS SIZED BY THE SAVED WATERMARK. A resumed model's dtStudied sits at its last studied bar, so Bars(dtStudied, now) ~ 0 and the resumed-model MI pre-scan built a zero-bar "complete" label cache - logged as "Buy: 0 | Sell: 0 | Neutral: 0". Train()'s own era start RESETS dtStudied to the training-window rule before computing its window; the pre-scan did not. The rule is now factored into TrainWindowStart() and both use it. The scan also refuses to arm before SERIES_SYNCHRONIZED (it ran in the same second as OnInit), and deployed models keep their watermark - for them it gates inference recency, not a training window. 2) THE HORIZON LATCHED ON AN INDICATOR WARM-UP. ComputeBarrierHorizonBars ran against a ZigZag with 0 calculated legs, fell back, and EnsureBarrierHorizon latched fallback(32) x slMult x tpMult = 384 for the process lifetime. A leg-starved horizon is now PROVISIONAL: re-resolved on the next rebuild, the label cache wiped if it moved (labels from two horizons answer different questions), and the geometry deriver refuses to run from it - a pair derived over a warm-up window would get PINNED. 3) THE DERIVED GEOMETRY WAS NEVER PERSISTED. The .cfg is written at model creation and at weights-reset - both BEFORE era 0 derives - so the measured pair lived only in memory: every restart read back zeros, adopted nothing, fell back to the enum barriers, and the era-0-only gate meant a resumed model could NEVER re-derive. A full day of training on 3.33/1.62 resumed as 2:6. Now: the settled pair is pinned to the .cfg the moment derivation completes (one-shot, atomic write), and the derive gate accepts any model with no pinned pair, not just era 0 - mid-run stability is carried by m_geometryDerived itself, which never allows a second derivation. Both build variants compile 0 errors, 0 warnings. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-09 22:40:43 -04:00
if(!m_trainingComplete)
dtStudied = TrainWindowStart(m_tuneStartTrainBar);
refactor: split CExpertSignalAIBase implementation by responsibility ExpertSignalAIBase.mqh was 8216 lines: the class declaration followed by 87 method bodies covering training, labelling, feature extraction, persistence, chart drawing, online learning, the GA auto-tuner and inference, all in one file. Train() alone is 1492 lines; a change to arrow drawing meant scrolling past the era loop. Moved the bodies into Expert\AIBase\, included at the bottom of the original after the class declaration: Training.mqh 1607 era loop, plateau ladder, checkpoint select, deploy Features.mqh 1093 indicator creation + per-bar input feature vector ChartUI.mqh 634 arrows, arrow persistence, status panel, cleanup Persistence.mqh 492 .stats/.cfg sidecars, CPU-inference validation, copy OnlineLearning.mqh 461 live continual learning, EMA shadow, OOS simulator Labels.mqh 309 ZigZag pivot labels, async label-cache prebuild AutoTune.mqh 275 genetic tuner (population, crossover, halving) Inference.mqh 235 softmax, prior calibration, class priors ExpertSignalAIBase.mqh 8216 -> 3131 (declaration + topology build only) This is a pure relocation - verified mechanically, not by eye: HEAD's file reconstructed from the eight partials plus the surviving remainder is byte-identical to HEAD, span for span (scratchpad verify_split.py). No declaration moved, no signature changed, no code rewritten, so behaviour is unchanged by construction. Compiles 0 errors, 0 warnings. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-29 00:42:45 -04:00
int barsNow = (int)MathMin(Bars(m_symbol.Name(), PERIOD_CURRENT, dtStudied, TimeCurrent()) + m_historyBars, Bars(m_symbol.Name(), PERIOD_CURRENT));
//--- Clamped for TWO reasons, only one of which is about labels (see ServableBars()). So an
//--- unclamped prebuild here would re-break the very feature block Train()'s clamp just
//--- repaired, from a path that looks unrelated to it.
refactor: split CExpertSignalAIBase implementation by responsibility ExpertSignalAIBase.mqh was 8216 lines: the class declaration followed by 87 method bodies covering training, labelling, feature extraction, persistence, chart drawing, online learning, the GA auto-tuner and inference, all in one file. Train() alone is 1492 lines; a change to arrow drawing meant scrolling past the era loop. Moved the bodies into Expert\AIBase\, included at the bottom of the original after the class declaration: Training.mqh 1607 era loop, plateau ladder, checkpoint select, deploy Features.mqh 1093 indicator creation + per-bar input feature vector ChartUI.mqh 634 arrows, arrow persistence, status panel, cleanup Persistence.mqh 492 .stats/.cfg sidecars, CPU-inference validation, copy OnlineLearning.mqh 461 live continual learning, EMA shadow, OOS simulator Labels.mqh 309 ZigZag pivot labels, async label-cache prebuild AutoTune.mqh 275 genetic tuner (population, crossover, halving) Inference.mqh 235 softmax, prior calibration, class priors ExpertSignalAIBase.mqh 8216 -> 3131 (declaration + topology build only) This is a pure relocation - verified mechanically, not by eye: HEAD's file reconstructed from the eight partials plus the surviving remainder is byte-identical to HEAD, span for span (scratchpad verify_split.py). No declaration moved, no signature changed, no code rewritten, so behaviour is unchanged by construction. Compiles 0 errors, 0 warnings. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-29 00:42:45 -04:00
if(!ResizeBuffers(barsNow) || !RefreshData())
fix(buffers): revert the MA +1 - it asked for a bar that does not exist and stopped every chart REGRESSION I INTRODUCED IN 1dda479, live for ~15 minutes. CSeries::BufferResize -> CheckLoadHistory -> CheckTerminalHistory succeeds only when Bars() >= size. Train() calls ResizeBuffers with barIndex == Bars(), so sizing the MA buffer to barIndex + 1 asks for one bar more than the symbol has and fails the WHOLE ResizeBuffers call. The log named it exactly: failed to get 50180 bars for USDJPY,PERIOD_H4 (Bars() = 50,179) failed to get 33983 bars for XAUUSD,PERIOD_H4 (Bars() = 33,982) StartLabelCachePrebuild() then bailed on the false return and stayed silent, so the only symptom was Train() reporting "arming the first label-cache prebuild" forever with labelCacheBars=0 - the panel's "getting ready". The premise was wrong, not just the arithmetic. The MA block reads GetData(idx) AND GetData(idx + 1), and at the OLDEST bar that second read is SUPPOSED to fail - there is no older bar to difference against. Rejecting that one bar is correct behaviour; buying it cost the entire history. Two more things, since the same defect had a second instance and no alarm: - The Ichimoku pair (closeBars and m_Ichimoku, both barIndex + ichiKijun) is the same bug with a far larger constant, latent only because the feature is off. Both are now clamped to Bars(). The oldest ichiKijun bars then have no cloud, which that block's EMPTY_VALUE guard already handles per-bar - the right outcome. - The prebuild's bare `return` on a false ResizeBuffers now says so once, naming the depth and Bars(). MQL5's own "failed to get N bars" was in the log the entire time, from a stack frame nothing connected to the prebuild. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-17 14:12:47 -04:00
{
//--- NEVER SILENT AGAIN. MQL5's own "failed to get N bars" line was in the log the whole time
//--- and belonged to a stack frame nothing connected to the prebuild. Say which depth, and
//--- say it is fatal here.
fix(buffers): revert the MA +1 - it asked for a bar that does not exist and stopped every chart REGRESSION I INTRODUCED IN 1dda479, live for ~15 minutes. CSeries::BufferResize -> CheckLoadHistory -> CheckTerminalHistory succeeds only when Bars() >= size. Train() calls ResizeBuffers with barIndex == Bars(), so sizing the MA buffer to barIndex + 1 asks for one bar more than the symbol has and fails the WHOLE ResizeBuffers call. The log named it exactly: failed to get 50180 bars for USDJPY,PERIOD_H4 (Bars() = 50,179) failed to get 33983 bars for XAUUSD,PERIOD_H4 (Bars() = 33,982) StartLabelCachePrebuild() then bailed on the false return and stayed silent, so the only symptom was Train() reporting "arming the first label-cache prebuild" forever with labelCacheBars=0 - the panel's "getting ready". The premise was wrong, not just the arithmetic. The MA block reads GetData(idx) AND GetData(idx + 1), and at the OLDEST bar that second read is SUPPOSED to fail - there is no older bar to difference against. Rejecting that one bar is correct behaviour; buying it cost the entire history. Two more things, since the same defect had a second instance and no alarm: - The Ichimoku pair (closeBars and m_Ichimoku, both barIndex + ichiKijun) is the same bug with a far larger constant, latent only because the feature is off. Both are now clamped to Bars(). The oldest ichiKijun bars then have no cloud, which that block's EMPTY_VALUE guard already handles per-bar - the right outcome. - The prebuild's bare `return` on a false ResizeBuffers now says so once, naming the depth and Bars(). MQL5's own "failed to get N bars" was in the log the entire time, from a stack frame nothing connected to the prebuild. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-17 14:12:47 -04:00
if(!m_prebuildBlockWarned)
{
m_prebuildBlockWarned = true;
PrintFormat("%s: label prebuild BLOCKED - buffers would not prepare for %d bars. If MQL5"
" printed 'failed to get %d bars' just above, a buffer is being sized beyond the"
" %d bars this symbol actually has, and no era can start until that is fixed.",
ID, barsNow, barsNow, Bars(m_symbol.Name(), PERIOD_CURRENT));
}
return; // m_labelCachePrebuilt stays false, retried next call
}
feat(depth): prime -> settle -> sweep, and name which handle is short "Max bars in chart" is set to Unlimited, so the static-terminal-limit reading in 1dda479 was wrong. Two other candidate causes are falsified too: the price series is fully downloaded and flat (USDJPY 50,162 -> 50,163 over 80 minutes, i.e. one new H4 bar), and the handles have been stable since 13:07 with zero windows for the 17 minutes after, so it is not download-in-progress and not handle churn. The MA period tops out at 200 (ADIndicatorTuner MA_PERIOD_PRESETS) against ~50k bars, so it is not indicator cost either. What IS verified stays verified: CopyBuffer past the calculated depth fails outright rather than short-reading, so the buffer holds nothing and every index reads EMPTY_VALUE; m_MA is the only CiCustom whose block REJECTS on that (m_ADZigZag neutral-fills, RSI/MACD/Ichimoku/ATR are built-ins); the wall is therefore feature 25 of every bar, exactly as the "24 of 832" stall lines said. And it is depth-correlated: 16k-bar charts train, 34k/50k get zero windows forever. So the WHY is still open, and this fix does not depend on it. Per the user's protocol: the request itself is the primer, so prime at full depth, then poll TunableBarsCalculated() every 3s and hold the sweep until it stops changing (3 steady probes), then use whatever it settled at. Bounded at 10 min, and a give-up is logged as a give-up so an abandoned depth is never mistaken for a settled one. This supersedes 1dda479's clamp on the two training paths, which snapshotted a value that may still have been climbing; the clamp remains for the paths that cannot wait (inference/online/rescan/export, see 7e63a8b). The load-bearing part is what does NOT happen while waiting: no sweep. A 50k-bar feature scan starves the indicator threads the request just woke, which is how the failure sustained itself for 40 minutes at a time - discard era, re-sweep, discard, which is the 0->100% oscillation on the panel. Also adds IndicatorDepthReport(): per-handle BarsCalculated() on the priming, cap and stall lines. The logs proved WHICH FEATURE died but never WHICH HANDLE was short, so the cause had to be inferred - and was guessed wrong twice. The next occurrence reads it off. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-17 13:44:27 -04:00
int settled = SettledBars(barsNow, "label prebuild");
if(settled <= 0)
return; // depth still moving - retried next call, same contract as the line above
if(settled < barsNow)
{
barsNow = settled;
if(!ResizeBuffers(barsNow) || !RefreshData())
return;
}
refactor: split CExpertSignalAIBase implementation by responsibility ExpertSignalAIBase.mqh was 8216 lines: the class declaration followed by 87 method bodies covering training, labelling, feature extraction, persistence, chart drawing, online learning, the GA auto-tuner and inference, all in one file. Train() alone is 1492 lines; a change to arrow drawing meant scrolling past the era loop. Moved the bodies into Expert\AIBase\, included at the bottom of the original after the class declaration: Training.mqh 1607 era loop, plateau ladder, checkpoint select, deploy Features.mqh 1093 indicator creation + per-bar input feature vector ChartUI.mqh 634 arrows, arrow persistence, status panel, cleanup Persistence.mqh 492 .stats/.cfg sidecars, CPU-inference validation, copy OnlineLearning.mqh 461 live continual learning, EMA shadow, OOS simulator Labels.mqh 309 ZigZag pivot labels, async label-cache prebuild AutoTune.mqh 275 genetic tuner (population, crossover, halving) Inference.mqh 235 softmax, prior calibration, class priors ExpertSignalAIBase.mqh 8216 -> 3131 (declaration + topology build only) This is a pure relocation - verified mechanically, not by eye: HEAD's file reconstructed from the eight partials plus the surviving remainder is byte-identical to HEAD, span for span (scratchpad verify_split.py). No declaration moved, no signature changed, no code rewritten, so behaviour is unchanged by construction. Compiles 0 errors, 0 warnings. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-29 00:42:45 -04:00
EnsureBarCachesCapacity(barsNow);
int totalIter = (int)MathMax(barsNow - MathMax(m_historyBars, 0), 0);
m_labelPrebuildBars = barsNow;
m_labelPrebuildOosCutoff = (int)(MathMax(0, MathMin(100, m_oosSplitPct)) / 100.0 * totalIter);
m_labelPrebuildIndex = (int)(barsNow - MathMax(m_historyBars, 0) - 1);
m_labelPrebuildBuyCount = 0;
m_labelPrebuildSellCount = 0;
m_labelPrebuildNeutralCount = 0;
feat(target): delete the barrier/geometry stack - the label is the verdict Step 3 of the swing-pivot plan, whole-hog. The swing label is now the ONE target and the era verdict is precision + recall per class against the label's own base rate - no win rate, no break-even, no expectancy, no geometry anywhere in training. DELETED - Expert/Excursion/ (4), Expert/BarrierHorizon/ (4), GeometrySweep, FirstPassageLadder, Labeling/TripleBarrier.mqh (CLabelOverlap survives in Labeling/LabelOverlap.mqh), 3 test EAs. - TripleBarrierLabel + walk, fractal label, geometry derivation/scan/ adoption, exit-policy replay, excursion MI targets, the drift verdict (DIRECTION_INTELLIGENT), the recall floor, balanced-accuracy telemetry, the barrier defines, the .cfg geometry adopt (slots kept as zeros for the positional layout), the derived-geometry live-order override. - TRAINING_TARGET input/enum: direction models are always swing; META2 re-keys the meta head onto label agreement (descriptor loses its two geometry slots). REWORKED - Labels.mqh (1795 -> ~370 lines): AdvanceSwingLabelState with FINALITY-GATED CACHING - an unresolved bar (pivot pair uncommitted) is never cached, so it can never freeze as a false Neutral; training, calibration, OOS scoring and online learning all skip unresolved bars. - SDeployVerdict: significance-only; SOosTally chance = larger directional class share; pooled gate poolability = timeframe (record v2). - Purge/embargo/declustering gaps: the measured mean label resolution lag (LabelResolutionBars), not a barrier horizon. - Pool purge key + backfill DB rows: marked at the bar the label resolved on (m_labelResolveAge), not a fabricated barrier touch. - Online learning frontier: finality, not a horizon delay. - m_bestBalancedOos -> m_bestSelectionScore, m_erasSinceBestBalanced -> m_erasSinceBest, ensemble vote outcome arrays -> label arrays. STEP 4 folded in: Entry_Multiplier / SL_Mode / TP_Mode / tradingdirection are inputs again - trade management is the tester GA's search space. Fingerprints: every direction model re-keys (TGT:SWG1 now unconditional, CUT token gone); META1 -> META2. Full retrain, as planned. Compile-verified in _claude_stage: Warrior_EA + both surviving test EAs, 0 errors, 0 warnings each. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-24 20:42:31 -04:00
//--- Reset WITH the cache, not once per process: lifespans measured under an older window answer
//--- a different question, and carrying them forward would deflate the new standard errors by the
//--- old overlap.
refactor(labeling): CTripleBarrier - one copy of the fill/barrier arithmetic Session B of the feature-selection/labeling refactor track. Extracts the two pieces of triple-barrier arithmetic that were genuinely duplicated or scattered, taking price/ATR/geometry as plain arguments - no chart, no indicator handle - so it is testable with synthetic numbers. CTripleBarrier::ComputeLevels() replaces the fill/barrier level arithmetic that TripleBarrierLabel() and SimulateTradeOutcome() each spelled out by hand; their own comments already called it "IDENTICAL... deliberately and by copy." One caller resolves both sides at once (the both-won tie-break needs both); the other selects the side its isLong argument names. Same for ApplyMinStopWidening(), the broker-minimum-stop floor both walks applied. Fuzzed 200k random (entry, spread, risk, reward, minStop, isLong) tuples against both original hand-written forms: 0 mismatches. CLabelOverlap replaces m_labelLifespanSum/m_labelLifespanCount - two members reset from three separate call sites (constructor, label-cache rebuild), the exact "N loose members cleared in more than one place" shape a candidate- geometry incident (7452bd1) turned into a live bug. One object, one Reset(), default-constructed like every other object member. MeanLabelLifespan() and EffectiveSampleSize() on the signal become thin forwarders with an unchanged signature - every one of their ~15 existing callers, direct and through the CAIBaseTrainingData adapter, is unaffected. SnapHorizonToLadder() forwards to CTripleBarrier::SnapToLadder(), the ladder array's one remaining copy; EffectiveHorizonMax() (the close-all cache) stays on the signal since that state has no clean argument form. NOT extracted: TripleBarrierLabel()'s ~200-line walk itself. It resolves both sides simultaneously, tracks the first-passage ladder, and feeds the label every live order is sized from; a rewrite of it cannot be checked without a compiler, so only the two pieces provably identical to their originals moved. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-08-23 18:57:17 -04:00
m_labelOverlap.Reset();
refactor: split CExpertSignalAIBase implementation by responsibility ExpertSignalAIBase.mqh was 8216 lines: the class declaration followed by 87 method bodies covering training, labelling, feature extraction, persistence, chart drawing, online learning, the GA auto-tuner and inference, all in one file. Train() alone is 1492 lines; a change to arrow drawing meant scrolling past the era loop. Moved the bodies into Expert\AIBase\, included at the bottom of the original after the class declaration: Training.mqh 1607 era loop, plateau ladder, checkpoint select, deploy Features.mqh 1093 indicator creation + per-bar input feature vector ChartUI.mqh 634 arrows, arrow persistence, status panel, cleanup Persistence.mqh 492 .stats/.cfg sidecars, CPU-inference validation, copy OnlineLearning.mqh 461 live continual learning, EMA shadow, OOS simulator Labels.mqh 309 ZigZag pivot labels, async label-cache prebuild AutoTune.mqh 275 genetic tuner (population, crossover, halving) Inference.mqh 235 softmax, prior calibration, class priors ExpertSignalAIBase.mqh 8216 -> 3131 (declaration + topology build only) This is a pure relocation - verified mechanically, not by eye: HEAD's file reconstructed from the eight partials plus the surviving remainder is byte-identical to HEAD, span for span (scratchpad verify_split.py). No declaration moved, no signature changed, no code rewritten, so behaviour is unchanged by construction. Compiles 0 errors, 0 warnings. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-29 00:42:45 -04:00
m_labelPrebuildActive = true;
}
//+------------------------------------------------------------------+
//| Advances the eager label-cache pre-build by up to a time budget, |
//| then yields (same chunking pattern as the era loop). Mirrors the |
//| era loop's own labeling eligibility gate (minus the dPrevSignal |
//| check, meaningless pre-first-feedForward). On completion, seeds |
//| m_prevEraTrueBuyCount/Sell/Neutral from the upfront IS-only tally |
//| so era 0's class priors are measured, not empty. |
refactor: split CExpertSignalAIBase implementation by responsibility ExpertSignalAIBase.mqh was 8216 lines: the class declaration followed by 87 method bodies covering training, labelling, feature extraction, persistence, chart drawing, online learning, the GA auto-tuner and inference, all in one file. Train() alone is 1492 lines; a change to arrow drawing meant scrolling past the era loop. Moved the bodies into Expert\AIBase\, included at the bottom of the original after the class declaration: Training.mqh 1607 era loop, plateau ladder, checkpoint select, deploy Features.mqh 1093 indicator creation + per-bar input feature vector ChartUI.mqh 634 arrows, arrow persistence, status panel, cleanup Persistence.mqh 492 .stats/.cfg sidecars, CPU-inference validation, copy OnlineLearning.mqh 461 live continual learning, EMA shadow, OOS simulator Labels.mqh 309 ZigZag pivot labels, async label-cache prebuild AutoTune.mqh 275 genetic tuner (population, crossover, halving) Inference.mqh 235 softmax, prior calibration, class priors ExpertSignalAIBase.mqh 8216 -> 3131 (declaration + topology build only) This is a pure relocation - verified mechanically, not by eye: HEAD's file reconstructed from the eight partials plus the surviving remainder is byte-identical to HEAD, span for span (scratchpad verify_split.py). No declaration moved, no signature changed, no code rewritten, so behaviour is unchanged by construction. Compiles 0 errors, 0 warnings. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-29 00:42:45 -04:00
//+------------------------------------------------------------------+
void CExpertSignalAIBase::AdvanceLabelCachePrebuild(void)
{
const uint PREBUILD_TIME_BUDGET_MS = 80;
uint chunkStartTick = GetTickCount();
int i;
for(i = m_labelPrebuildIndex; i >= 2; i--)
{
fix(chart): a purge that reports "zero leftovers" was only ever checking its own list 2026-08-17 21:58: all three charts hit "Abnormal termination" ~5.3 s into OnDeinit with NO cleanup-timings line - the teardown was starved again. The 22:00 init purge then removed 993 / 1373 / 1557 stranded objects and reported ZERO by-name leftovers on every chart, and the charts still came up with duplicated panels. "Nothing matching our prefixes remains" and "the chart is clean" are different statements and only the first was being made. Three changes, in the order they matter: 1. WHY the teardown starved, and it is a gap in ad80e0b. StartLabelCachePrebuild runs ResizeBuffers + RefreshData over the FULL study window (33,984 bars on XAUUSD), unchunked, and OnDeinit cannot begin until it returns. Normally a once-per-run cost. That night SP500 and XAUUSD LSTM were wedged in the "cache invalidated at era start" loop, which calls it on EVERY Train() call - two members re-preparing tens of thousands of bars indefinitely. The terminal closed into that. Guarded now, plus a resumable guard in the prebuild chunk loop (the tally pass after it is not chunked). 2. Catch-all "Warrior" prefix in WarriorChartPrefixes. Every family this EA creates is named Warrior* except the arrows (WarSig_), so one bare prefix covers the three named entries AND anything a rename or a stale .ex5 left under a name nobody remembers. Still a prefix delete, never ObjectsDeleteAll(chart) - the user's own drawings are not ours to remove. Does not defeat skipArrows: "WarSig_" does not start with "Warrior". 3. The init purge now REPORTS the residue it did not claim, by name (up to 12). Not deleted - an unmatched object may belong to the user or another indicator. If a Warrior panel is visible and appears in neither the removed count nor this list, the prefix list has drifted a third time and the name is in the journal instead of being inferred from a screenshot. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-17 22:07:09 -04:00
//--- Already chunked at 80 ms, so this costs at most one chunk - but the tally pass below is NOT
//--- chunked, and on a stop there is no reason to walk the rest of the window to reach it.
//--- Resumable by construction: m_labelPrebuildIndex is written before returning either way.
if(ShutdownRequested())
{
m_labelPrebuildIndex = i;
return;
}
refactor: split CExpertSignalAIBase implementation by responsibility ExpertSignalAIBase.mqh was 8216 lines: the class declaration followed by 87 method bodies covering training, labelling, feature extraction, persistence, chart drawing, online learning, the GA auto-tuner and inference, all in one file. Train() alone is 1492 lines; a change to arrow drawing meant scrolling past the era loop. Moved the bodies into Expert\AIBase\, included at the bottom of the original after the class declaration: Training.mqh 1607 era loop, plateau ladder, checkpoint select, deploy Features.mqh 1093 indicator creation + per-bar input feature vector ChartUI.mqh 634 arrows, arrow persistence, status panel, cleanup Persistence.mqh 492 .stats/.cfg sidecars, CPU-inference validation, copy OnlineLearning.mqh 461 live continual learning, EMA shadow, OOS simulator Labels.mqh 309 ZigZag pivot labels, async label-cache prebuild AutoTune.mqh 275 genetic tuner (population, crossover, halving) Inference.mqh 235 softmax, prior calibration, class priors ExpertSignalAIBase.mqh 8216 -> 3131 (declaration + topology build only) This is a pure relocation - verified mechanically, not by eye: HEAD's file reconstructed from the eight partials plus the surviving remainder is byte-identical to HEAD, span for span (scratchpad verify_split.py). No declaration moved, no signature changed, no code rewritten, so behaviour is unchanged by construction. Compiles 0 errors, 0 warnings. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-29 00:42:45 -04:00
if(GetTickCount() - chunkStartTick >= PREBUILD_TIME_BUDGET_MS)
{
m_labelPrebuildIndex = i;
return;
}
if(!(i < (int)(m_labelPrebuildBars - MathMax(m_historyBars, 0) - 1) && m_Time.GetData(i) > dtStudied))
continue;
if(!m_labelCacheHasValue[i])
feat(target): delete the barrier/geometry stack - the label is the verdict Step 3 of the swing-pivot plan, whole-hog. The swing label is now the ONE target and the era verdict is precision + recall per class against the label's own base rate - no win rate, no break-even, no expectancy, no geometry anywhere in training. DELETED - Expert/Excursion/ (4), Expert/BarrierHorizon/ (4), GeometrySweep, FirstPassageLadder, Labeling/TripleBarrier.mqh (CLabelOverlap survives in Labeling/LabelOverlap.mqh), 3 test EAs. - TripleBarrierLabel + walk, fractal label, geometry derivation/scan/ adoption, exit-policy replay, excursion MI targets, the drift verdict (DIRECTION_INTELLIGENT), the recall floor, balanced-accuracy telemetry, the barrier defines, the .cfg geometry adopt (slots kept as zeros for the positional layout), the derived-geometry live-order override. - TRAINING_TARGET input/enum: direction models are always swing; META2 re-keys the meta head onto label agreement (descriptor loses its two geometry slots). REWORKED - Labels.mqh (1795 -> ~370 lines): AdvanceSwingLabelState with FINALITY-GATED CACHING - an unresolved bar (pivot pair uncommitted) is never cached, so it can never freeze as a false Neutral; training, calibration, OOS scoring and online learning all skip unresolved bars. - SDeployVerdict: significance-only; SOosTally chance = larger directional class share; pooled gate poolability = timeframe (record v2). - Purge/embargo/declustering gaps: the measured mean label resolution lag (LabelResolutionBars), not a barrier horizon. - Pool purge key + backfill DB rows: marked at the bar the label resolved on (m_labelResolveAge), not a fabricated barrier touch. - Online learning frontier: finality, not a horizon delay. - m_bestBalancedOos -> m_bestSelectionScore, m_erasSinceBestBalanced -> m_erasSinceBest, ensemble vote outcome arrays -> label arrays. STEP 4 folded in: Entry_Multiplier / SL_Mode / TP_Mode / tradingdirection are inputs again - trade management is the tester GA's search space. Fingerprints: every direction model re-keys (TGT:SWG1 now unconditional, CUT token gone); META1 -> META2. Full retrain, as planned. Compile-verified in _claude_stage: Warrior_EA + both surviving test EAs, 0 errors, 0 warnings each. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-24 20:42:31 -04:00
AdvanceSwingLabelState(i, m_labelPrebuildBars);
refactor: split CExpertSignalAIBase implementation by responsibility ExpertSignalAIBase.mqh was 8216 lines: the class declaration followed by 87 method bodies covering training, labelling, feature extraction, persistence, chart drawing, online learning, the GA auto-tuner and inference, all in one file. Train() alone is 1492 lines; a change to arrow drawing meant scrolling past the era loop. Moved the bodies into Expert\AIBase\, included at the bottom of the original after the class declaration: Training.mqh 1607 era loop, plateau ladder, checkpoint select, deploy Features.mqh 1093 indicator creation + per-bar input feature vector ChartUI.mqh 634 arrows, arrow persistence, status panel, cleanup Persistence.mqh 492 .stats/.cfg sidecars, CPU-inference validation, copy OnlineLearning.mqh 461 live continual learning, EMA shadow, OOS simulator Labels.mqh 309 ZigZag pivot labels, async label-cache prebuild AutoTune.mqh 275 genetic tuner (population, crossover, halving) Inference.mqh 235 softmax, prior calibration, class priors ExpertSignalAIBase.mqh 8216 -> 3131 (declaration + topology build only) This is a pure relocation - verified mechanically, not by eye: HEAD's file reconstructed from the eight partials plus the surviving remainder is byte-identical to HEAD, span for span (scratchpad verify_split.py). No declaration moved, no signature changed, no code rewritten, so behaviour is unchanged by construction. Compiles 0 errors, 0 warnings. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-29 00:42:45 -04:00
}
feat(ai): triple-barrier labels replace exact-pivot ZigZag targets The 31:1 class imbalance was self-inflicted by the TARGET, not a property of the market. Labelling only the exact bar where a ZigZag pivot confirms gave Buy 1164 / Sell 1164 / Neutral 35841, and every correction mechanism this codebase accumulated sits downstream of that one choice: the logit-adjusted loss and its range cap, the prior EMA, the +-3.0 output-bias seed, balanced-accuracy-then-precision selection with its coverage floor, the recall floor and its catch-22, the alternation gate, NMS, and the four oversampling designs that collapsed before them. The reference this engine is built on (references/neuronetworksbook.pdf ch. 3.1/3.3) also uses ZigZag, but targets the DIRECTION TO THE NEXT EXTREMUM on every bar - ~50/50 by construction, with no imbalance to correct at all. It never had this problem because it never asked "is this the pivot bar". Labels are now the triple barrier (Lopez de Prado ch. 3), using the EA's OWN SL_Mode/TP_Mode: does a trade opened at this bar's close reach its target before its stop, within a horizon. Buy = long resolves, Sell = short resolves, Neutral = neither. Consequences: - dir-precision in the era line stops being a proxy and becomes the win rate of the strategy under its own exit rules. - Expected balance ~25/25/50 at the shipped 1:3 (gambler's ruin), i.e. ~2:1 instead of 31:1. Measured and logged at the end of the prebuild. - Spread is charged on both legs, so it is a NET win rate. - Intrabar ambiguity resolves to the STOP. OHLC cannot order two touches inside one bar and the optimistic reading is how a backtested edge becomes a live loss. ZigZag stays as input features (EnableSwingContext) and now also supplies the vertical barrier: the horizon is the median confirmed leg length, snapped to a coarse ladder. Derived, not configured, and deliberately kept out of the filename fingerprint - a filename keyed on a measured quantity orphans a trained model the moment the measurement moves. Removed, because the premise died with the old target: - the alternation gate. Correct for pivot labels (a ZigZag cannot emit two same-type pivots in a row, so a repeat was provably a false fire), and wrong for barrier labels, which answer each bar independently. It also took its worst consequence with it: a one-sided model previously got ONE trade per backtest, a hard blocker on marketplace validation. - SignalClusterWindow now defaults off - it de-duplicated repeats that are now real trades. Kept as an opt-in display control. - LABEL_WINDOW_BARS, the pivot-widening pass, ConfirmedZigZagLabel. - the era-0 output-bias seed now needs a genuinely dominant class (0.70) rather than 0.40; at ~50% Neutral a +-3.0 seed is a distortion, not a correction. Also fixed, both found while wiring the above: 1. RefreshConvergedSignal sized its buffers from a date delta (Bars(sym, period, dtStudied, TimeCurrent())). dtStudied is a training watermark; in the tester it is loaded from a live-chart save AHEAD of the simulated date, so the interval inverted, Bars() returned ~0, and the buffer came out at exactly m_historyBars - deep enough for the OHLC window and far too shallow for the Donchian-50 / 20-bar-return / SMA extension behind it. Inference silently computed DIFFERENT features from the ones training learned on, live as well as in the tester. Now sized from what the feature builder actually needs. 2. The barrier horizon is resolved on the deployed path too. A deployed model never enters Train(), so it never reached the prebuild, and OnlineLearnStep reads the horizon as its confirmation delay - left at the fallback it would have backpropped bars whose barriers had not resolved. Silent lookahead in the one place that writes to a live model. SL_Mode/TP_Mode join the weights fingerprint: they define the labels now, so a model trained at 1:3 must never be silently reused at 1:1. This re-keys every pre-existing model by design - none were trained on this task. Inference census extended with the vote gate. LongCondition/ShortCondition open with a readiness check the refresh counters never see; in the tester it reduces to "the seeded _optcache.nnw must have LOADED", and if it did not, every vote is hard-zeroed while the model still answers Buy. The old three counters would have read that as "the model says Neutral" - false, and a completely different fix. This is the leading candidate for the zero-direction backtest and the census can now name it in one run. Both builds compile 0 errors / 0 warnings. Forces a full retrain. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-31 20:39:49 -04:00
//--- Final tally pass (IS-only, matches isOOS = (i < oosCutoff) used by the era loop).
refactor: split CExpertSignalAIBase implementation by responsibility ExpertSignalAIBase.mqh was 8216 lines: the class declaration followed by 87 method bodies covering training, labelling, feature extraction, persistence, chart drawing, online learning, the GA auto-tuner and inference, all in one file. Train() alone is 1492 lines; a change to arrow drawing meant scrolling past the era loop. Moved the bodies into Expert\AIBase\, included at the bottom of the original after the class declaration: Training.mqh 1607 era loop, plateau ladder, checkpoint select, deploy Features.mqh 1093 indicator creation + per-bar input feature vector ChartUI.mqh 634 arrows, arrow persistence, status panel, cleanup Persistence.mqh 492 .stats/.cfg sidecars, CPU-inference validation, copy OnlineLearning.mqh 461 live continual learning, EMA shadow, OOS simulator Labels.mqh 309 ZigZag pivot labels, async label-cache prebuild AutoTune.mqh 275 genetic tuner (population, crossover, halving) Inference.mqh 235 softmax, prior calibration, class priors ExpertSignalAIBase.mqh 8216 -> 3131 (declaration + topology build only) This is a pure relocation - verified mechanically, not by eye: HEAD's file reconstructed from the eight partials plus the surviving remainder is byte-identical to HEAD, span for span (scratchpad verify_split.py). No declaration moved, no signature changed, no code rewritten, so behaviour is unchanged by construction. Compiles 0 errors, 0 warnings. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-29 00:42:45 -04:00
for(i = m_labelPrebuildBars - MathMax(m_historyBars, 0) - 1; i >= MathMax(2, m_labelPrebuildOosCutoff); i--)
{
if(!m_labelCacheHasValue[i])
feat(target): delete the barrier/geometry stack - the label is the verdict Step 3 of the swing-pivot plan, whole-hog. The swing label is now the ONE target and the era verdict is precision + recall per class against the label's own base rate - no win rate, no break-even, no expectancy, no geometry anywhere in training. DELETED - Expert/Excursion/ (4), Expert/BarrierHorizon/ (4), GeometrySweep, FirstPassageLadder, Labeling/TripleBarrier.mqh (CLabelOverlap survives in Labeling/LabelOverlap.mqh), 3 test EAs. - TripleBarrierLabel + walk, fractal label, geometry derivation/scan/ adoption, exit-policy replay, excursion MI targets, the drift verdict (DIRECTION_INTELLIGENT), the recall floor, balanced-accuracy telemetry, the barrier defines, the .cfg geometry adopt (slots kept as zeros for the positional layout), the derived-geometry live-order override. - TRAINING_TARGET input/enum: direction models are always swing; META2 re-keys the meta head onto label agreement (descriptor loses its two geometry slots). REWORKED - Labels.mqh (1795 -> ~370 lines): AdvanceSwingLabelState with FINALITY-GATED CACHING - an unresolved bar (pivot pair uncommitted) is never cached, so it can never freeze as a false Neutral; training, calibration, OOS scoring and online learning all skip unresolved bars. - SDeployVerdict: significance-only; SOosTally chance = larger directional class share; pooled gate poolability = timeframe (record v2). - Purge/embargo/declustering gaps: the measured mean label resolution lag (LabelResolutionBars), not a barrier horizon. - Pool purge key + backfill DB rows: marked at the bar the label resolved on (m_labelResolveAge), not a fabricated barrier touch. - Online learning frontier: finality, not a horizon delay. - m_bestBalancedOos -> m_bestSelectionScore, m_erasSinceBestBalanced -> m_erasSinceBest, ensemble vote outcome arrays -> label arrays. STEP 4 folded in: Entry_Multiplier / SL_Mode / TP_Mode / tradingdirection are inputs again - trade management is the tester GA's search space. Fingerprints: every direction model re-keys (TGT:SWG1 now unconditional, CUT token gone); META1 -> META2. Full retrain, as planned. Compile-verified in _claude_stage: Warrior_EA + both surviving test EAs, 0 errors, 0 warnings each. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-24 20:42:31 -04:00
continue; // unresolved, or outside the dtStudied/window-edge eligibility gate above
refactor: split CExpertSignalAIBase implementation by responsibility ExpertSignalAIBase.mqh was 8216 lines: the class declaration followed by 87 method bodies covering training, labelling, feature extraction, persistence, chart drawing, online learning, the GA auto-tuner and inference, all in one file. Train() alone is 1492 lines; a change to arrow drawing meant scrolling past the era loop. Moved the bodies into Expert\AIBase\, included at the bottom of the original after the class declaration: Training.mqh 1607 era loop, plateau ladder, checkpoint select, deploy Features.mqh 1093 indicator creation + per-bar input feature vector ChartUI.mqh 634 arrows, arrow persistence, status panel, cleanup Persistence.mqh 492 .stats/.cfg sidecars, CPU-inference validation, copy OnlineLearning.mqh 461 live continual learning, EMA shadow, OOS simulator Labels.mqh 309 ZigZag pivot labels, async label-cache prebuild AutoTune.mqh 275 genetic tuner (population, crossover, halving) Inference.mqh 235 softmax, prior calibration, class priors ExpertSignalAIBase.mqh 8216 -> 3131 (declaration + topology build only) This is a pure relocation - verified mechanically, not by eye: HEAD's file reconstructed from the eight partials plus the surviving remainder is byte-identical to HEAD, span for span (scratchpad verify_split.py). No declaration moved, no signature changed, no code rewritten, so behaviour is unchanged by construction. Compiles 0 errors, 0 warnings. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-29 00:42:45 -04:00
if(m_labelCacheBuy[i])
m_labelPrebuildBuyCount++;
else
if(m_labelCacheSell[i])
m_labelPrebuildSellCount++;
else
m_labelPrebuildNeutralCount++;
}
//--- Prebuild complete - seed era 0's class base rates from the real upfront tally instead of leaving
//--- UpdateClassPriors() nothing to measure (see m_prevEraTrueBuyCount's declaration comment).
//--- Consumed (and cleared) by Train()'s era-start block on era 0 specifically - m_prebuildSeedPending.
refactor: split CExpertSignalAIBase implementation by responsibility ExpertSignalAIBase.mqh was 8216 lines: the class declaration followed by 87 method bodies covering training, labelling, feature extraction, persistence, chart drawing, online learning, the GA auto-tuner and inference, all in one file. Train() alone is 1492 lines; a change to arrow drawing meant scrolling past the era loop. Moved the bodies into Expert\AIBase\, included at the bottom of the original after the class declaration: Training.mqh 1607 era loop, plateau ladder, checkpoint select, deploy Features.mqh 1093 indicator creation + per-bar input feature vector ChartUI.mqh 634 arrows, arrow persistence, status panel, cleanup Persistence.mqh 492 .stats/.cfg sidecars, CPU-inference validation, copy OnlineLearning.mqh 461 live continual learning, EMA shadow, OOS simulator Labels.mqh 309 ZigZag pivot labels, async label-cache prebuild AutoTune.mqh 275 genetic tuner (population, crossover, halving) Inference.mqh 235 softmax, prior calibration, class priors ExpertSignalAIBase.mqh 8216 -> 3131 (declaration + topology build only) This is a pure relocation - verified mechanically, not by eye: HEAD's file reconstructed from the eight partials plus the surviving remainder is byte-identical to HEAD, span for span (scratchpad verify_split.py). No declaration moved, no signature changed, no code rewritten, so behaviour is unchanged by construction. Compiles 0 errors, 0 warnings. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-29 00:42:45 -04:00
m_prevEraTrueBuyCount = m_labelPrebuildBuyCount;
m_prevEraTrueSellCount = m_labelPrebuildSellCount;
m_prevEraTrueNeutralCount = m_labelPrebuildNeutralCount;
m_prebuildSeedPending = true;
m_labelCachePrebuilt = true;
m_labelPrebuildActive = false;
feat(target): delete the barrier/geometry stack - the label is the verdict Step 3 of the swing-pivot plan, whole-hog. The swing label is now the ONE target and the era verdict is precision + recall per class against the label's own base rate - no win rate, no break-even, no expectancy, no geometry anywhere in training. DELETED - Expert/Excursion/ (4), Expert/BarrierHorizon/ (4), GeometrySweep, FirstPassageLadder, Labeling/TripleBarrier.mqh (CLabelOverlap survives in Labeling/LabelOverlap.mqh), 3 test EAs. - TripleBarrierLabel + walk, fractal label, geometry derivation/scan/ adoption, exit-policy replay, excursion MI targets, the drift verdict (DIRECTION_INTELLIGENT), the recall floor, balanced-accuracy telemetry, the barrier defines, the .cfg geometry adopt (slots kept as zeros for the positional layout), the derived-geometry live-order override. - TRAINING_TARGET input/enum: direction models are always swing; META2 re-keys the meta head onto label agreement (descriptor loses its two geometry slots). REWORKED - Labels.mqh (1795 -> ~370 lines): AdvanceSwingLabelState with FINALITY-GATED CACHING - an unresolved bar (pivot pair uncommitted) is never cached, so it can never freeze as a false Neutral; training, calibration, OOS scoring and online learning all skip unresolved bars. - SDeployVerdict: significance-only; SOosTally chance = larger directional class share; pooled gate poolability = timeframe (record v2). - Purge/embargo/declustering gaps: the measured mean label resolution lag (LabelResolutionBars), not a barrier horizon. - Pool purge key + backfill DB rows: marked at the bar the label resolved on (m_labelResolveAge), not a fabricated barrier touch. - Online learning frontier: finality, not a horizon delay. - m_bestBalancedOos -> m_bestSelectionScore, m_erasSinceBestBalanced -> m_erasSinceBest, ensemble vote outcome arrays -> label arrays. STEP 4 folded in: Entry_Multiplier / SL_Mode / TP_Mode / tradingdirection are inputs again - trade management is the tester GA's search space. Fingerprints: every direction model re-keys (TGT:SWG1 now unconditional, CUT token gone); META1 -> META2. Full retrain, as planned. Compile-verified in _claude_stage: Warrior_EA + both surviving test EAs, 0 errors, 0 warnings each. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-24 20:42:31 -04:00
//--- Measured-imbalance visibility. A log line must describe what the code DID: report the
//--- measured distribution, which is real and useful, and nothing else.
refactor: split CExpertSignalAIBase implementation by responsibility ExpertSignalAIBase.mqh was 8216 lines: the class declaration followed by 87 method bodies covering training, labelling, feature extraction, persistence, chart drawing, online learning, the GA auto-tuner and inference, all in one file. Train() alone is 1492 lines; a change to arrow drawing meant scrolling past the era loop. Moved the bodies into Expert\AIBase\, included at the bottom of the original after the class declaration: Training.mqh 1607 era loop, plateau ladder, checkpoint select, deploy Features.mqh 1093 indicator creation + per-bar input feature vector ChartUI.mqh 634 arrows, arrow persistence, status panel, cleanup Persistence.mqh 492 .stats/.cfg sidecars, CPU-inference validation, copy OnlineLearning.mqh 461 live continual learning, EMA shadow, OOS simulator Labels.mqh 309 ZigZag pivot labels, async label-cache prebuild AutoTune.mqh 275 genetic tuner (population, crossover, halving) Inference.mqh 235 softmax, prior calibration, class priors ExpertSignalAIBase.mqh 8216 -> 3131 (declaration + topology build only) This is a pure relocation - verified mechanically, not by eye: HEAD's file reconstructed from the eight partials plus the surviving remainder is byte-identical to HEAD, span for span (scratchpad verify_split.py). No declaration moved, no signature changed, no code rewritten, so behaviour is unchanged by construction. Compiles 0 errors, 0 warnings. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-29 00:42:45 -04:00
int prebuildMinDir = (int)MathMin(m_labelPrebuildBuyCount, m_labelPrebuildSellCount);
int prebuildMaxCls = (int)MathMax(m_labelPrebuildNeutralCount, MathMax(m_labelPrebuildBuyCount, m_labelPrebuildSellCount));
string prebuildRatioInfo = (prebuildMinDir > 0 && prebuildMaxCls > 0)
refactor(ai): nine class-imbalance inputs down to two The imbalance section offered nine controls for one job. Audited against the code, five of them did not do what their names said at the shipped defaults: AILogitPriorStrength DEAD - Inference.mqh's post-hoc prior early-returns whenever the adjusted loss is on, which is default. OversampleParity DEAD in training - Training.mqh gated the replay loop on !useLogitAdjustedLoss (correctly, citing Buda et al. 2018). Live only in the online-learning path. EnableMinorityReplay DEAD as replay. It survived ONLY as a focal-gamma damper - "replay minority bars through pass-2 oversampling" was a focal-loss switch. ConstrainReplay DEAD as a cap; it only chose damper 0.125 vs 0.25. UseStaticPrior An exact duplicate of FreezePriorCalibration - the two were OR'd together in the single place either is read. So they were not five mechanisms fighting; they were one mechanism plus eight knobs that mostly described machinery that no longer ran. That is worse than a real conflict, because the log agreed with the names: the label-cache line printed "reps up to 28x (90% parity) (seeding era 0's class-balance oversampling)" on every run, describing an oversampling pass that had been switched off. It is fixed here too - it cost this session a wrong diagnosis. The one genuine redundancy was focal loss, running at gamma*0.125 alongside the adjusted loss: two corrections on the same axis, the exact stacking failure this file already cited Buda et al. for in two other places, damped by a replay flag whose replay path was itself dead. Removed rather than re-tuned. The plateau ladder is unaffected - its escape is the learning-rate warm restart; the gamma anneal beside it only ever stepped toward zero. WHAT REMAINS is logit-adjusted loss (Menon et al. 2021) plus a prior freeze: LogitAdjustTau 0 = off; replaces the separate EnableLogitAdjusted- Loss boolean, since a strength dial where 0 already means off does not need an on/off switch beside it. FreezePriorCalibration unchanged. It is the only one of the six corrections with a consistency guarantee, and it is consistent for exactly the balanced-error metric checkpoint selection already ranks on - so the loss and the deploy decision optimize one thing. The online continual-learning path keeps its own alpha-balanced focal weight, now as constants pinned to the removed inputs' shipped defaults, so its behaviour is unchanged. It legitimately needs its own correction: ApplyLogitAdjustment() only runs inside a training run, so a deployed model that was reloaded carries no logit offsets and would otherwise stream 31:1 data into itself uncorrected. The weights-filename fingerprint is BYTE-IDENTICAL. The focal slot was a double fed to a %d conversion and had always emitted a literal 0; the |MR: segment is written as the constant its shipped defaults produced. Dropping either would have re-keyed every model and forced a from-scratch retrain of the one topology currently converged and trading. Also removed as orphans: FOCAL_GAMMA_PRESET, MAX_OVERSAMPLE_REPLICAS, OVERSAMPLE_PARITY_FRACTION, PLATEAU_GAMMA_STEP, and the now-unreachable "neutralized by prior correction" diagnostic. Both builds compile 0 errors, 0 warnings. No retrain forced. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-31 11:46:57 -04:00
? " | measured imbalance ~" + DoubleToString((double)prebuildMaxCls / prebuildMinDir, 1) + ":1"
refactor: split CExpertSignalAIBase implementation by responsibility ExpertSignalAIBase.mqh was 8216 lines: the class declaration followed by 87 method bodies covering training, labelling, feature extraction, persistence, chart drawing, online learning, the GA auto-tuner and inference, all in one file. Train() alone is 1492 lines; a change to arrow drawing meant scrolling past the era loop. Moved the bodies into Expert\AIBase\, included at the bottom of the original after the class declaration: Training.mqh 1607 era loop, plateau ladder, checkpoint select, deploy Features.mqh 1093 indicator creation + per-bar input feature vector ChartUI.mqh 634 arrows, arrow persistence, status panel, cleanup Persistence.mqh 492 .stats/.cfg sidecars, CPU-inference validation, copy OnlineLearning.mqh 461 live continual learning, EMA shadow, OOS simulator Labels.mqh 309 ZigZag pivot labels, async label-cache prebuild AutoTune.mqh 275 genetic tuner (population, crossover, halving) Inference.mqh 235 softmax, prior calibration, class priors ExpertSignalAIBase.mqh 8216 -> 3131 (declaration + topology build only) This is a pure relocation - verified mechanically, not by eye: HEAD's file reconstructed from the eight partials plus the surviving remainder is byte-identical to HEAD, span for span (scratchpad verify_split.py). No declaration moved, no signature changed, no code rewritten, so behaviour is unchanged by construction. Compiles 0 errors, 0 warnings. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-29 00:42:45 -04:00
: " | measured imbalance n/a (a directional class has no labeled bars in this window)";
feat(ai): triple-barrier labels replace exact-pivot ZigZag targets The 31:1 class imbalance was self-inflicted by the TARGET, not a property of the market. Labelling only the exact bar where a ZigZag pivot confirms gave Buy 1164 / Sell 1164 / Neutral 35841, and every correction mechanism this codebase accumulated sits downstream of that one choice: the logit-adjusted loss and its range cap, the prior EMA, the +-3.0 output-bias seed, balanced-accuracy-then-precision selection with its coverage floor, the recall floor and its catch-22, the alternation gate, NMS, and the four oversampling designs that collapsed before them. The reference this engine is built on (references/neuronetworksbook.pdf ch. 3.1/3.3) also uses ZigZag, but targets the DIRECTION TO THE NEXT EXTREMUM on every bar - ~50/50 by construction, with no imbalance to correct at all. It never had this problem because it never asked "is this the pivot bar". Labels are now the triple barrier (Lopez de Prado ch. 3), using the EA's OWN SL_Mode/TP_Mode: does a trade opened at this bar's close reach its target before its stop, within a horizon. Buy = long resolves, Sell = short resolves, Neutral = neither. Consequences: - dir-precision in the era line stops being a proxy and becomes the win rate of the strategy under its own exit rules. - Expected balance ~25/25/50 at the shipped 1:3 (gambler's ruin), i.e. ~2:1 instead of 31:1. Measured and logged at the end of the prebuild. - Spread is charged on both legs, so it is a NET win rate. - Intrabar ambiguity resolves to the STOP. OHLC cannot order two touches inside one bar and the optimistic reading is how a backtested edge becomes a live loss. ZigZag stays as input features (EnableSwingContext) and now also supplies the vertical barrier: the horizon is the median confirmed leg length, snapped to a coarse ladder. Derived, not configured, and deliberately kept out of the filename fingerprint - a filename keyed on a measured quantity orphans a trained model the moment the measurement moves. Removed, because the premise died with the old target: - the alternation gate. Correct for pivot labels (a ZigZag cannot emit two same-type pivots in a row, so a repeat was provably a false fire), and wrong for barrier labels, which answer each bar independently. It also took its worst consequence with it: a one-sided model previously got ONE trade per backtest, a hard blocker on marketplace validation. - SignalClusterWindow now defaults off - it de-duplicated repeats that are now real trades. Kept as an opt-in display control. - LABEL_WINDOW_BARS, the pivot-widening pass, ConfirmedZigZagLabel. - the era-0 output-bias seed now needs a genuinely dominant class (0.70) rather than 0.40; at ~50% Neutral a +-3.0 seed is a distortion, not a correction. Also fixed, both found while wiring the above: 1. RefreshConvergedSignal sized its buffers from a date delta (Bars(sym, period, dtStudied, TimeCurrent())). dtStudied is a training watermark; in the tester it is loaded from a live-chart save AHEAD of the simulated date, so the interval inverted, Bars() returned ~0, and the buffer came out at exactly m_historyBars - deep enough for the OHLC window and far too shallow for the Donchian-50 / 20-bar-return / SMA extension behind it. Inference silently computed DIFFERENT features from the ones training learned on, live as well as in the tester. Now sized from what the feature builder actually needs. 2. The barrier horizon is resolved on the deployed path too. A deployed model never enters Train(), so it never reached the prebuild, and OnlineLearnStep reads the horizon as its confirmation delay - left at the fallback it would have backpropped bars whose barriers had not resolved. Silent lookahead in the one place that writes to a live model. SL_Mode/TP_Mode join the weights fingerprint: they define the labels now, so a model trained at 1:3 must never be silently reused at 1:1. This re-keys every pre-existing model by design - none were trained on this task. Inference census extended with the vote gate. LongCondition/ShortCondition open with a readiness check the refresh counters never see; in the tester it reduces to "the seeded _optcache.nnw must have LOADED", and if it did not, every vote is hard-zeroed while the model still answers Buy. The old three counters would have read that as "the model says Neutral" - false, and a completely different fix. This is the leading candidate for the zero-direction backtest and the census can now name it in one run. Both builds compile 0 errors / 0 warnings. Forces a full retrain. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-31 20:39:49 -04:00
int prebuildTotal = m_labelPrebuildBuyCount + m_labelPrebuildSellCount + m_labelPrebuildNeutralCount;
string prebuildShare = (prebuildTotal > 0)
? " | share Buy " + DoubleToString(100.0 * m_labelPrebuildBuyCount / prebuildTotal, 1) +
"% Sell " + DoubleToString(100.0 * m_labelPrebuildSellCount / prebuildTotal, 1) +
"% Neutral " + DoubleToString(100.0 * m_labelPrebuildNeutralCount / prebuildTotal, 1) + "%"
: "";
//--- LABEL OVERLAP, printed with the distribution because it is a property of the same
feat(label): pivot-EVENT target replaces direction-to-next-pivot The old target asked "which way is the next pivot", which every bar of a ~13-20 bar leg answers identically - so the net could not tell a fresh turn from mid-trend and learned the prevailing direction instead. Its own zero-skill reference showed it: chance sat at 56/44, i.e. the label WAS the drift, and the gate's standing warning ("a model that only reproduces it has found the drift, not an edge") applied to the target itself. Buy now means a swing LOW commits within PIVOT_LABEL_TOLERANCE_BARS bars, Sell a swing HIGH, Neutral no turn that close. Pivot type is read from ZigZagBuffer[p] == Low[p], exact by construction in ZigZag.mq5. The existing P1-final-once-P2-commits rule is kept and now also settles the NEGATIVE verdict, so the Neutral majority is permanent rather than provisional. Measured on a full fresh run, all 6 charts: class balance 56/44/~0 -> 13.7/13.7/72.6 (imbalance 5.3:1) label overlap ~31 bars -> 5 bars independent obs 368-1086 -> 2331-7032 weights/obs 9.2-26.2 -> 1.1-4.2 coverage 100% of bars -> 17-48% 23 of 24 models fire all three classes at precision 18-32% vs 13-15% chance; SP500's ensemble reaches DEPLOYABLE (32.3% vs a 24.0% bar). Two bindings had to move with the label: - The capacity deflator. m_swingLifespan fed EstimatedInSampleBars() as raw/31, measured from the legs. Overlap is now a property of the LABEL - one turn is callable by exactly the tolerance window - so it is the window, not a leg measurement. Missing this would have kept every model sized for a sixth of its real evidence. - A dormant cold-start seed. Labels.mqh seeds the output bias toward the dominant class above COLD_START_SEED_MIN_DOMINANCE (0.70); at 56/44 it never armed, at 72.6% Neutral it does - writing a fixed +-3.0 against a true prior spread of ~1.75, which would start every net predicting Neutral ~95% of the time. Now seeds the measured log-prior, zero-centred and capped by the same guard rail the logit adjustment uses (Lin et al. 2017). TGT:SWG1 -> TGT:PVT1:<tolerance>, with the window in the token because it is part of the label: every .nnw is invalidated and the fleet retrains. Depth is still gated, and now for a precise reason: the first dense layer stays at FIRST_LAYER_MIN_WIDTH because budget = effN/(inputWidth+1) is 11.2 at input 624. Reaching the next rung needs inputWidth <= ~218, i.e. feature pruning - not architecture. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-26 00:16:16 -04:00
//--- measurement and because every standard error downstream is divided by it. Under the
//--- pivot-event target the overlap is the TOLERANCE WINDOW - the bars that can call one turn -
//--- not the resolution lag, which is a statement about when a label may be trusted rather than
//--- about how much independent evidence it carries. See SwingPivotDirectionLabel().
string prebuildOverlap = "";
if(m_labelOverlap.Count() > 0)
feat(label): pivot-EVENT target replaces direction-to-next-pivot The old target asked "which way is the next pivot", which every bar of a ~13-20 bar leg answers identically - so the net could not tell a fresh turn from mid-trend and learned the prevailing direction instead. Its own zero-skill reference showed it: chance sat at 56/44, i.e. the label WAS the drift, and the gate's standing warning ("a model that only reproduces it has found the drift, not an edge") applied to the target itself. Buy now means a swing LOW commits within PIVOT_LABEL_TOLERANCE_BARS bars, Sell a swing HIGH, Neutral no turn that close. Pivot type is read from ZigZagBuffer[p] == Low[p], exact by construction in ZigZag.mq5. The existing P1-final-once-P2-commits rule is kept and now also settles the NEGATIVE verdict, so the Neutral majority is permanent rather than provisional. Measured on a full fresh run, all 6 charts: class balance 56/44/~0 -> 13.7/13.7/72.6 (imbalance 5.3:1) label overlap ~31 bars -> 5 bars independent obs 368-1086 -> 2331-7032 weights/obs 9.2-26.2 -> 1.1-4.2 coverage 100% of bars -> 17-48% 23 of 24 models fire all three classes at precision 18-32% vs 13-15% chance; SP500's ensemble reaches DEPLOYABLE (32.3% vs a 24.0% bar). Two bindings had to move with the label: - The capacity deflator. m_swingLifespan fed EstimatedInSampleBars() as raw/31, measured from the legs. Overlap is now a property of the LABEL - one turn is callable by exactly the tolerance window - so it is the window, not a leg measurement. Missing this would have kept every model sized for a sixth of its real evidence. - A dormant cold-start seed. Labels.mqh seeds the output bias toward the dominant class above COLD_START_SEED_MIN_DOMINANCE (0.70); at 56/44 it never armed, at 72.6% Neutral it does - writing a fixed +-3.0 against a true prior spread of ~1.75, which would start every net predicting Neutral ~95% of the time. Now seeds the measured log-prior, zero-centred and capped by the same guard rail the logit adjustment uses (Lin et al. 2017). TGT:SWG1 -> TGT:PVT1:<tolerance>, with the window in the token because it is part of the label: every .nnw is invalidated and the fleet retrains. Depth is still gated, and now for a precise reason: the first dense layer stays at FIRST_LAYER_MIN_WIDTH because budget = effN/(inputWidth+1) is 11.2 at input 624. Reaching the next rung needs inputWidth <= ~218, i.e. feature pruning - not architecture. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-26 00:16:16 -04:00
prebuildOverlap = StringFormat(" | label overlap %.1f bars (the window of bars that can call one pivot) -> "
"%d labels are worth ~%d independent ones (every SE below is sized on that)",
MeanLabelLifespan(), (int)m_labelOverlap.Count(),
(int)EffectiveSampleSize((double)m_labelOverlap.Count()));
Print(ID + ": label cache pre-built - IS true-label distribution -> Buy: " + IntegerToString(m_labelPrebuildBuyCount) +
" | Sell: " + IntegerToString(m_labelPrebuildSellCount) + " | Neutral: " + IntegerToString(m_labelPrebuildNeutralCount) +
feat(target): delete the barrier/geometry stack - the label is the verdict Step 3 of the swing-pivot plan, whole-hog. The swing label is now the ONE target and the era verdict is precision + recall per class against the label's own base rate - no win rate, no break-even, no expectancy, no geometry anywhere in training. DELETED - Expert/Excursion/ (4), Expert/BarrierHorizon/ (4), GeometrySweep, FirstPassageLadder, Labeling/TripleBarrier.mqh (CLabelOverlap survives in Labeling/LabelOverlap.mqh), 3 test EAs. - TripleBarrierLabel + walk, fractal label, geometry derivation/scan/ adoption, exit-policy replay, excursion MI targets, the drift verdict (DIRECTION_INTELLIGENT), the recall floor, balanced-accuracy telemetry, the barrier defines, the .cfg geometry adopt (slots kept as zeros for the positional layout), the derived-geometry live-order override. - TRAINING_TARGET input/enum: direction models are always swing; META2 re-keys the meta head onto label agreement (descriptor loses its two geometry slots). REWORKED - Labels.mqh (1795 -> ~370 lines): AdvanceSwingLabelState with FINALITY-GATED CACHING - an unresolved bar (pivot pair uncommitted) is never cached, so it can never freeze as a false Neutral; training, calibration, OOS scoring and online learning all skip unresolved bars. - SDeployVerdict: significance-only; SOosTally chance = larger directional class share; pooled gate poolability = timeframe (record v2). - Purge/embargo/declustering gaps: the measured mean label resolution lag (LabelResolutionBars), not a barrier horizon. - Pool purge key + backfill DB rows: marked at the bar the label resolved on (m_labelResolveAge), not a fabricated barrier touch. - Online learning frontier: finality, not a horizon delay. - m_bestBalancedOos -> m_bestSelectionScore, m_erasSinceBestBalanced -> m_erasSinceBest, ensemble vote outcome arrays -> label arrays. STEP 4 folded in: Entry_Multiplier / SL_Mode / TP_Mode / tradingdirection are inputs again - trade management is the tester GA's search space. Fingerprints: every direction model re-keys (TGT:SWG1 now unconditional, CUT token gone); META1 -> META2. Full retrain, as planned. Compile-verified in _claude_stage: Warrior_EA + both surviving test EAs, 0 errors, 0 warnings each. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-24 20:42:31 -04:00
prebuildRatioInfo + prebuildShare + prebuildOverlap +
feat(label): pivot-EVENT target replaces direction-to-next-pivot The old target asked "which way is the next pivot", which every bar of a ~13-20 bar leg answers identically - so the net could not tell a fresh turn from mid-trend and learned the prevailing direction instead. Its own zero-skill reference showed it: chance sat at 56/44, i.e. the label WAS the drift, and the gate's standing warning ("a model that only reproduces it has found the drift, not an edge") applied to the target itself. Buy now means a swing LOW commits within PIVOT_LABEL_TOLERANCE_BARS bars, Sell a swing HIGH, Neutral no turn that close. Pivot type is read from ZigZagBuffer[p] == Low[p], exact by construction in ZigZag.mq5. The existing P1-final-once-P2-commits rule is kept and now also settles the NEGATIVE verdict, so the Neutral majority is permanent rather than provisional. Measured on a full fresh run, all 6 charts: class balance 56/44/~0 -> 13.7/13.7/72.6 (imbalance 5.3:1) label overlap ~31 bars -> 5 bars independent obs 368-1086 -> 2331-7032 weights/obs 9.2-26.2 -> 1.1-4.2 coverage 100% of bars -> 17-48% 23 of 24 models fire all three classes at precision 18-32% vs 13-15% chance; SP500's ensemble reaches DEPLOYABLE (32.3% vs a 24.0% bar). Two bindings had to move with the label: - The capacity deflator. m_swingLifespan fed EstimatedInSampleBars() as raw/31, measured from the legs. Overlap is now a property of the LABEL - one turn is callable by exactly the tolerance window - so it is the window, not a leg measurement. Missing this would have kept every model sized for a sixth of its real evidence. - A dormant cold-start seed. Labels.mqh seeds the output bias toward the dominant class above COLD_START_SEED_MIN_DOMINANCE (0.70); at 56/44 it never armed, at 72.6% Neutral it does - writing a fixed +-3.0 against a true prior spread of ~1.75, which would start every net predicting Neutral ~95% of the time. Now seeds the measured log-prior, zero-centred and capped by the same guard rail the logit adjustment uses (Lin et al. 2017). TGT:SWG1 -> TGT:PVT1:<tolerance>, with the window in the token because it is part of the label: every .nnw is invalidated and the fleet retrains. Depth is still gated, and now for a precise reason: the first dense layer stays at FIRST_LAYER_MIN_WIDTH because budget = effN/(inputWidth+1) is 11.2 at input 624. Reaching the next rung needs inputWidth <= ~218, i.e. feature pruning - not architecture. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-26 00:16:16 -04:00
(m_eraCount == 0 ? StringFormat(" (seeding era 0 - PIVOT-EVENT target: Buy/Sell mean 'a swing low/high commits"
" within %d bars', Neutral means no turn that close, which is most bars)",
(int)PIVOT_LABEL_TOLERANCE_BARS)
refactor: split CExpertSignalAIBase implementation by responsibility ExpertSignalAIBase.mqh was 8216 lines: the class declaration followed by 87 method bodies covering training, labelling, feature extraction, persistence, chart drawing, online learning, the GA auto-tuner and inference, all in one file. Train() alone is 1492 lines; a change to arrow drawing meant scrolling past the era loop. Moved the bodies into Expert\AIBase\, included at the bottom of the original after the class declaration: Training.mqh 1607 era loop, plateau ladder, checkpoint select, deploy Features.mqh 1093 indicator creation + per-bar input feature vector ChartUI.mqh 634 arrows, arrow persistence, status panel, cleanup Persistence.mqh 492 .stats/.cfg sidecars, CPU-inference validation, copy OnlineLearning.mqh 461 live continual learning, EMA shadow, OOS simulator Labels.mqh 309 ZigZag pivot labels, async label-cache prebuild AutoTune.mqh 275 genetic tuner (population, crossover, halving) Inference.mqh 235 softmax, prior calibration, class priors ExpertSignalAIBase.mqh 8216 -> 3131 (declaration + topology build only) This is a pure relocation - verified mechanically, not by eye: HEAD's file reconstructed from the eight partials plus the surviving remainder is byte-identical to HEAD, span for span (scratchpad verify_split.py). No declaration moved, no signature changed, no code rewritten, so behaviour is unchanged by construction. Compiles 0 errors, 0 warnings. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-29 00:42:45 -04:00
: " (mid-run rebuild after new-bar cache invalidation - era " + IntegerToString(m_eraCount) + " resumes on the relabeled window)"));
//--- Cold-start fix: a freshly-initialized (random-weight) network's argmax is close to uniform
//--- noise across the 3 classes, so on this typically heavily-skewed label distribution it fires
//--- far more non-majority-class calls at the very start of era 0 than the true base rate
//--- warrants, until enough backProp steps correct it.
refactor: split CExpertSignalAIBase implementation by responsibility ExpertSignalAIBase.mqh was 8216 lines: the class declaration followed by 87 method bodies covering training, labelling, feature extraction, persistence, chart drawing, online learning, the GA auto-tuner and inference, all in one file. Train() alone is 1492 lines; a change to arrow drawing meant scrolling past the era loop. Moved the bodies into Expert\AIBase\, included at the bottom of the original after the class declaration: Training.mqh 1607 era loop, plateau ladder, checkpoint select, deploy Features.mqh 1093 indicator creation + per-bar input feature vector ChartUI.mqh 634 arrows, arrow persistence, status panel, cleanup Persistence.mqh 492 .stats/.cfg sidecars, CPU-inference validation, copy OnlineLearning.mqh 461 live continual learning, EMA shadow, OOS simulator Labels.mqh 309 ZigZag pivot labels, async label-cache prebuild AutoTune.mqh 275 genetic tuner (population, crossover, halving) Inference.mqh 235 softmax, prior calibration, class priors ExpertSignalAIBase.mqh 8216 -> 3131 (declaration + topology build only) This is a pure relocation - verified mechanically, not by eye: HEAD's file reconstructed from the eight partials plus the surviving remainder is byte-identical to HEAD, span for span (scratchpad verify_split.py). No declaration moved, no signature changed, no code rewritten, so behaviour is unchanged by construction. Compiles 0 errors, 0 warnings. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-29 00:42:45 -04:00
if(m_outputNeuronsCount == 3 && m_eraCount == 0)
{
int dominant = 2; // Neutral
int dominantCount = m_labelPrebuildNeutralCount;
if(m_labelPrebuildBuyCount > dominantCount)
{
dominant = 0;
dominantCount = m_labelPrebuildBuyCount;
}
if(m_labelPrebuildSellCount > dominantCount)
{
dominant = 1;
dominantCount = m_labelPrebuildSellCount;
}
int totalLabeled = m_labelPrebuildBuyCount + m_labelPrebuildSellCount + m_labelPrebuildNeutralCount;
feat(ai): triple-barrier labels replace exact-pivot ZigZag targets The 31:1 class imbalance was self-inflicted by the TARGET, not a property of the market. Labelling only the exact bar where a ZigZag pivot confirms gave Buy 1164 / Sell 1164 / Neutral 35841, and every correction mechanism this codebase accumulated sits downstream of that one choice: the logit-adjusted loss and its range cap, the prior EMA, the +-3.0 output-bias seed, balanced-accuracy-then-precision selection with its coverage floor, the recall floor and its catch-22, the alternation gate, NMS, and the four oversampling designs that collapsed before them. The reference this engine is built on (references/neuronetworksbook.pdf ch. 3.1/3.3) also uses ZigZag, but targets the DIRECTION TO THE NEXT EXTREMUM on every bar - ~50/50 by construction, with no imbalance to correct at all. It never had this problem because it never asked "is this the pivot bar". Labels are now the triple barrier (Lopez de Prado ch. 3), using the EA's OWN SL_Mode/TP_Mode: does a trade opened at this bar's close reach its target before its stop, within a horizon. Buy = long resolves, Sell = short resolves, Neutral = neither. Consequences: - dir-precision in the era line stops being a proxy and becomes the win rate of the strategy under its own exit rules. - Expected balance ~25/25/50 at the shipped 1:3 (gambler's ruin), i.e. ~2:1 instead of 31:1. Measured and logged at the end of the prebuild. - Spread is charged on both legs, so it is a NET win rate. - Intrabar ambiguity resolves to the STOP. OHLC cannot order two touches inside one bar and the optimistic reading is how a backtested edge becomes a live loss. ZigZag stays as input features (EnableSwingContext) and now also supplies the vertical barrier: the horizon is the median confirmed leg length, snapped to a coarse ladder. Derived, not configured, and deliberately kept out of the filename fingerprint - a filename keyed on a measured quantity orphans a trained model the moment the measurement moves. Removed, because the premise died with the old target: - the alternation gate. Correct for pivot labels (a ZigZag cannot emit two same-type pivots in a row, so a repeat was provably a false fire), and wrong for barrier labels, which answer each bar independently. It also took its worst consequence with it: a one-sided model previously got ONE trade per backtest, a hard blocker on marketplace validation. - SignalClusterWindow now defaults off - it de-duplicated repeats that are now real trades. Kept as an opt-in display control. - LABEL_WINDOW_BARS, the pivot-widening pass, ConfirmedZigZagLabel. - the era-0 output-bias seed now needs a genuinely dominant class (0.70) rather than 0.40; at ~50% Neutral a +-3.0 seed is a distortion, not a correction. Also fixed, both found while wiring the above: 1. RefreshConvergedSignal sized its buffers from a date delta (Bars(sym, period, dtStudied, TimeCurrent())). dtStudied is a training watermark; in the tester it is loaded from a live-chart save AHEAD of the simulated date, so the interval inverted, Bars() returned ~0, and the buffer came out at exactly m_historyBars - deep enough for the OHLC window and far too shallow for the Donchian-50 / 20-bar-return / SMA extension behind it. Inference silently computed DIFFERENT features from the ones training learned on, live as well as in the tester. Now sized from what the feature builder actually needs. 2. The barrier horizon is resolved on the deployed path too. A deployed model never enters Train(), so it never reached the prebuild, and OnlineLearnStep reads the horizon as its confirmation delay - left at the fallback it would have backpropped bars whose barriers had not resolved. Silent lookahead in the one place that writes to a live model. SL_Mode/TP_Mode join the weights fingerprint: they define the labels now, so a model trained at 1:3 must never be silently reused at 1:1. This re-keys every pre-existing model by design - none were trained on this task. Inference census extended with the vote gate. LongCondition/ShortCondition open with a readiness check the refresh counters never see; in the tester it reduces to "the seeded _optcache.nnw must have LOADED", and if it did not, every vote is hard-zeroed while the model still answers Buy. The old three counters would have read that as "the model says Neutral" - false, and a completely different fix. This is the leading candidate for the zero-direction backtest and the census can now name it in one run. Both builds compile 0 errors / 0 warnings. Forces a full retrain. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-31 20:39:49 -04:00
if(totalLabeled > 0 && (double)dominantCount / totalLabeled > COLD_START_SEED_MIN_DOMINANCE)
refactor: split CExpertSignalAIBase implementation by responsibility ExpertSignalAIBase.mqh was 8216 lines: the class declaration followed by 87 method bodies covering training, labelling, feature extraction, persistence, chart drawing, online learning, the GA auto-tuner and inference, all in one file. Train() alone is 1492 lines; a change to arrow drawing meant scrolling past the era loop. Moved the bodies into Expert\AIBase\, included at the bottom of the original after the class declaration: Training.mqh 1607 era loop, plateau ladder, checkpoint select, deploy Features.mqh 1093 indicator creation + per-bar input feature vector ChartUI.mqh 634 arrows, arrow persistence, status panel, cleanup Persistence.mqh 492 .stats/.cfg sidecars, CPU-inference validation, copy OnlineLearning.mqh 461 live continual learning, EMA shadow, OOS simulator Labels.mqh 309 ZigZag pivot labels, async label-cache prebuild AutoTune.mqh 275 genetic tuner (population, crossover, halving) Inference.mqh 235 softmax, prior calibration, class priors ExpertSignalAIBase.mqh 8216 -> 3131 (declaration + topology build only) This is a pure relocation - verified mechanically, not by eye: HEAD's file reconstructed from the eight partials plus the surviving remainder is byte-identical to HEAD, span for span (scratchpad verify_split.py). No declaration moved, no signature changed, no code rewritten, so behaviour is unchanged by construction. Compiles 0 errors, 0 warnings. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-29 00:42:45 -04:00
{
feat(label): pivot-EVENT target replaces direction-to-next-pivot The old target asked "which way is the next pivot", which every bar of a ~13-20 bar leg answers identically - so the net could not tell a fresh turn from mid-trend and learned the prevailing direction instead. Its own zero-skill reference showed it: chance sat at 56/44, i.e. the label WAS the drift, and the gate's standing warning ("a model that only reproduces it has found the drift, not an edge") applied to the target itself. Buy now means a swing LOW commits within PIVOT_LABEL_TOLERANCE_BARS bars, Sell a swing HIGH, Neutral no turn that close. Pivot type is read from ZigZagBuffer[p] == Low[p], exact by construction in ZigZag.mq5. The existing P1-final-once-P2-commits rule is kept and now also settles the NEGATIVE verdict, so the Neutral majority is permanent rather than provisional. Measured on a full fresh run, all 6 charts: class balance 56/44/~0 -> 13.7/13.7/72.6 (imbalance 5.3:1) label overlap ~31 bars -> 5 bars independent obs 368-1086 -> 2331-7032 weights/obs 9.2-26.2 -> 1.1-4.2 coverage 100% of bars -> 17-48% 23 of 24 models fire all three classes at precision 18-32% vs 13-15% chance; SP500's ensemble reaches DEPLOYABLE (32.3% vs a 24.0% bar). Two bindings had to move with the label: - The capacity deflator. m_swingLifespan fed EstimatedInSampleBars() as raw/31, measured from the legs. Overlap is now a property of the LABEL - one turn is callable by exactly the tolerance window - so it is the window, not a leg measurement. Missing this would have kept every model sized for a sixth of its real evidence. - A dormant cold-start seed. Labels.mqh seeds the output bias toward the dominant class above COLD_START_SEED_MIN_DOMINANCE (0.70); at 56/44 it never armed, at 72.6% Neutral it does - writing a fixed +-3.0 against a true prior spread of ~1.75, which would start every net predicting Neutral ~95% of the time. Now seeds the measured log-prior, zero-centred and capped by the same guard rail the logit adjustment uses (Lin et al. 2017). TGT:SWG1 -> TGT:PVT1:<tolerance>, with the window in the token because it is part of the label: every .nnw is invalidated and the fleet retrains. Depth is still gated, and now for a precise reason: the first dense layer stays at FIRST_LAYER_MIN_WIDTH because budget = effN/(inputWidth+1) is 11.2 at input 624. Reaching the next rung needs inputWidth <= ~218, i.e. feature pruning - not architecture. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-26 00:16:16 -04:00
//--- SEED THE MEASURED LOG-PRIOR, NOT A FIXED MAGNITUDE. This used to write +-3.0
//--- (softmax ~0.95/0.05) toward whichever class dominated. That never executed while the
//--- target was direction-to-next-pivot - the split there was ~56/44/0, under
//--- COLD_START_SEED_MIN_DOMINANCE - and the pivot-event target is the first label to arm
//--- it, at ~12/12/75. A +-3 seed is a 6-logit spread against a TRUE prior spread of
//--- log(0.75/0.125) ~= 1.79: it would start the net predicting Neutral ~95% of the time,
//--- roughly 3.4x more skewed than the data, and the two rare directional classes then have
//--- to climb out of that on top of the residual imbalance the capped logit adjustment
//--- already leaves them (ApplyLogitAdjustment's tau pins the correction at 1.2 logits).
//--- That is the shape of the collapse this project already paid for once (1b5a412).
//---
//--- Initialising the output bias to the class log-prior is the standard prescription for
//--- exactly this rare-event regime (Lin et al. 2017, focal loss, sec. 4.1 "prior"): it
//--- makes the untrained net predict the base rate instead of uniform noise, which is what
//--- the original comment below wanted, without overshooting the base rate it is matching.
//--- Zero-centred because softmax is shift-invariant - only the differences are real.
double priors[3];
priors[0] = (double)m_labelPrebuildBuyCount / totalLabeled;
priors[1] = (double)m_labelPrebuildSellCount / totalLabeled;
priors[2] = (double)m_labelPrebuildNeutralCount / totalLabeled;
//--- Floor: an empty class must not seed a -inf bias. 1e-4 is well below any share that
//--- survives the MIN_OOS_CLASS_SAMPLES_FOR_GATE-scale counts this runs on.
double logs[3], mean = 0.0;
for(int c = 0; c < 3; c++)
{
logs[c] = MathLog(MathMax(priors[c], 1e-4));
mean += logs[c];
}
mean /= 3.0;
//--- Same guard rail the logit adjustment uses: never consume more than
//--- LOGIT_ADJUST_MAX_RANGE_FRACTION of the head's usable logit range.
double seedCap = LOGIT_ADJUST_MAX_RANGE_FRACTION * CLASS_LOGIT_SCALE;
double biasValues[3];
for(int c = 0; c < 3; c++)
biasValues[c] = MathMax(-seedCap, MathMin(seedCap, logs[c] - mean));
refactor: split CExpertSignalAIBase implementation by responsibility ExpertSignalAIBase.mqh was 8216 lines: the class declaration followed by 87 method bodies covering training, labelling, feature extraction, persistence, chart drawing, online learning, the GA auto-tuner and inference, all in one file. Train() alone is 1492 lines; a change to arrow drawing meant scrolling past the era loop. Moved the bodies into Expert\AIBase\, included at the bottom of the original after the class declaration: Training.mqh 1607 era loop, plateau ladder, checkpoint select, deploy Features.mqh 1093 indicator creation + per-bar input feature vector ChartUI.mqh 634 arrows, arrow persistence, status panel, cleanup Persistence.mqh 492 .stats/.cfg sidecars, CPU-inference validation, copy OnlineLearning.mqh 461 live continual learning, EMA shadow, OOS simulator Labels.mqh 309 ZigZag pivot labels, async label-cache prebuild AutoTune.mqh 275 genetic tuner (population, crossover, halving) Inference.mqh 235 softmax, prior calibration, class priors ExpertSignalAIBase.mqh 8216 -> 3131 (declaration + topology build only) This is a pure relocation - verified mechanically, not by eye: HEAD's file reconstructed from the eight partials plus the surviving remainder is byte-identical to HEAD, span for span (scratchpad verify_split.py). No declaration moved, no signature changed, no code rewritten, so behaviour is unchanged by construction. Compiles 0 errors, 0 warnings. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-29 00:42:45 -04:00
if(Net.SeedOutputLayerBias(biasValues))
feat(label): pivot-EVENT target replaces direction-to-next-pivot The old target asked "which way is the next pivot", which every bar of a ~13-20 bar leg answers identically - so the net could not tell a fresh turn from mid-trend and learned the prevailing direction instead. Its own zero-skill reference showed it: chance sat at 56/44, i.e. the label WAS the drift, and the gate's standing warning ("a model that only reproduces it has found the drift, not an edge") applied to the target itself. Buy now means a swing LOW commits within PIVOT_LABEL_TOLERANCE_BARS bars, Sell a swing HIGH, Neutral no turn that close. Pivot type is read from ZigZagBuffer[p] == Low[p], exact by construction in ZigZag.mq5. The existing P1-final-once-P2-commits rule is kept and now also settles the NEGATIVE verdict, so the Neutral majority is permanent rather than provisional. Measured on a full fresh run, all 6 charts: class balance 56/44/~0 -> 13.7/13.7/72.6 (imbalance 5.3:1) label overlap ~31 bars -> 5 bars independent obs 368-1086 -> 2331-7032 weights/obs 9.2-26.2 -> 1.1-4.2 coverage 100% of bars -> 17-48% 23 of 24 models fire all three classes at precision 18-32% vs 13-15% chance; SP500's ensemble reaches DEPLOYABLE (32.3% vs a 24.0% bar). Two bindings had to move with the label: - The capacity deflator. m_swingLifespan fed EstimatedInSampleBars() as raw/31, measured from the legs. Overlap is now a property of the LABEL - one turn is callable by exactly the tolerance window - so it is the window, not a leg measurement. Missing this would have kept every model sized for a sixth of its real evidence. - A dormant cold-start seed. Labels.mqh seeds the output bias toward the dominant class above COLD_START_SEED_MIN_DOMINANCE (0.70); at 56/44 it never armed, at 72.6% Neutral it does - writing a fixed +-3.0 against a true prior spread of ~1.75, which would start every net predicting Neutral ~95% of the time. Now seeds the measured log-prior, zero-centred and capped by the same guard rail the logit adjustment uses (Lin et al. 2017). TGT:SWG1 -> TGT:PVT1:<tolerance>, with the window in the token because it is part of the label: every .nnw is invalidated and the fleet retrains. Depth is still gated, and now for a precise reason: the first dense layer stays at FIRST_LAYER_MIN_WIDTH because budget = effN/(inputWidth+1) is 11.2 at input 624. Reaching the next rung needs inputWidth <= ~218, i.e. feature pruning - not architecture. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-26 00:16:16 -04:00
PrintVerbose(ID + StringFormat(": seeded output layer bias to the measured class log-prior"
" B/S/N = %+.2f/%+.2f/%+.2f (shares %.1f%%/%.1f%%/%.1f%%, dominant %s at"
" %.1f%% > %.0f%% seed threshold) - era 0 cold-start fix, so the untrained"
" net starts at the base rate rather than at uniform noise.",
biasValues[0], biasValues[1], biasValues[2],
100.0 * priors[0], 100.0 * priors[1], 100.0 * priors[2],
EnumToString((ENUM_SIGNAL)(dominant == 0 ? Buy : dominant == 1 ? Sell : Neutral)),
100.0 * dominantCount / totalLabeled, 100.0 * COLD_START_SEED_MIN_DOMINANCE));
refactor: split CExpertSignalAIBase implementation by responsibility ExpertSignalAIBase.mqh was 8216 lines: the class declaration followed by 87 method bodies covering training, labelling, feature extraction, persistence, chart drawing, online learning, the GA auto-tuner and inference, all in one file. Train() alone is 1492 lines; a change to arrow drawing meant scrolling past the era loop. Moved the bodies into Expert\AIBase\, included at the bottom of the original after the class declaration: Training.mqh 1607 era loop, plateau ladder, checkpoint select, deploy Features.mqh 1093 indicator creation + per-bar input feature vector ChartUI.mqh 634 arrows, arrow persistence, status panel, cleanup Persistence.mqh 492 .stats/.cfg sidecars, CPU-inference validation, copy OnlineLearning.mqh 461 live continual learning, EMA shadow, OOS simulator Labels.mqh 309 ZigZag pivot labels, async label-cache prebuild AutoTune.mqh 275 genetic tuner (population, crossover, halving) Inference.mqh 235 softmax, prior calibration, class priors ExpertSignalAIBase.mqh 8216 -> 3131 (declaration + topology build only) This is a pure relocation - verified mechanically, not by eye: HEAD's file reconstructed from the eight partials plus the surviving remainder is byte-identical to HEAD, span for span (scratchpad verify_split.py). No declaration moved, no signature changed, no code rewritten, so behaviour is unchanged by construction. Compiles 0 errors, 0 warnings. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-29 00:42:45 -04:00
}
}
}
#endif // WARRIOR_AIBASE_LABELS_MQH