refactor: split CExpertSignalAIBase implementation by responsibility
ExpertSignalAIBase.mqh was 8216 lines: the class declaration followed by 87
method bodies covering training, labelling, feature extraction, persistence,
chart drawing, online learning, the GA auto-tuner and inference, all in one
file. Train() alone is 1492 lines; a change to arrow drawing meant scrolling
past the era loop.
Moved the bodies into Expert\AIBase\, included at the bottom of the original
after the class declaration:
Training.mqh 1607 era loop, plateau ladder, checkpoint select, deploy
Features.mqh 1093 indicator creation + per-bar input feature vector
ChartUI.mqh 634 arrows, arrow persistence, status panel, cleanup
Persistence.mqh 492 .stats/.cfg sidecars, CPU-inference validation, copy
OnlineLearning.mqh 461 live continual learning, EMA shadow, OOS simulator
Labels.mqh 309 ZigZag pivot labels, async label-cache prebuild
AutoTune.mqh 275 genetic tuner (population, crossover, halving)
Inference.mqh 235 softmax, prior calibration, class priors
ExpertSignalAIBase.mqh 8216 -> 3131 (declaration + topology build only)
This is a pure relocation - verified mechanically, not by eye: HEAD's file
reconstructed from the eight partials plus the surviving remainder is
byte-identical to HEAD, span for span (scratchpad verify_split.py). No
declaration moved, no signature changed, no code rewritten, so behaviour is
unchanged by construction.
Compiles 0 errors, 0 warnings.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-29 00:42:45 -04:00
|
|
|
//+------------------------------------------------------------------+
|
|
|
|
|
//| Warrior_EA |
|
|
|
|
|
//| AnimateDread |
|
|
|
|
|
//| |
|
|
|
|
|
//| Indicator creation and the per-bar input feature vector. |
|
|
|
|
|
//| |
|
|
|
|
|
//| PARTIAL IMPLEMENTATION FILE - not standalone. |
|
|
|
|
|
//| This holds CExpertSignalAIBase method BODIES only. The class |
|
|
|
|
|
//| declaration lives in Expert\ExpertSignalAIBase.mqh, which |
|
|
|
|
|
//| #includes this file at the bottom, after the declaration. Do not |
|
|
|
|
|
//| include it anywhere else and do not compile it on its own. |
|
|
|
|
|
//| |
|
|
|
|
|
//| Split out purely to make the 8216-line original navigable; the |
|
|
|
|
|
//| code inside was moved verbatim, not rewritten. |
|
|
|
|
|
//+------------------------------------------------------------------+
|
|
|
|
|
#ifndef WARRIOR_AIBASE_FEATURES_MQH
|
|
|
|
|
#define WARRIOR_AIBASE_FEATURES_MQH
|
2026-08-02 01:09:18 -04:00
|
|
|
//--- Plausibility ceiling for any single input value, enforced once over the whole bar at the end of
|
|
|
|
|
//--- BufferTempDataCompute(). Deliberately far above every clamp used inside that function (the widest
|
|
|
|
|
//--- is +/-10) - this is not a normalization knob, it is the "no legitimate feature looks like this"
|
|
|
|
|
//--- line. See the sanitize loop at the end of BufferTempDataCompute() for what it protects.
|
|
|
|
|
#define FEATURE_ABS_MAX 1.0e4
|
2026-07-29 00:46:01 -04:00
|
|
|
//+------------------------------------------------------------------+
|
refactor: split CExpertSignalAIBase implementation by responsibility
ExpertSignalAIBase.mqh was 8216 lines: the class declaration followed by 87
method bodies covering training, labelling, feature extraction, persistence,
chart drawing, online learning, the GA auto-tuner and inference, all in one
file. Train() alone is 1492 lines; a change to arrow drawing meant scrolling
past the era loop.
Moved the bodies into Expert\AIBase\, included at the bottom of the original
after the class declaration:
Training.mqh 1607 era loop, plateau ladder, checkpoint select, deploy
Features.mqh 1093 indicator creation + per-bar input feature vector
ChartUI.mqh 634 arrows, arrow persistence, status panel, cleanup
Persistence.mqh 492 .stats/.cfg sidecars, CPU-inference validation, copy
OnlineLearning.mqh 461 live continual learning, EMA shadow, OOS simulator
Labels.mqh 309 ZigZag pivot labels, async label-cache prebuild
AutoTune.mqh 275 genetic tuner (population, crossover, halving)
Inference.mqh 235 softmax, prior calibration, class priors
ExpertSignalAIBase.mqh 8216 -> 3131 (declaration + topology build only)
This is a pure relocation - verified mechanically, not by eye: HEAD's file
reconstructed from the eight partials plus the surviving remainder is
byte-identical to HEAD, span for span (scratchpad verify_split.py). No
declaration moved, no signature changed, no code rewritten, so behaviour is
unchanged by construction.
Compiles 0 errors, 0 warnings.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-29 00:42:45 -04:00
|
|
|
//| Rebuilds only the enabled AD* CiCustom handles in place, so a new |
|
|
|
|
|
//| trial's member-struct param values take effect. Re-Create()-ing |
|
|
|
|
|
//| the existing CiCustom object (rather than removing/re-adding it |
|
|
|
|
|
//| to indicators) avoids adding the same pointer into the CIndicators|
|
|
|
|
|
//| collection twice, which would risk it being deleted twice on |
|
|
|
|
|
//| teardown - MQL5's CIndicators has no documented single-item |
|
fix: the indicator re-init leaked a terminal handle per candidate
This is what killed CONV and LSTM on 2026-08-07. Terminal journal:
19:19:40 6664 x "VirtualAlloc failed in large allocator"
19:19:40.829 expert Warrior_EA (SP500,H1) removed <- CONV
19:29:55 2048 x "VirtualAlloc failed in large allocator"
19:29:55.359 expert Warrior_EA (SP500,H1) removed <- LSTM
50ms and 71ms after each printed its "logit adjustment" line, i.e. the
instant era 0 tried to allocate its training queues. They did not hang -
MT5 shot them for running out of memory.
ReInitADIndicators() re-Create()s every enabled indicator and released
nothing. The comment above it asserted "CiCustom.Create() already
releases its old handle"; MQL5's CIndicator::Create is
m_handle = IndicatorCreate(symbol, period, type, num_params, params);
a plain overwrite, whose only success-path IndicatorRelease is in
~CIndicator. IndicatorRelease appeared nowhere in this codebase.
That function is the indicator tuner's inner loop. AutoTuneIndicators
scored 324 candidates per model on SP500 H1, so ~324 x 6 orphaned
terminal-side instances, each holding a full-history buffer set -
ADWyckoffEventStream is 14 buffers x ~38k bars x 8 bytes = ~4.3 MB each.
Gigabytes. Confirmed in the shutdown teardown, where a single surviving
expert still held 70 x ADWES, 53 x ADWyckoffEventStream, 23 x
WFS(48,3,1.80,1.10), 19 x ADMovingAverage, 18 x WYSB(162). Those
parameter sets are the tuner's candidate grids.
Release is UNCONDITIONAL, not gated on the handle having changed: MT5
refcounts instances by (symbol, period, params), so re-creating with
IDENTICAL params returns the SAME handle with the count incremented -
the "23 x WFS(48,3,1.80,1.10)" pattern. Either way Create() added one
reference and we hold one handle, so one release is owed.
Released AFTER the re-creates, never before: dropping the terminal's
last reference first would tear the instance down, so an identical-params
Create() would rebuild it from scratch instead of reusing the live one -
turning a refcount bump into a full recalculation over all history, 324
times over.
PAI was unaffected because it has no AD/Wyckoff indicators enabled (35
candidates, built-ins only). HYBRID survived on luck: CONV and LSTM died
2 and 12 minutes before its own sweep finished, freeing the memory.
Compiles clean: 0 errors, 0 warnings.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-08 15:42:03 -04:00
|
|
|
//| remove. |
|
|
|
|
|
//| |
|
|
|
|
|
//| This used to end "...and CiCustom.Create() already releases its |
|
|
|
|
|
//| old handle." IT DOES NOT, and that sentence cost two models. See |
|
|
|
|
|
//| the handle-release block in the definition below. |
|
refactor: split CExpertSignalAIBase implementation by responsibility
ExpertSignalAIBase.mqh was 8216 lines: the class declaration followed by 87
method bodies covering training, labelling, feature extraction, persistence,
chart drawing, online learning, the GA auto-tuner and inference, all in one
file. Train() alone is 1492 lines; a change to arrow drawing meant scrolling
past the era loop.
Moved the bodies into Expert\AIBase\, included at the bottom of the original
after the class declaration:
Training.mqh 1607 era loop, plateau ladder, checkpoint select, deploy
Features.mqh 1093 indicator creation + per-bar input feature vector
ChartUI.mqh 634 arrows, arrow persistence, status panel, cleanup
Persistence.mqh 492 .stats/.cfg sidecars, CPU-inference validation, copy
OnlineLearning.mqh 461 live continual learning, EMA shadow, OOS simulator
Labels.mqh 309 ZigZag pivot labels, async label-cache prebuild
AutoTune.mqh 275 genetic tuner (population, crossover, halving)
Inference.mqh 235 softmax, prior calibration, class priors
ExpertSignalAIBase.mqh 8216 -> 3131 (declaration + topology build only)
This is a pure relocation - verified mechanically, not by eye: HEAD's file
reconstructed from the eight partials plus the surviving remainder is
byte-identical to HEAD, span for span (scratchpad verify_split.py). No
declaration moved, no signature changed, no code rewritten, so behaviour is
unchanged by construction.
Compiles 0 errors, 0 warnings.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-29 00:42:45 -04:00
|
|
|
//+------------------------------------------------------------------+
|
fix: make the indicator tuner actually measure, and gate what it installs
ROOT CAUSE of the zero spread measured on SP500 H1 2026-08-07 (all 17 candidates
returned exactly 0.00359 nats): the tune loop re-inits the indicators and then
scores, with no RefreshData() between.
ReInitADIndicators() does its part - Create() builds a NEW handle carrying the
new parameters, and the feature cache is flagged stale so features really are
recomputed. But BufferTempDataCompute() reads the CIndicatorBuffer objects, and
only Refresh() copies data out of a handle into those. So every candidate was
scored on values still held from the PREVIOUS handle. My earlier guess in the
diagnostic ("suspect the feature cache") was wrong: the cache invalidation works.
Two things land together, because neither is safe alone:
1. RefreshData() after the re-init, so a candidate is scored on its own features.
2. A SELECTION GATE on the install. bestScore is a MAXIMUM over candidates, and
the maximum of N draws from a null beats its incumbent almost every time - so
"it beat the incumbent" installs noise. This selector is the highest-stakes of
the three found in this audit because it ACTS: it overwrites the user's
configured indicator settings and forces BuildFreshTopology(), so the network
then trains on whatever the noise picked. Fixing (1) without (2) would have
made a dormant bug actively harmful.
The gate draws the winner's own permutation null once, then corrects the p-value
for having chosen it out of N with Sidak: p_family = 1 - (1-p)^N. Sidak rather
than the max-of-N resample used by the geometry scan because each candidate here
has a DIFFERENT feature set, so their draws cannot be pooled; Sidak needs only
the one null. Exact under independence, mildly anti-conservative under positive
dependence - stated in the comment rather than hidden. A rejected winner restores
the configured settings, which best[] cannot do since the descent mutates it.
Also reports the least-ready tunable handle's BarsCalculated(). IndicatorCreate()
calculates asynchronously, so if the spread is STILL zero the handles simply are
not done and the tuner needs to yield between candidates rather than score them
back to back - a state machine like the label prebuild. That distinction is now
readable from the log instead of requiring another guess.
No input, topology or label change: no retrain.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-07 09:31:06 -04:00
|
|
|
//+------------------------------------------------------------------+
|
|
|
|
|
//| See the declaration. Minimum over the enabled tunable indicators, |
|
|
|
|
|
//| because the feature vector is only as ready as its least-ready |
|
|
|
|
|
//| component; -1 when nothing tunable is switched on. |
|
|
|
|
|
//+------------------------------------------------------------------+
|
|
|
|
|
int CExpertSignalAIBase::TunableBarsCalculated(void)
|
|
|
|
|
{
|
|
|
|
|
int worst = INT_MAX;
|
|
|
|
|
if(m_useMA)
|
|
|
|
|
worst = (int)MathMin(worst, m_MA.BarsCalculated());
|
|
|
|
|
if(m_useRSI)
|
|
|
|
|
worst = (int)MathMin(worst, m_RSI.BarsCalculated());
|
|
|
|
|
if(m_useMACD)
|
|
|
|
|
worst = (int)MathMin(worst, m_MACDFeature.BarsCalculated());
|
|
|
|
|
if(m_useIchimoku)
|
|
|
|
|
worst = (int)MathMin(worst, m_Ichimoku.BarsCalculated());
|
|
|
|
|
if(m_useADCumulativeDelta)
|
|
|
|
|
worst = (int)MathMin(worst, m_ADCumulativeDelta.BarsCalculated());
|
|
|
|
|
if(m_useADShorteningOfThrust)
|
|
|
|
|
worst = (int)MathMin(worst, m_ADShorteningOfThrust.BarsCalculated());
|
|
|
|
|
if(m_useADWyckoffEventStream)
|
|
|
|
|
worst = (int)MathMin(worst, m_ADWyckoffEventStream.BarsCalculated());
|
|
|
|
|
if(m_useADWyckoffFailedStructure)
|
|
|
|
|
worst = (int)MathMin(worst, m_ADWyckoffFailedStructure.BarsCalculated());
|
|
|
|
|
if(m_useADWyckoffSignificantBarInversion)
|
|
|
|
|
worst = (int)MathMin(worst, m_ADWyckoffSignificantBarInversion.BarsCalculated());
|
|
|
|
|
return (worst == INT_MAX) ? -1 : worst;
|
|
|
|
|
}
|
|
|
|
|
//+------------------------------------------------------------------+
|
2026-08-13 10:23:11 -04:00
|
|
|
//+------------------------------------------------------------------+
|
|
|
|
|
//| Adopt a saved indicator-param set, rebuilding handles only on a |
|
|
|
|
|
//| REAL change. |
|
|
|
|
|
//| |
|
|
|
|
|
//| The resume path restores the params a model was trained with and |
|
|
|
|
|
//| used to call ReInitADIndicators unconditionally. In the common |
|
|
|
|
|
//| case the saved set is byte-identical to the values the indicators |
|
|
|
|
|
//| were created with a few hundred milliseconds earlier (the MI |
|
|
|
|
|
//| tuner usually keeps the configured settings), so the "rebuild" |
|
|
|
|
|
//| destroyed five working, already-calculating indicator instances |
|
|
|
|
|
//| to recreate them with the same inputs - at process start, with |
|
|
|
|
|
//| history still syncing. On a memory-starved box (2026-08-13: |
|
|
|
|
|
//| 1 GB free of 31) the replacements stayed cold for 6+ minutes and |
|
|
|
|
|
//| the resumed model could not train a single era. A no-change adopt |
|
|
|
|
|
//| now only aligns the tuner state and leaves the live instances |
|
|
|
|
|
//| alone. |
|
|
|
|
|
//+------------------------------------------------------------------+
|
|
|
|
|
bool CExpertSignalAIBase::AdoptIndicatorParams(const double &loaded[], CIndicators *indicators)
|
|
|
|
|
{
|
|
|
|
|
double current[];
|
|
|
|
|
m_indicatorTuner.Flatten(current);
|
|
|
|
|
bool changed = (ArraySize(current) != ArraySize(loaded));
|
|
|
|
|
if(!changed)
|
|
|
|
|
for(int k = 0; k < ArraySize(loaded); k++)
|
|
|
|
|
if(current[k] != loaded[k])
|
|
|
|
|
{
|
|
|
|
|
changed = true;
|
|
|
|
|
break;
|
|
|
|
|
}
|
|
|
|
|
//--- the tuner mirrors the model's params either way - it feeds the .nnw save and the fingerprint
|
|
|
|
|
m_indicatorTuner.Unflatten(loaded);
|
|
|
|
|
if(!changed)
|
|
|
|
|
{
|
|
|
|
|
PrintVerbose(ID + ": saved indicator params match the live indicators - keeping the existing"
|
|
|
|
|
" instances (no handle rebuild).");
|
|
|
|
|
return true;
|
|
|
|
|
}
|
|
|
|
|
Print(ID + ": saved indicator params differ from the live defaults - rebuilding the tunable"
|
|
|
|
|
" indicator handles to match the model they trained.");
|
|
|
|
|
return ReInitADIndicators(indicators);
|
|
|
|
|
}
|
|
|
|
|
//+------------------------------------------------------------------+
|
refactor: split CExpertSignalAIBase implementation by responsibility
ExpertSignalAIBase.mqh was 8216 lines: the class declaration followed by 87
method bodies covering training, labelling, feature extraction, persistence,
chart drawing, online learning, the GA auto-tuner and inference, all in one
file. Train() alone is 1492 lines; a change to arrow drawing meant scrolling
past the era loop.
Moved the bodies into Expert\AIBase\, included at the bottom of the original
after the class declaration:
Training.mqh 1607 era loop, plateau ladder, checkpoint select, deploy
Features.mqh 1093 indicator creation + per-bar input feature vector
ChartUI.mqh 634 arrows, arrow persistence, status panel, cleanup
Persistence.mqh 492 .stats/.cfg sidecars, CPU-inference validation, copy
OnlineLearning.mqh 461 live continual learning, EMA shadow, OOS simulator
Labels.mqh 309 ZigZag pivot labels, async label-cache prebuild
AutoTune.mqh 275 genetic tuner (population, crossover, halving)
Inference.mqh 235 softmax, prior calibration, class priors
ExpertSignalAIBase.mqh 8216 -> 3131 (declaration + topology build only)
This is a pure relocation - verified mechanically, not by eye: HEAD's file
reconstructed from the eight partials plus the surviving remainder is
byte-identical to HEAD, span for span (scratchpad verify_split.py). No
declaration moved, no signature changed, no code rewritten, so behaviour is
unchanged by construction.
Compiles 0 errors, 0 warnings.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-29 00:42:45 -04:00
|
|
|
bool CExpertSignalAIBase::ReInitADIndicators(CIndicators *indicators)
|
|
|
|
|
{
|
|
|
|
|
bool result = true;
|
fix: the indicator re-init leaked a terminal handle per candidate
This is what killed CONV and LSTM on 2026-08-07. Terminal journal:
19:19:40 6664 x "VirtualAlloc failed in large allocator"
19:19:40.829 expert Warrior_EA (SP500,H1) removed <- CONV
19:29:55 2048 x "VirtualAlloc failed in large allocator"
19:29:55.359 expert Warrior_EA (SP500,H1) removed <- LSTM
50ms and 71ms after each printed its "logit adjustment" line, i.e. the
instant era 0 tried to allocate its training queues. They did not hang -
MT5 shot them for running out of memory.
ReInitADIndicators() re-Create()s every enabled indicator and released
nothing. The comment above it asserted "CiCustom.Create() already
releases its old handle"; MQL5's CIndicator::Create is
m_handle = IndicatorCreate(symbol, period, type, num_params, params);
a plain overwrite, whose only success-path IndicatorRelease is in
~CIndicator. IndicatorRelease appeared nowhere in this codebase.
That function is the indicator tuner's inner loop. AutoTuneIndicators
scored 324 candidates per model on SP500 H1, so ~324 x 6 orphaned
terminal-side instances, each holding a full-history buffer set -
ADWyckoffEventStream is 14 buffers x ~38k bars x 8 bytes = ~4.3 MB each.
Gigabytes. Confirmed in the shutdown teardown, where a single surviving
expert still held 70 x ADWES, 53 x ADWyckoffEventStream, 23 x
WFS(48,3,1.80,1.10), 19 x ADMovingAverage, 18 x WYSB(162). Those
parameter sets are the tuner's candidate grids.
Release is UNCONDITIONAL, not gated on the handle having changed: MT5
refcounts instances by (symbol, period, params), so re-creating with
IDENTICAL params returns the SAME handle with the count incremented -
the "23 x WFS(48,3,1.80,1.10)" pattern. Either way Create() added one
reference and we hold one handle, so one release is owed.
Released AFTER the re-creates, never before: dropping the terminal's
last reference first would tear the instance down, so an identical-params
Create() would rebuild it from scratch instead of reusing the live one -
turning a refcount bump into a full recalculation over all history, 324
times over.
PAI was unaffected because it has no AD/Wyckoff indicators enabled (35
candidates, built-ins only). HYBRID survived on luck: CONV and LSTM died
2 and 12 minutes before its own sweep finished, freeing the memory.
Compiles clean: 0 errors, 0 warnings.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-08 15:42:03 -04:00
|
|
|
//--- RELEASE THE HANDLE EACH Create() IS ABOUT TO REPLACE. Without this every call here leaks one
|
|
|
|
|
//--- terminal-side indicator instance per enabled indicator, and this function is the tuner's inner
|
|
|
|
|
//--- loop - AutoTuneIndicators scored 324 candidates on SP500 H1, so ~324 x 6 orphaned instances per
|
|
|
|
|
//--- model, each holding a full-history buffer set (ADWyckoffEventStream is 14 buffers x ~38k bars x
|
|
|
|
|
//--- 8 bytes = ~4.3 MB EACH). That is gigabytes, and it is what killed CONV and LSTM on 2026-08-07:
|
|
|
|
|
//--- 6664 and 2048 "VirtualAlloc failed in large allocator" lines in the terminal journal, then
|
|
|
|
|
//--- "expert Warrior_EA (SP500,H1) removed", 50ms and 71ms after each finished its sweep and era 0
|
|
|
|
|
//--- tried to allocate. HYBRID only survived because those two died first and freed the memory.
|
|
|
|
|
//---
|
|
|
|
|
//--- THE COMMENT THAT USED TO SIT HERE SAID Create() "already releases its old handle". It does not.
|
|
|
|
|
//--- MQL5's CIndicator::Create (Include\Indicators\Indicator.mqh) is:
|
|
|
|
|
//--- m_handle = IndicatorCreate(symbol, period, type, num_params, params);
|
|
|
|
|
//--- - a plain overwrite. Its only success-path IndicatorRelease is in ~CIndicator. Nothing else in
|
|
|
|
|
//--- this codebase called IndicatorRelease at all.
|
|
|
|
|
//---
|
|
|
|
|
//--- UNCONDITIONAL, not "only when the handle changed". MT5 refcounts indicator instances by
|
|
|
|
|
//--- (symbol, period, params): re-creating with IDENTICAL params hands back the SAME handle with the
|
|
|
|
|
//--- count incremented, so skipping the release there would leak a reference just as surely - which is
|
|
|
|
|
//--- the "23 x WFS(48,3,1.80,1.10)" pattern in the journal, next to the distinct-parameter leaks from
|
|
|
|
|
//--- the candidate grid. Either way Create() added exactly one reference and we still hold exactly one
|
|
|
|
|
//--- handle, so exactly one release is owed.
|
|
|
|
|
int hCD = m_useADCumulativeDelta ? m_ADCumulativeDelta.Handle() : INVALID_HANDLE;
|
|
|
|
|
int hSOT = m_useADShorteningOfThrust ? m_ADShorteningOfThrust.Handle() : INVALID_HANDLE;
|
|
|
|
|
int hWES = m_useADWyckoffEventStream ? m_ADWyckoffEventStream.Handle() : INVALID_HANDLE;
|
|
|
|
|
int hWFS = m_useADWyckoffFailedStructure ? m_ADWyckoffFailedStructure.Handle() : INVALID_HANDLE;
|
|
|
|
|
int hWSBI = m_useADWyckoffSignificantBarInversion ? m_ADWyckoffSignificantBarInversion.Handle() : INVALID_HANDLE;
|
|
|
|
|
int hMA = m_useMA ? m_MA.Handle() : INVALID_HANDLE;
|
|
|
|
|
int hRSI = m_useRSI ? m_RSI.Handle() : INVALID_HANDLE;
|
|
|
|
|
int hMACD = m_useMACD ? m_MACDFeature.Handle() : INVALID_HANDLE;
|
|
|
|
|
int hIchi = m_useIchimoku ? m_Ichimoku.Handle() : INVALID_HANDLE;
|
refactor: split CExpertSignalAIBase implementation by responsibility
ExpertSignalAIBase.mqh was 8216 lines: the class declaration followed by 87
method bodies covering training, labelling, feature extraction, persistence,
chart drawing, online learning, the GA auto-tuner and inference, all in one
file. Train() alone is 1492 lines; a change to arrow drawing meant scrolling
past the era loop.
Moved the bodies into Expert\AIBase\, included at the bottom of the original
after the class declaration:
Training.mqh 1607 era loop, plateau ladder, checkpoint select, deploy
Features.mqh 1093 indicator creation + per-bar input feature vector
ChartUI.mqh 634 arrows, arrow persistence, status panel, cleanup
Persistence.mqh 492 .stats/.cfg sidecars, CPU-inference validation, copy
OnlineLearning.mqh 461 live continual learning, EMA shadow, OOS simulator
Labels.mqh 309 ZigZag pivot labels, async label-cache prebuild
AutoTune.mqh 275 genetic tuner (population, crossover, halving)
Inference.mqh 235 softmax, prior calibration, class priors
ExpertSignalAIBase.mqh 8216 -> 3131 (declaration + topology build only)
This is a pure relocation - verified mechanically, not by eye: HEAD's file
reconstructed from the eight partials plus the surviving remainder is
byte-identical to HEAD, span for span (scratchpad verify_split.py). No
declaration moved, no signature changed, no code rewritten, so behaviour is
unchanged by construction.
Compiles 0 errors, 0 warnings.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-29 00:42:45 -04:00
|
|
|
if(m_useADCumulativeDelta)
|
|
|
|
|
result = InitADCumulativeDelta(indicators, false) && result;
|
|
|
|
|
if(m_useADShorteningOfThrust)
|
|
|
|
|
result = InitADShorteningOfThrust(indicators, false) && result;
|
|
|
|
|
if(m_useADWyckoffEventStream)
|
|
|
|
|
result = InitADWyckoffEventStream(indicators, false) && result;
|
|
|
|
|
if(m_useADWyckoffFailedStructure)
|
|
|
|
|
result = InitADWyckoffFailedStructure(indicators, false) && result;
|
|
|
|
|
if(m_useADWyckoffSignificantBarInversion)
|
|
|
|
|
result = InitADWyckoffSignificantBarInversion(indicators, false) && result;
|
|
|
|
|
if(m_useMA)
|
|
|
|
|
result = InitMA(indicators, false) && result;
|
|
|
|
|
if(m_useRSI)
|
|
|
|
|
result = InitRSI(indicators, false) && result;
|
|
|
|
|
if(m_useMACD)
|
|
|
|
|
result = InitMACDFeature(indicators, false) && result;
|
|
|
|
|
if(m_useIchimoku)
|
|
|
|
|
result = InitIchimoku(indicators, false) && result;
|
fix: the indicator re-init leaked a terminal handle per candidate
This is what killed CONV and LSTM on 2026-08-07. Terminal journal:
19:19:40 6664 x "VirtualAlloc failed in large allocator"
19:19:40.829 expert Warrior_EA (SP500,H1) removed <- CONV
19:29:55 2048 x "VirtualAlloc failed in large allocator"
19:29:55.359 expert Warrior_EA (SP500,H1) removed <- LSTM
50ms and 71ms after each printed its "logit adjustment" line, i.e. the
instant era 0 tried to allocate its training queues. They did not hang -
MT5 shot them for running out of memory.
ReInitADIndicators() re-Create()s every enabled indicator and released
nothing. The comment above it asserted "CiCustom.Create() already
releases its old handle"; MQL5's CIndicator::Create is
m_handle = IndicatorCreate(symbol, period, type, num_params, params);
a plain overwrite, whose only success-path IndicatorRelease is in
~CIndicator. IndicatorRelease appeared nowhere in this codebase.
That function is the indicator tuner's inner loop. AutoTuneIndicators
scored 324 candidates per model on SP500 H1, so ~324 x 6 orphaned
terminal-side instances, each holding a full-history buffer set -
ADWyckoffEventStream is 14 buffers x ~38k bars x 8 bytes = ~4.3 MB each.
Gigabytes. Confirmed in the shutdown teardown, where a single surviving
expert still held 70 x ADWES, 53 x ADWyckoffEventStream, 23 x
WFS(48,3,1.80,1.10), 19 x ADMovingAverage, 18 x WYSB(162). Those
parameter sets are the tuner's candidate grids.
Release is UNCONDITIONAL, not gated on the handle having changed: MT5
refcounts instances by (symbol, period, params), so re-creating with
IDENTICAL params returns the SAME handle with the count incremented -
the "23 x WFS(48,3,1.80,1.10)" pattern. Either way Create() added one
reference and we hold one handle, so one release is owed.
Released AFTER the re-creates, never before: dropping the terminal's
last reference first would tear the instance down, so an identical-params
Create() would rebuild it from scratch instead of reusing the live one -
turning a refcount bump into a full recalculation over all history, 324
times over.
PAI was unaffected because it has no AD/Wyckoff indicators enabled (35
candidates, built-ins only). HYBRID survived on luck: CONV and LSTM died
2 and 12 minutes before its own sweep finished, freeing the memory.
Compiles clean: 0 errors, 0 warnings.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-08 15:42:03 -04:00
|
|
|
//--- AFTER the re-creates, never before: releasing first can drop the terminal's last reference and
|
|
|
|
|
//--- make it tear the instance down, so an identical-params Create() would then rebuild it from
|
|
|
|
|
//--- scratch instead of re-using the live one - turning a refcount bump into a full recalculation over
|
|
|
|
|
//--- the whole history, 324 times over. Released here, the old instance survives until its
|
|
|
|
|
//--- replacement exists.
|
|
|
|
|
if(hCD != INVALID_HANDLE)
|
|
|
|
|
IndicatorRelease(hCD);
|
|
|
|
|
if(hSOT != INVALID_HANDLE)
|
|
|
|
|
IndicatorRelease(hSOT);
|
|
|
|
|
if(hWES != INVALID_HANDLE)
|
|
|
|
|
IndicatorRelease(hWES);
|
|
|
|
|
if(hWFS != INVALID_HANDLE)
|
|
|
|
|
IndicatorRelease(hWFS);
|
|
|
|
|
if(hWSBI != INVALID_HANDLE)
|
|
|
|
|
IndicatorRelease(hWSBI);
|
|
|
|
|
if(hMA != INVALID_HANDLE)
|
|
|
|
|
IndicatorRelease(hMA);
|
|
|
|
|
if(hRSI != INVALID_HANDLE)
|
|
|
|
|
IndicatorRelease(hRSI);
|
|
|
|
|
if(hMACD != INVALID_HANDLE)
|
|
|
|
|
IndicatorRelease(hMACD);
|
|
|
|
|
if(hIchi != INVALID_HANDLE)
|
|
|
|
|
IndicatorRelease(hIchi);
|
refactor: split CExpertSignalAIBase implementation by responsibility
ExpertSignalAIBase.mqh was 8216 lines: the class declaration followed by 87
method bodies covering training, labelling, feature extraction, persistence,
chart drawing, online learning, the GA auto-tuner and inference, all in one
file. Train() alone is 1492 lines; a change to arrow drawing meant scrolling
past the era loop.
Moved the bodies into Expert\AIBase\, included at the bottom of the original
after the class declaration:
Training.mqh 1607 era loop, plateau ladder, checkpoint select, deploy
Features.mqh 1093 indicator creation + per-bar input feature vector
ChartUI.mqh 634 arrows, arrow persistence, status panel, cleanup
Persistence.mqh 492 .stats/.cfg sidecars, CPU-inference validation, copy
OnlineLearning.mqh 461 live continual learning, EMA shadow, OOS simulator
Labels.mqh 309 ZigZag pivot labels, async label-cache prebuild
AutoTune.mqh 275 genetic tuner (population, crossover, halving)
Inference.mqh 235 softmax, prior calibration, class priors
ExpertSignalAIBase.mqh 8216 -> 3131 (declaration + topology build only)
This is a pure relocation - verified mechanically, not by eye: HEAD's file
reconstructed from the eight partials plus the surviving remainder is
byte-identical to HEAD, span for span (scratchpad verify_split.py). No
declaration moved, no signature changed, no code rewritten, so behaviour is
unchanged by construction.
Compiles 0 errors, 0 warnings.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-29 00:42:45 -04:00
|
|
|
//--- Indicator params just changed, so every cached feature row is now stale (the feature values
|
|
|
|
|
//--- depend on these indicators; the LABELS do not - they come from ADZigZag - so the label cache is
|
|
|
|
|
//--- deliberately left intact and reused). Without this, a tuner candidate would silently train and be
|
|
|
|
|
//--- scored on the PREVIOUS candidate's features. Cheap: just flags rows for lazy recompute on next read.
|
|
|
|
|
ArrayInitialize(m_featureCacheHasValue, false);
|
|
|
|
|
return result;
|
|
|
|
|
}
|
|
|
|
|
//+------------------------------------------------------------------+
|
|
|
|
|
//| |
|
|
|
|
|
//+------------------------------------------------------------------+
|
|
|
|
|
bool CExpertSignalAIBase::ResizeBuffers(int barIndex)
|
|
|
|
|
{
|
|
|
|
|
// The Ichimoku feature's Chikou term reads m_Close at idx + ichiKijun (see its block in
|
|
|
|
|
// BufferTempDataCompute() for why that direction, and only that direction, is lookahead-free), which
|
|
|
|
|
// is further back than any other consumer of the close series reaches. Grow the close buffer to match
|
|
|
|
|
// when that feature is on, so the oldest requested bars resolve from real data instead of tripping
|
|
|
|
|
// that block's EMPTY_VALUE guard and being rejected as unusable examples.
|
|
|
|
|
int closeBars = m_useIchimoku ? barIndex + m_indicatorTuner.ichiKijun : barIndex;
|
|
|
|
|
if(!m_Open.BufferResize(barIndex) || !m_Close.BufferResize(closeBars) || !m_High.BufferResize(barIndex) || !m_Low.BufferResize(barIndex))
|
|
|
|
|
return false;
|
|
|
|
|
if(m_useVolumes)
|
|
|
|
|
{
|
|
|
|
|
if(!m_Volumes.BufferResize(barIndex))
|
|
|
|
|
return false;
|
|
|
|
|
}
|
|
|
|
|
// Unconditional - see InitTime()'s call site in InitIndicators() for why m_Time must always be live.
|
|
|
|
|
if(!m_Time.BufferResize(barIndex))
|
|
|
|
|
return false;
|
|
|
|
|
if(m_useMA)
|
|
|
|
|
{
|
|
|
|
|
if(!m_MA.BufferResize(barIndex))
|
|
|
|
|
return false;
|
|
|
|
|
}
|
|
|
|
|
if(m_useRSI)
|
|
|
|
|
{
|
|
|
|
|
if(!m_RSI.BufferResize(barIndex))
|
|
|
|
|
return false;
|
|
|
|
|
}
|
|
|
|
|
if(m_useMACD)
|
|
|
|
|
{
|
|
|
|
|
if(!m_MACDFeature.BufferResize(barIndex))
|
|
|
|
|
return false;
|
|
|
|
|
}
|
|
|
|
|
if(m_useIchimoku)
|
|
|
|
|
{
|
|
|
|
|
// + m_indicatorTuner.ichiKijun: the cloud reads reach that many bars FURTHER back than every other
|
|
|
|
|
// indicator here does (see the m_useIchimoku feature block for why the offset exists), so sizing
|
|
|
|
|
// this buffer to barIndex alone would leave the oldest requested bars' cloud values unavailable.
|
|
|
|
|
if(!m_Ichimoku.BufferResize(barIndex + m_indicatorTuner.ichiKijun))
|
|
|
|
|
return false;
|
|
|
|
|
}
|
|
|
|
|
// Unconditional (not gated by m_useATR): the ATR-normalization in BufferTempData() reads
|
|
|
|
|
// m_ATR.Main() regardless of whether ATR is enabled as an explicit extra input feature -
|
|
|
|
|
// m_useATR only controls that feature-count opt-in (see InitIndicators()'s "already init in the
|
|
|
|
|
// base class" comment), not whether ATR data itself needs to be kept live.
|
|
|
|
|
if(!m_ATR.BufferResize(barIndex))
|
|
|
|
|
return false;
|
feat(ai): triple-barrier labels replace exact-pivot ZigZag targets
The 31:1 class imbalance was self-inflicted by the TARGET, not a property
of the market. Labelling only the exact bar where a ZigZag pivot confirms
gave Buy 1164 / Sell 1164 / Neutral 35841, and every correction mechanism
this codebase accumulated sits downstream of that one choice: the
logit-adjusted loss and its range cap, the prior EMA, the +-3.0 output-bias
seed, balanced-accuracy-then-precision selection with its coverage floor,
the recall floor and its catch-22, the alternation gate, NMS, and the four
oversampling designs that collapsed before them.
The reference this engine is built on (references/neuronetworksbook.pdf
ch. 3.1/3.3) also uses ZigZag, but targets the DIRECTION TO THE NEXT
EXTREMUM on every bar - ~50/50 by construction, with no imbalance to
correct at all. It never had this problem because it never asked "is this
the pivot bar".
Labels are now the triple barrier (Lopez de Prado ch. 3), using the EA's
OWN SL_Mode/TP_Mode: does a trade opened at this bar's close reach its
target before its stop, within a horizon. Buy = long resolves, Sell =
short resolves, Neutral = neither. Consequences:
- dir-precision in the era line stops being a proxy and becomes the win
rate of the strategy under its own exit rules.
- Expected balance ~25/25/50 at the shipped 1:3 (gambler's ruin), i.e.
~2:1 instead of 31:1. Measured and logged at the end of the prebuild.
- Spread is charged on both legs, so it is a NET win rate.
- Intrabar ambiguity resolves to the STOP. OHLC cannot order two touches
inside one bar and the optimistic reading is how a backtested edge
becomes a live loss.
ZigZag stays as input features (EnableSwingContext) and now also supplies
the vertical barrier: the horizon is the median confirmed leg length,
snapped to a coarse ladder. Derived, not configured, and deliberately kept
out of the filename fingerprint - a filename keyed on a measured quantity
orphans a trained model the moment the measurement moves.
Removed, because the premise died with the old target:
- the alternation gate. Correct for pivot labels (a ZigZag cannot emit two
same-type pivots in a row, so a repeat was provably a false fire), and
wrong for barrier labels, which answer each bar independently. It also
took its worst consequence with it: a one-sided model previously got ONE
trade per backtest, a hard blocker on marketplace validation.
- SignalClusterWindow now defaults off - it de-duplicated repeats that are
now real trades. Kept as an opt-in display control.
- LABEL_WINDOW_BARS, the pivot-widening pass, ConfirmedZigZagLabel.
- the era-0 output-bias seed now needs a genuinely dominant class (0.70)
rather than 0.40; at ~50% Neutral a +-3.0 seed is a distortion, not a
correction.
Also fixed, both found while wiring the above:
1. RefreshConvergedSignal sized its buffers from a date delta
(Bars(sym, period, dtStudied, TimeCurrent())). dtStudied is a training
watermark; in the tester it is loaded from a live-chart save AHEAD of
the simulated date, so the interval inverted, Bars() returned ~0, and
the buffer came out at exactly m_historyBars - deep enough for the OHLC
window and far too shallow for the Donchian-50 / 20-bar-return / SMA
extension behind it. Inference silently computed DIFFERENT features
from the ones training learned on, live as well as in the tester. Now
sized from what the feature builder actually needs.
2. The barrier horizon is resolved on the deployed path too. A deployed
model never enters Train(), so it never reached the prebuild, and
OnlineLearnStep reads the horizon as its confirmation delay - left at
the fallback it would have backpropped bars whose barriers had not
resolved. Silent lookahead in the one place that writes to a live model.
SL_Mode/TP_Mode join the weights fingerprint: they define the labels now,
so a model trained at 1:3 must never be silently reused at 1:1. This
re-keys every pre-existing model by design - none were trained on this task.
Inference census extended with the vote gate. LongCondition/ShortCondition
open with a readiness check the refresh counters never see; in the tester it
reduces to "the seeded _optcache.nnw must have LOADED", and if it did not,
every vote is hard-zeroed while the model still answers Buy. The old three
counters would have read that as "the model says Neutral" - false, and a
completely different fix. This is the leading candidate for the
zero-direction backtest and the census can now name it in one run.
Both builds compile 0 errors / 0 warnings. Forces a full retrain.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-31 20:39:49 -04:00
|
|
|
// Unconditional, same reasoning as m_ATR above - m_ADZigZag drives the swing-context features AND
|
|
|
|
|
// ComputeBarrierHorizonBars()'s measurement, not an opt-in feature, so it's never gated by an
|
|
|
|
|
// m_use* flag. (It was also the training-label source until the 2026-08-01 triple-barrier relabel.)
|
refactor: split CExpertSignalAIBase implementation by responsibility
ExpertSignalAIBase.mqh was 8216 lines: the class declaration followed by 87
method bodies covering training, labelling, feature extraction, persistence,
chart drawing, online learning, the GA auto-tuner and inference, all in one
file. Train() alone is 1492 lines; a change to arrow drawing meant scrolling
past the era loop.
Moved the bodies into Expert\AIBase\, included at the bottom of the original
after the class declaration:
Training.mqh 1607 era loop, plateau ladder, checkpoint select, deploy
Features.mqh 1093 indicator creation + per-bar input feature vector
ChartUI.mqh 634 arrows, arrow persistence, status panel, cleanup
Persistence.mqh 492 .stats/.cfg sidecars, CPU-inference validation, copy
OnlineLearning.mqh 461 live continual learning, EMA shadow, OOS simulator
Labels.mqh 309 ZigZag pivot labels, async label-cache prebuild
AutoTune.mqh 275 genetic tuner (population, crossover, halving)
Inference.mqh 235 softmax, prior calibration, class priors
ExpertSignalAIBase.mqh 8216 -> 3131 (declaration + topology build only)
This is a pure relocation - verified mechanically, not by eye: HEAD's file
reconstructed from the eight partials plus the surviving remainder is
byte-identical to HEAD, span for span (scratchpad verify_split.py). No
declaration moved, no signature changed, no code rewritten, so behaviour is
unchanged by construction.
Compiles 0 errors, 0 warnings.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-29 00:42:45 -04:00
|
|
|
if(!m_ADZigZag.BufferResize(barIndex))
|
|
|
|
|
return false;
|
|
|
|
|
if(m_useADCumulativeDelta)
|
|
|
|
|
{
|
|
|
|
|
if(!m_ADCumulativeDelta.BufferResize(barIndex))
|
|
|
|
|
return false;
|
|
|
|
|
}
|
|
|
|
|
if(m_useADShorteningOfThrust)
|
|
|
|
|
{
|
|
|
|
|
if(!m_ADShorteningOfThrust.BufferResize(barIndex))
|
|
|
|
|
return false;
|
|
|
|
|
}
|
|
|
|
|
if(m_useADWyckoffEventStream)
|
|
|
|
|
{
|
|
|
|
|
if(!m_ADWyckoffEventStream.BufferResize(barIndex))
|
|
|
|
|
return false;
|
|
|
|
|
}
|
|
|
|
|
if(m_useADWyckoffFailedStructure)
|
|
|
|
|
{
|
|
|
|
|
if(!m_ADWyckoffFailedStructure.BufferResize(barIndex))
|
|
|
|
|
return false;
|
|
|
|
|
}
|
|
|
|
|
if(m_useADWyckoffSignificantBarInversion)
|
|
|
|
|
{
|
|
|
|
|
if(!m_ADWyckoffSignificantBarInversion.BufferResize(barIndex))
|
|
|
|
|
return false;
|
|
|
|
|
}
|
|
|
|
|
return true;
|
|
|
|
|
}
|
|
|
|
|
//+------------------------------------------------------------------+
|
|
|
|
|
//| |
|
|
|
|
|
//+------------------------------------------------------------------+
|
|
|
|
|
bool CExpertSignalAIBase::RefreshData()
|
|
|
|
|
{
|
|
|
|
|
// CSeries/CIndicator::Refresh() is void - there is no per-call success/failure signal to
|
|
|
|
|
// propagate here. The real data-validity check happens downstream, per value, in
|
|
|
|
|
// BufferTempDataCompute() (EMPTY_VALUE / atr<=0 guards) - this function's job is only to ask
|
|
|
|
|
// every buffer to refresh, unconditionally, before that per-value check runs.
|
|
|
|
|
m_Open.Refresh(OBJ_ALL_PERIODS);
|
|
|
|
|
m_Close.Refresh(OBJ_ALL_PERIODS);
|
|
|
|
|
m_High.Refresh(OBJ_ALL_PERIODS);
|
|
|
|
|
m_Low.Refresh(OBJ_ALL_PERIODS);
|
|
|
|
|
if(m_useVolumes)
|
|
|
|
|
{
|
|
|
|
|
m_Volumes.Refresh(OBJ_ALL_PERIODS);
|
|
|
|
|
}
|
|
|
|
|
// Unconditional - see InitTime()'s call site in InitIndicators() for why m_Time must always be live.
|
|
|
|
|
m_Time.Refresh(OBJ_ALL_PERIODS);
|
|
|
|
|
if(m_useMA)
|
|
|
|
|
{
|
|
|
|
|
m_MA.Refresh(OBJ_ALL_PERIODS);
|
|
|
|
|
}
|
|
|
|
|
if(m_useRSI)
|
|
|
|
|
{
|
|
|
|
|
m_RSI.Refresh(OBJ_ALL_PERIODS);
|
|
|
|
|
}
|
|
|
|
|
if(m_useMACD)
|
|
|
|
|
{
|
|
|
|
|
m_MACDFeature.Refresh(OBJ_ALL_PERIODS);
|
|
|
|
|
}
|
|
|
|
|
if(m_useIchimoku)
|
|
|
|
|
{
|
|
|
|
|
m_Ichimoku.Refresh(OBJ_ALL_PERIODS);
|
|
|
|
|
}
|
|
|
|
|
// Unconditional - see the matching BufferResize() comment above.
|
|
|
|
|
m_ATR.Refresh(OBJ_ALL_PERIODS);
|
|
|
|
|
m_ADZigZag.Refresh(OBJ_ALL_PERIODS);
|
|
|
|
|
if(m_useADCumulativeDelta)
|
|
|
|
|
{
|
|
|
|
|
m_ADCumulativeDelta.Refresh(OBJ_ALL_PERIODS);
|
|
|
|
|
}
|
|
|
|
|
if(m_useADShorteningOfThrust)
|
|
|
|
|
{
|
|
|
|
|
m_ADShorteningOfThrust.Refresh(OBJ_ALL_PERIODS);
|
|
|
|
|
}
|
|
|
|
|
if(m_useADWyckoffEventStream)
|
|
|
|
|
{
|
|
|
|
|
m_ADWyckoffEventStream.Refresh(OBJ_ALL_PERIODS);
|
|
|
|
|
}
|
|
|
|
|
if(m_useADWyckoffFailedStructure)
|
|
|
|
|
{
|
|
|
|
|
m_ADWyckoffFailedStructure.Refresh(OBJ_ALL_PERIODS);
|
|
|
|
|
}
|
|
|
|
|
if(m_useADWyckoffSignificantBarInversion)
|
|
|
|
|
{
|
|
|
|
|
m_ADWyckoffSignificantBarInversion.Refresh(OBJ_ALL_PERIODS);
|
|
|
|
|
}
|
|
|
|
|
return true;
|
|
|
|
|
}
|
|
|
|
|
//+------------------------------------------------------------------+
|
|
|
|
|
//| |
|
|
|
|
|
//+------------------------------------------------------------------+
|
|
|
|
|
//+------------------------------------------------------------------+
|
|
|
|
|
//| Cache-or-compute wrapper around BufferTempDataCompute(): a given |
|
|
|
|
|
//| now-relative bar index's feature vector is invariant until the |
|
|
|
|
|
//| next candle close (see m_featureCache's declaration comment), so |
|
|
|
|
|
//| a cache hit just replays the m_neuronsCount values already |
|
|
|
|
|
//| computed for this idx straight into TempData instead of re- |
|
|
|
|
|
//| deriving them from price/ATR/AD-indicator buffers again. |
|
|
|
|
|
//+------------------------------------------------------------------+
|
|
|
|
|
bool CExpertSignalAIBase::BufferTempData(int idx)
|
|
|
|
|
{
|
|
|
|
|
int width = m_neuronsCount;
|
|
|
|
|
bool cacheable = (idx >= 0 && idx < ArraySize(m_featureCacheHasValue) && width > 0);
|
|
|
|
|
if(cacheable && m_featureCacheHasValue[idx])
|
|
|
|
|
{
|
|
|
|
|
if(!m_featureCacheValid[idx])
|
|
|
|
|
return false;
|
|
|
|
|
int base = idx * width;
|
|
|
|
|
for(int f = 0; f < width; f++)
|
|
|
|
|
if(!TempData.Add(m_featureCache[base + f]))
|
|
|
|
|
return false;
|
|
|
|
|
return true;
|
|
|
|
|
}
|
|
|
|
|
int startTotal = TempData.Total();
|
|
|
|
|
bool ok = BufferTempDataCompute(idx);
|
diag: name the cause when every feature window fails, and enforce the width contract
Era 0 stalls with "NOT ONE of 54681 scanned bars produced a usable
feature window, windows ok=0 failed=54681" and nothing else. That line
reads identically for a cold ATR, a conditionally-missing optional
feature block and an out-of-range index, so it cannot be diagnosed
without one restart per hypothesis.
Two changes:
1. WIDTH CONTRACT in BufferTempData. Every enabled block must emit
exactly m_neuronsCount values on EVERY bar. A block that emits its
values on some bars and skips them on others (indicator, panel or
series unavailable for that bar) does not merely shorten the window -
it SHIFTS every feature after it into the wrong slot, and the net
then trains on silently misaligned inputs that still look like a
valid window to everything downstream. Now rejected, rolled back and
reported once, naming the optional blocks (XA / SPR / swing context)
as the ones carrying an availability test. Worth having independently
of the current stall.
2. BuildFeatureWindow records WHICH lookback slot rejected and how much
of the window was assembled, and the pass-1 stall report renders it:
"slot 0 of 20 REJECTED (window had 0 of 760)" is an indicator warm-up
or history-edge read; "every lookback bar ACCEPTED and the window was
still short: 640 of 760" is a missing 6-value block.
No behaviour change on a healthy run: the width check is an equality
that already holds, and the diagnostics render only inside the
total-failure branch.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-11 10:31:56 -04:00
|
|
|
//--- WIDTH CONTRACT. Every enabled feature block must emit exactly the number of values
|
|
|
|
|
//--- m_neuronsCount was computed from, on every bar, unconditionally - a block that emits its
|
|
|
|
|
//--- values on some bars and skips them on others (because an indicator, panel or series was
|
|
|
|
|
//--- unavailable for THAT bar) does not merely shorten the window: it SHIFTS every feature after it
|
|
|
|
|
//--- into the wrong slot, and the net then trains on silently misaligned inputs that still look like
|
|
|
|
|
//--- a valid window to everything downstream. Caught here rather than left to surface as
|
|
|
|
|
//--- BuildFeatureWindow's length check, which cannot say which bar or which block was responsible.
|
|
|
|
|
if(ok)
|
|
|
|
|
{
|
|
|
|
|
int produced = TempData.Total() - startTotal;
|
|
|
|
|
if(produced != width)
|
|
|
|
|
{
|
|
|
|
|
ok = false;
|
|
|
|
|
m_featureFailTransient = false; // a width fault is structural, never "not ready yet"
|
|
|
|
|
if(!m_featureWidthWarned)
|
|
|
|
|
{
|
|
|
|
|
m_featureWidthWarned = true;
|
|
|
|
|
PrintFormat("%s: FEATURE WIDTH MISMATCH at bar %d - the enabled blocks produced %d values"
|
|
|
|
|
" but m_neuronsCount says %d. Every feature after the short block would have"
|
|
|
|
|
" landed in the wrong slot, so the bar is rejected rather than trained on."
|
|
|
|
|
" A block that can be conditionally unavailable must emit neutral values, not"
|
|
|
|
|
" nothing. Check the optional blocks first (cross-asset XA, spread SPR, swing"
|
|
|
|
|
" context) - those are the ones with an availability test.",
|
|
|
|
|
ID, idx, produced, width);
|
|
|
|
|
}
|
|
|
|
|
//--- Roll back the partial bar so the caller's window cannot contain half of it.
|
|
|
|
|
while(TempData.Total() > startTotal)
|
|
|
|
|
TempData.Delete(TempData.Total() - 1);
|
|
|
|
|
}
|
|
|
|
|
}
|
2026-08-11 11:30:02 -04:00
|
|
|
//--- ONLY SUCCESSES ARE CACHED. A miss is never stored, in any form.
|
|
|
|
|
//---
|
|
|
|
|
//--- The previous rule cached a miss whenever it was not flagged transient, and flagged exactly TWO
|
|
|
|
|
//--- guards - the EMPTY_VALUE open and the cold ATR. That was the half-fix: every OTHER rejection in
|
|
|
|
|
//--- BufferTempDataCompute (an indicator buffer not yet calculated, a panel not yet built, a series
|
|
|
|
|
//--- not yet loaded, a failed Add) still cached as PERMANENT, so one early sweep across cold
|
|
|
|
|
//--- indicators poisoned those bars for the rest of the process. Observed 2026-08-11: the MI
|
|
|
|
|
//--- pre-scan runs ~3 s after OnInit, touches all 54k bars while the indicators are still warming,
|
|
|
|
|
//--- and the run then reported "0 samples" and never trained again - the same failure the two-guard
|
|
|
|
|
//--- version was written to prevent, arriving through the guards it did not cover.
|
|
|
|
|
//---
|
|
|
|
|
//--- Enumerating which rejections are "really" permanent is the wrong shape of fix: it is a list that
|
|
|
|
|
//--- has to be re-audited every time a feature block is added, and being wrong once costs the whole
|
|
|
|
|
//--- run silently. Caching only successes needs no list and cannot be wrong. The cost is bounded and
|
|
|
|
|
//--- small: in steady state the only bars that still fail are the handful at the deep end of history
|
|
|
|
|
//--- inside the indicators' own warm-up, so an era recomputes ~ind_Periods bars rather than 54k.
|
|
|
|
|
//---
|
|
|
|
|
//--- m_featureCacheValid is now always true where m_featureCacheHasValue is true. Both are kept
|
|
|
|
|
//--- rather than collapsed into one array: the pair is written and read in several places, and a
|
|
|
|
|
//--- silent meaning change is exactly how the last version of this drifted.
|
|
|
|
|
if(cacheable && ok)
|
refactor: split CExpertSignalAIBase implementation by responsibility
ExpertSignalAIBase.mqh was 8216 lines: the class declaration followed by 87
method bodies covering training, labelling, feature extraction, persistence,
chart drawing, online learning, the GA auto-tuner and inference, all in one
file. Train() alone is 1492 lines; a change to arrow drawing meant scrolling
past the era loop.
Moved the bodies into Expert\AIBase\, included at the bottom of the original
after the class declaration:
Training.mqh 1607 era loop, plateau ladder, checkpoint select, deploy
Features.mqh 1093 indicator creation + per-bar input feature vector
ChartUI.mqh 634 arrows, arrow persistence, status panel, cleanup
Persistence.mqh 492 .stats/.cfg sidecars, CPU-inference validation, copy
OnlineLearning.mqh 461 live continual learning, EMA shadow, OOS simulator
Labels.mqh 309 ZigZag pivot labels, async label-cache prebuild
AutoTune.mqh 275 genetic tuner (population, crossover, halving)
Inference.mqh 235 softmax, prior calibration, class priors
ExpertSignalAIBase.mqh 8216 -> 3131 (declaration + topology build only)
This is a pure relocation - verified mechanically, not by eye: HEAD's file
reconstructed from the eight partials plus the surviving remainder is
byte-identical to HEAD, span for span (scratchpad verify_split.py). No
declaration moved, no signature changed, no code rewritten, so behaviour is
unchanged by construction.
Compiles 0 errors, 0 warnings.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-29 00:42:45 -04:00
|
|
|
{
|
|
|
|
|
m_featureCacheHasValue[idx] = true;
|
2026-08-11 11:30:02 -04:00
|
|
|
m_featureCacheValid[idx] = true;
|
|
|
|
|
int base = idx * width;
|
|
|
|
|
int count = TempData.Total() - startTotal;
|
|
|
|
|
for(int f = 0; f < count && f < width; f++)
|
|
|
|
|
m_featureCache[base + f] = TempData.At(startTotal + f);
|
refactor: split CExpertSignalAIBase implementation by responsibility
ExpertSignalAIBase.mqh was 8216 lines: the class declaration followed by 87
method bodies covering training, labelling, feature extraction, persistence,
chart drawing, online learning, the GA auto-tuner and inference, all in one
file. Train() alone is 1492 lines; a change to arrow drawing meant scrolling
past the era loop.
Moved the bodies into Expert\AIBase\, included at the bottom of the original
after the class declaration:
Training.mqh 1607 era loop, plateau ladder, checkpoint select, deploy
Features.mqh 1093 indicator creation + per-bar input feature vector
ChartUI.mqh 634 arrows, arrow persistence, status panel, cleanup
Persistence.mqh 492 .stats/.cfg sidecars, CPU-inference validation, copy
OnlineLearning.mqh 461 live continual learning, EMA shadow, OOS simulator
Labels.mqh 309 ZigZag pivot labels, async label-cache prebuild
AutoTune.mqh 275 genetic tuner (population, crossover, halving)
Inference.mqh 235 softmax, prior calibration, class priors
ExpertSignalAIBase.mqh 8216 -> 3131 (declaration + topology build only)
This is a pure relocation - verified mechanically, not by eye: HEAD's file
reconstructed from the eight partials plus the surviving remainder is
byte-identical to HEAD, span for span (scratchpad verify_split.py). No
declaration moved, no signature changed, no code rewritten, so behaviour is
unchanged by construction.
Compiles 0 errors, 0 warnings.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-29 00:42:45 -04:00
|
|
|
}
|
|
|
|
|
return ok;
|
|
|
|
|
}
|
|
|
|
|
//+------------------------------------------------------------------+
|
fix: the sequence models were reading the window backwards
BuildFeatureWindow() replaces eight hand-rolled copies of the same loop
and feeds the window OLDEST BAR FIRST. Every copy fed it newest-first,
because MQL5 timeseries indices run backwards and `r + b` with b ascending
walks into the past.
Harmless for PAI and CONV - a dense layer learns a weight per position
either way, a conv learns time-mirrored kernels. Not harmless for the
recurrent stacks:
- LSTM_SeqStepForward reads `inputs + t*Iw`, so step t is block t.
- It writes output[] only when t == steps-1: the visible output IS the
last hidden state.
- c_t = f*c_{t-1} + i*g decays toward the start of the sequence.
lstm_seq_flowcheck.cpp measured block 0's influence on the output at
1.2e-2 of block T-1's, at the shipped forget bias of 1.0.
So the bar being PREDICTED sat at the far end of the decay and the output
was handed to the OLDEST bar in the window - the exact inverse of what the
window is for. ~80x backwards on LSTM and HYBRID, on all three tiers
(OpenCL kernel, CPU DLL, pure-MQL5 inference), which is why it never
surfaced as a backend discrepancy.
This does not create edge - the MI diagnostics read at the noise floor
(p=0.4975) with a working positive control. It makes the one hypothesis
those diagnostics explicitly do NOT cover testable: they are marginal and
per-bar, and state they "cannot rule out one that only exists in
combination or across time". The sequence model is the instrument for
across-time structure and it has been crippled, so that hypothesis has
never been honestly tested.
Fingerprint gets an unconditional |WIN:2 - the vector keeps its shape and
its features, so a stale .nnw would load cleanly and run a model fitted to
one ordering against the other, silently. Re-keying every config is the
point, not collateral damage. FORCES A FULL RETRAIN.
Also: the now-relative bar caches are re-keyed on the two live paths.
EnsureBarCachesCapacity() was only ever called from training paths, but
once m_trainingComplete is set ScheduleTrainingIfNeeded() routes every bar
to RefreshConvergedSignal() and Train() is never re-entered - so nothing
cleared the feature cache again for the life of the process. A chart that
trained to convergence kept replaying the rows computed for the last
training era's bar grid: the live signal froze at its convergence-time
value, and OnlineLearnStep() backpropped those stale features against
freshly resolved labels. Backtests were never affected (an inference-only
process never allocates the arrays, so every read recomputes).
Compiles clean: 0 errors, 0 warnings.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-07 18:28:44 -04:00
|
|
|
//| THE ONE PLACE a feature WINDOW is assembled. Every consumer - |
|
|
|
|
|
//| training pass 1/2/3, live inference, online learning, the OOS |
|
|
|
|
|
//| continual simulation, the chart rescan and the CPU-inference |
|
|
|
|
|
//| self-check - goes through here, because the thing this function |
|
|
|
|
|
//| fixes is a contract that eight hand-rolled copies of the same |
|
|
|
|
|
//| loop cannot hold on their own. |
|
|
|
|
|
//| |
|
|
|
|
|
//| ORDER IS CHRONOLOGICAL: OLDEST BAR FIRST, bar `r` (the bar being |
|
|
|
|
|
//| predicted) LAST. That is the whole point of this function. |
|
|
|
|
|
//| |
|
|
|
|
|
//| MQL5 timeseries indices run BACKWARDS - index 0 is the newest bar |
|
|
|
|
|
//| and increasing index walks into the past. So the obvious loop, |
|
|
|
|
|
//| `for(b = 0..T-1) BufferTempData(r + b)`, appends the window in |
|
|
|
|
|
//| REVERSE chronological order: the newest bar lands in block 0 and |
|
|
|
|
|
//| the oldest in block T-1. That is what every call site used to do. |
|
|
|
|
|
//| |
|
|
|
|
|
//| For the dense (PAI) and convolutional stacks it is harmless - a |
|
|
|
|
|
//| dense layer learns a weight per position either way, and a conv |
|
|
|
|
|
//| just learns time-mirrored kernels. For the RECURRENT stacks it is |
|
|
|
|
|
//| not, and it is not a subtlety: |
|
|
|
|
|
//| - CNeuronLSTMOCL walks steps t = 0..T-1 reading `inputs + t*Iw` |
|
|
|
|
|
//| (AI\Network.cl, LSTM_SeqStepForward), so step t consumes the |
|
|
|
|
|
//| t-th block in buffer order. |
|
|
|
|
|
//| - Its visible output is the LAST hidden state only - the kernel |
|
|
|
|
|
//| writes `output[id]` solely when `t == steps - 1`. |
|
|
|
|
|
//| - The cell state decays toward the start of the sequence: |
|
|
|
|
|
//| c_t = f*c_{t-1} + i*g. DirectML\lstm_seq_flowcheck.cpp measured |
|
|
|
|
|
//| block 0's influence on the output, relative to block T-1, at |
|
|
|
|
|
//| 1.2e-2 for the shipped LSTM_FORGET_BIAS_INIT of 1.0 (see that |
|
|
|
|
|
//| constant's comment for the full sweep). |
|
|
|
|
|
//| Fed newest-first, that put the bar being PREDICTED at the far end |
|
|
|
|
|
//| of the decay and handed the output to the OLDEST bar in the window |
|
|
|
|
|
//| - roughly 80x backwards, and the exact inverse of what the window |
|
|
|
|
|
//| exists for ("everything known as of this bar's close", see Train() |
|
|
|
|
|
//| 's r comment). Reversing it here makes the final timestep the |
|
|
|
|
|
//| current bar, which is the standard arrangement and the one the |
|
|
|
|
|
//| forget-bias sweep was implicitly reasoning about. |
|
|
|
|
|
//| |
|
|
|
|
|
//| Nothing downstream reads a fixed block position, so this is safe |
|
|
|
|
|
//| for every topology; it re-keys the weight fingerprint (see |
|
|
|
|
|
//| ConfigFingerprint's WIN token) precisely BECAUSE the input vector |
|
|
|
|
|
//| now means something different, and models trained under the old |
|
|
|
|
|
//| order must never load into it. |
|
|
|
|
|
//| |
|
|
|
|
|
//| Returns true only when the COMPLETE, correctly-sized window is in |
|
|
|
|
|
//| TempData - callers must not feedForward on a partial one (a stale |
|
|
|
|
|
//| output layer would be scored against this bar's label). |
|
refactor: split CExpertSignalAIBase implementation by responsibility
ExpertSignalAIBase.mqh was 8216 lines: the class declaration followed by 87
method bodies covering training, labelling, feature extraction, persistence,
chart drawing, online learning, the GA auto-tuner and inference, all in one
file. Train() alone is 1492 lines; a change to arrow drawing meant scrolling
past the era loop.
Moved the bodies into Expert\AIBase\, included at the bottom of the original
after the class declaration:
Training.mqh 1607 era loop, plateau ladder, checkpoint select, deploy
Features.mqh 1093 indicator creation + per-bar input feature vector
ChartUI.mqh 634 arrows, arrow persistence, status panel, cleanup
Persistence.mqh 492 .stats/.cfg sidecars, CPU-inference validation, copy
OnlineLearning.mqh 461 live continual learning, EMA shadow, OOS simulator
Labels.mqh 309 ZigZag pivot labels, async label-cache prebuild
AutoTune.mqh 275 genetic tuner (population, crossover, halving)
Inference.mqh 235 softmax, prior calibration, class priors
ExpertSignalAIBase.mqh 8216 -> 3131 (declaration + topology build only)
This is a pure relocation - verified mechanically, not by eye: HEAD's file
reconstructed from the eight partials plus the surviving remainder is
byte-identical to HEAD, span for span (scratchpad verify_split.py). No
declaration moved, no signature changed, no code rewritten, so behaviour is
unchanged by construction.
Compiles 0 errors, 0 warnings.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-29 00:42:45 -04:00
|
|
|
//+------------------------------------------------------------------+
|
fix: the sequence models were reading the window backwards
BuildFeatureWindow() replaces eight hand-rolled copies of the same loop
and feeds the window OLDEST BAR FIRST. Every copy fed it newest-first,
because MQL5 timeseries indices run backwards and `r + b` with b ascending
walks into the past.
Harmless for PAI and CONV - a dense layer learns a weight per position
either way, a conv learns time-mirrored kernels. Not harmless for the
recurrent stacks:
- LSTM_SeqStepForward reads `inputs + t*Iw`, so step t is block t.
- It writes output[] only when t == steps-1: the visible output IS the
last hidden state.
- c_t = f*c_{t-1} + i*g decays toward the start of the sequence.
lstm_seq_flowcheck.cpp measured block 0's influence on the output at
1.2e-2 of block T-1's, at the shipped forget bias of 1.0.
So the bar being PREDICTED sat at the far end of the decay and the output
was handed to the OLDEST bar in the window - the exact inverse of what the
window is for. ~80x backwards on LSTM and HYBRID, on all three tiers
(OpenCL kernel, CPU DLL, pure-MQL5 inference), which is why it never
surfaced as a backend discrepancy.
This does not create edge - the MI diagnostics read at the noise floor
(p=0.4975) with a working positive control. It makes the one hypothesis
those diagnostics explicitly do NOT cover testable: they are marginal and
per-bar, and state they "cannot rule out one that only exists in
combination or across time". The sequence model is the instrument for
across-time structure and it has been crippled, so that hypothesis has
never been honestly tested.
Fingerprint gets an unconditional |WIN:2 - the vector keeps its shape and
its features, so a stale .nnw would load cleanly and run a model fitted to
one ordering against the other, silently. Re-keying every config is the
point, not collateral damage. FORCES A FULL RETRAIN.
Also: the now-relative bar caches are re-keyed on the two live paths.
EnsureBarCachesCapacity() was only ever called from training paths, but
once m_trainingComplete is set ScheduleTrainingIfNeeded() routes every bar
to RefreshConvergedSignal() and Train() is never re-entered - so nothing
cleared the feature cache again for the life of the process. A chart that
trained to convergence kept replaying the rows computed for the last
training era's bar grid: the live signal froze at its convergence-time
value, and OnlineLearnStep() backpropped those stale features against
freshly resolved labels. Backtests were never affected (an inference-only
process never allocates the arrays, so every read recomputes).
Compiles clean: 0 errors, 0 warnings.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-07 18:28:44 -04:00
|
|
|
bool CExpertSignalAIBase::BuildFeatureWindow(int r)
|
|
|
|
|
{
|
|
|
|
|
int width = (int)m_historyBars * m_neuronsCount;
|
|
|
|
|
TempData.Clear();
|
|
|
|
|
TempData.Reserve(width);
|
|
|
|
|
if(r < 0 || m_historyBars <= 0 || m_neuronsCount <= 0)
|
|
|
|
|
return false;
|
|
|
|
|
//--- b counts bars BACK from r, so (m_historyBars - 1 - b) emits the deepest lookback first and
|
|
|
|
|
//--- lands on r itself on the final iteration. Identical set of bars as before, opposite order.
|
|
|
|
|
for(int b = 0; b < (int)m_historyBars; b++)
|
|
|
|
|
if(!BufferTempData(r + ((int)m_historyBars - 1 - b)))
|
diag: name the cause when every feature window fails, and enforce the width contract
Era 0 stalls with "NOT ONE of 54681 scanned bars produced a usable
feature window, windows ok=0 failed=54681" and nothing else. That line
reads identically for a cold ATR, a conditionally-missing optional
feature block and an out-of-range index, so it cannot be diagnosed
without one restart per hypothesis.
Two changes:
1. WIDTH CONTRACT in BufferTempData. Every enabled block must emit
exactly m_neuronsCount values on EVERY bar. A block that emits its
values on some bars and skips them on others (indicator, panel or
series unavailable for that bar) does not merely shorten the window -
it SHIFTS every feature after it into the wrong slot, and the net
then trains on silently misaligned inputs that still look like a
valid window to everything downstream. Now rejected, rolled back and
reported once, naming the optional blocks (XA / SPR / swing context)
as the ones carrying an availability test. Worth having independently
of the current stall.
2. BuildFeatureWindow records WHICH lookback slot rejected and how much
of the window was assembled, and the pass-1 stall report renders it:
"slot 0 of 20 REJECTED (window had 0 of 760)" is an indicator warm-up
or history-edge read; "every lookback bar ACCEPTED and the window was
still short: 640 of 760" is a missing 6-value block.
No behaviour change on a healthy run: the width check is an equality
that already holds, and the diagnostics render only inside the
total-failure branch.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-11 10:31:56 -04:00
|
|
|
{
|
|
|
|
|
//--- Which lookback slot rejected, and how much of the window had been assembled. Without
|
|
|
|
|
//--- this the pass-1 stall report can only say "0 of 54681 usable", which is true of a cold
|
|
|
|
|
//--- ATR, a missing optional block and an out-of-range index alike.
|
|
|
|
|
m_windowFailSlot = b;
|
|
|
|
|
m_windowFailTotal = TempData.Total();
|
fix: the sequence models were reading the window backwards
BuildFeatureWindow() replaces eight hand-rolled copies of the same loop
and feeds the window OLDEST BAR FIRST. Every copy fed it newest-first,
because MQL5 timeseries indices run backwards and `r + b` with b ascending
walks into the past.
Harmless for PAI and CONV - a dense layer learns a weight per position
either way, a conv learns time-mirrored kernels. Not harmless for the
recurrent stacks:
- LSTM_SeqStepForward reads `inputs + t*Iw`, so step t is block t.
- It writes output[] only when t == steps-1: the visible output IS the
last hidden state.
- c_t = f*c_{t-1} + i*g decays toward the start of the sequence.
lstm_seq_flowcheck.cpp measured block 0's influence on the output at
1.2e-2 of block T-1's, at the shipped forget bias of 1.0.
So the bar being PREDICTED sat at the far end of the decay and the output
was handed to the OLDEST bar in the window - the exact inverse of what the
window is for. ~80x backwards on LSTM and HYBRID, on all three tiers
(OpenCL kernel, CPU DLL, pure-MQL5 inference), which is why it never
surfaced as a backend discrepancy.
This does not create edge - the MI diagnostics read at the noise floor
(p=0.4975) with a working positive control. It makes the one hypothesis
those diagnostics explicitly do NOT cover testable: they are marginal and
per-bar, and state they "cannot rule out one that only exists in
combination or across time". The sequence model is the instrument for
across-time structure and it has been crippled, so that hypothesis has
never been honestly tested.
Fingerprint gets an unconditional |WIN:2 - the vector keeps its shape and
its features, so a stale .nnw would load cleanly and run a model fitted to
one ordering against the other, silently. Re-keying every config is the
point, not collateral damage. FORCES A FULL RETRAIN.
Also: the now-relative bar caches are re-keyed on the two live paths.
EnsureBarCachesCapacity() was only ever called from training paths, but
once m_trainingComplete is set ScheduleTrainingIfNeeded() routes every bar
to RefreshConvergedSignal() and Train() is never re-entered - so nothing
cleared the feature cache again for the life of the process. A chart that
trained to convergence kept replaying the rows computed for the last
training era's bar grid: the live signal froze at its convergence-time
value, and OnlineLearnStep() backpropped those stale features against
freshly resolved labels. Backtests were never affected (an inference-only
process never allocates the arrays, so every read recomputes).
Compiles clean: 0 errors, 0 warnings.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-07 18:28:44 -04:00
|
|
|
return false;
|
diag: name the cause when every feature window fails, and enforce the width contract
Era 0 stalls with "NOT ONE of 54681 scanned bars produced a usable
feature window, windows ok=0 failed=54681" and nothing else. That line
reads identically for a cold ATR, a conditionally-missing optional
feature block and an out-of-range index, so it cannot be diagnosed
without one restart per hypothesis.
Two changes:
1. WIDTH CONTRACT in BufferTempData. Every enabled block must emit
exactly m_neuronsCount values on EVERY bar. A block that emits its
values on some bars and skips them on others (indicator, panel or
series unavailable for that bar) does not merely shorten the window -
it SHIFTS every feature after it into the wrong slot, and the net
then trains on silently misaligned inputs that still look like a
valid window to everything downstream. Now rejected, rolled back and
reported once, naming the optional blocks (XA / SPR / swing context)
as the ones carrying an availability test. Worth having independently
of the current stall.
2. BuildFeatureWindow records WHICH lookback slot rejected and how much
of the window was assembled, and the pass-1 stall report renders it:
"slot 0 of 20 REJECTED (window had 0 of 760)" is an indicator warm-up
or history-edge read; "every lookback bar ACCEPTED and the window was
still short: 640 of 760" is a missing 6-value block.
No behaviour change on a healthy run: the width check is an equality
that already holds, and the diagnostics render only inside the
total-failure branch.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-11 10:31:56 -04:00
|
|
|
}
|
|
|
|
|
if(TempData.Total() < width)
|
|
|
|
|
{
|
|
|
|
|
//--- Nothing rejected the bar and the window is still short. Distinct fault from a guard
|
|
|
|
|
//--- rejection and it used to be indistinguishable from one; the per-bar width contract in
|
|
|
|
|
//--- BufferTempData should now catch this first, so reaching here means the shortfall is in the
|
|
|
|
|
//--- window assembly itself rather than in one bar's blocks.
|
|
|
|
|
m_windowFailSlot = -1;
|
|
|
|
|
m_windowFailTotal = TempData.Total();
|
|
|
|
|
return false;
|
|
|
|
|
}
|
|
|
|
|
return true;
|
fix: the sequence models were reading the window backwards
BuildFeatureWindow() replaces eight hand-rolled copies of the same loop
and feeds the window OLDEST BAR FIRST. Every copy fed it newest-first,
because MQL5 timeseries indices run backwards and `r + b` with b ascending
walks into the past.
Harmless for PAI and CONV - a dense layer learns a weight per position
either way, a conv learns time-mirrored kernels. Not harmless for the
recurrent stacks:
- LSTM_SeqStepForward reads `inputs + t*Iw`, so step t is block t.
- It writes output[] only when t == steps-1: the visible output IS the
last hidden state.
- c_t = f*c_{t-1} + i*g decays toward the start of the sequence.
lstm_seq_flowcheck.cpp measured block 0's influence on the output at
1.2e-2 of block T-1's, at the shipped forget bias of 1.0.
So the bar being PREDICTED sat at the far end of the decay and the output
was handed to the OLDEST bar in the window - the exact inverse of what the
window is for. ~80x backwards on LSTM and HYBRID, on all three tiers
(OpenCL kernel, CPU DLL, pure-MQL5 inference), which is why it never
surfaced as a backend discrepancy.
This does not create edge - the MI diagnostics read at the noise floor
(p=0.4975) with a working positive control. It makes the one hypothesis
those diagnostics explicitly do NOT cover testable: they are marginal and
per-bar, and state they "cannot rule out one that only exists in
combination or across time". The sequence model is the instrument for
across-time structure and it has been crippled, so that hypothesis has
never been honestly tested.
Fingerprint gets an unconditional |WIN:2 - the vector keeps its shape and
its features, so a stale .nnw would load cleanly and run a model fitted to
one ordering against the other, silently. Re-keying every config is the
point, not collateral damage. FORCES A FULL RETRAIN.
Also: the now-relative bar caches are re-keyed on the two live paths.
EnsureBarCachesCapacity() was only ever called from training paths, but
once m_trainingComplete is set ScheduleTrainingIfNeeded() routes every bar
to RefreshConvergedSignal() and Train() is never re-entered - so nothing
cleared the feature cache again for the life of the process. A chart that
trained to convergence kept replaying the rows computed for the last
training era's bar grid: the live signal froze at its convergence-time
value, and OnlineLearnStep() backpropped those stale features against
freshly resolved labels. Backtests were never affected (an inference-only
process never allocates the arrays, so every read recomputes).
Compiles clean: 0 errors, 0 warnings.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-07 18:28:44 -04:00
|
|
|
}
|
fix(signals): revive a dead MA model, and demote Sanyaku from state to event
Two defects surfaced by research/test_classic.py, both verified fixed by re-running the
transcription against 178k bars of EURUSD H1.
CSignalMA model 1 could never fire. For any recursive average - and MA_TYPE_EMA is the
shipped default - MA(i) = a*Close(i) + (1-a)*MA(i+1), so
DiffMA(i) = a * (Close(i) - MA(i+1))
DiffCloseMA(i) = (1-a) * (Close(i) - MA(i+1))
are positive multiples of one quantity and always share a sign. Model 1 asks for a close
BELOW a RISING average, which is precisely the combination that identity forbids: 0.000%
of bars, either direction, any symbol. The MQL5 standard library this was ported from
defaults to MODE_SMA, where the two are merely correlated - the bug arrived with the EMA
default, not with the port. Reading the slope one bar back (DiffMAPrev) breaks the tie for
every MA type while keeping the model's stated meaning. Now fires on 7.92% of bars.
CSignalIchimoku model 11 fired on 27% of bars at weight 100. Sanyaku is three standing
STATES conjoined with no transition term, so it held across long stretches - and being
last in the if-chain at the top weight, the module's highest-conviction reading was also
its most common one, overwriting all eight event models below it on a quarter of all bars.
The old comment rejected an event form because "demanding all three flip on the same bar
would fire almost never" - true, but that is not the alternative. Kouten is the TURN: the
ALIGNMENT transitions, and only one role need change for it to. Testing !Sanyaku(idx+1)
fires once per aligned stretch. Now 2.17%, in line with Kumo breakout (2.4%) and the
strong TK cross (1.1%). DataReady() extended one bar deeper to cover the lookback.
Neither pattern showed edge before or after; this is about the models meaning what they
say and the vote not being dominated by a constant.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-01 17:14:34 -04:00
|
|
|
//+------------------------------------------------------------------+
|
|
|
|
|
//| (Re)build the cross-asset panel over `bars` bars. |
|
|
|
|
|
//| |
|
|
|
|
|
//| Called from the same places that size the price buffers, because |
|
|
|
|
|
//| the panel is aligned to exactly that bar grid and a stale panel |
|
|
|
|
|
//| would silently mis-index. Cheap to call redundantly: Build() is |
|
|
|
|
|
//| one CopyClose per reference pair, not per bar. |
|
|
|
|
|
//| |
|
|
|
|
|
//| A failure here is NOT fatal. The panel logs its own reason and |
|
|
|
|
|
//| every Features() call then 0-fills, so the run continues without |
|
|
|
|
|
//| the cross-asset block instead of refusing to train. |
|
|
|
|
|
//+------------------------------------------------------------------+
|
|
|
|
|
bool CExpertSignalAIBase::BuildCrossAssetPanel(int bars)
|
|
|
|
|
{
|
|
|
|
|
if(!m_useCrossAsset)
|
|
|
|
|
return true;
|
|
|
|
|
if(bars <= 0)
|
|
|
|
|
return false;
|
feat(ai): spread as a volatility-regime feature, and fix a stale-index cache in both new blocks
Adds spread/ATR and the spread change ratio as network inputs (EnableSpreadFeature,
default on). Spread is the one microstructure channel that is both FX-available and
genuinely historical in the Strategy Tester - "during testing, the spread is not modeled
but is taken from historical data" - so unlike swap, signed tick flow or depth of market it
is something a backtest can honestly validate.
What it encodes, stated precisely because the raw measurement overstates it.
research/test_spread.py found spr/atr the strongest single feature in this codebase, on 5
of 8 instrument/geometry cells at 2-4x any volume feature. But the barrier LABEL charges
the spread inside its own barriers, so a wide-spread bar is mechanically likelier to
resolve as a loss and the feature would partly be predicting its own cost model. Relabelling
at zero cost and re-measuring the identical feature showed 20-40% of it WAS that tautology
and the majority was not (XAUUSD retained 97%). What survives is a volatility-regime
reading: spread is near-fixed while ATR is not, so the ratio runs high exactly when
realised volatility is below its own ATR estimate, which genuinely predicts whether
ATR-scaled barriers get reached. It is UNSIGNED - Neutral-vs-directional only, never a side.
Also fixes a stale-index bug I introduced with the cross-asset panel and had just repeated
in the spread series. Both cached on length alone:
if(m_crossAsset.Bars() >= bars) return true;
MQL5 series indices are relative to NOW, so one new closed candle shifts every index by
one. Keyed only on length, the panel keeps serving its index 0 as a bar that is no longer
the newest, and every cross-asset value is read one bar out of step with the price features
sitting beside it in the same vector - silently, with no error and no shape change. This is
the same class of defect as the dtStudied watermark behind the zero-direction backtests.
Both now carry a datetime anchor on m_Time.GetData(0), the same invalidation key the
label/feature bar caches already use.
And a performance fix that fell out of it: with correct invalidation the panel rebuilds on
every new bar, and RefreshConvergedSignal runs per bar - which in the tester would mean one
full multi-symbol resample per simulated bar at training depth. Inference only reads bars
0..m_historyBars-1 plus the panel's own slow window, so it now requests exactly that. The
cache check is >=, so a deeper panel left from training still satisfies it.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-01 17:42:40 -04:00
|
|
|
//--- Deep enough AND anchored to the current newest bar. Depth alone would leave the panel's
|
|
|
|
|
//--- index 0 pointing at a bar that is no longer the newest as soon as one candle closes, so
|
|
|
|
|
//--- every cross-asset feature would be read one bar out of step with the price features beside
|
|
|
|
|
//--- it - see m_crossAssetAnchor's declaration comment.
|
|
|
|
|
datetime anchor = m_Time.GetData(0);
|
|
|
|
|
if(m_crossAsset.IsReady() && m_crossAsset.Bars() >= bars && m_crossAssetAnchor == anchor && anchor > 0)
|
fix(signals): revive a dead MA model, and demote Sanyaku from state to event
Two defects surfaced by research/test_classic.py, both verified fixed by re-running the
transcription against 178k bars of EURUSD H1.
CSignalMA model 1 could never fire. For any recursive average - and MA_TYPE_EMA is the
shipped default - MA(i) = a*Close(i) + (1-a)*MA(i+1), so
DiffMA(i) = a * (Close(i) - MA(i+1))
DiffCloseMA(i) = (1-a) * (Close(i) - MA(i+1))
are positive multiples of one quantity and always share a sign. Model 1 asks for a close
BELOW a RISING average, which is precisely the combination that identity forbids: 0.000%
of bars, either direction, any symbol. The MQL5 standard library this was ported from
defaults to MODE_SMA, where the two are merely correlated - the bug arrived with the EMA
default, not with the port. Reading the slope one bar back (DiffMAPrev) breaks the tie for
every MA type while keeping the model's stated meaning. Now fires on 7.92% of bars.
CSignalIchimoku model 11 fired on 27% of bars at weight 100. Sanyaku is three standing
STATES conjoined with no transition term, so it held across long stretches - and being
last in the if-chain at the top weight, the module's highest-conviction reading was also
its most common one, overwriting all eight event models below it on a quarter of all bars.
The old comment rejected an event form because "demanding all three flip on the same bar
would fire almost never" - true, but that is not the alternative. Kouten is the TURN: the
ALIGNMENT transitions, and only one role need change for it to. Testing !Sanyaku(idx+1)
fires once per aligned stretch. Now 2.17%, in line with Kumo breakout (2.4%) and the
strong TK cross (1.1%). DataReady() extended one bar deeper to cover the lookback.
Neither pattern showed edge before or after; this is about the models meaning what they
say and the vote not being dominated by a constant.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-01 17:14:34 -04:00
|
|
|
return true;
|
2026-08-11 21:29:14 -04:00
|
|
|
//--- A trained model builds from the pair set it was trained on (adopted from the .cfg), never
|
|
|
|
|
//--- from whatever Market Watch holds today - see m_crossAssetPairsPinned.
|
|
|
|
|
if(m_crossAssetPairsPinned != "" && !m_crossAsset.HasPinnedPairs())
|
|
|
|
|
m_crossAsset.SetPinnedPairs(m_crossAssetPairsPinned);
|
feat(ai): spread as a volatility-regime feature, and fix a stale-index cache in both new blocks
Adds spread/ATR and the spread change ratio as network inputs (EnableSpreadFeature,
default on). Spread is the one microstructure channel that is both FX-available and
genuinely historical in the Strategy Tester - "during testing, the spread is not modeled
but is taken from historical data" - so unlike swap, signed tick flow or depth of market it
is something a backtest can honestly validate.
What it encodes, stated precisely because the raw measurement overstates it.
research/test_spread.py found spr/atr the strongest single feature in this codebase, on 5
of 8 instrument/geometry cells at 2-4x any volume feature. But the barrier LABEL charges
the spread inside its own barriers, so a wide-spread bar is mechanically likelier to
resolve as a loss and the feature would partly be predicting its own cost model. Relabelling
at zero cost and re-measuring the identical feature showed 20-40% of it WAS that tautology
and the majority was not (XAUUSD retained 97%). What survives is a volatility-regime
reading: spread is near-fixed while ATR is not, so the ratio runs high exactly when
realised volatility is below its own ATR estimate, which genuinely predicts whether
ATR-scaled barriers get reached. It is UNSIGNED - Neutral-vs-directional only, never a side.
Also fixes a stale-index bug I introduced with the cross-asset panel and had just repeated
in the spread series. Both cached on length alone:
if(m_crossAsset.Bars() >= bars) return true;
MQL5 series indices are relative to NOW, so one new closed candle shifts every index by
one. Keyed only on length, the panel keeps serving its index 0 as a bar that is no longer
the newest, and every cross-asset value is read one bar out of step with the price features
sitting beside it in the same vector - silently, with no error and no shape change. This is
the same class of defect as the dtStudied watermark behind the zero-direction backtests.
Both now carry a datetime anchor on m_Time.GetData(0), the same invalidation key the
label/feature bar caches already use.
And a performance fix that fell out of it: with correct invalidation the panel rebuilds on
every new bar, and RefreshConvergedSignal runs per bar - which in the tester would mean one
full multi-symbol resample per simulated bar at training depth. Inference only reads bars
0..m_historyBars-1 plus the panel's own slow window, so it now requests exactly that. The
cache check is >=, so a deeper panel left from training still satisfies it.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-01 17:42:40 -04:00
|
|
|
if(!m_crossAsset.Build(m_symbol.Name(), (ENUM_TIMEFRAMES)m_period, bars))
|
|
|
|
|
{
|
|
|
|
|
m_crossAssetAnchor = 0;
|
|
|
|
|
return false;
|
|
|
|
|
}
|
|
|
|
|
m_crossAssetAnchor = anchor;
|
2026-08-11 21:29:14 -04:00
|
|
|
//--- FIRST successful build of a model with no pinned set yet: this pair set is now this model's
|
|
|
|
|
//--- pair set for life. Stamp it and pin it to the .cfg one-shot, exactly like the derived barrier
|
|
|
|
|
//--- pair (see Labels.mqh's m_geometryCfgSaved block) - the .cfg was written at model creation,
|
|
|
|
|
//--- BEFORE the panel could possibly have built, so without this re-save the pin would live only
|
|
|
|
|
//--- in memory and every restart would silently fall back to discovery.
|
|
|
|
|
if(m_crossAssetPairsPinned == "" && m_crossAsset.UsedPairsCsv() != "")
|
|
|
|
|
{
|
|
|
|
|
m_crossAssetPairsPinned = m_crossAsset.UsedPairsCsv();
|
|
|
|
|
m_crossAsset.SetPinnedPairs(m_crossAssetPairsPinned);
|
|
|
|
|
if(!m_crossAssetCfgSaved && m_activeFileName != "")
|
|
|
|
|
{
|
|
|
|
|
m_crossAssetCfgSaved = true;
|
|
|
|
|
if(SaveTopologyConfiguration(m_activeFileName, m_initialNeuronsCount, m_hiddenLayersCount,
|
|
|
|
|
m_neuronsReduction, m_minNeuronsCount, m_optimizationAlgo,
|
|
|
|
|
m_historyBars, m_outputNeuronsCount, m_neuronsCount,
|
|
|
|
|
LEGACY_STUDY_PERIOD_SLOT, m_minTrainYear, m_isInitialized,
|
|
|
|
|
LEGACY_CONVERGE_WR_SLOT, m_fractalPeriods, m_convFilterCount,
|
|
|
|
|
m_lstmHiddenSize, m_activeFileCommon))
|
|
|
|
|
Print(ID + ": cross-asset pair set PINNED to the .cfg - [" + m_crossAssetPairsPinned +
|
|
|
|
|
"]. Restarts and redeploys now build the panel from exactly this set; Market Watch "
|
|
|
|
|
"changes no longer alter what a trained model's features mean.");
|
|
|
|
|
else
|
|
|
|
|
Print(ID + ": WARNING - failed to pin the cross-asset pair set to the .cfg; a restart "
|
|
|
|
|
"will re-discover Market Watch instead of adopting the trained set.");
|
|
|
|
|
}
|
|
|
|
|
}
|
feat(ai): spread as a volatility-regime feature, and fix a stale-index cache in both new blocks
Adds spread/ATR and the spread change ratio as network inputs (EnableSpreadFeature,
default on). Spread is the one microstructure channel that is both FX-available and
genuinely historical in the Strategy Tester - "during testing, the spread is not modeled
but is taken from historical data" - so unlike swap, signed tick flow or depth of market it
is something a backtest can honestly validate.
What it encodes, stated precisely because the raw measurement overstates it.
research/test_spread.py found spr/atr the strongest single feature in this codebase, on 5
of 8 instrument/geometry cells at 2-4x any volume feature. But the barrier LABEL charges
the spread inside its own barriers, so a wide-spread bar is mechanically likelier to
resolve as a loss and the feature would partly be predicting its own cost model. Relabelling
at zero cost and re-measuring the identical feature showed 20-40% of it WAS that tautology
and the majority was not (XAUUSD retained 97%). What survives is a volatility-regime
reading: spread is near-fixed while ATR is not, so the ratio runs high exactly when
realised volatility is below its own ATR estimate, which genuinely predicts whether
ATR-scaled barriers get reached. It is UNSIGNED - Neutral-vs-directional only, never a side.
Also fixes a stale-index bug I introduced with the cross-asset panel and had just repeated
in the spread series. Both cached on length alone:
if(m_crossAsset.Bars() >= bars) return true;
MQL5 series indices are relative to NOW, so one new closed candle shifts every index by
one. Keyed only on length, the panel keeps serving its index 0 as a bar that is no longer
the newest, and every cross-asset value is read one bar out of step with the price features
sitting beside it in the same vector - silently, with no error and no shape change. This is
the same class of defect as the dtStudied watermark behind the zero-direction backtests.
Both now carry a datetime anchor on m_Time.GetData(0), the same invalidation key the
label/feature bar caches already use.
And a performance fix that fell out of it: with correct invalidation the panel rebuilds on
every new bar, and RefreshConvergedSignal runs per bar - which in the tester would mean one
full multi-symbol resample per simulated bar at training depth. Inference only reads bars
0..m_historyBars-1 plus the panel's own slow window, so it now requests exactly that. The
cache check is >=, so a deeper panel left from training still satisfies it.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-01 17:42:40 -04:00
|
|
|
return true;
|
|
|
|
|
}
|
|
|
|
|
//+------------------------------------------------------------------+
|
|
|
|
|
//| Copy the historical spread series onto the current bar grid. |
|
|
|
|
|
//| |
|
|
|
|
|
//| CopySpread is a RANGE call, so this runs once wherever the price |
|
|
|
|
|
//| buffers are sized - never per bar. Values are in POINTS (int); |
|
|
|
|
|
//| the feature block converts with m_symbol.Point(). |
|
|
|
|
|
//| |
|
|
|
|
|
//| Non-fatal: a short or failed copy leaves m_spreadSeriesBars at |
|
|
|
|
|
//| whatever was actually obtained and the feature block 0-fills past |
|
|
|
|
|
//| it, matching the degraded-but-usable convention used by the swing |
|
|
|
|
|
//| and cross-asset blocks. Refusing to train because one auxiliary |
|
|
|
|
|
//| series came up short would be a far worse failure. |
|
|
|
|
|
//+------------------------------------------------------------------+
|
|
|
|
|
bool CExpertSignalAIBase::EnsureSpreadSeries(int bars)
|
|
|
|
|
{
|
feat: S2 meta-labeling head - binary trade-quality model over the classic-candidate corpus
The NN now has a target that is not per-bar direction (closed, best-of-999
p=1.0000): P(win | this journaled candidate, at the EA's own SL/TP, net of
cost). One net for all 52 pattern-sides, AIType=AI_META.
- NetForward.mqh: the host-side softmax+CE gradient generalized total==3 ->
2||3 on both backprop paths; a 2-class softmax IS a logistic head, and no
compute backend changes.
- SignalMETA.mqh (new): corpus loaded read-only from the LARGEST signal DB on
disk (decoupled from the config fingerprint that burned four S1 runs); the
GMT->server offset is measured PER ROW against entryPrice vs bar open
(DST-immune, histogram logged); a window-span regime filter drops the
pre-2017 daily-backfill rows; 31-feature setup descriptor appended at the
input (26 one-hot + side + tanh netVote + SL/TP ATR + spread/ATR).
- Training.mqh: candidate-queued pass 1, binary-target pass 2, per-candidate
calibration (2.5) and OOS (3) walks. Counter mapping win->Buy / loss->Sell
lets checkpoint selection, the edge floor, the plateau ladder and the
family-wise deploy gate run UNCHANGED: precision reads as win rate among
traded candidates, chance as the base win rate, recalls as sensitivity/
specificity. Era-end META line: coverage x (p - break-even) vs the null.
- Labels are the side-conditional triple-barrier win caches - never the DB's
stop-and-reverse outcome. Logit adjustment deliberately skipped (~40% base
rate). Live inference + online learning guarded off until S3.
- Fingerprint: conditional |TGT:META1; State\META\ folder + 2-output filename
slot keep meta models fully separate from direction models.
Compiles clean (0 errors, 0 warnings). S2 run = attach a chart with
AIType=AI_META; S3 wires the votes via the per-side hooks.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-13 06:52:31 -04:00
|
|
|
//--- The meta target's setup descriptor reads spread/ATR at the candidate's fire bar regardless of
|
|
|
|
|
//--- whether spread is enabled as a per-bar WINDOW feature, so the series must exist for it.
|
|
|
|
|
if(!m_useSpreadFeature && !IsMetaTarget())
|
feat(ai): spread as a volatility-regime feature, and fix a stale-index cache in both new blocks
Adds spread/ATR and the spread change ratio as network inputs (EnableSpreadFeature,
default on). Spread is the one microstructure channel that is both FX-available and
genuinely historical in the Strategy Tester - "during testing, the spread is not modeled
but is taken from historical data" - so unlike swap, signed tick flow or depth of market it
is something a backtest can honestly validate.
What it encodes, stated precisely because the raw measurement overstates it.
research/test_spread.py found spr/atr the strongest single feature in this codebase, on 5
of 8 instrument/geometry cells at 2-4x any volume feature. But the barrier LABEL charges
the spread inside its own barriers, so a wide-spread bar is mechanically likelier to
resolve as a loss and the feature would partly be predicting its own cost model. Relabelling
at zero cost and re-measuring the identical feature showed 20-40% of it WAS that tautology
and the majority was not (XAUUSD retained 97%). What survives is a volatility-regime
reading: spread is near-fixed while ATR is not, so the ratio runs high exactly when
realised volatility is below its own ATR estimate, which genuinely predicts whether
ATR-scaled barriers get reached. It is UNSIGNED - Neutral-vs-directional only, never a side.
Also fixes a stale-index bug I introduced with the cross-asset panel and had just repeated
in the spread series. Both cached on length alone:
if(m_crossAsset.Bars() >= bars) return true;
MQL5 series indices are relative to NOW, so one new closed candle shifts every index by
one. Keyed only on length, the panel keeps serving its index 0 as a bar that is no longer
the newest, and every cross-asset value is read one bar out of step with the price features
sitting beside it in the same vector - silently, with no error and no shape change. This is
the same class of defect as the dtStudied watermark behind the zero-direction backtests.
Both now carry a datetime anchor on m_Time.GetData(0), the same invalidation key the
label/feature bar caches already use.
And a performance fix that fell out of it: with correct invalidation the panel rebuilds on
every new bar, and RefreshConvergedSignal runs per bar - which in the tester would mean one
full multi-symbol resample per simulated bar at training depth. Inference only reads bars
0..m_historyBars-1 plus the panel's own slow window, so it now requests exactly that. The
cache check is >=, so a deeper panel left from training still satisfies it.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-01 17:42:40 -04:00
|
|
|
return true;
|
|
|
|
|
if(bars <= 0)
|
|
|
|
|
return false;
|
|
|
|
|
//--- Length alone is NOT a sufficient cache key - see m_spreadSeriesAnchor's declaration comment.
|
|
|
|
|
datetime anchor = m_Time.GetData(0);
|
|
|
|
|
if(m_spreadSeriesBars >= bars && m_spreadSeriesAnchor == anchor && anchor > 0)
|
|
|
|
|
return true;
|
|
|
|
|
ArraySetAsSeries(m_spreadSeries, true); // index 0 = newest, matching every other buffer here
|
|
|
|
|
int got = CopySpread(m_symbol.Name(), (ENUM_TIMEFRAMES)m_period, 0, bars, m_spreadSeries);
|
|
|
|
|
if(got <= 0)
|
|
|
|
|
{
|
|
|
|
|
m_spreadSeriesBars = 0;
|
|
|
|
|
m_spreadSeriesAnchor = 0;
|
|
|
|
|
Print(__FUNCTION__ + ": CopySpread returned " + IntegerToString(got) + " for " + m_symbol.Name() +
|
|
|
|
|
" - spread features 0-filled this run.");
|
|
|
|
|
return false;
|
|
|
|
|
}
|
|
|
|
|
m_spreadSeriesBars = got;
|
|
|
|
|
m_spreadSeriesAnchor = anchor;
|
|
|
|
|
return true;
|
fix(signals): revive a dead MA model, and demote Sanyaku from state to event
Two defects surfaced by research/test_classic.py, both verified fixed by re-running the
transcription against 178k bars of EURUSD H1.
CSignalMA model 1 could never fire. For any recursive average - and MA_TYPE_EMA is the
shipped default - MA(i) = a*Close(i) + (1-a)*MA(i+1), so
DiffMA(i) = a * (Close(i) - MA(i+1))
DiffCloseMA(i) = (1-a) * (Close(i) - MA(i+1))
are positive multiples of one quantity and always share a sign. Model 1 asks for a close
BELOW a RISING average, which is precisely the combination that identity forbids: 0.000%
of bars, either direction, any symbol. The MQL5 standard library this was ported from
defaults to MODE_SMA, where the two are merely correlated - the bug arrived with the EMA
default, not with the port. Reading the slope one bar back (DiffMAPrev) breaks the tie for
every MA type while keeping the model's stated meaning. Now fires on 7.92% of bars.
CSignalIchimoku model 11 fired on 27% of bars at weight 100. Sanyaku is three standing
STATES conjoined with no transition term, so it held across long stretches - and being
last in the if-chain at the top weight, the module's highest-conviction reading was also
its most common one, overwriting all eight event models below it on a quarter of all bars.
The old comment rejected an event form because "demanding all three flip on the same bar
would fire almost never" - true, but that is not the alternative. Kouten is the TURN: the
ALIGNMENT transitions, and only one role need change for it to. Testing !Sanyaku(idx+1)
fires once per aligned stretch. Now 2.17%, in line with Kumo breakout (2.4%) and the
strong TK cross (1.1%). DataReady() extended one bar deeper to cover the lookback.
Neither pattern showed edge before or after; this is about the models meaning what they
say and the vote not being dominated by a constant.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-01 17:14:34 -04:00
|
|
|
}
|
|
|
|
|
//+------------------------------------------------------------------+
|
refactor: split CExpertSignalAIBase implementation by responsibility
ExpertSignalAIBase.mqh was 8216 lines: the class declaration followed by 87
method bodies covering training, labelling, feature extraction, persistence,
chart drawing, online learning, the GA auto-tuner and inference, all in one
file. Train() alone is 1492 lines; a change to arrow drawing meant scrolling
past the era loop.
Moved the bodies into Expert\AIBase\, included at the bottom of the original
after the class declaration:
Training.mqh 1607 era loop, plateau ladder, checkpoint select, deploy
Features.mqh 1093 indicator creation + per-bar input feature vector
ChartUI.mqh 634 arrows, arrow persistence, status panel, cleanup
Persistence.mqh 492 .stats/.cfg sidecars, CPU-inference validation, copy
OnlineLearning.mqh 461 live continual learning, EMA shadow, OOS simulator
Labels.mqh 309 ZigZag pivot labels, async label-cache prebuild
AutoTune.mqh 275 genetic tuner (population, crossover, halving)
Inference.mqh 235 softmax, prior calibration, class priors
ExpertSignalAIBase.mqh 8216 -> 3131 (declaration + topology build only)
This is a pure relocation - verified mechanically, not by eye: HEAD's file
reconstructed from the eight partials plus the surviving remainder is
byte-identical to HEAD, span for span (scratchpad verify_split.py). No
declaration moved, no signature changed, no code rewritten, so behaviour is
unchanged by construction.
Compiles 0 errors, 0 warnings.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-29 00:42:45 -04:00
|
|
|
bool CExpertSignalAIBase::BufferTempDataCompute(int idx)
|
|
|
|
|
{
|
2026-08-02 01:09:18 -04:00
|
|
|
//--- Where THIS bar's block starts. The function appends m_neuronsCount values below; remembering
|
|
|
|
|
//--- the offset lets the whole vector be validated in one place at the end instead of at each of
|
|
|
|
|
//--- the ~60 Add() call sites.
|
|
|
|
|
int featureStart = TempData.Total();
|
fix: a resumed model cached a cold ATR as permanent, so it never trained
BufferTempData cached EVERY failure - m_featureCacheHasValue[idx]=true
with m_featureCacheValid[idx]=false - and the cache never re-tries a
miss. So a single feature read taken before the terminal had finished
calculating the indicator buffers marked those bars unusable for the
rest of the process, even though the data arrived milliseconds later.
MT5 fills an indicator's buffers asynchronously after the handle is
created, and a cold ATR returns 0 for EVERY index, not just its warm-up
tail. BufferTempDataCompute rejects a bar with no ATR (correctly - the
price features would be meaningless), so the whole window failed, and
the whole cache was poisoned.
Only resumed models were hit, because only they read features that
early. Topology.mqh sets m_warmupPassesRemaining = netLoaded ? 0 : 3:
a fresh start sits through three separately-scheduled Train() calls
before anything touches a feature, which is exactly what those passes
are for. A resumed one skips them and TuneIndicatorsAndTrain drives
StartLabelCachePrebuild and the MI report from the first chart event.
Its rationale - "a restart already has a proven-synced history" - holds
for HISTORY and not for INDICATORS, which are recreated every process
start.
Downstream: BuildFeatureWindow failed on every bar of every era, so
add_loop never went true, so pass 2, pass 3, the era counter and the
checkpoint were all skipped and pass 1 swept 0->100% forever. The
"0 samples" MI report line at startup was the same failure, four
seconds earlier, already visible in the log.
- a miss is now cached only when it is PERMANENT; the two "not ready
yet" guards mark m_featureFailTransient and are recomputed on the
next visit. Steady-state cost is ~ind_Periods bars per era, not 54k.
- an era that discards itself now drops the feature cache before
restarting, so any remaining cause of this state self-heals instead
of looping.
Deleting the .nnw "fixed" this only by turning the model back into a
fresh one.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-10 11:41:31 -04:00
|
|
|
//--- Cleared here, set by the two NOT-READY-YET guards below. See BufferTempData() for what it
|
|
|
|
|
//--- controls: a rejection caused by data that has not arrived yet must not be cached, because the
|
|
|
|
|
//--- cache never re-tries a miss.
|
|
|
|
|
m_featureFailTransient = false;
|
refactor: split CExpertSignalAIBase implementation by responsibility
ExpertSignalAIBase.mqh was 8216 lines: the class declaration followed by 87
method bodies covering training, labelling, feature extraction, persistence,
chart drawing, online learning, the GA auto-tuner and inference, all in one
file. Train() alone is 1492 lines; a change to arrow drawing meant scrolling
past the era loop.
Moved the bodies into Expert\AIBase\, included at the bottom of the original
after the class declaration:
Training.mqh 1607 era loop, plateau ladder, checkpoint select, deploy
Features.mqh 1093 indicator creation + per-bar input feature vector
ChartUI.mqh 634 arrows, arrow persistence, status panel, cleanup
Persistence.mqh 492 .stats/.cfg sidecars, CPU-inference validation, copy
OnlineLearning.mqh 461 live continual learning, EMA shadow, OOS simulator
Labels.mqh 309 ZigZag pivot labels, async label-cache prebuild
AutoTune.mqh 275 genetic tuner (population, crossover, halving)
Inference.mqh 235 softmax, prior calibration, class priors
ExpertSignalAIBase.mqh 8216 -> 3131 (declaration + topology build only)
This is a pure relocation - verified mechanically, not by eye: HEAD's file
reconstructed from the eight partials plus the surviving remainder is
byte-identical to HEAD, span for span (scratchpad verify_split.py). No
declaration moved, no signature changed, no code rewritten, so behaviour is
unchanged by construction.
Compiles 0 errors, 0 warnings.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-29 00:42:45 -04:00
|
|
|
double open = m_Open.GetData(idx);
|
|
|
|
|
double close = m_Close.GetData(idx);
|
|
|
|
|
double high = m_High.GetData(idx);
|
|
|
|
|
double low = m_Low.GetData(idx);
|
|
|
|
|
MqlDateTime sTime;
|
|
|
|
|
TimeToStruct(m_Time.GetData(idx), sTime);
|
|
|
|
|
if(open == EMPTY_VALUE)
|
fix: a resumed model cached a cold ATR as permanent, so it never trained
BufferTempData cached EVERY failure - m_featureCacheHasValue[idx]=true
with m_featureCacheValid[idx]=false - and the cache never re-tries a
miss. So a single feature read taken before the terminal had finished
calculating the indicator buffers marked those bars unusable for the
rest of the process, even though the data arrived milliseconds later.
MT5 fills an indicator's buffers asynchronously after the handle is
created, and a cold ATR returns 0 for EVERY index, not just its warm-up
tail. BufferTempDataCompute rejects a bar with no ATR (correctly - the
price features would be meaningless), so the whole window failed, and
the whole cache was poisoned.
Only resumed models were hit, because only they read features that
early. Topology.mqh sets m_warmupPassesRemaining = netLoaded ? 0 : 3:
a fresh start sits through three separately-scheduled Train() calls
before anything touches a feature, which is exactly what those passes
are for. A resumed one skips them and TuneIndicatorsAndTrain drives
StartLabelCachePrebuild and the MI report from the first chart event.
Its rationale - "a restart already has a proven-synced history" - holds
for HISTORY and not for INDICATORS, which are recreated every process
start.
Downstream: BuildFeatureWindow failed on every bar of every era, so
add_loop never went true, so pass 2, pass 3, the era counter and the
checkpoint were all skipped and pass 1 swept 0->100% forever. The
"0 samples" MI report line at startup was the same failure, four
seconds earlier, already visible in the log.
- a miss is now cached only when it is PERMANENT; the two "not ready
yet" guards mark m_featureFailTransient and are recomputed on the
next visit. Steady-state cost is ~ind_Periods bars per era, not 54k.
- an era that discards itself now drops the feature cache before
restarting, so any remaining cause of this state self-heals instead
of looping.
Deleting the .nnw "fixed" this only by turning the model back into a
fresh one.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-10 11:41:31 -04:00
|
|
|
{
|
|
|
|
|
m_featureFailTransient = true;
|
refactor: split CExpertSignalAIBase implementation by responsibility
ExpertSignalAIBase.mqh was 8216 lines: the class declaration followed by 87
method bodies covering training, labelling, feature extraction, persistence,
chart drawing, online learning, the GA auto-tuner and inference, all in one
file. Train() alone is 1492 lines; a change to arrow drawing meant scrolling
past the era loop.
Moved the bodies into Expert\AIBase\, included at the bottom of the original
after the class declaration:
Training.mqh 1607 era loop, plateau ladder, checkpoint select, deploy
Features.mqh 1093 indicator creation + per-bar input feature vector
ChartUI.mqh 634 arrows, arrow persistence, status panel, cleanup
Persistence.mqh 492 .stats/.cfg sidecars, CPU-inference validation, copy
OnlineLearning.mqh 461 live continual learning, EMA shadow, OOS simulator
Labels.mqh 309 ZigZag pivot labels, async label-cache prebuild
AutoTune.mqh 275 genetic tuner (population, crossover, halving)
Inference.mqh 235 softmax, prior calibration, class priors
ExpertSignalAIBase.mqh 8216 -> 3131 (declaration + topology build only)
This is a pure relocation - verified mechanically, not by eye: HEAD's file
reconstructed from the eight partials plus the surviving remainder is
byte-identical to HEAD, span for span (scratchpad verify_split.py). No
declaration moved, no signature changed, no code rewritten, so behaviour is
unchanged by construction.
Compiles 0 errors, 0 warnings.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-29 00:42:45 -04:00
|
|
|
return false;
|
fix: a resumed model cached a cold ATR as permanent, so it never trained
BufferTempData cached EVERY failure - m_featureCacheHasValue[idx]=true
with m_featureCacheValid[idx]=false - and the cache never re-tries a
miss. So a single feature read taken before the terminal had finished
calculating the indicator buffers marked those bars unusable for the
rest of the process, even though the data arrived milliseconds later.
MT5 fills an indicator's buffers asynchronously after the handle is
created, and a cold ATR returns 0 for EVERY index, not just its warm-up
tail. BufferTempDataCompute rejects a bar with no ATR (correctly - the
price features would be meaningless), so the whole window failed, and
the whole cache was poisoned.
Only resumed models were hit, because only they read features that
early. Topology.mqh sets m_warmupPassesRemaining = netLoaded ? 0 : 3:
a fresh start sits through three separately-scheduled Train() calls
before anything touches a feature, which is exactly what those passes
are for. A resumed one skips them and TuneIndicatorsAndTrain drives
StartLabelCachePrebuild and the MI report from the first chart event.
Its rationale - "a restart already has a proven-synced history" - holds
for HISTORY and not for INDICATORS, which are recreated every process
start.
Downstream: BuildFeatureWindow failed on every bar of every era, so
add_loop never went true, so pass 2, pass 3, the era counter and the
checkpoint were all skipped and pass 1 swept 0->100% forever. The
"0 samples" MI report line at startup was the same failure, four
seconds earlier, already visible in the log.
- a miss is now cached only when it is PERMANENT; the two "not ready
yet" guards mark m_featureFailTransient and are recomputed on the
next visit. Steady-state cost is ~ind_Periods bars per era, not 54k.
- an era that discards itself now drops the feature cache before
restarting, so any remaining cause of this state self-heals instead
of looping.
Deleting the .nnw "fixed" this only by turning the model back into a
fresh one.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-10 11:41:31 -04:00
|
|
|
}
|
refactor: split CExpertSignalAIBase implementation by responsibility
ExpertSignalAIBase.mqh was 8216 lines: the class declaration followed by 87
method bodies covering training, labelling, feature extraction, persistence,
chart drawing, online learning, the GA auto-tuner and inference, all in one
file. Train() alone is 1492 lines; a change to arrow drawing meant scrolling
past the era loop.
Moved the bodies into Expert\AIBase\, included at the bottom of the original
after the class declaration:
Training.mqh 1607 era loop, plateau ladder, checkpoint select, deploy
Features.mqh 1093 indicator creation + per-bar input feature vector
ChartUI.mqh 634 arrows, arrow persistence, status panel, cleanup
Persistence.mqh 492 .stats/.cfg sidecars, CPU-inference validation, copy
OnlineLearning.mqh 461 live continual learning, EMA shadow, OOS simulator
Labels.mqh 309 ZigZag pivot labels, async label-cache prebuild
AutoTune.mqh 275 genetic tuner (population, crossover, halving)
Inference.mqh 235 softmax, prior calibration, class priors
ExpertSignalAIBase.mqh 8216 -> 3131 (declaration + topology build only)
This is a pure relocation - verified mechanically, not by eye: HEAD's file
reconstructed from the eight partials plus the surviving remainder is
byte-identical to HEAD, span for span (scratchpad verify_split.py). No
declaration moved, no signature changed, no code rewritten, so behaviour is
unchanged by construction.
Compiles 0 errors, 0 warnings.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-29 00:42:45 -04:00
|
|
|
// ATR-normalize every raw-price-unit feature below instead of feeding e.g. 0.0005 on EURUSD vs.
|
|
|
|
|
// 50.0 on a JPY pair or an index straight into the network - with Adam and hardcoded, scale-
|
|
|
|
|
// sensitive activations (TANH saturates, PRELU's 0.01 leak only means anything relative to the
|
|
|
|
|
// input's own scale), an unnormalized feature either vanishes into rounding noise or dominates
|
|
|
|
|
// the weighted sum depending on which symbol/timeframe happens to be loaded. Dividing by the
|
|
|
|
|
// bar's own ATR expresses every price-based feature as "fraction of typical volatility", which
|
|
|
|
|
// is comparable across symbols/timeframes and centered near zero. No ATR reading yet (e.g. the
|
|
|
|
|
// first few bars of history) means every price feature this bar would be meaningless - reject
|
|
|
|
|
// the bar via the same "return false" convention as the EMPTY_VALUE check above.
|
|
|
|
|
double atr = m_ATR.Main(idx);
|
|
|
|
|
if(atr <= 0.0 || atr == EMPTY_VALUE)
|
fix: a resumed model cached a cold ATR as permanent, so it never trained
BufferTempData cached EVERY failure - m_featureCacheHasValue[idx]=true
with m_featureCacheValid[idx]=false - and the cache never re-tries a
miss. So a single feature read taken before the terminal had finished
calculating the indicator buffers marked those bars unusable for the
rest of the process, even though the data arrived milliseconds later.
MT5 fills an indicator's buffers asynchronously after the handle is
created, and a cold ATR returns 0 for EVERY index, not just its warm-up
tail. BufferTempDataCompute rejects a bar with no ATR (correctly - the
price features would be meaningless), so the whole window failed, and
the whole cache was poisoned.
Only resumed models were hit, because only they read features that
early. Topology.mqh sets m_warmupPassesRemaining = netLoaded ? 0 : 3:
a fresh start sits through three separately-scheduled Train() calls
before anything touches a feature, which is exactly what those passes
are for. A resumed one skips them and TuneIndicatorsAndTrain drives
StartLabelCachePrebuild and the MI report from the first chart event.
Its rationale - "a restart already has a proven-synced history" - holds
for HISTORY and not for INDICATORS, which are recreated every process
start.
Downstream: BuildFeatureWindow failed on every bar of every era, so
add_loop never went true, so pass 2, pass 3, the era counter and the
checkpoint were all skipped and pass 1 swept 0->100% forever. The
"0 samples" MI report line at startup was the same failure, four
seconds earlier, already visible in the log.
- a miss is now cached only when it is PERMANENT; the two "not ready
yet" guards mark m_featureFailTransient and are recomputed on the
next visit. Steady-state cost is ~ind_Periods bars per era, not 54k.
- an era that discards itself now drops the feature cache before
restarting, so any remaining cause of this state self-heals instead
of looping.
Deleting the .nnw "fixed" this only by turning the model back into a
fresh one.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-10 11:41:31 -04:00
|
|
|
{
|
|
|
|
|
//--- TRANSIENT BY NATURE, and the reason resumed models could never train. MT5 calculates an
|
|
|
|
|
//--- indicator's buffers asynchronously after the handle is created, so a call made before ATR
|
|
|
|
|
//--- has filled returns 0 for EVERY index, not just the warm-up tail. A FRESH model never saw
|
|
|
|
|
//--- this: it sits through m_warmupPassesRemaining separately-scheduled Train() calls before
|
|
|
|
|
//--- anything touches a feature, which is exactly what those passes are for. A RESUMED model
|
|
|
|
|
//--- skips them - TuneIndicatorsAndTrain drives StartLabelCachePrebuild and the MI report from
|
|
|
|
|
//--- the very first chart event, milliseconds after OnInit - so it read a cold ATR, every bar
|
|
|
|
|
//--- was rejected, and BufferTempData cached all of it as permanent misses. From then on
|
|
|
|
|
//--- BuildFeatureWindow failed on every bar of every era, add_loop never went true, and pass 1
|
|
|
|
|
//--- swept 0->100% forever with nothing in the journal (2026-08-10; deleting the .nnw "fixed"
|
|
|
|
|
//--- it only by turning the model back into a fresh one).
|
|
|
|
|
m_featureFailTransient = true;
|
refactor: split CExpertSignalAIBase implementation by responsibility
ExpertSignalAIBase.mqh was 8216 lines: the class declaration followed by 87
method bodies covering training, labelling, feature extraction, persistence,
chart drawing, online learning, the GA auto-tuner and inference, all in one
file. Train() alone is 1492 lines; a change to arrow drawing meant scrolling
past the era loop.
Moved the bodies into Expert\AIBase\, included at the bottom of the original
after the class declaration:
Training.mqh 1607 era loop, plateau ladder, checkpoint select, deploy
Features.mqh 1093 indicator creation + per-bar input feature vector
ChartUI.mqh 634 arrows, arrow persistence, status panel, cleanup
Persistence.mqh 492 .stats/.cfg sidecars, CPU-inference validation, copy
OnlineLearning.mqh 461 live continual learning, EMA shadow, OOS simulator
Labels.mqh 309 ZigZag pivot labels, async label-cache prebuild
AutoTune.mqh 275 genetic tuner (population, crossover, halving)
Inference.mqh 235 softmax, prior calibration, class priors
ExpertSignalAIBase.mqh 8216 -> 3131 (declaration + topology build only)
This is a pure relocation - verified mechanically, not by eye: HEAD's file
reconstructed from the eight partials plus the surviving remainder is
byte-identical to HEAD, span for span (scratchpad verify_split.py). No
declaration moved, no signature changed, no code rewritten, so behaviour is
unchanged by construction.
Compiles 0 errors, 0 warnings.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-29 00:42:45 -04:00
|
|
|
return false;
|
fix: a resumed model cached a cold ATR as permanent, so it never trained
BufferTempData cached EVERY failure - m_featureCacheHasValue[idx]=true
with m_featureCacheValid[idx]=false - and the cache never re-tries a
miss. So a single feature read taken before the terminal had finished
calculating the indicator buffers marked those bars unusable for the
rest of the process, even though the data arrived milliseconds later.
MT5 fills an indicator's buffers asynchronously after the handle is
created, and a cold ATR returns 0 for EVERY index, not just its warm-up
tail. BufferTempDataCompute rejects a bar with no ATR (correctly - the
price features would be meaningless), so the whole window failed, and
the whole cache was poisoned.
Only resumed models were hit, because only they read features that
early. Topology.mqh sets m_warmupPassesRemaining = netLoaded ? 0 : 3:
a fresh start sits through three separately-scheduled Train() calls
before anything touches a feature, which is exactly what those passes
are for. A resumed one skips them and TuneIndicatorsAndTrain drives
StartLabelCachePrebuild and the MI report from the first chart event.
Its rationale - "a restart already has a proven-synced history" - holds
for HISTORY and not for INDICATORS, which are recreated every process
start.
Downstream: BuildFeatureWindow failed on every bar of every era, so
add_loop never went true, so pass 2, pass 3, the era counter and the
checkpoint were all skipped and pass 1 swept 0->100% forever. The
"0 samples" MI report line at startup was the same failure, four
seconds earlier, already visible in the log.
- a miss is now cached only when it is PERMANENT; the two "not ready
yet" guards mark m_featureFailTransient and are recomputed on the
next visit. Steady-state cost is ~ind_Periods bars per era, not 54k.
- an era that discards itself now drops the feature cache before
restarting, so any remaining cause of this state self-heals instead
of looping.
Deleting the .nnw "fixed" this only by turning the model back into a
fresh one.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-10 11:41:31 -04:00
|
|
|
}
|
refactor: split CExpertSignalAIBase implementation by responsibility
ExpertSignalAIBase.mqh was 8216 lines: the class declaration followed by 87
method bodies covering training, labelling, feature extraction, persistence,
chart drawing, online learning, the GA auto-tuner and inference, all in one
file. Train() alone is 1492 lines; a change to arrow drawing meant scrolling
past the era loop.
Moved the bodies into Expert\AIBase\, included at the bottom of the original
after the class declaration:
Training.mqh 1607 era loop, plateau ladder, checkpoint select, deploy
Features.mqh 1093 indicator creation + per-bar input feature vector
ChartUI.mqh 634 arrows, arrow persistence, status panel, cleanup
Persistence.mqh 492 .stats/.cfg sidecars, CPU-inference validation, copy
OnlineLearning.mqh 461 live continual learning, EMA shadow, OOS simulator
Labels.mqh 309 ZigZag pivot labels, async label-cache prebuild
AutoTune.mqh 275 genetic tuner (population, crossover, halving)
Inference.mqh 235 softmax, prior calibration, class priors
ExpertSignalAIBase.mqh 8216 -> 3131 (declaration + topology build only)
This is a pure relocation - verified mechanically, not by eye: HEAD's file
reconstructed from the eight partials plus the surviving remainder is
byte-identical to HEAD, span for span (scratchpad verify_split.py). No
declaration moved, no signature changed, no code rewritten, so behaviour is
unchanged by construction.
Compiles 0 errors, 0 warnings.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-29 00:42:45 -04:00
|
|
|
if(!TempData.Add((close - open) / atr) ||
|
|
|
|
|
!TempData.Add((high - open) / atr) ||
|
|
|
|
|
!TempData.Add((low - open) / atr) ||
|
|
|
|
|
// Explicit bullish/bearish flag - (close-open)/atr already encodes direction *and* magnitude
|
|
|
|
|
// together, which asks the network to disentangle "which way" from "how much" out of a single
|
|
|
|
|
// continuous value. Giving direction its own clean +1/-1/0 signal removes that ambiguity.
|
|
|
|
|
!TempData.Add(close > open ? 1.0 : (close < open ? -1.0 : 0.0)))
|
|
|
|
|
{
|
|
|
|
|
return false;
|
|
|
|
|
}
|
|
|
|
|
if(m_useSwingContext)
|
|
|
|
|
{
|
|
|
|
|
// Most recent CONFIRMED swing pivot as of bar idx - "confirmed" meaning at least
|
feat(ai): triple-barrier labels replace exact-pivot ZigZag targets
The 31:1 class imbalance was self-inflicted by the TARGET, not a property
of the market. Labelling only the exact bar where a ZigZag pivot confirms
gave Buy 1164 / Sell 1164 / Neutral 35841, and every correction mechanism
this codebase accumulated sits downstream of that one choice: the
logit-adjusted loss and its range cap, the prior EMA, the +-3.0 output-bias
seed, balanced-accuracy-then-precision selection with its coverage floor,
the recall floor and its catch-22, the alternation gate, NMS, and the four
oversampling designs that collapsed before them.
The reference this engine is built on (references/neuronetworksbook.pdf
ch. 3.1/3.3) also uses ZigZag, but targets the DIRECTION TO THE NEXT
EXTREMUM on every bar - ~50/50 by construction, with no imbalance to
correct at all. It never had this problem because it never asked "is this
the pivot bar".
Labels are now the triple barrier (Lopez de Prado ch. 3), using the EA's
OWN SL_Mode/TP_Mode: does a trade opened at this bar's close reach its
target before its stop, within a horizon. Buy = long resolves, Sell =
short resolves, Neutral = neither. Consequences:
- dir-precision in the era line stops being a proxy and becomes the win
rate of the strategy under its own exit rules.
- Expected balance ~25/25/50 at the shipped 1:3 (gambler's ruin), i.e.
~2:1 instead of 31:1. Measured and logged at the end of the prebuild.
- Spread is charged on both legs, so it is a NET win rate.
- Intrabar ambiguity resolves to the STOP. OHLC cannot order two touches
inside one bar and the optimistic reading is how a backtested edge
becomes a live loss.
ZigZag stays as input features (EnableSwingContext) and now also supplies
the vertical barrier: the horizon is the median confirmed leg length,
snapped to a coarse ladder. Derived, not configured, and deliberately kept
out of the filename fingerprint - a filename keyed on a measured quantity
orphans a trained model the moment the measurement moves.
Removed, because the premise died with the old target:
- the alternation gate. Correct for pivot labels (a ZigZag cannot emit two
same-type pivots in a row, so a repeat was provably a false fire), and
wrong for barrier labels, which answer each bar independently. It also
took its worst consequence with it: a one-sided model previously got ONE
trade per backtest, a hard blocker on marketplace validation.
- SignalClusterWindow now defaults off - it de-duplicated repeats that are
now real trades. Kept as an opt-in display control.
- LABEL_WINDOW_BARS, the pivot-widening pass, ConfirmedZigZagLabel.
- the era-0 output-bias seed now needs a genuinely dominant class (0.70)
rather than 0.40; at ~50% Neutral a +-3.0 seed is a distortion, not a
correction.
Also fixed, both found while wiring the above:
1. RefreshConvergedSignal sized its buffers from a date delta
(Bars(sym, period, dtStudied, TimeCurrent())). dtStudied is a training
watermark; in the tester it is loaded from a live-chart save AHEAD of
the simulated date, so the interval inverted, Bars() returned ~0, and
the buffer came out at exactly m_historyBars - deep enough for the OHLC
window and far too shallow for the Donchian-50 / 20-bar-return / SMA
extension behind it. Inference silently computed DIFFERENT features
from the ones training learned on, live as well as in the tester. Now
sized from what the feature builder actually needs.
2. The barrier horizon is resolved on the deployed path too. A deployed
model never enters Train(), so it never reached the prebuild, and
OnlineLearnStep reads the horizon as its confirmation delay - left at
the fallback it would have backpropped bars whose barriers had not
resolved. Silent lookahead in the one place that writes to a live model.
SL_Mode/TP_Mode join the weights fingerprint: they define the labels now,
so a model trained at 1:3 must never be silently reused at 1:1. This
re-keys every pre-existing model by design - none were trained on this task.
Inference census extended with the vote gate. LongCondition/ShortCondition
open with a readiness check the refresh counters never see; in the tester it
reduces to "the seeded _optcache.nnw must have LOADED", and if it did not,
every vote is hard-zeroed while the model still answers Buy. The old three
counters would have read that as "the model says Neutral" - false, and a
completely different fix. This is the leading candidate for the
zero-direction backtest and the census can now name it in one run.
Both builds compile 0 errors / 0 warnings. Forces a full retrain.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-31 20:39:49 -04:00
|
|
|
// m_swingConfirmationBars MORE bars have closed after it (see m_swingConfirmationBars' and
|
|
|
|
|
// m_useSwingContext's declaration comments). This is now the ONLY consumer of that embargo: the
|
|
|
|
|
// label side stopped needing it when the target became the triple barrier, whose own lookahead is
|
|
|
|
|
// m_barrierHorizonBars. Skipping this embargo here - e.g. reading
|
refactor: split CExpertSignalAIBase implementation by responsibility
ExpertSignalAIBase.mqh was 8216 lines: the class declaration followed by 87
method bodies covering training, labelling, feature extraction, persistence,
chart drawing, online learning, the GA auto-tuner and inference, all in one
file. Train() alone is 1492 lines; a change to arrow drawing meant scrolling
past the era loop.
Moved the bodies into Expert\AIBase\, included at the bottom of the original
after the class declaration:
Training.mqh 1607 era loop, plateau ladder, checkpoint select, deploy
Features.mqh 1093 indicator creation + per-bar input feature vector
ChartUI.mqh 634 arrows, arrow persistence, status panel, cleanup
Persistence.mqh 492 .stats/.cfg sidecars, CPU-inference validation, copy
OnlineLearning.mqh 461 live continual learning, EMA shadow, OOS simulator
Labels.mqh 309 ZigZag pivot labels, async label-cache prebuild
AutoTune.mqh 275 genetic tuner (population, crossover, halving)
Inference.mqh 235 softmax, prior calibration, class priors
ExpertSignalAIBase.mqh 8216 -> 3131 (declaration + topology build only)
This is a pure relocation - verified mechanically, not by eye: HEAD's file
reconstructed from the eight partials plus the surviving remainder is
byte-identical to HEAD, span for span (scratchpad verify_split.py). No
declaration moved, no signature changed, no code rewritten, so behaviour is
unchanged by construction.
Compiles 0 errors, 0 warnings.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-29 00:42:45 -04:00
|
|
|
// m_ADZigZag's raw current buffer value instead - would leak information a live bar at idx
|
|
|
|
|
// could never actually have had yet, since ZigZag's most recent 1-3 legs are still provisional
|
|
|
|
|
// and can be revised as new bars arrive.
|
|
|
|
|
int pivotIdx = -1;
|
|
|
|
|
double pivotPrice = 0.0;
|
|
|
|
|
bool pivotIsLow = false;
|
|
|
|
|
if(!FindConfirmedZigZagPivot(idx + MathMax(m_swingConfirmationBars, 1), pivotIdx, pivotPrice, pivotIsLow))
|
|
|
|
|
{
|
|
|
|
|
// No confirmed pivot within the scan cap (e.g. right at the start of available history) -
|
|
|
|
|
// this is legitimately "no swing context yet", not bad/missing data, so a neutral 0-fill
|
|
|
|
|
// keeps the bar usable rather than rejecting it outright like the ATR/EMPTY_VALUE guards do.
|
|
|
|
|
if(!TempData.Add(0.0) || !TempData.Add(0.0) || !TempData.Add(0.0) || !TempData.Add(0.0) || !TempData.Add(0.0))
|
|
|
|
|
return false;
|
|
|
|
|
}
|
|
|
|
|
else
|
|
|
|
|
{
|
|
|
|
|
// Direction of the CURRENT leg: the last confirmed pivot being a bottom means price has been
|
|
|
|
|
// rising away from it (an up-leg) ever since, and vice versa - same +1/-1 convention as the
|
|
|
|
|
// bullish/bearish flag above, just at swing scale instead of single-bar scale.
|
|
|
|
|
double direction = pivotIsLow ? 1.0 : -1.0;
|
|
|
|
|
// How far price has travelled since that pivot, ATR-normalized and signed (+ve above the
|
|
|
|
|
// pivot price, -ve below) - clamped generously since an extended trending leg has no natural
|
|
|
|
|
// ceiling the way a single bar's range does.
|
|
|
|
|
double distSincePivot = MathMax(-10.0, MathMin(10.0, (close - pivotPrice) / atr));
|
|
|
|
|
// Magnitude of the PRIOR completed leg (the pivot immediately before pivotIdx) - a scale
|
|
|
|
|
// reference for "is the current move big or small relative to the last full swing". No
|
|
|
|
|
// additional embargo needed here (see FindConfirmedZigZagPivot()'s declaration comment) -
|
|
|
|
|
// anything at or before an already-confirmed pivot is necessarily even older.
|
|
|
|
|
int priorPivotIdx = -1;
|
|
|
|
|
double priorPivotPrice = 0.0;
|
|
|
|
|
bool priorPivotIsLow = false;
|
|
|
|
|
bool havePrior = FindConfirmedZigZagPivot(pivotIdx + 1, priorPivotIdx, priorPivotPrice, priorPivotIsLow);
|
|
|
|
|
double priorLegMagnitude = havePrior ? MathMax(0.0, MathMin(10.0, MathAbs(pivotPrice - priorPivotPrice) / atr)) : 0.0;
|
|
|
|
|
// Retracement/extension ratio (current distance relative to the prior leg's own size) -
|
|
|
|
|
// Fibonacci-style relative position, often more informative than either raw magnitude alone
|
|
|
|
|
// since it's comparable across both quiet and volatile regimes. 0 when there's no prior leg
|
|
|
|
|
// to compare against yet.
|
|
|
|
|
double retracementRatio = (havePrior && priorLegMagnitude > 0.0001) ?
|
|
|
|
|
MathMax(-5.0, MathMin(5.0, distSincePivot / priorLegMagnitude)) : 0.0;
|
|
|
|
|
// Swing age (bars since the pivot) - a maturity/exhaustion proxy, same +/- style clamp
|
|
|
|
|
// convention as the volume-ratio feature below.
|
|
|
|
|
double barsSincePivot = MathMax(0.0, MathMin(5.0, (double)(pivotIdx - idx) / 100.0));
|
|
|
|
|
if(!TempData.Add(direction) ||
|
|
|
|
|
!TempData.Add(distSincePivot) ||
|
|
|
|
|
!TempData.Add(priorLegMagnitude) ||
|
|
|
|
|
!TempData.Add(retracementRatio) ||
|
|
|
|
|
!TempData.Add(barsSincePivot))
|
|
|
|
|
return false;
|
|
|
|
|
}
|
|
|
|
|
// Recent price-action context (4 features), computed from CLOSED bars at idx or older only -
|
|
|
|
|
// no ZigZag confirmation, so no repainting and NO embargo, and never stale, unlike the five
|
|
|
|
|
// pivot-anchored features above whose confirmed anchor is always >= m_swingConfirmationBars
|
|
|
|
|
// (~100) bars old. Those describe the OLD structure well but say nothing about the recent leg
|
|
|
|
|
// the bar actually sits in - which is exactly what's needed to tell a genuine reversal at a
|
|
|
|
|
// range extreme from a mid-trend bar that merely looks like a bottom/top (the "clustered
|
|
|
|
|
// counter-trend signals" failure mode). These locate the bar within its recent range and
|
|
|
|
|
// trend so the network can learn that a directional call belongs at an extreme of an extended
|
|
|
|
|
// move, not anywhere the local candle shape resembles a pivot. All windows walk toward OLDER
|
|
|
|
|
// bars (increasing index), so nothing here can see the future.
|
|
|
|
|
double hi20 = high, lo20 = low, hi50 = high, lo50 = low;
|
|
|
|
|
double sum20 = close, oldestClose20 = close;
|
|
|
|
|
int cnt20 = 1;
|
|
|
|
|
for(int w = 1; w < 50; w++)
|
|
|
|
|
{
|
|
|
|
|
int j = idx + w;
|
|
|
|
|
double jc = m_Close.GetData(j);
|
|
|
|
|
double jh = m_High.GetData(j);
|
|
|
|
|
double jl = m_Low.GetData(j);
|
|
|
|
|
// ran off the oldest edge of loaded history (out-of-range reads back as 0/EMPTY_VALUE) -
|
|
|
|
|
// use whatever window we gathered so far rather than rejecting the bar; a shorter early-
|
|
|
|
|
// history window is degraded-but-usable, same spirit as the pivot 0-fill above.
|
|
|
|
|
if(jh == EMPTY_VALUE || jh <= 0.0 || jl <= 0.0)
|
|
|
|
|
break;
|
|
|
|
|
if(jh > hi50)
|
|
|
|
|
hi50 = jh;
|
|
|
|
|
if(jl < lo50)
|
|
|
|
|
lo50 = jl;
|
|
|
|
|
if(w < 20)
|
|
|
|
|
{
|
|
|
|
|
if(jh > hi20)
|
|
|
|
|
hi20 = jh;
|
|
|
|
|
if(jl < lo20)
|
|
|
|
|
lo20 = jl;
|
|
|
|
|
sum20 += jc;
|
|
|
|
|
oldestClose20 = jc;
|
|
|
|
|
cnt20++;
|
|
|
|
|
}
|
|
|
|
|
}
|
|
|
|
|
// Donchian position: where close sits inside the recent high/low range, rescaled to [-1,+1]
|
|
|
|
|
// (-1 = at the range low / bottom candidate, +1 = at the range high / top candidate, 0 = mid-
|
|
|
|
|
// range / mid-trend). Two scales - a short 20-bar and a medium 50-bar view - so the network
|
|
|
|
|
// sees both local and swing-scale extremity. 0 (mid) when the range is degenerate.
|
|
|
|
|
double range20 = hi20 - lo20;
|
|
|
|
|
double range50 = hi50 - lo50;
|
|
|
|
|
double donchPos20 = (range20 > 0.0) ? ((close - lo20) / range20 - 0.5) * 2.0 : 0.0;
|
|
|
|
|
double donchPos50 = (range50 > 0.0) ? ((close - lo50) / range50 - 0.5) * 2.0 : 0.0;
|
|
|
|
|
// Net directional displacement over the recent window, ATR-normalized and signed - the
|
|
|
|
|
// prevailing-trend strength/direction the counter-trend clusters were ignoring.
|
|
|
|
|
double recentReturn = MathMax(-10.0, MathMin(10.0, (close - oldestClose20) / atr));
|
|
|
|
|
// Distance from the recent mean (SMA), ATR-normalized - a stretch/exhaustion proxy distinct
|
|
|
|
|
// from the net return (a move can be far from its mean with little net displacement, or vice
|
|
|
|
|
// versa); genuine reversals tend to be over-extended from equilibrium.
|
|
|
|
|
double smaExtension = MathMax(-10.0, MathMin(10.0, (close - sum20 / cnt20) / atr));
|
|
|
|
|
if(!TempData.Add(donchPos20) ||
|
|
|
|
|
!TempData.Add(donchPos50) ||
|
|
|
|
|
!TempData.Add(recentReturn) ||
|
|
|
|
|
!TempData.Add(smaExtension))
|
|
|
|
|
return false;
|
|
|
|
|
}
|
|
|
|
|
if(m_useVolumes)
|
|
|
|
|
{
|
feat(ai): widen the volume feature block from 1 value to 4
The block fed exactly one number: (v[i] - v[i-1]) / v[i-1]. That is the first difference,
and it cannot express three things that matter - the LEVEL relative to a baseline (two
dead bars and two frantic bars both read ~0 change), and the two volume-vs-range
interactions, where heavy participation that went NOWHERE (absorption) and heavy
participation that travelled (continuation) mean opposite things and currently collapse
onto the same value.
research/test_volume.py measures each candidate's mutual information with the triple-
barrier label across 3 instruments x 2 geometries, against a BLOCK-permutation null -
blocks sized to the barrier horizon, because adjacent labels share almost their entire
outcome window and a free shuffle yields a null so tight that everything looks
significant. Finite-sample MI bias (~7/n here) is reported alongside rather than
subtracted, since the permutation null already absorbs it.
Result: volLevel beats the shipped change ratio outright on 4 of 6 cells (EURUSD 2:3
+0.000118 excess at p=0.006, USDJPY 1:2 +0.000284 at p=0.002); absorption is the single
strongest reading anywhere in the sweep at EURUSD 1:2 (+0.000404, p=0.002) though it is
null on XAUUSD; vol x range clears on 4 of 6. The shipped change ratio is itself
significant on 5 of 6, so it stays.
Kept OUT: a session-relative z-score against the same hour-of-day's own recent history.
It was the weakest candidate - null on both EURUSD cells - and it is the only one needing
per-hour rolling bookkeeping in MQL5. Not worth the state for a reading that did not
survive its own null on the primary instrument.
Magnitudes, stated plainly because they are the point: the excess MI is ~2e-4 nats against
a label entropy near 1.05. That is under a tenth of one percent of the label's
uncertainty. It is real, it repeats across instruments, and it is nowhere near an edge -
this is worth having because it costs one 50-bar loop, not because it changes the answer.
Prior work stands: the whole single-series feature family measured at the noise floor.
m_neuronsCount is already in the fingerprint, so the width change re-keys existing caches
by itself, which is correct - the input vector genuinely changed shape.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-01 17:34:33 -04:00
|
|
|
// FOUR values, not one. This block used to feed only the bar-over-bar change ratio below.
|
|
|
|
|
// research/test_volume.py measured all four against the barrier label with a block-permutation
|
|
|
|
|
// null (blocks = the barrier horizon, because adjacent labels share almost their whole outcome
|
|
|
|
|
// window and a free shuffle produces a null far too tight): the LEVEL and the two
|
|
|
|
|
// volume-vs-range interactions each carry information the first difference does not, and the
|
|
|
|
|
// level beats the shipped feature outright on 4 of 6 instrument/geometry cells.
|
|
|
|
|
//
|
|
|
|
|
// Read the magnitudes before expecting much: the excess mutual information is ~2e-4 nats
|
|
|
|
|
// against a label entropy near 1.05, i.e. well under a tenth of one percent of the label's
|
|
|
|
|
// uncertainty. This is real and repeatable across instruments, and it is nowhere near an edge.
|
|
|
|
|
// It is worth having because it costs one 50-bar loop, not because it changes the answer.
|
|
|
|
|
double vNow = m_Volumes.Main(idx);
|
refactor: split CExpertSignalAIBase implementation by responsibility
ExpertSignalAIBase.mqh was 8216 lines: the class declaration followed by 87
method bodies covering training, labelling, feature extraction, persistence,
chart drawing, online learning, the GA auto-tuner and inference, all in one
file. Train() alone is 1492 lines; a change to arrow drawing meant scrolling
past the era loop.
Moved the bodies into Expert\AIBase\, included at the bottom of the original
after the class declaration:
Training.mqh 1607 era loop, plateau ladder, checkpoint select, deploy
Features.mqh 1093 indicator creation + per-bar input feature vector
ChartUI.mqh 634 arrows, arrow persistence, status panel, cleanup
Persistence.mqh 492 .stats/.cfg sidecars, CPU-inference validation, copy
OnlineLearning.mqh 461 live continual learning, EMA shadow, OOS simulator
Labels.mqh 309 ZigZag pivot labels, async label-cache prebuild
AutoTune.mqh 275 genetic tuner (population, crossover, halving)
Inference.mqh 235 softmax, prior calibration, class priors
ExpertSignalAIBase.mqh 8216 -> 3131 (declaration + topology build only)
This is a pure relocation - verified mechanically, not by eye: HEAD's file
reconstructed from the eight partials plus the surviving remainder is
byte-identical to HEAD, span for span (scratchpad verify_split.py). No
declaration moved, no signature changed, no code rewritten, so behaviour is
unchanged by construction.
Compiles 0 errors, 0 warnings.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-29 00:42:45 -04:00
|
|
|
double prevVolume = m_Volumes.Main(idx + 1);
|
feat(ai): widen the volume feature block from 1 value to 4
The block fed exactly one number: (v[i] - v[i-1]) / v[i-1]. That is the first difference,
and it cannot express three things that matter - the LEVEL relative to a baseline (two
dead bars and two frantic bars both read ~0 change), and the two volume-vs-range
interactions, where heavy participation that went NOWHERE (absorption) and heavy
participation that travelled (continuation) mean opposite things and currently collapse
onto the same value.
research/test_volume.py measures each candidate's mutual information with the triple-
barrier label across 3 instruments x 2 geometries, against a BLOCK-permutation null -
blocks sized to the barrier horizon, because adjacent labels share almost their entire
outcome window and a free shuffle yields a null so tight that everything looks
significant. Finite-sample MI bias (~7/n here) is reported alongside rather than
subtracted, since the permutation null already absorbs it.
Result: volLevel beats the shipped change ratio outright on 4 of 6 cells (EURUSD 2:3
+0.000118 excess at p=0.006, USDJPY 1:2 +0.000284 at p=0.002); absorption is the single
strongest reading anywhere in the sweep at EURUSD 1:2 (+0.000404, p=0.002) though it is
null on XAUUSD; vol x range clears on 4 of 6. The shipped change ratio is itself
significant on 5 of 6, so it stays.
Kept OUT: a session-relative z-score against the same hour-of-day's own recent history.
It was the weakest candidate - null on both EURUSD cells - and it is the only one needing
per-hour rolling bookkeeping in MQL5. Not worth the state for a reading that did not
survive its own null on the primary instrument.
Magnitudes, stated plainly because they are the point: the excess MI is ~2e-4 nats against
a label entropy near 1.05. That is under a tenth of one percent of the label's
uncertainty. It is real, it repeats across instruments, and it is nowhere near an edge -
this is worth having because it costs one 50-bar loop, not because it changes the answer.
Prior work stands: the whole single-series feature family measured at the noise floor.
m_neuronsCount is already in the fingerprint, so the width change re-keys existing caches
by itself, which is correct - the input vector genuinely changed shape.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-01 17:34:33 -04:00
|
|
|
double volumeDelta = vNow - prevVolume;
|
|
|
|
|
// Relative change - trading activity magnitude varies wildly across symbols/timeframes, so the
|
|
|
|
|
// previous bar's own volume is the scale reference, same logic as ATR-normalizing price above.
|
|
|
|
|
// Guard against a zero previous-bar volume (e.g. a holiday-thin session) instead of dividing by
|
|
|
|
|
// it. Clamped to +/-5: unlike the ATR-normalized price features this ratio has no natural
|
|
|
|
|
// ceiling (a 1-tick bar followed by a normal one produces a huge outlier).
|
refactor: split CExpertSignalAIBase implementation by responsibility
ExpertSignalAIBase.mqh was 8216 lines: the class declaration followed by 87
method bodies covering training, labelling, feature extraction, persistence,
chart drawing, online learning, the GA auto-tuner and inference, all in one
file. Train() alone is 1492 lines; a change to arrow drawing meant scrolling
past the era loop.
Moved the bodies into Expert\AIBase\, included at the bottom of the original
after the class declaration:
Training.mqh 1607 era loop, plateau ladder, checkpoint select, deploy
Features.mqh 1093 indicator creation + per-bar input feature vector
ChartUI.mqh 634 arrows, arrow persistence, status panel, cleanup
Persistence.mqh 492 .stats/.cfg sidecars, CPU-inference validation, copy
OnlineLearning.mqh 461 live continual learning, EMA shadow, OOS simulator
Labels.mqh 309 ZigZag pivot labels, async label-cache prebuild
AutoTune.mqh 275 genetic tuner (population, crossover, halving)
Inference.mqh 235 softmax, prior calibration, class priors
ExpertSignalAIBase.mqh 8216 -> 3131 (declaration + topology build only)
This is a pure relocation - verified mechanically, not by eye: HEAD's file
reconstructed from the eight partials plus the surviving remainder is
byte-identical to HEAD, span for span (scratchpad verify_split.py). No
declaration moved, no signature changed, no code rewritten, so behaviour is
unchanged by construction.
Compiles 0 errors, 0 warnings.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-29 00:42:45 -04:00
|
|
|
double volumeChangeRatio = prevVolume > 0.0 ? volumeDelta / prevVolume : 0.0;
|
feat(ai): widen the volume feature block from 1 value to 4
The block fed exactly one number: (v[i] - v[i-1]) / v[i-1]. That is the first difference,
and it cannot express three things that matter - the LEVEL relative to a baseline (two
dead bars and two frantic bars both read ~0 change), and the two volume-vs-range
interactions, where heavy participation that went NOWHERE (absorption) and heavy
participation that travelled (continuation) mean opposite things and currently collapse
onto the same value.
research/test_volume.py measures each candidate's mutual information with the triple-
barrier label across 3 instruments x 2 geometries, against a BLOCK-permutation null -
blocks sized to the barrier horizon, because adjacent labels share almost their entire
outcome window and a free shuffle yields a null so tight that everything looks
significant. Finite-sample MI bias (~7/n here) is reported alongside rather than
subtracted, since the permutation null already absorbs it.
Result: volLevel beats the shipped change ratio outright on 4 of 6 cells (EURUSD 2:3
+0.000118 excess at p=0.006, USDJPY 1:2 +0.000284 at p=0.002); absorption is the single
strongest reading anywhere in the sweep at EURUSD 1:2 (+0.000404, p=0.002) though it is
null on XAUUSD; vol x range clears on 4 of 6. The shipped change ratio is itself
significant on 5 of 6, so it stays.
Kept OUT: a session-relative z-score against the same hour-of-day's own recent history.
It was the weakest candidate - null on both EURUSD cells - and it is the only one needing
per-hour rolling bookkeeping in MQL5. Not worth the state for a reading that did not
survive its own null on the primary instrument.
Magnitudes, stated plainly because they are the point: the excess MI is ~2e-4 nats against
a label entropy near 1.05. That is under a tenth of one percent of the label's
uncertainty. It is real, it repeats across instruments, and it is nowhere near an edge -
this is worth having because it costs one 50-bar loop, not because it changes the answer.
Prior work stands: the whole single-series feature family measured at the noise floor.
m_neuronsCount is already in the fingerprint, so the width change re-keys existing caches
by itself, which is correct - the input vector genuinely changed shape.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-01 17:34:33 -04:00
|
|
|
// Baseline over the trailing 50 bars, walking toward OLDER bars only (increasing index), so
|
|
|
|
|
// nothing here can see the future. Degraded-but-usable at the oldest edge, same convention as
|
|
|
|
|
// the swing-context window above: a short early-history baseline beats rejecting the bar.
|
|
|
|
|
double volSum = vNow;
|
|
|
|
|
int volCnt = 1;
|
|
|
|
|
for(int w = 1; w < 50; w++)
|
|
|
|
|
{
|
|
|
|
|
double jv = m_Volumes.Main(idx + w);
|
|
|
|
|
if(jv <= 0.0)
|
|
|
|
|
break;
|
|
|
|
|
volSum += jv;
|
|
|
|
|
volCnt++;
|
|
|
|
|
}
|
|
|
|
|
double volBase = volSum / volCnt;
|
|
|
|
|
// LEVEL: is this an active bar or a dead one? The change ratio cannot express this at all -
|
|
|
|
|
// two consecutive dead bars and two consecutive frantic ones both read as ~0 change.
|
|
|
|
|
double volLevel = (volBase > 0.0) ? vNow / volBase : 1.0;
|
|
|
|
|
double rangeAtr = (high - low) / atr;
|
|
|
|
|
// ABSORPTION: range delivered per unit of activity. A low value means heavy participation that
|
|
|
|
|
// went nowhere - supply meeting demand - which is a categorically different bar from heavy
|
|
|
|
|
// participation that travelled. The single change ratio conflates the two.
|
|
|
|
|
double absorption = (volLevel > 0.05) ? rangeAtr / volLevel : 0.0;
|
|
|
|
|
// ...and its converse, effort AND result together, which is the continuation reading.
|
|
|
|
|
double volXrange = volLevel * rangeAtr;
|
|
|
|
|
if(!TempData.Add(MathMax(-5.0, MathMin(5.0, volumeChangeRatio))) ||
|
|
|
|
|
!TempData.Add(MathMax(0.0, MathMin(5.0, volLevel))) ||
|
|
|
|
|
!TempData.Add(MathMax(0.0, MathMin(5.0, absorption))) ||
|
|
|
|
|
!TempData.Add(MathMax(0.0, MathMin(5.0, volXrange))))
|
refactor: split CExpertSignalAIBase implementation by responsibility
ExpertSignalAIBase.mqh was 8216 lines: the class declaration followed by 87
method bodies covering training, labelling, feature extraction, persistence,
chart drawing, online learning, the GA auto-tuner and inference, all in one
file. Train() alone is 1492 lines; a change to arrow drawing meant scrolling
past the era loop.
Moved the bodies into Expert\AIBase\, included at the bottom of the original
after the class declaration:
Training.mqh 1607 era loop, plateau ladder, checkpoint select, deploy
Features.mqh 1093 indicator creation + per-bar input feature vector
ChartUI.mqh 634 arrows, arrow persistence, status panel, cleanup
Persistence.mqh 492 .stats/.cfg sidecars, CPU-inference validation, copy
OnlineLearning.mqh 461 live continual learning, EMA shadow, OOS simulator
Labels.mqh 309 ZigZag pivot labels, async label-cache prebuild
AutoTune.mqh 275 genetic tuner (population, crossover, halving)
Inference.mqh 235 softmax, prior calibration, class priors
ExpertSignalAIBase.mqh 8216 -> 3131 (declaration + topology build only)
This is a pure relocation - verified mechanically, not by eye: HEAD's file
reconstructed from the eight partials plus the surviving remainder is
byte-identical to HEAD, span for span (scratchpad verify_split.py). No
declaration moved, no signature changed, no code rewritten, so behaviour is
unchanged by construction.
Compiles 0 errors, 0 warnings.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-29 00:42:45 -04:00
|
|
|
return false;
|
|
|
|
|
}
|
|
|
|
|
if(m_useTime)
|
|
|
|
|
{
|
|
|
|
|
// Normalize time (cyclical encoding)
|
|
|
|
|
if(!TempData.Add(sin(2 * M_PI * sTime.hour / 24.0)))
|
|
|
|
|
return false;
|
|
|
|
|
if(!TempData.Add(cos(2 * M_PI * sTime.hour / 24.0)))
|
|
|
|
|
return false;
|
|
|
|
|
if(!TempData.Add(sin(2 * M_PI * sTime.day_of_week / 7.0)))
|
|
|
|
|
return false;
|
|
|
|
|
if(!TempData.Add(cos(2 * M_PI * sTime.day_of_week / 7.0)))
|
|
|
|
|
return false;
|
|
|
|
|
if(!TempData.Add(sin(2 * M_PI * sTime.mon / 12.0)))
|
|
|
|
|
return false;
|
|
|
|
|
if(!TempData.Add(cos(2 * M_PI * sTime.mon / 12.0)))
|
|
|
|
|
return false;
|
|
|
|
|
}
|
|
|
|
|
if(m_useATR)
|
|
|
|
|
{
|
|
|
|
|
// ATR/close (volatility as a fraction of price), not raw ATR - the raw absolute value is
|
|
|
|
|
// itself unnormalized (e.g. ~0.0012 on EURUSD vs. ~1.5 on gold, and drifts over time even on
|
|
|
|
|
// one symbol as its price level changes), which is exactly the kind of scale-dependent
|
|
|
|
|
// feature this whole normalization pass is fixing everywhere else.
|
|
|
|
|
if(!TempData.Add(close != 0.0 ? atr / close : 0.0))
|
|
|
|
|
return false;
|
|
|
|
|
}
|
|
|
|
|
if(m_useMA)
|
|
|
|
|
{
|
|
|
|
|
// Same ATR-normalized distance-from-level convention as the base OHLC-from-open features above,
|
|
|
|
|
// just measured against the MA instead of the bar's own open - lets the network read where
|
|
|
|
|
// price sits relative to the same MA Signals\SignalMA.mqh votes on. Plus the MA's own
|
|
|
|
|
// bar-over-bar change (also ATR-normalized, since the MA lives in price units and ATR is
|
|
|
|
|
// already this codebase's scale reference for that - see m_useMA's declaration comment for why
|
|
|
|
|
// this isn't volume's previous-bar-ratio scheme instead).
|
|
|
|
|
double maNow = m_MA.GetData(0, idx);
|
|
|
|
|
double maPrev = m_MA.GetData(0, idx + 1);
|
|
|
|
|
if(maNow == EMPTY_VALUE || maPrev == EMPTY_VALUE)
|
|
|
|
|
return false;
|
|
|
|
|
if(!TempData.Add((open - maNow) / atr) ||
|
|
|
|
|
!TempData.Add((high - maNow) / atr) ||
|
|
|
|
|
!TempData.Add((low - maNow) / atr) ||
|
|
|
|
|
!TempData.Add((close - maNow) / atr) ||
|
|
|
|
|
!TempData.Add((maNow - maPrev) / atr))
|
|
|
|
|
return false;
|
|
|
|
|
}
|
|
|
|
|
if(m_useRSI)
|
|
|
|
|
{
|
|
|
|
|
// Already a 0-100 oscillator - /100 is the only transform needed to match the rest of the
|
|
|
|
|
// feature vector's scale (see m_useRSI's declaration comment).
|
|
|
|
|
double rsiNow = m_RSI.Main(idx);
|
|
|
|
|
if(rsiNow == EMPTY_VALUE)
|
|
|
|
|
return false;
|
|
|
|
|
if(!TempData.Add(rsiNow / 100.0))
|
|
|
|
|
return false;
|
|
|
|
|
}
|
|
|
|
|
if(m_useMACD)
|
|
|
|
|
{
|
|
|
|
|
// Main and signal lines are price-domain differences of two EMAs, so the same ATR normalization
|
|
|
|
|
// every other price-unit feature here uses applies unchanged. The third value is the histogram
|
|
|
|
|
// (main - signal): algebraically derivable from the first two, but handed over explicitly for the
|
|
|
|
|
// same reason the bullish/bearish flag is handed to the network alongside (close-open)/atr - a
|
|
|
|
|
// value the network would otherwise have to learn to subtract is better given directly, and the
|
|
|
|
|
// histogram (momentum ACCELERATION) is the one term nothing else in this vector carries.
|
|
|
|
|
double macdMain = m_MACDFeature.Main(idx);
|
|
|
|
|
double macdSignal = m_MACDFeature.Signal(idx);
|
|
|
|
|
if(macdMain == EMPTY_VALUE || macdSignal == EMPTY_VALUE)
|
|
|
|
|
return false;
|
|
|
|
|
if(!TempData.Add(macdMain / atr) ||
|
|
|
|
|
!TempData.Add(macdSignal / atr) ||
|
|
|
|
|
!TempData.Add((macdMain - macdSignal) / atr))
|
|
|
|
|
return false;
|
|
|
|
|
}
|
|
|
|
|
if(m_useIchimoku)
|
|
|
|
|
{
|
|
|
|
|
// LOOKAHEAD, the one thing that matters in this block. MT5's iIchimoku does NOT pre-shift its
|
|
|
|
|
// buffers - it stores raw per-bar values and shifts only the DRAWING (Ichimoku.mq5 sets
|
|
|
|
|
// PLOT_SHIFT=+Kijun on the Senkou A/B cloud plot and -Kijun on the Chikou plot). In series
|
|
|
|
|
// indexing that means:
|
|
|
|
|
// - SenkouSpan*(i) is computed FROM bar i and drawn Kijun bars into the FUTURE, so the cloud
|
|
|
|
|
// actually sitting under bar idx is SenkouSpan*(idx + kijun) - built from bar idx+kijun and
|
|
|
|
|
// older, hence strictly past data. Reading SenkouSpan*(idx) as "the cloud here" is the classic
|
|
|
|
|
// Ichimoku backtest bug and would leak Kijun bars of future information into every example.
|
|
|
|
|
// - SenkouSpan*(idx) with NO offset IS legitimate as the PROJECTED cloud - the part of the chart
|
|
|
|
|
// already drawn ahead of the current bar. A live bar at idx genuinely knows it (it is computed
|
|
|
|
|
// from bar idx), which is why it appears below as its own feature rather than being avoided.
|
|
|
|
|
// - ChinkouSpan(i) is just Close(i) drawn at i+Kijun, so the Chikou plotted AT bar idx would be
|
|
|
|
|
// Close(idx - kijun) - a FUTURE bar. It is never read. The lookahead-free statement of the same
|
|
|
|
|
// reading is "how far is this close from the close Kijun bars ago", the last feature below.
|
|
|
|
|
// Signals\SignalIchimoku.mqh's class comment documents the identical convention for the vote side.
|
|
|
|
|
int kijunShift = m_indicatorTuner.ichiKijun;
|
|
|
|
|
double tenkan = m_Ichimoku.TenkanSen(idx);
|
|
|
|
|
double kijun = m_Ichimoku.KijunSen(idx);
|
|
|
|
|
double spanA = m_Ichimoku.SenkouSpanA(idx + kijunShift); // cloud AS PLOTTED AT bar idx
|
|
|
|
|
double spanB = m_Ichimoku.SenkouSpanB(idx + kijunShift);
|
|
|
|
|
double futureSpanA = m_Ichimoku.SenkouSpanA(idx); // cloud projected AHEAD of bar idx
|
|
|
|
|
double futureSpanB = m_Ichimoku.SenkouSpanB(idx);
|
|
|
|
|
double closeLagRef = m_Close.GetData(idx + kijunShift); // Chikou reference, never idx - kijunShift
|
|
|
|
|
if(tenkan == EMPTY_VALUE || kijun == EMPTY_VALUE ||
|
|
|
|
|
spanA == EMPTY_VALUE || spanB == EMPTY_VALUE ||
|
|
|
|
|
futureSpanA == EMPTY_VALUE || futureSpanB == EMPTY_VALUE ||
|
|
|
|
|
closeLagRef == EMPTY_VALUE || closeLagRef <= 0.0)
|
|
|
|
|
return false;
|
|
|
|
|
if(!TempData.Add((close - tenkan) / atr) || // distance to the fast line
|
|
|
|
|
!TempData.Add((close - kijun) / atr) || // distance to the equilibrium line
|
|
|
|
|
!TempData.Add((tenkan - kijun) / atr) || // TK spread: sign = cross state, size = conviction
|
|
|
|
|
!TempData.Add((close - spanA) / atr) || // distance to each cloud edge, so the network can
|
|
|
|
|
!TempData.Add((close - spanB) / atr) || // place price above / inside / below the cloud
|
|
|
|
|
!TempData.Add((spanA - spanB) / atr) || // signed cloud thickness here: sign = regime, size = strength
|
|
|
|
|
!TempData.Add((futureSpanA - futureSpanB) / atr) || // same for the projected cloud - the "twist" ahead
|
|
|
|
|
!TempData.Add((close - closeLagRef) / atr)) // Chikou displacement, in its lookahead-free form
|
|
|
|
|
return false;
|
|
|
|
|
}
|
|
|
|
|
if(m_useNews)
|
|
|
|
|
{
|
|
|
|
|
// Event proximity + impact only - see this member's declaration comment and
|
|
|
|
|
// System\NewsRelevance.mqh's ImpactWeightedProximity() for why the forward-looking half
|
|
|
|
|
// (searchForward=true) isn't lookahead bias despite being computed for a historical bar.
|
fix(signals): revive a dead MA model, and demote Sanyaku from state to event
Two defects surfaced by research/test_classic.py, both verified fixed by re-running the
transcription against 178k bars of EURUSD H1.
CSignalMA model 1 could never fire. For any recursive average - and MA_TYPE_EMA is the
shipped default - MA(i) = a*Close(i) + (1-a)*MA(i+1), so
DiffMA(i) = a * (Close(i) - MA(i+1))
DiffCloseMA(i) = (1-a) * (Close(i) - MA(i+1))
are positive multiples of one quantity and always share a sign. Model 1 asks for a close
BELOW a RISING average, which is precisely the combination that identity forbids: 0.000%
of bars, either direction, any symbol. The MQL5 standard library this was ported from
defaults to MODE_SMA, where the two are merely correlated - the bug arrived with the EMA
default, not with the port. Reading the slope one bar back (DiffMAPrev) breaks the tie for
every MA type while keeping the model's stated meaning. Now fires on 7.92% of bars.
CSignalIchimoku model 11 fired on 27% of bars at weight 100. Sanyaku is three standing
STATES conjoined with no transition term, so it held across long stretches - and being
last in the if-chain at the top weight, the module's highest-conviction reading was also
its most common one, overwriting all eight event models below it on a quarter of all bars.
The old comment rejected an event form because "demanding all three flip on the same bar
would fire almost never" - true, but that is not the alternative. Kouten is the TURN: the
ALIGNMENT transitions, and only one role need change for it to. Testing !Sanyaku(idx+1)
fires once per aligned stretch. Now 2.17%, in line with Kumo breakout (2.4%) and the
strong TK cross (1.1%). DataReady() extended one bar deeper to cover the lookback.
Neither pattern showed edge before or after; this is about the models meaning what they
say and the vote not being dominated by a constant.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-01 17:14:34 -04:00
|
|
|
// What is DELIBERATELY not here is actual-vs-forecast surprise. Release TIMES are published
|
|
|
|
|
// in advance and never revised, so reading them for a historical bar is legitimate; released
|
|
|
|
|
// VALUES are neither. MqlCalendarValue.actual_value returns the FINAL figure, and the calendar
|
|
|
|
|
// keeps no as-of-release snapshot (revised_prev_value exists precisely because revisions
|
|
|
|
|
// happen), so a surprise feature computed for a 2019 bar would be built from a number nobody
|
|
|
|
|
// had in 2019. That is the same class of leak that made the RSI/MACD divergence models read
|
|
|
|
|
// +4 sigma in research/test_classic.py until two bars of lookahead were closed - except this
|
|
|
|
|
// one would survive into production and be paid for in real money.
|
refactor: split CExpertSignalAIBase implementation by responsibility
ExpertSignalAIBase.mqh was 8216 lines: the class declaration followed by 87
method bodies covering training, labelling, feature extraction, persistence,
chart drawing, online learning, the GA auto-tuner and inference, all in one
file. Train() alone is 1492 lines; a change to arrow drawing meant scrolling
past the era loop.
Moved the bodies into Expert\AIBase\, included at the bottom of the original
after the class declaration:
Training.mqh 1607 era loop, plateau ladder, checkpoint select, deploy
Features.mqh 1093 indicator creation + per-bar input feature vector
ChartUI.mqh 634 arrows, arrow persistence, status panel, cleanup
Persistence.mqh 492 .stats/.cfg sidecars, CPU-inference validation, copy
OnlineLearning.mqh 461 live continual learning, EMA shadow, OOS simulator
Labels.mqh 309 ZigZag pivot labels, async label-cache prebuild
AutoTune.mqh 275 genetic tuner (population, crossover, halving)
Inference.mqh 235 softmax, prior calibration, class priors
ExpertSignalAIBase.mqh 8216 -> 3131 (declaration + topology build only)
This is a pure relocation - verified mechanically, not by eye: HEAD's file
reconstructed from the eight partials plus the surviving remainder is
byte-identical to HEAD, span for span (scratchpad verify_split.py). No
declaration moved, no signature changed, no code rewritten, so behaviour is
unchanged by construction.
Compiles 0 errors, 0 warnings.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-29 00:42:45 -04:00
|
|
|
datetime barTime = m_Time.GetData(idx);
|
|
|
|
|
double newsRecency = ImpactWeightedProximity(m_symbol.Name(), barTime, m_newsFeatureWindowMinutes, false);
|
|
|
|
|
double newsProximity = ImpactWeightedProximity(m_symbol.Name(), barTime, m_newsFeatureWindowMinutes, true);
|
|
|
|
|
if(!TempData.Add(newsRecency) || !TempData.Add(newsProximity))
|
|
|
|
|
return false;
|
|
|
|
|
}
|
feat(ai): spread as a volatility-regime feature, and fix a stale-index cache in both new blocks
Adds spread/ATR and the spread change ratio as network inputs (EnableSpreadFeature,
default on). Spread is the one microstructure channel that is both FX-available and
genuinely historical in the Strategy Tester - "during testing, the spread is not modeled
but is taken from historical data" - so unlike swap, signed tick flow or depth of market it
is something a backtest can honestly validate.
What it encodes, stated precisely because the raw measurement overstates it.
research/test_spread.py found spr/atr the strongest single feature in this codebase, on 5
of 8 instrument/geometry cells at 2-4x any volume feature. But the barrier LABEL charges
the spread inside its own barriers, so a wide-spread bar is mechanically likelier to
resolve as a loss and the feature would partly be predicting its own cost model. Relabelling
at zero cost and re-measuring the identical feature showed 20-40% of it WAS that tautology
and the majority was not (XAUUSD retained 97%). What survives is a volatility-regime
reading: spread is near-fixed while ATR is not, so the ratio runs high exactly when
realised volatility is below its own ATR estimate, which genuinely predicts whether
ATR-scaled barriers get reached. It is UNSIGNED - Neutral-vs-directional only, never a side.
Also fixes a stale-index bug I introduced with the cross-asset panel and had just repeated
in the spread series. Both cached on length alone:
if(m_crossAsset.Bars() >= bars) return true;
MQL5 series indices are relative to NOW, so one new closed candle shifts every index by
one. Keyed only on length, the panel keeps serving its index 0 as a bar that is no longer
the newest, and every cross-asset value is read one bar out of step with the price features
sitting beside it in the same vector - silently, with no error and no shape change. This is
the same class of defect as the dtStudied watermark behind the zero-direction backtests.
Both now carry a datetime anchor on m_Time.GetData(0), the same invalidation key the
label/feature bar caches already use.
And a performance fix that fell out of it: with correct invalidation the panel rebuilds on
every new bar, and RefreshConvergedSignal runs per bar - which in the tester would mean one
full multi-symbol resample per simulated bar at training depth. Inference only reads bars
0..m_historyBars-1 plus the panel's own slow window, so it now requests exactly that. The
cache check is >=, so a deeper panel left from training still satisfies it.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-01 17:42:40 -04:00
|
|
|
if(m_useSpreadFeature)
|
|
|
|
|
{
|
|
|
|
|
// TWO values. What this block actually encodes is worth stating precisely, because the raw
|
|
|
|
|
// measurement overstates it.
|
|
|
|
|
//
|
|
|
|
|
// spr/atr measured as the single strongest feature in this codebase (research/test_spread.py):
|
|
|
|
|
// significant on 5 of 8 instrument/geometry cells at 2-4x any volume feature. But the barrier
|
|
|
|
|
// LABEL is computed with the spread charged inside it, so a high-spread bar has its barriers
|
|
|
|
|
// shifted adversely and is mechanically likelier to resolve as a loss - the feature would
|
|
|
|
|
// partly be predicting its own cost model, which is not tradeable. Re-labelling at zero cost
|
|
|
|
|
// and re-measuring the identical feature showed 20-40% of the signal WAS that tautology and
|
|
|
|
|
// the majority was not (XAUUSD kept 97%).
|
|
|
|
|
//
|
|
|
|
|
// What survives is a VOLATILITY-REGIME reading: the spread is near-fixed while ATR is not, so
|
|
|
|
|
// this ratio runs high exactly when realised volatility is below its own ATR estimate - which
|
|
|
|
|
// genuinely predicts whether ATR-scaled barriers get reached at all. Note it is UNSIGNED, like
|
|
|
|
|
// volume: it informs Neutral-vs-directional and can never pick a side.
|
|
|
|
|
double sprRatio = 0.0, sprChange = 0.0;
|
|
|
|
|
if(idx + 1 < m_spreadSeriesBars)
|
|
|
|
|
{
|
|
|
|
|
double sNow = (double)m_spreadSeries[idx] * m_symbol.Point();
|
|
|
|
|
double sPrev = (double)m_spreadSeries[idx + 1] * m_symbol.Point();
|
|
|
|
|
sprRatio = sNow / atr;
|
|
|
|
|
if(sPrev > 0.0)
|
|
|
|
|
sprChange = (sNow - sPrev) / sPrev;
|
|
|
|
|
}
|
|
|
|
|
if(!TempData.Add(MathMax(0.0, MathMin(5.0, sprRatio))) ||
|
|
|
|
|
!TempData.Add(MathMax(-5.0, MathMin(5.0, sprChange))))
|
|
|
|
|
return false;
|
|
|
|
|
}
|
fix(signals): revive a dead MA model, and demote Sanyaku from state to event
Two defects surfaced by research/test_classic.py, both verified fixed by re-running the
transcription against 178k bars of EURUSD H1.
CSignalMA model 1 could never fire. For any recursive average - and MA_TYPE_EMA is the
shipped default - MA(i) = a*Close(i) + (1-a)*MA(i+1), so
DiffMA(i) = a * (Close(i) - MA(i+1))
DiffCloseMA(i) = (1-a) * (Close(i) - MA(i+1))
are positive multiples of one quantity and always share a sign. Model 1 asks for a close
BELOW a RISING average, which is precisely the combination that identity forbids: 0.000%
of bars, either direction, any symbol. The MQL5 standard library this was ported from
defaults to MODE_SMA, where the two are merely correlated - the bug arrived with the EMA
default, not with the port. Reading the slope one bar back (DiffMAPrev) breaks the tie for
every MA type while keeping the model's stated meaning. Now fires on 7.92% of bars.
CSignalIchimoku model 11 fired on 27% of bars at weight 100. Sanyaku is three standing
STATES conjoined with no transition term, so it held across long stretches - and being
last in the if-chain at the top weight, the module's highest-conviction reading was also
its most common one, overwriting all eight event models below it on a quarter of all bars.
The old comment rejected an event form because "demanding all three flip on the same bar
would fire almost never" - true, but that is not the alternative. Kouten is the TURN: the
ALIGNMENT transitions, and only one role need change for it to. Testing !Sanyaku(idx+1)
fires once per aligned stretch. Now 2.17%, in line with Kumo breakout (2.4%) and the
strong TK cross (1.1%). DataReady() extended one bar deeper to cover the lookback.
Neither pattern showed edge before or after; this is about the models meaning what they
say and the vote not being dominated by a constant.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-01 17:14:34 -04:00
|
|
|
if(m_useCrossAsset)
|
|
|
|
|
{
|
|
|
|
|
// What every OTHER instrument was doing at this bar's timestamp - the one feature block here
|
|
|
|
|
// that is not a function of this symbol's own series. See System\CrossAsset.mqh.
|
|
|
|
|
// A panel that failed to build (no Market Watch pairs, unsynchronised history) yields a
|
|
|
|
|
// neutral 0-fill rather than rejecting the bar: the block is additive context, and losing
|
|
|
|
|
// every bar of training because a reference symbol was missing would be a far worse failure
|
|
|
|
|
// than training without the context. Features() reports that by returning false, which is
|
|
|
|
|
// logged once at build time rather than per bar.
|
|
|
|
|
double xa[];
|
|
|
|
|
m_crossAsset.Features(idx, xa);
|
|
|
|
|
for(int k = 0; k < CROSSASSET_FEATURES; k++)
|
|
|
|
|
if(!TempData.Add(xa[k]))
|
|
|
|
|
return false;
|
|
|
|
|
}
|
refactor: split CExpertSignalAIBase implementation by responsibility
ExpertSignalAIBase.mqh was 8216 lines: the class declaration followed by 87
method bodies covering training, labelling, feature extraction, persistence,
chart drawing, online learning, the GA auto-tuner and inference, all in one
file. Train() alone is 1492 lines; a change to arrow drawing meant scrolling
past the era loop.
Moved the bodies into Expert\AIBase\, included at the bottom of the original
after the class declaration:
Training.mqh 1607 era loop, plateau ladder, checkpoint select, deploy
Features.mqh 1093 indicator creation + per-bar input feature vector
ChartUI.mqh 634 arrows, arrow persistence, status panel, cleanup
Persistence.mqh 492 .stats/.cfg sidecars, CPU-inference validation, copy
OnlineLearning.mqh 461 live continual learning, EMA shadow, OOS simulator
Labels.mqh 309 ZigZag pivot labels, async label-cache prebuild
AutoTune.mqh 275 genetic tuner (population, crossover, halving)
Inference.mqh 235 softmax, prior calibration, class priors
ExpertSignalAIBase.mqh 8216 -> 3131 (declaration + topology build only)
This is a pure relocation - verified mechanically, not by eye: HEAD's file
reconstructed from the eight partials plus the surviving remainder is
byte-identical to HEAD, span for span (scratchpad verify_split.py). No
declaration moved, no signature changed, no code rewritten, so behaviour is
unchanged by construction.
Compiles 0 errors, 0 warnings.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-29 00:42:45 -04:00
|
|
|
if(m_useADCumulativeDelta)
|
|
|
|
|
{
|
fix: the trailing incumbent read the future across eras; cold AD blocks cached zeros as truth
Three findings from the 2026-08-11 audit:
1. The excursion head's trailing-quantile ring was deliberately never cleared
between eras ("a rolling estimate of the market, not of the era") - but
pass 3 re-walks the SAME OOS window every era, so at each walk's restart
the ring still held the outcome masks of the newest OOS bars from the
previous walk: the chronological FUTURE of the bars about to be scored.
For the first ~window+horizon pushes of every era the "trailing" incumbent
was partly a leading one - conservative for the gate (an informed incumbent
is a harder hurdle) but exactly the self-made-artifact class 06d4785 hunts.
The ring now clears at era-score reset; the warm-up bars simply don't score
the trail race, which the m_excTrailN gating already accounts for.
2. skillTrail compared the head's FULL-block Brier (pro-rated by coverage)
against the incumbent's subset sum - valid only if head skill is uniform
across the OOS walk, while the trail-scored subset systematically excludes
each era's warm-up bars. The audit also found m_excBrierHeadD/BaseD/
m_excOosHitsD declared, zeroed and never accumulated (dead since e2c9593
made every scored bar disjoint). The dead trio is replaced by
m_excBrierHeadT: the head's Brier accumulated only on the bars the warm
incumbent also scored, so the race now compares both predictors on an
identical bar set.
3. The AD/Wyckoff feature blocks read GetData with no EMPTY_VALUE guard; a
cold (still-calculating) indicator returns EMPTY_VALUE everywhere, the
sanitize loop rewrote that to 0.0, and the bar SUCCEEDED - so
BufferTempData cached an all-zero Wyckoff block as a success for the whole
bar frame: the one path the f6150ee only-cache-successes rule cannot see,
because it never fails (the ba13eef class, arriving through values that
never fail; a resumed model's era-0 prebuild starts milliseconds after
OnInit). ADIndicatorCold() probes the NEWEST bar - EMPTY_VALUE there means
async warm-up (transient reject, retried), while deep bars beyond the
buffered depth keep the sanitize loop's neutral-fill so degraded history
still trains. Also fixed m_featureCacheValid's declaration comment, which
still described the pre-f6150ee cached-miss semantics.
Compile: 0 errors, 0 warnings.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-11 18:20:24 -04:00
|
|
|
//--- COLD IS TRANSIENT, NOT ZERO (2026-08-11). These raw GetData reads have no EMPTY_VALUE
|
|
|
|
|
//--- guard of their own; a not-yet-calculated indicator returns EMPTY_VALUE for EVERY index,
|
|
|
|
|
//--- the sanitize loop at the bottom rewrites that to 0.0, the bar then SUCCEEDS - and
|
|
|
|
|
//--- BufferTempData caches it as a success for the whole bar frame. That is the one path the
|
|
|
|
|
//--- f6150ee only-cache-successes rule cannot see, because it never fails: on a resumed model
|
|
|
|
|
//--- the era-0 prebuild/MI report start milliseconds after OnInit and could train on all-zero
|
|
|
|
|
//--- Wyckoff/AD blocks for up to a full bar (the ba13eef failure class, arriving through
|
|
|
|
|
//--- values that never fail). ADIndicatorCold probes the NEWEST bar: EMPTY_VALUE there means
|
|
|
|
|
//--- the async calculation hasn't filled yet -> transient reject (never cached, retried like
|
|
|
|
|
//--- the cold-ATR guard above). A warm indicator whose DEEP bars read EMPTY_VALUE (beyond its
|
|
|
|
|
//--- buffer depth) is different - that stays the sanitize loop's neutral-fill, since rejecting
|
|
|
|
|
//--- those bars would starve training of legitimately degraded history.
|
|
|
|
|
if(ADIndicatorCold(m_ADCumulativeDelta))
|
|
|
|
|
return false;
|
refactor: split CExpertSignalAIBase implementation by responsibility
ExpertSignalAIBase.mqh was 8216 lines: the class declaration followed by 87
method bodies covering training, labelling, feature extraction, persistence,
chart drawing, online learning, the GA auto-tuner and inference, all in one
file. Train() alone is 1492 lines; a change to arrow drawing meant scrolling
past the era loop.
Moved the bodies into Expert\AIBase\, included at the bottom of the original
after the class declaration:
Training.mqh 1607 era loop, plateau ladder, checkpoint select, deploy
Features.mqh 1093 indicator creation + per-bar input feature vector
ChartUI.mqh 634 arrows, arrow persistence, status panel, cleanup
Persistence.mqh 492 .stats/.cfg sidecars, CPU-inference validation, copy
OnlineLearning.mqh 461 live continual learning, EMA shadow, OOS simulator
Labels.mqh 309 ZigZag pivot labels, async label-cache prebuild
AutoTune.mqh 275 genetic tuner (population, crossover, halving)
Inference.mqh 235 softmax, prior calibration, class priors
ExpertSignalAIBase.mqh 8216 -> 3131 (declaration + topology build only)
This is a pure relocation - verified mechanically, not by eye: HEAD's file
reconstructed from the eight partials plus the surviving remainder is
byte-identical to HEAD, span for span (scratchpad verify_split.py). No
declaration moved, no signature changed, no code rewritten, so behaviour is
unchanged by construction.
Compiles 0 errors, 0 warnings.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-29 00:42:45 -04:00
|
|
|
// buffers: 0=Pressure, 1=CumulativeDelta, 2=BullishPressure, 3=BearishPressure, 4=Absorption, 5=Initiative.
|
|
|
|
|
// CumulativeDelta (buffer 1) is now cumulativeDelta/sumVolume clamped +/-2 (same scale as every
|
|
|
|
|
// other buffer here, see ADCumulativeDelta.mq5) - a pure order-flow-imbalance ratio, distinct from
|
|
|
|
|
// Pressure (buffer 0), which is this same term further adjusted by Initiative/Absorption.
|
|
|
|
|
if(!TempData.Add(m_ADCumulativeDelta.GetData(0, idx)) || // Pressure
|
|
|
|
|
!TempData.Add(m_ADCumulativeDelta.GetData(1, idx)) || // CumulativeDelta
|
|
|
|
|
!TempData.Add(m_ADCumulativeDelta.GetData(2, idx)) || // BullishPressure
|
|
|
|
|
!TempData.Add(m_ADCumulativeDelta.GetData(3, idx)) || // BearishPressure
|
|
|
|
|
!TempData.Add(m_ADCumulativeDelta.GetData(4, idx)) || // Absorption
|
|
|
|
|
!TempData.Add(m_ADCumulativeDelta.GetData(5, idx))) // Initiative
|
|
|
|
|
return false;
|
|
|
|
|
}
|
|
|
|
|
if(m_useADShorteningOfThrust)
|
|
|
|
|
{
|
fix: the trailing incumbent read the future across eras; cold AD blocks cached zeros as truth
Three findings from the 2026-08-11 audit:
1. The excursion head's trailing-quantile ring was deliberately never cleared
between eras ("a rolling estimate of the market, not of the era") - but
pass 3 re-walks the SAME OOS window every era, so at each walk's restart
the ring still held the outcome masks of the newest OOS bars from the
previous walk: the chronological FUTURE of the bars about to be scored.
For the first ~window+horizon pushes of every era the "trailing" incumbent
was partly a leading one - conservative for the gate (an informed incumbent
is a harder hurdle) but exactly the self-made-artifact class 06d4785 hunts.
The ring now clears at era-score reset; the warm-up bars simply don't score
the trail race, which the m_excTrailN gating already accounts for.
2. skillTrail compared the head's FULL-block Brier (pro-rated by coverage)
against the incumbent's subset sum - valid only if head skill is uniform
across the OOS walk, while the trail-scored subset systematically excludes
each era's warm-up bars. The audit also found m_excBrierHeadD/BaseD/
m_excOosHitsD declared, zeroed and never accumulated (dead since e2c9593
made every scored bar disjoint). The dead trio is replaced by
m_excBrierHeadT: the head's Brier accumulated only on the bars the warm
incumbent also scored, so the race now compares both predictors on an
identical bar set.
3. The AD/Wyckoff feature blocks read GetData with no EMPTY_VALUE guard; a
cold (still-calculating) indicator returns EMPTY_VALUE everywhere, the
sanitize loop rewrote that to 0.0, and the bar SUCCEEDED - so
BufferTempData cached an all-zero Wyckoff block as a success for the whole
bar frame: the one path the f6150ee only-cache-successes rule cannot see,
because it never fails (the ba13eef class, arriving through values that
never fail; a resumed model's era-0 prebuild starts milliseconds after
OnInit). ADIndicatorCold() probes the NEWEST bar - EMPTY_VALUE there means
async warm-up (transient reject, retried), while deep bars beyond the
buffered depth keep the sanitize loop's neutral-fill so degraded history
still trains. Also fixed m_featureCacheValid's declaration comment, which
still described the pre-f6150ee cached-miss semantics.
Compile: 0 errors, 0 warnings.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-11 18:20:24 -04:00
|
|
|
if(ADIndicatorCold(m_ADShorteningOfThrust)) // see the CumulativeDelta block's comment
|
|
|
|
|
return false;
|
refactor: split CExpertSignalAIBase implementation by responsibility
ExpertSignalAIBase.mqh was 8216 lines: the class declaration followed by 87
method bodies covering training, labelling, feature extraction, persistence,
chart drawing, online learning, the GA auto-tuner and inference, all in one
file. Train() alone is 1492 lines; a change to arrow drawing meant scrolling
past the era loop.
Moved the bodies into Expert\AIBase\, included at the bottom of the original
after the class declaration:
Training.mqh 1607 era loop, plateau ladder, checkpoint select, deploy
Features.mqh 1093 indicator creation + per-bar input feature vector
ChartUI.mqh 634 arrows, arrow persistence, status panel, cleanup
Persistence.mqh 492 .stats/.cfg sidecars, CPU-inference validation, copy
OnlineLearning.mqh 461 live continual learning, EMA shadow, OOS simulator
Labels.mqh 309 ZigZag pivot labels, async label-cache prebuild
AutoTune.mqh 275 genetic tuner (population, crossover, halving)
Inference.mqh 235 softmax, prior calibration, class priors
ExpertSignalAIBase.mqh 8216 -> 3131 (declaration + topology build only)
This is a pure relocation - verified mechanically, not by eye: HEAD's file
reconstructed from the eight partials plus the surviving remainder is
byte-identical to HEAD, span for span (scratchpad verify_split.py). No
declaration moved, no signature changed, no code rewritten, so behaviour is
unchanged by construction.
Compiles 0 errors, 0 warnings.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-29 00:42:45 -04:00
|
|
|
// buffers: 0=SOT, 1=SOTEffortRegime, 2=SOTConfirmation, 3=SOTPushRegime
|
|
|
|
|
if(!TempData.Add(m_ADShorteningOfThrust.GetData(0, idx)) || // SOT
|
|
|
|
|
!TempData.Add(m_ADShorteningOfThrust.GetData(1, idx)) || // SOTEffortRegime
|
|
|
|
|
!TempData.Add(m_ADShorteningOfThrust.GetData(2, idx)) || // SOTConfirmation
|
|
|
|
|
!TempData.Add(m_ADShorteningOfThrust.GetData(3, idx))) // SOTPushRegime
|
|
|
|
|
return false;
|
|
|
|
|
}
|
|
|
|
|
if(m_useADWyckoffEventStream)
|
|
|
|
|
{
|
|
|
|
|
// buffers: 0=EventCode, 1=EventPhase, 2=ZoneTop, 3=ZoneBottom, 4=EventPrice, 5=StructuralPhase,
|
|
|
|
|
// 6=CHoCHTrendToRange, 7=CHoCHRangeToTrend, 8=SlopeAccumulationBullish, 9=SlopeAccumulationBearish,
|
|
|
|
|
// 10=SlopeDistributionBullish, 11=SlopeDistributionBearish, 12=Reaccumulation, 13=Redistribution.
|
|
|
|
|
// Buffer 4 (EventPrice) is deliberately skipped below - per the indicator's own source
|
|
|
|
|
// (ADWyckoffEventStream.mq5: "BufColor[wi]=(ev!=0)?C[i]:0;"), it's just this bar's close price
|
|
|
|
|
// echoed back when an event fires (0 otherwise), kept only so a charting/backtesting tool like
|
|
|
|
|
// StrategyQuant can anchor an arrow to a price. It carries no information the network doesn't
|
|
|
|
|
// already have (EventCode already flags whether an event fired; the close is already in the
|
|
|
|
|
// base OHLC features), and normalizing it as a "distance from close" like ZoneTop/ZoneBottom
|
|
|
|
|
// below would be actively wrong: it's close-close=0 on event bars but 0-close=-close (a raw,
|
|
|
|
|
// ATR-blown-up price) on every other bar - a huge, meaningless outlier feature.
|
2026-08-02 17:08:48 -04:00
|
|
|
// Buffer 1 (EventPhase) USED to be skipped for the same kind of reason - it was written as
|
|
|
|
|
// "BufPhase[wi]=(double)ev;", a byte-for-byte copy of EventCode. The 2026-08-02 rewrite made it a
|
|
|
|
|
// real reading: "BufPhase[wi]=(double)(phaseNow*((dirNow>=0)?1:-1))", i.e. the live range's own
|
|
|
|
|
// Wyckoff phase 1..5 signed by whether that range is accumulation (+) or distribution (-). That is
|
|
|
|
|
// NOT what StructuralPhase (buffer 5) carries: StructuralPhase is derived from the EVENT on this
|
|
|
|
|
// bar (MapStructuralPhase(ev)) and so is 0 on every bar where nothing fires, while EventPhase
|
|
|
|
|
// persists for the whole life of the range. The pair gives the network both "an event just put us
|
|
|
|
|
// in phase C" and "we are still in phase C" - so it is included below.
|
feat: mini-batch gradient accumulation (F4), front-end-aware capacity budget (F6), split Wyckoff categoricals (N1)
Completes the 2026-08-09 training audit. FORCES A RETRAIN of every
Wyckoff-enabled config (N1 re-keys the fingerprint), and BOTH DLLs must be
redeployed alongside the .ex5 - they carry new exports.
F4 - mini-batch accumulation, TRAIN_BATCH_SIZE=32. Training was pure online
SGD (one weight update per bar), which is the mechanical source of the
era-to-era whipsaw every downstream guard was built to cope with. The O(n^2)
outer product is native - AccumulateWeightGrad / AccumulateWeightGradConv /
AccumulateBufferInto in Network.cl, WarriorCPU and WarriorDML - while the
optimizer step is host-side MQL5 shared by all tiers (ApplyAccumToBlock), so
there is one Adam/SGD implementation instead of four that can drift.
- the LSTM needs no outer-product kernel (WeightsGradient already holds the
sample's full dW) but could NOT simply be left un-zeroed between samples:
CPU_LSTMSeqBackward/DML_LSTMSeqBackward memset it on entry. Hence a
separate accumulator plus an elementwise add.
- batch-norm gamma/beta accumulate in host arrays, not new BatchOptions
slots - BN_OPT_STRIDE is baked into every persisted .nnw.
- scoped to pass 2; online learning keeps immediate updates. Every save /
checkpoint / scoring boundary flushes, scaling by the real sample count.
- degrades to per-sample updates (one log line) on a tier that cannot
accumulate, so old devices and DLL-free builds are unaffected.
- verified offline: DirectML/batch_accum_check.cpp drives the real exports
against an independent reference; at B=1 the accumulator matches the
shipped unbatched kernel's own gradient to 1.1e-16. Math only - the
in-situ check remains the per-layer dW/W report on a real era.
F6 - ComputeFirstLayerWidth budgeted against the RAW input width even where a
conv/LSTM front end had already reduced it, so an LSTM's dense stack was
charged for 1,280 inputs when it receives 64. Confirmed from the deployed
.cfg files: CONV, LSTM and HYBRID were all pinned at the 16-unit floor. Now
budgeted against the front-end output and capped at it (never fan out), with
the derivation reordered so both stages settle first.
N1 - EventCode/EventPhase/StructuralPhase are signed categoricals packing
direction and Wyckoff stage into one scalar across a sign discontinuity. Split
into direction + [0,1] magnitude, the same convention the base OHLC block uses.
Information-preserving; 13 readings now occupy 16 inputs.
Compiled clean (0 errors, 0 warnings); both DLLs rebuilt.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-09 11:48:03 -04:00
|
|
|
// SIGN AND MAGNITUDE SPLIT (2026-08-09 audit, N1). All three of these buffers are signed
|
|
|
|
|
// categoricals of the form (stage * direction), packed into one scalar:
|
|
|
|
|
// EventCode +-1..7 sign = accumulation/distribution, |v| = the Wyckoff schematic stage
|
|
|
|
|
// (1 PS, 2 SC, 3 AR, 4 ST, 5 Spring/UTAD, 6 LPS/LPSY, 7 SOS/SOW)
|
|
|
|
|
// EventPhase +-1..5 the LIVE range's phase, same sign convention, persists between events
|
|
|
|
|
// StructuralPhase +-1..5 this bar's event mapped to a phase, 0 when nothing fired
|
|
|
|
|
// Fed raw, each one asks the network to disentangle "which way" from "how far through the
|
|
|
|
|
// schematic" out of a single continuous value - and to do it across a sign change, where the
|
|
|
|
|
// ordinal jumps from -1 to +1 with nothing in between. That is the exact ambiguity the base OHLC
|
|
|
|
|
// block calls out and fixes by handing direction its own +1/-1/0 flag beside (close-open)/atr;
|
|
|
|
|
// these are the same shape of value and get the same treatment.
|
|
|
|
|
// Information-preserving: (dir, mag) reconstructs the original exactly, so this is a re-encoding
|
|
|
|
|
// and not a feature change. Magnitudes are scaled onto [0,1] by their own maxima so they sit in
|
|
|
|
|
// the same range as the rest of the vector instead of reaching 7.
|
|
|
|
|
// The magnitudes ARE meaningfully ordinal - the schematic is a sequence, not a set of unrelated
|
|
|
|
|
// labels - which is why they stay scalars rather than being one-hot expanded across 7 inputs.
|
fix: the trailing incumbent read the future across eras; cold AD blocks cached zeros as truth
Three findings from the 2026-08-11 audit:
1. The excursion head's trailing-quantile ring was deliberately never cleared
between eras ("a rolling estimate of the market, not of the era") - but
pass 3 re-walks the SAME OOS window every era, so at each walk's restart
the ring still held the outcome masks of the newest OOS bars from the
previous walk: the chronological FUTURE of the bars about to be scored.
For the first ~window+horizon pushes of every era the "trailing" incumbent
was partly a leading one - conservative for the gate (an informed incumbent
is a harder hurdle) but exactly the self-made-artifact class 06d4785 hunts.
The ring now clears at era-score reset; the warm-up bars simply don't score
the trail race, which the m_excTrailN gating already accounts for.
2. skillTrail compared the head's FULL-block Brier (pro-rated by coverage)
against the incumbent's subset sum - valid only if head skill is uniform
across the OOS walk, while the trail-scored subset systematically excludes
each era's warm-up bars. The audit also found m_excBrierHeadD/BaseD/
m_excOosHitsD declared, zeroed and never accumulated (dead since e2c9593
made every scored bar disjoint). The dead trio is replaced by
m_excBrierHeadT: the head's Brier accumulated only on the bars the warm
incumbent also scored, so the race now compares both predictors on an
identical bar set.
3. The AD/Wyckoff feature blocks read GetData with no EMPTY_VALUE guard; a
cold (still-calculating) indicator returns EMPTY_VALUE everywhere, the
sanitize loop rewrote that to 0.0, and the bar SUCCEEDED - so
BufferTempData cached an all-zero Wyckoff block as a success for the whole
bar frame: the one path the f6150ee only-cache-successes rule cannot see,
because it never fails (the ba13eef class, arriving through values that
never fail; a resumed model's era-0 prebuild starts milliseconds after
OnInit). ADIndicatorCold() probes the NEWEST bar - EMPTY_VALUE there means
async warm-up (transient reject, retried), while deep bars beyond the
buffered depth keep the sanitize loop's neutral-fill so degraded history
still trains. Also fixed m_featureCacheValid's declaration comment, which
still described the pre-f6150ee cached-miss semantics.
Compile: 0 errors, 0 warnings.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-11 18:20:24 -04:00
|
|
|
if(ADIndicatorCold(m_ADWyckoffEventStream)) // see the CumulativeDelta block's comment
|
|
|
|
|
return false;
|
feat: mini-batch gradient accumulation (F4), front-end-aware capacity budget (F6), split Wyckoff categoricals (N1)
Completes the 2026-08-09 training audit. FORCES A RETRAIN of every
Wyckoff-enabled config (N1 re-keys the fingerprint), and BOTH DLLs must be
redeployed alongside the .ex5 - they carry new exports.
F4 - mini-batch accumulation, TRAIN_BATCH_SIZE=32. Training was pure online
SGD (one weight update per bar), which is the mechanical source of the
era-to-era whipsaw every downstream guard was built to cope with. The O(n^2)
outer product is native - AccumulateWeightGrad / AccumulateWeightGradConv /
AccumulateBufferInto in Network.cl, WarriorCPU and WarriorDML - while the
optimizer step is host-side MQL5 shared by all tiers (ApplyAccumToBlock), so
there is one Adam/SGD implementation instead of four that can drift.
- the LSTM needs no outer-product kernel (WeightsGradient already holds the
sample's full dW) but could NOT simply be left un-zeroed between samples:
CPU_LSTMSeqBackward/DML_LSTMSeqBackward memset it on entry. Hence a
separate accumulator plus an elementwise add.
- batch-norm gamma/beta accumulate in host arrays, not new BatchOptions
slots - BN_OPT_STRIDE is baked into every persisted .nnw.
- scoped to pass 2; online learning keeps immediate updates. Every save /
checkpoint / scoring boundary flushes, scaling by the real sample count.
- degrades to per-sample updates (one log line) on a tier that cannot
accumulate, so old devices and DLL-free builds are unaffected.
- verified offline: DirectML/batch_accum_check.cpp drives the real exports
against an independent reference; at B=1 the accumulator matches the
shipped unbatched kernel's own gradient to 1.1e-16. Math only - the
in-situ check remains the per-layer dW/W report on a real era.
F6 - ComputeFirstLayerWidth budgeted against the RAW input width even where a
conv/LSTM front end had already reduced it, so an LSTM's dense stack was
charged for 1,280 inputs when it receives 64. Confirmed from the deployed
.cfg files: CONV, LSTM and HYBRID were all pinned at the 16-unit floor. Now
budgeted against the front-end output and capped at it (never fan out), with
the derivation reordered so both stages settle first.
N1 - EventCode/EventPhase/StructuralPhase are signed categoricals packing
direction and Wyckoff stage into one scalar across a sign discontinuity. Split
into direction + [0,1] magnitude, the same convention the base OHLC block uses.
Information-preserving; 13 readings now occupy 16 inputs.
Compiled clean (0 errors, 0 warnings); both DLLs rebuilt.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-09 11:48:03 -04:00
|
|
|
double wesEvent = m_ADWyckoffEventStream.GetData(0, idx);
|
|
|
|
|
double wesLivePhase = m_ADWyckoffEventStream.GetData(1, idx);
|
|
|
|
|
double wesStructPhase = m_ADWyckoffEventStream.GetData(5, idx);
|
|
|
|
|
if(!TempData.Add(wesEvent > 0 ? 1.0 : (wesEvent < 0 ? -1.0 : 0.0)) || // event direction
|
|
|
|
|
!TempData.Add(MathMin(1.0, MathAbs(wesEvent) / 7.0)) || // event stage
|
|
|
|
|
!TempData.Add(wesLivePhase > 0 ? 1.0 : (wesLivePhase < 0 ? -1.0 : 0.0)) || // live-range direction
|
|
|
|
|
!TempData.Add(MathMin(1.0, MathAbs(wesLivePhase) / 5.0)) || // live-range phase
|
refactor: split CExpertSignalAIBase implementation by responsibility
ExpertSignalAIBase.mqh was 8216 lines: the class declaration followed by 87
method bodies covering training, labelling, feature extraction, persistence,
chart drawing, online learning, the GA auto-tuner and inference, all in one
file. Train() alone is 1492 lines; a change to arrow drawing meant scrolling
past the era loop.
Moved the bodies into Expert\AIBase\, included at the bottom of the original
after the class declaration:
Training.mqh 1607 era loop, plateau ladder, checkpoint select, deploy
Features.mqh 1093 indicator creation + per-bar input feature vector
ChartUI.mqh 634 arrows, arrow persistence, status panel, cleanup
Persistence.mqh 492 .stats/.cfg sidecars, CPU-inference validation, copy
OnlineLearning.mqh 461 live continual learning, EMA shadow, OOS simulator
Labels.mqh 309 ZigZag pivot labels, async label-cache prebuild
AutoTune.mqh 275 genetic tuner (population, crossover, halving)
Inference.mqh 235 softmax, prior calibration, class priors
ExpertSignalAIBase.mqh 8216 -> 3131 (declaration + topology build only)
This is a pure relocation - verified mechanically, not by eye: HEAD's file
reconstructed from the eight partials plus the surviving remainder is
byte-identical to HEAD, span for span (scratchpad verify_split.py). No
declaration moved, no signature changed, no code rewritten, so behaviour is
unchanged by construction.
Compiles 0 errors, 0 warnings.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-29 00:42:45 -04:00
|
|
|
!TempData.Add((m_ADWyckoffEventStream.GetData(2, idx) - close) / atr) || // ZoneTop
|
|
|
|
|
!TempData.Add((m_ADWyckoffEventStream.GetData(3, idx) - close) / atr) || // ZoneBottom
|
feat: mini-batch gradient accumulation (F4), front-end-aware capacity budget (F6), split Wyckoff categoricals (N1)
Completes the 2026-08-09 training audit. FORCES A RETRAIN of every
Wyckoff-enabled config (N1 re-keys the fingerprint), and BOTH DLLs must be
redeployed alongside the .ex5 - they carry new exports.
F4 - mini-batch accumulation, TRAIN_BATCH_SIZE=32. Training was pure online
SGD (one weight update per bar), which is the mechanical source of the
era-to-era whipsaw every downstream guard was built to cope with. The O(n^2)
outer product is native - AccumulateWeightGrad / AccumulateWeightGradConv /
AccumulateBufferInto in Network.cl, WarriorCPU and WarriorDML - while the
optimizer step is host-side MQL5 shared by all tiers (ApplyAccumToBlock), so
there is one Adam/SGD implementation instead of four that can drift.
- the LSTM needs no outer-product kernel (WeightsGradient already holds the
sample's full dW) but could NOT simply be left un-zeroed between samples:
CPU_LSTMSeqBackward/DML_LSTMSeqBackward memset it on entry. Hence a
separate accumulator plus an elementwise add.
- batch-norm gamma/beta accumulate in host arrays, not new BatchOptions
slots - BN_OPT_STRIDE is baked into every persisted .nnw.
- scoped to pass 2; online learning keeps immediate updates. Every save /
checkpoint / scoring boundary flushes, scaling by the real sample count.
- degrades to per-sample updates (one log line) on a tier that cannot
accumulate, so old devices and DLL-free builds are unaffected.
- verified offline: DirectML/batch_accum_check.cpp drives the real exports
against an independent reference; at B=1 the accumulator matches the
shipped unbatched kernel's own gradient to 1.1e-16. Math only - the
in-situ check remains the per-layer dW/W report on a real era.
F6 - ComputeFirstLayerWidth budgeted against the RAW input width even where a
conv/LSTM front end had already reduced it, so an LSTM's dense stack was
charged for 1,280 inputs when it receives 64. Confirmed from the deployed
.cfg files: CONV, LSTM and HYBRID were all pinned at the 16-unit floor. Now
budgeted against the front-end output and capped at it (never fan out), with
the derivation reordered so both stages settle first.
N1 - EventCode/EventPhase/StructuralPhase are signed categoricals packing
direction and Wyckoff stage into one scalar across a sign discontinuity. Split
into direction + [0,1] magnitude, the same convention the base OHLC block uses.
Information-preserving; 13 readings now occupy 16 inputs.
Compiled clean (0 errors, 0 warnings); both DLLs rebuilt.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-09 11:48:03 -04:00
|
|
|
!TempData.Add(wesStructPhase > 0 ? 1.0 : (wesStructPhase < 0 ? -1.0 : 0.0)) || // struct direction
|
|
|
|
|
!TempData.Add(MathMin(1.0, MathAbs(wesStructPhase) / 5.0)) || // struct phase
|
refactor: split CExpertSignalAIBase implementation by responsibility
ExpertSignalAIBase.mqh was 8216 lines: the class declaration followed by 87
method bodies covering training, labelling, feature extraction, persistence,
chart drawing, online learning, the GA auto-tuner and inference, all in one
file. Train() alone is 1492 lines; a change to arrow drawing meant scrolling
past the era loop.
Moved the bodies into Expert\AIBase\, included at the bottom of the original
after the class declaration:
Training.mqh 1607 era loop, plateau ladder, checkpoint select, deploy
Features.mqh 1093 indicator creation + per-bar input feature vector
ChartUI.mqh 634 arrows, arrow persistence, status panel, cleanup
Persistence.mqh 492 .stats/.cfg sidecars, CPU-inference validation, copy
OnlineLearning.mqh 461 live continual learning, EMA shadow, OOS simulator
Labels.mqh 309 ZigZag pivot labels, async label-cache prebuild
AutoTune.mqh 275 genetic tuner (population, crossover, halving)
Inference.mqh 235 softmax, prior calibration, class priors
ExpertSignalAIBase.mqh 8216 -> 3131 (declaration + topology build only)
This is a pure relocation - verified mechanically, not by eye: HEAD's file
reconstructed from the eight partials plus the surviving remainder is
byte-identical to HEAD, span for span (scratchpad verify_split.py). No
declaration moved, no signature changed, no code rewritten, so behaviour is
unchanged by construction.
Compiles 0 errors, 0 warnings.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-29 00:42:45 -04:00
|
|
|
!TempData.Add(m_ADWyckoffEventStream.GetData(6, idx)) || // CHoCHTrendToRange
|
|
|
|
|
!TempData.Add(m_ADWyckoffEventStream.GetData(7, idx)) || // CHoCHRangeToTrend
|
|
|
|
|
!TempData.Add(m_ADWyckoffEventStream.GetData(8, idx)) || // SlopeAccumulationBullish
|
|
|
|
|
!TempData.Add(m_ADWyckoffEventStream.GetData(9, idx)) || // SlopeAccumulationBearish
|
|
|
|
|
!TempData.Add(m_ADWyckoffEventStream.GetData(10, idx)) || // SlopeDistributionBullish
|
|
|
|
|
!TempData.Add(m_ADWyckoffEventStream.GetData(11, idx)) || // SlopeDistributionBearish
|
|
|
|
|
!TempData.Add(m_ADWyckoffEventStream.GetData(12, idx)) || // Reaccumulation
|
|
|
|
|
!TempData.Add(m_ADWyckoffEventStream.GetData(13, idx))) // Redistribution
|
|
|
|
|
return false;
|
|
|
|
|
}
|
|
|
|
|
if(m_useADWyckoffFailedStructure)
|
|
|
|
|
{
|
fix: the trailing incumbent read the future across eras; cold AD blocks cached zeros as truth
Three findings from the 2026-08-11 audit:
1. The excursion head's trailing-quantile ring was deliberately never cleared
between eras ("a rolling estimate of the market, not of the era") - but
pass 3 re-walks the SAME OOS window every era, so at each walk's restart
the ring still held the outcome masks of the newest OOS bars from the
previous walk: the chronological FUTURE of the bars about to be scored.
For the first ~window+horizon pushes of every era the "trailing" incumbent
was partly a leading one - conservative for the gate (an informed incumbent
is a harder hurdle) but exactly the self-made-artifact class 06d4785 hunts.
The ring now clears at era-score reset; the warm-up bars simply don't score
the trail race, which the m_excTrailN gating already accounts for.
2. skillTrail compared the head's FULL-block Brier (pro-rated by coverage)
against the incumbent's subset sum - valid only if head skill is uniform
across the OOS walk, while the trail-scored subset systematically excludes
each era's warm-up bars. The audit also found m_excBrierHeadD/BaseD/
m_excOosHitsD declared, zeroed and never accumulated (dead since e2c9593
made every scored bar disjoint). The dead trio is replaced by
m_excBrierHeadT: the head's Brier accumulated only on the bars the warm
incumbent also scored, so the race now compares both predictors on an
identical bar set.
3. The AD/Wyckoff feature blocks read GetData with no EMPTY_VALUE guard; a
cold (still-calculating) indicator returns EMPTY_VALUE everywhere, the
sanitize loop rewrote that to 0.0, and the bar SUCCEEDED - so
BufferTempData cached an all-zero Wyckoff block as a success for the whole
bar frame: the one path the f6150ee only-cache-successes rule cannot see,
because it never fails (the ba13eef class, arriving through values that
never fail; a resumed model's era-0 prebuild starts milliseconds after
OnInit). ADIndicatorCold() probes the NEWEST bar - EMPTY_VALUE there means
async warm-up (transient reject, retried), while deep bars beyond the
buffered depth keep the sanitize loop's neutral-fill so degraded history
still trains. Also fixed m_featureCacheValid's declaration comment, which
still described the pre-f6150ee cached-miss semantics.
Compile: 0 errors, 0 warnings.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-11 18:20:24 -04:00
|
|
|
if(ADIndicatorCold(m_ADWyckoffFailedStructure)) // see the CumulativeDelta block's comment
|
|
|
|
|
return false;
|
refactor: split CExpertSignalAIBase implementation by responsibility
ExpertSignalAIBase.mqh was 8216 lines: the class declaration followed by 87
method bodies covering training, labelling, feature extraction, persistence,
chart drawing, online learning, the GA auto-tuner and inference, all in one
file. Train() alone is 1492 lines; a change to arrow drawing meant scrolling
past the era loop.
Moved the bodies into Expert\AIBase\, included at the bottom of the original
after the class declaration:
Training.mqh 1607 era loop, plateau ladder, checkpoint select, deploy
Features.mqh 1093 indicator creation + per-bar input feature vector
ChartUI.mqh 634 arrows, arrow persistence, status panel, cleanup
Persistence.mqh 492 .stats/.cfg sidecars, CPU-inference validation, copy
OnlineLearning.mqh 461 live continual learning, EMA shadow, OOS simulator
Labels.mqh 309 ZigZag pivot labels, async label-cache prebuild
AutoTune.mqh 275 genetic tuner (population, crossover, halving)
Inference.mqh 235 softmax, prior calibration, class priors
ExpertSignalAIBase.mqh 8216 -> 3131 (declaration + topology build only)
This is a pure relocation - verified mechanically, not by eye: HEAD's file
reconstructed from the eight partials plus the surviving remainder is
byte-identical to HEAD, span for span (scratchpad verify_split.py). No
declaration moved, no signature changed, no code rewritten, so behaviour is
unchanged by construction.
Compiles 0 errors, 0 warnings.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-29 00:42:45 -04:00
|
|
|
// buffers: 0=Value, 1=BullishStructuralFailure, 2=BearishStructuralFailure, 3=FailedAccumulation, 4=FailedDistribution
|
|
|
|
|
if(!TempData.Add(m_ADWyckoffFailedStructure.GetData(0, idx)) || // Value
|
|
|
|
|
!TempData.Add(m_ADWyckoffFailedStructure.GetData(1, idx)) || // BullishStructuralFailure
|
|
|
|
|
!TempData.Add(m_ADWyckoffFailedStructure.GetData(2, idx)) || // BearishStructuralFailure
|
|
|
|
|
!TempData.Add(m_ADWyckoffFailedStructure.GetData(3, idx)) || // FailedAccumulation
|
|
|
|
|
!TempData.Add(m_ADWyckoffFailedStructure.GetData(4, idx))) // FailedDistribution
|
|
|
|
|
return false;
|
|
|
|
|
}
|
|
|
|
|
if(m_useADWyckoffSignificantBarInversion)
|
|
|
|
|
{
|
fix: the trailing incumbent read the future across eras; cold AD blocks cached zeros as truth
Three findings from the 2026-08-11 audit:
1. The excursion head's trailing-quantile ring was deliberately never cleared
between eras ("a rolling estimate of the market, not of the era") - but
pass 3 re-walks the SAME OOS window every era, so at each walk's restart
the ring still held the outcome masks of the newest OOS bars from the
previous walk: the chronological FUTURE of the bars about to be scored.
For the first ~window+horizon pushes of every era the "trailing" incumbent
was partly a leading one - conservative for the gate (an informed incumbent
is a harder hurdle) but exactly the self-made-artifact class 06d4785 hunts.
The ring now clears at era-score reset; the warm-up bars simply don't score
the trail race, which the m_excTrailN gating already accounts for.
2. skillTrail compared the head's FULL-block Brier (pro-rated by coverage)
against the incumbent's subset sum - valid only if head skill is uniform
across the OOS walk, while the trail-scored subset systematically excludes
each era's warm-up bars. The audit also found m_excBrierHeadD/BaseD/
m_excOosHitsD declared, zeroed and never accumulated (dead since e2c9593
made every scored bar disjoint). The dead trio is replaced by
m_excBrierHeadT: the head's Brier accumulated only on the bars the warm
incumbent also scored, so the race now compares both predictors on an
identical bar set.
3. The AD/Wyckoff feature blocks read GetData with no EMPTY_VALUE guard; a
cold (still-calculating) indicator returns EMPTY_VALUE everywhere, the
sanitize loop rewrote that to 0.0, and the bar SUCCEEDED - so
BufferTempData cached an all-zero Wyckoff block as a success for the whole
bar frame: the one path the f6150ee only-cache-successes rule cannot see,
because it never fails (the ba13eef class, arriving through values that
never fail; a resumed model's era-0 prebuild starts milliseconds after
OnInit). ADIndicatorCold() probes the NEWEST bar - EMPTY_VALUE there means
async warm-up (transient reject, retried), while deep bars beyond the
buffered depth keep the sanitize loop's neutral-fill so degraded history
still trains. Also fixed m_featureCacheValid's declaration comment, which
still described the pre-f6150ee cached-miss semantics.
Compile: 0 errors, 0 warnings.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-11 18:20:24 -04:00
|
|
|
if(ADIndicatorCold(m_ADWyckoffSignificantBarInversion)) // see the CumulativeDelta block's comment
|
|
|
|
|
return false;
|
refactor: split CExpertSignalAIBase implementation by responsibility
ExpertSignalAIBase.mqh was 8216 lines: the class declaration followed by 87
method bodies covering training, labelling, feature extraction, persistence,
chart drawing, online learning, the GA auto-tuner and inference, all in one
file. Train() alone is 1492 lines; a change to arrow drawing meant scrolling
past the era loop.
Moved the bodies into Expert\AIBase\, included at the bottom of the original
after the class declaration:
Training.mqh 1607 era loop, plateau ladder, checkpoint select, deploy
Features.mqh 1093 indicator creation + per-bar input feature vector
ChartUI.mqh 634 arrows, arrow persistence, status panel, cleanup
Persistence.mqh 492 .stats/.cfg sidecars, CPU-inference validation, copy
OnlineLearning.mqh 461 live continual learning, EMA shadow, OOS simulator
Labels.mqh 309 ZigZag pivot labels, async label-cache prebuild
AutoTune.mqh 275 genetic tuner (population, crossover, halving)
Inference.mqh 235 softmax, prior calibration, class priors
ExpertSignalAIBase.mqh 8216 -> 3131 (declaration + topology build only)
This is a pure relocation - verified mechanically, not by eye: HEAD's file
reconstructed from the eight partials plus the surviving remainder is
byte-identical to HEAD, span for span (scratchpad verify_split.py). No
declaration moved, no signature changed, no code rewritten, so behaviour is
unchanged by construction.
Compiles 0 errors, 0 warnings.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-29 00:42:45 -04:00
|
|
|
// buffers: 0=SignificantBarQuality, 1=BullishSignificantBar, 2=BearishSignificantBar, 3=BullishControlFlip, 4=BearishControlFlip
|
|
|
|
|
if(!TempData.Add(m_ADWyckoffSignificantBarInversion.GetData(0, idx)) || // SignificantBarQuality
|
|
|
|
|
!TempData.Add(m_ADWyckoffSignificantBarInversion.GetData(1, idx)) || // BullishSignificantBar
|
|
|
|
|
!TempData.Add(m_ADWyckoffSignificantBarInversion.GetData(2, idx)) || // BearishSignificantBar
|
|
|
|
|
!TempData.Add(m_ADWyckoffSignificantBarInversion.GetData(3, idx)) || // BullishControlFlip
|
|
|
|
|
!TempData.Add(m_ADWyckoffSignificantBarInversion.GetData(4, idx))) // BearishControlFlip
|
|
|
|
|
return false;
|
|
|
|
|
}
|
2026-08-02 01:09:18 -04:00
|
|
|
//--- ONE finiteness/plausibility gate for the whole bar, rather than 60-odd individually guarded
|
|
|
|
|
//--- Add() calls. Most blocks above already clamp their own output; the AD/Wyckoff blocks
|
|
|
|
|
//--- deliberately do not, because those indicators emit plain readings with no natural range. But
|
|
|
|
|
//--- MQL5's CDoubleBuffer::At() returns EMPTY_VALUE (DBL_MAX) for any index it holds no data for,
|
|
|
|
|
//--- and (EMPTY_VALUE - close) / atr is ~1e307: still FINITE, so it sails through every downstream
|
|
|
|
|
//--- isfinite() check, and still large enough to overflow the first batch-norm layer's running
|
|
|
|
|
//--- variance and latch that layer to NaN permanently (see NormalizeHost in AI\NeuronBatchNorm.mqh -
|
|
|
|
|
//--- that is the 2026-08-02 "BufferWrite failed for buffer 3" run). Neutral-fill rather than reject
|
|
|
|
|
//--- the bar: "this indicator has no reading here" is the degraded-but-usable case that the swing,
|
|
|
|
|
//--- cross-asset and spread blocks above all already handle the same way.
|
|
|
|
|
int featureEnd = TempData.Total();
|
|
|
|
|
for(int f = featureStart; f < featureEnd; f++)
|
|
|
|
|
{
|
|
|
|
|
double v = TempData.At(f);
|
|
|
|
|
if((!MathIsValidNumber(v) || MathAbs(v) > FEATURE_ABS_MAX) && !TempData.Update(f, 0.0))
|
|
|
|
|
return false;
|
|
|
|
|
}
|
refactor: split CExpertSignalAIBase implementation by responsibility
ExpertSignalAIBase.mqh was 8216 lines: the class declaration followed by 87
method bodies covering training, labelling, feature extraction, persistence,
chart drawing, online learning, the GA auto-tuner and inference, all in one
file. Train() alone is 1492 lines; a change to arrow drawing meant scrolling
past the era loop.
Moved the bodies into Expert\AIBase\, included at the bottom of the original
after the class declaration:
Training.mqh 1607 era loop, plateau ladder, checkpoint select, deploy
Features.mqh 1093 indicator creation + per-bar input feature vector
ChartUI.mqh 634 arrows, arrow persistence, status panel, cleanup
Persistence.mqh 492 .stats/.cfg sidecars, CPU-inference validation, copy
OnlineLearning.mqh 461 live continual learning, EMA shadow, OOS simulator
Labels.mqh 309 ZigZag pivot labels, async label-cache prebuild
AutoTune.mqh 275 genetic tuner (population, crossover, halving)
Inference.mqh 235 softmax, prior calibration, class priors
ExpertSignalAIBase.mqh 8216 -> 3131 (declaration + topology build only)
This is a pure relocation - verified mechanically, not by eye: HEAD's file
reconstructed from the eight partials plus the surviving remainder is
byte-identical to HEAD, span for span (scratchpad verify_split.py). No
declaration moved, no signature changed, no code rewritten, so behaviour is
unchanged by construction.
Compiles 0 errors, 0 warnings.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-29 00:42:45 -04:00
|
|
|
return true;
|
|
|
|
|
}
|
|
|
|
|
//+------------------------------------------------------------------+
|
fix: the trailing incumbent read the future across eras; cold AD blocks cached zeros as truth
Three findings from the 2026-08-11 audit:
1. The excursion head's trailing-quantile ring was deliberately never cleared
between eras ("a rolling estimate of the market, not of the era") - but
pass 3 re-walks the SAME OOS window every era, so at each walk's restart
the ring still held the outcome masks of the newest OOS bars from the
previous walk: the chronological FUTURE of the bars about to be scored.
For the first ~window+horizon pushes of every era the "trailing" incumbent
was partly a leading one - conservative for the gate (an informed incumbent
is a harder hurdle) but exactly the self-made-artifact class 06d4785 hunts.
The ring now clears at era-score reset; the warm-up bars simply don't score
the trail race, which the m_excTrailN gating already accounts for.
2. skillTrail compared the head's FULL-block Brier (pro-rated by coverage)
against the incumbent's subset sum - valid only if head skill is uniform
across the OOS walk, while the trail-scored subset systematically excludes
each era's warm-up bars. The audit also found m_excBrierHeadD/BaseD/
m_excOosHitsD declared, zeroed and never accumulated (dead since e2c9593
made every scored bar disjoint). The dead trio is replaced by
m_excBrierHeadT: the head's Brier accumulated only on the bars the warm
incumbent also scored, so the race now compares both predictors on an
identical bar set.
3. The AD/Wyckoff feature blocks read GetData with no EMPTY_VALUE guard; a
cold (still-calculating) indicator returns EMPTY_VALUE everywhere, the
sanitize loop rewrote that to 0.0, and the bar SUCCEEDED - so
BufferTempData cached an all-zero Wyckoff block as a success for the whole
bar frame: the one path the f6150ee only-cache-successes rule cannot see,
because it never fails (the ba13eef class, arriving through values that
never fail; a resumed model's era-0 prebuild starts milliseconds after
OnInit). ADIndicatorCold() probes the NEWEST bar - EMPTY_VALUE there means
async warm-up (transient reject, retried), while deep bars beyond the
buffered depth keep the sanitize loop's neutral-fill so degraded history
still trains. Also fixed m_featureCacheValid's declaration comment, which
still described the pre-f6150ee cached-miss semantics.
Compile: 0 errors, 0 warnings.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-11 18:20:24 -04:00
|
|
|
//| "Is this AD indicator still calculating?" MT5 fills custom- |
|
|
|
|
|
//| indicator buffers asynchronously after the handle is created, and |
|
|
|
|
|
//| a cold one returns EMPTY_VALUE for EVERY index - including the |
|
|
|
|
|
//| newest bar, which a warm indicator always has. So probing buffer |
|
|
|
|
|
//| 0 at index 0 cleanly separates "async calc hasn't run yet" (cold: |
|
|
|
|
|
//| transient reject, retried next call like the cold-ATR guard) from |
|
|
|
|
|
//| "this deep bar is beyond the buffered depth" (warm: neutral-fill |
|
|
|
|
|
//| by the sanitize loop, since that history is degraded-but-usable). |
|
|
|
|
|
//| Cost is one array read per block per bar. (2026-08-11) |
|
|
|
|
|
//+------------------------------------------------------------------+
|
|
|
|
|
bool CExpertSignalAIBase::ADIndicatorCold(CiCustom &ind)
|
|
|
|
|
{
|
|
|
|
|
if(ind.GetData(0, 0) != EMPTY_VALUE)
|
|
|
|
|
return false;
|
|
|
|
|
m_featureFailTransient = true;
|
|
|
|
|
return true;
|
|
|
|
|
}
|
|
|
|
|
//+------------------------------------------------------------------+
|
refactor: split CExpertSignalAIBase implementation by responsibility
ExpertSignalAIBase.mqh was 8216 lines: the class declaration followed by 87
method bodies covering training, labelling, feature extraction, persistence,
chart drawing, online learning, the GA auto-tuner and inference, all in one
file. Train() alone is 1492 lines; a change to arrow drawing meant scrolling
past the era loop.
Moved the bodies into Expert\AIBase\, included at the bottom of the original
after the class declaration:
Training.mqh 1607 era loop, plateau ladder, checkpoint select, deploy
Features.mqh 1093 indicator creation + per-bar input feature vector
ChartUI.mqh 634 arrows, arrow persistence, status panel, cleanup
Persistence.mqh 492 .stats/.cfg sidecars, CPU-inference validation, copy
OnlineLearning.mqh 461 live continual learning, EMA shadow, OOS simulator
Labels.mqh 309 ZigZag pivot labels, async label-cache prebuild
AutoTune.mqh 275 genetic tuner (population, crossover, halving)
Inference.mqh 235 softmax, prior calibration, class priors
ExpertSignalAIBase.mqh 8216 -> 3131 (declaration + topology build only)
This is a pure relocation - verified mechanically, not by eye: HEAD's file
reconstructed from the eight partials plus the surviving remainder is
byte-identical to HEAD, span for span (scratchpad verify_split.py). No
declaration moved, no signature changed, no code rewritten, so behaviour is
unchanged by construction.
Compiles 0 errors, 0 warnings.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-29 00:42:45 -04:00
|
|
|
//| Initialize Open indicators. |
|
|
|
|
|
//+------------------------------------------------------------------+
|
|
|
|
|
bool CExpertSignalAIBase::InitOpen(CIndicators * indicators)
|
|
|
|
|
{
|
|
|
|
|
//--- check pointer
|
|
|
|
|
if(indicators == NULL)
|
|
|
|
|
return (false);
|
|
|
|
|
//--- add object to collection
|
|
|
|
|
if(!indicators.Add(GetPointer(m_Open)))
|
|
|
|
|
{
|
|
|
|
|
printf(__FUNCTION__ + ": error adding object");
|
|
|
|
|
return (false);
|
|
|
|
|
}
|
|
|
|
|
//--- initialize object
|
|
|
|
|
if(!m_Open.Create(m_symbol.Name(), m_period))
|
|
|
|
|
{
|
|
|
|
|
printf(__FUNCTION__ + ": error initializing object");
|
|
|
|
|
return (false);
|
|
|
|
|
}
|
|
|
|
|
//--- ok
|
|
|
|
|
return (true);
|
|
|
|
|
}
|
|
|
|
|
//+------------------------------------------------------------------+
|
|
|
|
|
//| Initialize Close indicators. |
|
|
|
|
|
//+------------------------------------------------------------------+
|
|
|
|
|
bool CExpertSignalAIBase::InitClose(CIndicators * indicators)
|
|
|
|
|
{
|
|
|
|
|
//--- check pointer
|
|
|
|
|
if(indicators == NULL)
|
|
|
|
|
return (false);
|
|
|
|
|
//--- add object to collection
|
|
|
|
|
if(!indicators.Add(GetPointer(m_Close)))
|
|
|
|
|
{
|
|
|
|
|
printf(__FUNCTION__ + ": error adding object");
|
|
|
|
|
return (false);
|
|
|
|
|
}
|
|
|
|
|
//--- initialize object
|
|
|
|
|
if(!m_Close.Create(m_symbol.Name(), m_period))
|
|
|
|
|
{
|
|
|
|
|
printf(__FUNCTION__ + ": error initializing object");
|
|
|
|
|
return (false);
|
|
|
|
|
}
|
|
|
|
|
//--- ok
|
|
|
|
|
return (true);
|
|
|
|
|
}
|
|
|
|
|
//+------------------------------------------------------------------+
|
|
|
|
|
//| Initialize High indicators. |
|
|
|
|
|
//+------------------------------------------------------------------+
|
|
|
|
|
bool CExpertSignalAIBase::InitHigh(CIndicators * indicators)
|
|
|
|
|
{
|
|
|
|
|
//--- check pointer
|
|
|
|
|
if(indicators == NULL)
|
|
|
|
|
return (false);
|
|
|
|
|
//--- add object to collection
|
|
|
|
|
if(!indicators.Add(GetPointer(m_High)))
|
|
|
|
|
{
|
|
|
|
|
printf(__FUNCTION__ + ": error adding object");
|
|
|
|
|
return (false);
|
|
|
|
|
}
|
|
|
|
|
//--- initialize object
|
|
|
|
|
if(!m_High.Create(m_symbol.Name(), m_period))
|
|
|
|
|
{
|
|
|
|
|
printf(__FUNCTION__ + ": error initializing object");
|
|
|
|
|
return (false);
|
|
|
|
|
}
|
|
|
|
|
//--- ok
|
|
|
|
|
return (true);
|
|
|
|
|
}
|
|
|
|
|
//+------------------------------------------------------------------+
|
|
|
|
|
//| Initialize Low indicators. |
|
|
|
|
|
//+------------------------------------------------------------------+
|
|
|
|
|
bool CExpertSignalAIBase::InitLow(CIndicators * indicators)
|
|
|
|
|
{
|
|
|
|
|
//--- check pointer
|
|
|
|
|
if(indicators == NULL)
|
|
|
|
|
return (false);
|
|
|
|
|
//--- add object to collection
|
|
|
|
|
if(!indicators.Add(GetPointer(m_Low)))
|
|
|
|
|
{
|
|
|
|
|
printf(__FUNCTION__ + ": error adding object");
|
|
|
|
|
return (false);
|
|
|
|
|
}
|
|
|
|
|
//--- initialize object
|
|
|
|
|
if(!m_Low.Create(m_symbol.Name(), m_period))
|
|
|
|
|
{
|
|
|
|
|
printf(__FUNCTION__ + ": error initializing object");
|
|
|
|
|
return (false);
|
|
|
|
|
}
|
|
|
|
|
//--- ok
|
|
|
|
|
return (true);
|
|
|
|
|
}
|
|
|
|
|
//+------------------------------------------------------------------+
|
|
|
|
|
//| Initialize Time indicators. |
|
|
|
|
|
//+------------------------------------------------------------------+
|
|
|
|
|
bool CExpertSignalAIBase::InitTime(CIndicators * indicators)
|
|
|
|
|
{
|
|
|
|
|
//--- check pointer
|
|
|
|
|
if(indicators == NULL)
|
|
|
|
|
return (false);
|
|
|
|
|
//--- add object to collection
|
|
|
|
|
if(!indicators.Add(GetPointer(m_Time)))
|
|
|
|
|
{
|
|
|
|
|
printf(__FUNCTION__ + ": error adding object");
|
|
|
|
|
return (false);
|
|
|
|
|
}
|
|
|
|
|
//--- initialize object
|
|
|
|
|
if(!m_Time.Create(m_symbol.Name(), m_period))
|
|
|
|
|
{
|
|
|
|
|
printf(__FUNCTION__ + ": error initializing object");
|
|
|
|
|
return (false);
|
|
|
|
|
}
|
|
|
|
|
//--- ok
|
|
|
|
|
return (true);
|
|
|
|
|
}
|
|
|
|
|
//+------------------------------------------------------------------+
|
|
|
|
|
//| Initialize Volumes indicators. |
|
|
|
|
|
//+------------------------------------------------------------------+
|
|
|
|
|
bool CExpertSignalAIBase::InitVolumes(CIndicators * indicators)
|
|
|
|
|
{
|
|
|
|
|
//--- check pointer
|
|
|
|
|
if(indicators == NULL)
|
|
|
|
|
return (false);
|
|
|
|
|
//--- add object to collection
|
|
|
|
|
if(!indicators.Add(GetPointer(m_Volumes)))
|
|
|
|
|
{
|
|
|
|
|
printf(__FUNCTION__ + ": error adding object");
|
|
|
|
|
return (false);
|
|
|
|
|
}
|
|
|
|
|
//--- initialize object
|
|
|
|
|
if(!m_Volumes.Create(m_symbol.Name(), m_period, VolumeData))
|
|
|
|
|
{
|
|
|
|
|
printf(__FUNCTION__ + ": error initializing object");
|
|
|
|
|
return (false);
|
|
|
|
|
}
|
|
|
|
|
//--- ok
|
|
|
|
|
return (true);
|
|
|
|
|
}
|
|
|
|
|
//+------------------------------------------------------------------+
|
|
|
|
|
//| Initialize MA indicator (feature use - see m_useMA). Period comes |
|
|
|
|
|
//| from m_indicatorTuner.maPeriod, not the raw PeriodMA input - it |
|
|
|
|
|
//| starts equal to it (see CADIndicatorTuner's constructor) but may |
|
|
|
|
|
//| diverge once AutoTuneIndicators actually searches a trial. The |
|
|
|
|
|
//| Classic Signals MA vote is unaffected - see m_useMA's declaration |
|
|
|
|
|
//| comment. |
|
|
|
|
|
//+------------------------------------------------------------------+
|
|
|
|
|
bool CExpertSignalAIBase::InitMA(CIndicators * indicators, bool addToCollection)
|
|
|
|
|
{
|
|
|
|
|
if(indicators == NULL)
|
|
|
|
|
return (false);
|
|
|
|
|
if(addToCollection && !indicators.Add(GetPointer(m_MA)))
|
|
|
|
|
{
|
|
|
|
|
printf(__FUNCTION__ + ": error adding object");
|
|
|
|
|
return (false);
|
|
|
|
|
}
|
|
|
|
|
//--- unified MA custom indicator (CustomIndicators\ADMovingAverage.mq5); type AND period are both tuner-
|
|
|
|
|
//--- driven (m_indicatorTuner.maType/maPeriod). params[1..] mirror the indicator's own input order.
|
|
|
|
|
MqlParam params[9];
|
|
|
|
|
params[0].type = TYPE_STRING; params[0].string_value = WARRIOR_CI("ADMovingAverage");
|
|
|
|
|
params[1].type = TYPE_INT; params[1].integer_value = m_indicatorTuner.maType; // InpType
|
|
|
|
|
params[2].type = TYPE_INT; params[2].integer_value = m_indicatorTuner.maPeriod; // InpPeriod
|
|
|
|
|
params[3].type = TYPE_INT; params[3].integer_value = PRICE_CLOSE; // InpAppliedPrice
|
|
|
|
|
params[4].type = TYPE_DOUBLE; params[4].double_value = 0.85; // InpOffset (ALMA)
|
|
|
|
|
params[5].type = TYPE_DOUBLE; params[5].double_value = 6.0; // InpSigma (ALMA)
|
|
|
|
|
params[6].type = TYPE_DOUBLE; params[6].double_value = 0.7; // InpVolumeFactor (T3)
|
|
|
|
|
params[7].type = TYPE_DOUBLE; params[7].double_value = 0.001; // InpProcessNoise (Kalman)
|
|
|
|
|
params[8].type = TYPE_DOUBLE; params[8].double_value = 0.1; // InpMeasurementNoise (Kalman)
|
|
|
|
|
if(!m_MA.Create(m_symbol.Name(), m_period, IND_CUSTOM, 9, params))
|
|
|
|
|
{
|
|
|
|
|
printf(__FUNCTION__ + ": error initializing object");
|
|
|
|
|
return (false);
|
|
|
|
|
}
|
|
|
|
|
m_MA.NumBuffers(1);
|
|
|
|
|
return (true);
|
|
|
|
|
}
|
|
|
|
|
//+------------------------------------------------------------------+
|
|
|
|
|
//| Initialize RSI indicator (feature use - see m_useRSI). Period |
|
|
|
|
|
//| comes from m_indicatorTuner.rsiPeriod - see InitMA()'s comment. |
|
|
|
|
|
//+------------------------------------------------------------------+
|
|
|
|
|
bool CExpertSignalAIBase::InitRSI(CIndicators * indicators, bool addToCollection)
|
|
|
|
|
{
|
|
|
|
|
if(indicators == NULL)
|
|
|
|
|
return (false);
|
|
|
|
|
if(addToCollection && !indicators.Add(GetPointer(m_RSI)))
|
|
|
|
|
{
|
|
|
|
|
printf(__FUNCTION__ + ": error adding object");
|
|
|
|
|
return (false);
|
|
|
|
|
}
|
|
|
|
|
if(!m_RSI.Create(m_symbol.Name(), m_period, m_indicatorTuner.rsiPeriod, PRICE_CLOSE))
|
|
|
|
|
{
|
|
|
|
|
printf(__FUNCTION__ + ": error initializing object");
|
|
|
|
|
return (false);
|
|
|
|
|
}
|
|
|
|
|
return (true);
|
|
|
|
|
}
|
|
|
|
|
//+------------------------------------------------------------------+
|
|
|
|
|
//| Initialize MACD indicator (feature use - see m_useMACD). Periods |
|
|
|
|
|
//| come from m_indicatorTuner.macdFast/macdSlow/macdSignal - see |
|
|
|
|
|
//| InitMA()'s comment for the "starts at the input, may diverge once |
|
|
|
|
|
//| the tuner searches" split, and note the Classic Signals MACD vote |
|
|
|
|
|
//| (Signals\SignalMACD.mqh) keeps its own separate instance. |
|
|
|
|
|
//+------------------------------------------------------------------+
|
|
|
|
|
bool CExpertSignalAIBase::InitMACDFeature(CIndicators * indicators, bool addToCollection)
|
|
|
|
|
{
|
|
|
|
|
if(indicators == NULL)
|
|
|
|
|
return (false);
|
|
|
|
|
if(addToCollection && !indicators.Add(GetPointer(m_MACDFeature)))
|
|
|
|
|
{
|
|
|
|
|
printf(__FUNCTION__ + ": error adding object");
|
|
|
|
|
return (false);
|
|
|
|
|
}
|
|
|
|
|
if(!m_MACDFeature.Create(m_symbol.Name(), m_period, m_indicatorTuner.macdFast, m_indicatorTuner.macdSlow,
|
|
|
|
|
m_indicatorTuner.macdSignal, PRICE_CLOSE))
|
|
|
|
|
{
|
|
|
|
|
printf(__FUNCTION__ + ": error initializing object");
|
|
|
|
|
return (false);
|
|
|
|
|
}
|
|
|
|
|
return (true);
|
|
|
|
|
}
|
|
|
|
|
//+------------------------------------------------------------------+
|
|
|
|
|
//| Initialize Ichimoku indicator (feature use - see m_useIchimoku). |
|
|
|
|
|
//| Periods come from m_indicatorTuner.ichiTenkan/ichiKijun/ |
|
|
|
|
|
//| ichiSenkou - see InitMA()'s comment. The Classic Signals Ichimoku |
|
|
|
|
|
//| vote (Signals\SignalIchimoku.mqh) keeps its own instance. |
|
|
|
|
|
//+------------------------------------------------------------------+
|
|
|
|
|
bool CExpertSignalAIBase::InitIchimoku(CIndicators * indicators, bool addToCollection)
|
|
|
|
|
{
|
|
|
|
|
if(indicators == NULL)
|
|
|
|
|
return (false);
|
|
|
|
|
if(addToCollection && !indicators.Add(GetPointer(m_Ichimoku)))
|
|
|
|
|
{
|
|
|
|
|
printf(__FUNCTION__ + ": error adding object");
|
|
|
|
|
return (false);
|
|
|
|
|
}
|
|
|
|
|
if(!m_Ichimoku.Create(m_symbol.Name(), m_period, m_indicatorTuner.ichiTenkan, m_indicatorTuner.ichiKijun,
|
|
|
|
|
m_indicatorTuner.ichiSenkou))
|
|
|
|
|
{
|
|
|
|
|
printf(__FUNCTION__ + ": error initializing object");
|
|
|
|
|
return (false);
|
|
|
|
|
}
|
|
|
|
|
return (true);
|
|
|
|
|
}
|
|
|
|
|
//+------------------------------------------------------------------+
|
|
|
|
|
//| Initialize AD Cumulative Delta (CustomIndicators\ADCumulativeDelta.mq5) |
|
|
|
|
|
//| Loaded via iCustom/CiCustom, not a built-in Ci* class - the compiled |
|
|
|
|
|
//| indicator must be present under MQL5\Indicators\ (see |
|
|
|
|
|
//| ExtractCustomIndicators() in Warrior_EA.mq5). Uses the indicator's |
|
|
|
|
|
//| own input defaults; 6 output buffers, one TempData feature each. |
|
|
|
|
|
//+------------------------------------------------------------------+
|
|
|
|
|
bool CExpertSignalAIBase::InitADCumulativeDelta(CIndicators * indicators, bool addToCollection)
|
|
|
|
|
{
|
|
|
|
|
//--- check pointer
|
|
|
|
|
if(indicators == NULL)
|
|
|
|
|
return (false);
|
|
|
|
|
//--- add object to collection
|
|
|
|
|
if(addToCollection && !indicators.Add(GetPointer(m_ADCumulativeDelta)))
|
|
|
|
|
{
|
|
|
|
|
printf(__FUNCTION__ + ": error adding object");
|
|
|
|
|
return (false);
|
|
|
|
|
}
|
|
|
|
|
//--- initialize object; params[1..] mirror ADCumulativeDelta.mq5's own input order exactly
|
|
|
|
|
MqlParam params[11];
|
|
|
|
|
params[0].type = TYPE_STRING;
|
|
|
|
|
params[0].string_value = WARRIOR_CI("ADCumulativeDelta");
|
|
|
|
|
params[1].type = TYPE_INT;
|
|
|
|
|
params[1].integer_value = m_indicatorTuner.adCumDelta.lookback; // InpLookbackPeriod
|
|
|
|
|
params[2].type = TYPE_DOUBLE;
|
|
|
|
|
params[2].double_value = m_indicatorTuner.adCumDelta.volClimax; // InpVolumeClimaxMultiplier
|
|
|
|
|
params[3].type = TYPE_DOUBLE;
|
|
|
|
|
params[3].double_value = m_indicatorTuner.adCumDelta.volHigh; // InpVolumeHighMultiplier
|
|
|
|
|
params[4].type = TYPE_DOUBLE;
|
|
|
|
|
params[4].double_value = m_indicatorTuner.adCumDelta.rangeClimax; // InpRangeClimaxMultiplier
|
|
|
|
|
params[5].type = TYPE_DOUBLE;
|
|
|
|
|
params[5].double_value = m_indicatorTuner.adCumDelta.rangeSignificant; // InpRangeSignificantMult
|
|
|
|
|
params[6].type = TYPE_DOUBLE;
|
|
|
|
|
params[6].double_value = m_indicatorTuner.adCumDelta.stVolRatio; // InpSTVolumeRatio
|
|
|
|
|
params[7].type = TYPE_DOUBLE;
|
|
|
|
|
params[7].double_value = m_indicatorTuner.adCumDelta.atrMult; // InpATRMultiplier
|
|
|
|
|
params[8].type = TYPE_INT;
|
|
|
|
|
params[8].integer_value = 0; // InpContextMode - DO NOT tune
|
|
|
|
|
params[9].type = TYPE_INT;
|
|
|
|
|
params[9].integer_value = 5; // InpSessionType - DO NOT tune
|
|
|
|
|
params[10].type = TYPE_INT;
|
|
|
|
|
params[10].integer_value = 1; // InpSessionCount - DO NOT tune
|
|
|
|
|
if(!m_ADCumulativeDelta.Create(m_symbol.Name(), m_period, IND_CUSTOM, 11, params))
|
|
|
|
|
{
|
|
|
|
|
printf(__FUNCTION__ + ": error initializing object");
|
|
|
|
|
return (false);
|
|
|
|
|
}
|
|
|
|
|
m_ADCumulativeDelta.NumBuffers(6);
|
|
|
|
|
//--- ok
|
|
|
|
|
return (true);
|
|
|
|
|
}
|
|
|
|
|
//+------------------------------------------------------------------+
|
|
|
|
|
//| Initialize AD Shortening of Thrust (CustomIndicators\ADShorteningOfThrust.mq5) |
|
|
|
|
|
//+------------------------------------------------------------------+
|
|
|
|
|
bool CExpertSignalAIBase::InitADShorteningOfThrust(CIndicators * indicators, bool addToCollection)
|
|
|
|
|
{
|
|
|
|
|
//--- check pointer
|
|
|
|
|
if(indicators == NULL)
|
|
|
|
|
return (false);
|
|
|
|
|
//--- add object to collection
|
|
|
|
|
if(addToCollection && !indicators.Add(GetPointer(m_ADShorteningOfThrust)))
|
|
|
|
|
{
|
|
|
|
|
printf(__FUNCTION__ + ": error adding object");
|
|
|
|
|
return (false);
|
|
|
|
|
}
|
|
|
|
|
//--- initialize object; params[1..] mirror ADShorteningOfThrust.mq5's own input order exactly
|
|
|
|
|
MqlParam params[7];
|
|
|
|
|
params[0].type = TYPE_STRING;
|
|
|
|
|
params[0].string_value = WARRIOR_CI("ADShorteningOfThrust");
|
|
|
|
|
params[1].type = TYPE_INT;
|
|
|
|
|
params[1].integer_value = m_indicatorTuner.adSOT.thrustLookback; // InpThrustLookback
|
|
|
|
|
params[2].type = TYPE_INT;
|
|
|
|
|
params[2].integer_value = m_indicatorTuner.adSOT.minImpulses; // InpMinImpulses
|
|
|
|
|
params[3].type = TYPE_DOUBLE;
|
|
|
|
|
params[3].double_value = m_indicatorTuner.adSOT.sotThreshold; // InpSOTThreshold
|
|
|
|
|
params[4].type = TYPE_INT;
|
|
|
|
|
params[4].integer_value = 0; // InpContextMode - DO NOT tune
|
|
|
|
|
params[5].type = TYPE_INT;
|
|
|
|
|
params[5].integer_value = 5; // InpSessionType - DO NOT tune
|
|
|
|
|
params[6].type = TYPE_INT;
|
|
|
|
|
params[6].integer_value = 1; // InpSessionCount - DO NOT tune
|
|
|
|
|
if(!m_ADShorteningOfThrust.Create(m_symbol.Name(), m_period, IND_CUSTOM, 7, params))
|
|
|
|
|
{
|
|
|
|
|
printf(__FUNCTION__ + ": error initializing object");
|
|
|
|
|
return (false);
|
|
|
|
|
}
|
|
|
|
|
m_ADShorteningOfThrust.NumBuffers(4);
|
|
|
|
|
//--- ok
|
|
|
|
|
return (true);
|
|
|
|
|
}
|
|
|
|
|
//+------------------------------------------------------------------+
|
|
|
|
|
//| Initialize AD Wyckoff Event Stream (CustomIndicators\ADWyckoffEventStream.mq5) |
|
|
|
|
|
//+------------------------------------------------------------------+
|
|
|
|
|
bool CExpertSignalAIBase::InitADWyckoffEventStream(CIndicators * indicators, bool addToCollection)
|
|
|
|
|
{
|
|
|
|
|
//--- check pointer
|
|
|
|
|
if(indicators == NULL)
|
|
|
|
|
return (false);
|
|
|
|
|
//--- add object to collection
|
|
|
|
|
if(addToCollection && !indicators.Add(GetPointer(m_ADWyckoffEventStream)))
|
|
|
|
|
{
|
|
|
|
|
printf(__FUNCTION__ + ": error adding object");
|
|
|
|
|
return (false);
|
|
|
|
|
}
|
2026-08-02 17:08:48 -04:00
|
|
|
//--- initialize object; params[1..] mirror ADWyckoffEventStream.mq5's own input order exactly.
|
|
|
|
|
//--- NOTE the order is NOT grouped by meaning: the three range-lifecycle knobs the indicator gained on
|
|
|
|
|
//--- 2026-08-02 were appended AFTER the session inputs, not next to the thresholds they belong with.
|
|
|
|
|
//--- MqlParam is positional, so this list follows the indicator's declaration order, not a tidier one.
|
|
|
|
|
MqlParam params[17];
|
refactor: split CExpertSignalAIBase implementation by responsibility
ExpertSignalAIBase.mqh was 8216 lines: the class declaration followed by 87
method bodies covering training, labelling, feature extraction, persistence,
chart drawing, online learning, the GA auto-tuner and inference, all in one
file. Train() alone is 1492 lines; a change to arrow drawing meant scrolling
past the era loop.
Moved the bodies into Expert\AIBase\, included at the bottom of the original
after the class declaration:
Training.mqh 1607 era loop, plateau ladder, checkpoint select, deploy
Features.mqh 1093 indicator creation + per-bar input feature vector
ChartUI.mqh 634 arrows, arrow persistence, status panel, cleanup
Persistence.mqh 492 .stats/.cfg sidecars, CPU-inference validation, copy
OnlineLearning.mqh 461 live continual learning, EMA shadow, OOS simulator
Labels.mqh 309 ZigZag pivot labels, async label-cache prebuild
AutoTune.mqh 275 genetic tuner (population, crossover, halving)
Inference.mqh 235 softmax, prior calibration, class priors
ExpertSignalAIBase.mqh 8216 -> 3131 (declaration + topology build only)
This is a pure relocation - verified mechanically, not by eye: HEAD's file
reconstructed from the eight partials plus the surviving remainder is
byte-identical to HEAD, span for span (scratchpad verify_split.py). No
declaration moved, no signature changed, no code rewritten, so behaviour is
unchanged by construction.
Compiles 0 errors, 0 warnings.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-29 00:42:45 -04:00
|
|
|
params[0].type = TYPE_STRING;
|
|
|
|
|
params[0].string_value = WARRIOR_CI("ADWyckoffEventStream");
|
|
|
|
|
params[1].type = TYPE_INT;
|
|
|
|
|
params[1].integer_value = m_indicatorTuner.adWES.lookback; // InpLookback
|
|
|
|
|
params[2].type = TYPE_INT;
|
|
|
|
|
params[2].integer_value = m_indicatorTuner.adWES.zigzag; // InpZigZag
|
|
|
|
|
params[3].type = TYPE_DOUBLE;
|
|
|
|
|
params[3].double_value = m_indicatorTuner.adWES.volClimax; // InpVolClimax
|
|
|
|
|
params[4].type = TYPE_DOUBLE;
|
|
|
|
|
params[4].double_value = m_indicatorTuner.adWES.volHigh; // InpVolHigh
|
|
|
|
|
params[5].type = TYPE_DOUBLE;
|
|
|
|
|
params[5].double_value = m_indicatorTuner.adWES.rangeClimax; // InpRangeClimax
|
|
|
|
|
params[6].type = TYPE_DOUBLE;
|
|
|
|
|
params[6].double_value = m_indicatorTuner.adWES.rangeSignificant; // InpRangeSignificant
|
|
|
|
|
params[7].type = TYPE_DOUBLE;
|
|
|
|
|
params[7].double_value = m_indicatorTuner.adWES.stVolRatio; // InpSTVolRatio
|
|
|
|
|
params[8].type = TYPE_DOUBLE;
|
|
|
|
|
params[8].double_value = m_indicatorTuner.adWES.atr; // InpATR
|
|
|
|
|
params[9].type = TYPE_INT;
|
|
|
|
|
params[9].integer_value = 0; // InpContextMode - DO NOT tune
|
|
|
|
|
params[10].type = TYPE_INT;
|
|
|
|
|
params[10].integer_value = 5; // InpSessionType - DO NOT tune
|
|
|
|
|
params[11].type = TYPE_INT;
|
|
|
|
|
params[11].integer_value = 1; // InpSessionCount - DO NOT tune
|
2026-08-02 17:08:48 -04:00
|
|
|
params[12].type = TYPE_DOUBLE;
|
|
|
|
|
params[12].double_value = m_indicatorTuner.adWES.touchATR; // InpTouchATR
|
|
|
|
|
params[13].type = TYPE_DOUBLE;
|
|
|
|
|
params[13].double_value = m_indicatorTuner.adWES.arMinATR; // InpARMinATR
|
|
|
|
|
params[14].type = TYPE_INT;
|
|
|
|
|
params[14].integer_value = m_indicatorTuner.adWES.maxRangeBars; // InpMaxRangeBars
|
|
|
|
|
//--- InpShowLabels/InpShowZones - forced OFF, and deliberately NOT tunable. This handle exists purely
|
|
|
|
|
//--- to read buffers as network features; it is never the user's chart indicator. Left at their own
|
|
|
|
|
//--- `true` defaults the indicator would litter the traded chart with AWY_-prefixed labels and range
|
|
|
|
|
//--- rectangles that the EA does not own and its OnDeinit chart sweep does not know to remove.
|
|
|
|
|
params[15].type = TYPE_BOOL;
|
|
|
|
|
params[15].integer_value = 0; // InpShowLabels
|
|
|
|
|
params[16].type = TYPE_BOOL;
|
|
|
|
|
params[16].integer_value = 0; // InpShowZones
|
|
|
|
|
if(!m_ADWyckoffEventStream.Create(m_symbol.Name(), m_period, IND_CUSTOM, 17, params))
|
refactor: split CExpertSignalAIBase implementation by responsibility
ExpertSignalAIBase.mqh was 8216 lines: the class declaration followed by 87
method bodies covering training, labelling, feature extraction, persistence,
chart drawing, online learning, the GA auto-tuner and inference, all in one
file. Train() alone is 1492 lines; a change to arrow drawing meant scrolling
past the era loop.
Moved the bodies into Expert\AIBase\, included at the bottom of the original
after the class declaration:
Training.mqh 1607 era loop, plateau ladder, checkpoint select, deploy
Features.mqh 1093 indicator creation + per-bar input feature vector
ChartUI.mqh 634 arrows, arrow persistence, status panel, cleanup
Persistence.mqh 492 .stats/.cfg sidecars, CPU-inference validation, copy
OnlineLearning.mqh 461 live continual learning, EMA shadow, OOS simulator
Labels.mqh 309 ZigZag pivot labels, async label-cache prebuild
AutoTune.mqh 275 genetic tuner (population, crossover, halving)
Inference.mqh 235 softmax, prior calibration, class priors
ExpertSignalAIBase.mqh 8216 -> 3131 (declaration + topology build only)
This is a pure relocation - verified mechanically, not by eye: HEAD's file
reconstructed from the eight partials plus the surviving remainder is
byte-identical to HEAD, span for span (scratchpad verify_split.py). No
declaration moved, no signature changed, no code rewritten, so behaviour is
unchanged by construction.
Compiles 0 errors, 0 warnings.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-29 00:42:45 -04:00
|
|
|
{
|
|
|
|
|
printf(__FUNCTION__ + ": error initializing object");
|
|
|
|
|
return (false);
|
|
|
|
|
}
|
|
|
|
|
m_ADWyckoffEventStream.NumBuffers(14);
|
|
|
|
|
//--- ok
|
|
|
|
|
return (true);
|
|
|
|
|
}
|
|
|
|
|
//+------------------------------------------------------------------+
|
|
|
|
|
//| Initialize AD Wyckoff Failed Structure (CustomIndicators\ADWyckoffFailedStructure.mq5) |
|
|
|
|
|
//+------------------------------------------------------------------+
|
|
|
|
|
bool CExpertSignalAIBase::InitADWyckoffFailedStructure(CIndicators * indicators, bool addToCollection)
|
|
|
|
|
{
|
|
|
|
|
//--- check pointer
|
|
|
|
|
if(indicators == NULL)
|
|
|
|
|
return (false);
|
|
|
|
|
//--- add object to collection
|
|
|
|
|
if(addToCollection && !indicators.Add(GetPointer(m_ADWyckoffFailedStructure)))
|
|
|
|
|
{
|
|
|
|
|
printf(__FUNCTION__ + ": error adding object");
|
|
|
|
|
return (false);
|
|
|
|
|
}
|
|
|
|
|
//--- initialize object; params[1..] mirror ADWyckoffFailedStructure.mq5's own input order exactly
|
|
|
|
|
MqlParam params[12];
|
|
|
|
|
params[0].type = TYPE_STRING;
|
|
|
|
|
params[0].string_value = WARRIOR_CI("ADWyckoffFailedStructure");
|
|
|
|
|
params[1].type = TYPE_INT;
|
|
|
|
|
params[1].integer_value = m_indicatorTuner.adWFS.lookback; // InpLookbackPeriod
|
|
|
|
|
params[2].type = TYPE_INT;
|
|
|
|
|
params[2].integer_value = m_indicatorTuner.adWFS.zigzagStrength; // InpZigZagStrength
|
|
|
|
|
params[3].type = TYPE_DOUBLE;
|
|
|
|
|
params[3].double_value = m_indicatorTuner.adWFS.volClimax; // InpVolumeClimaxMultiplier
|
|
|
|
|
params[4].type = TYPE_DOUBLE;
|
|
|
|
|
params[4].double_value = m_indicatorTuner.adWFS.volHigh; // InpVolumeHighMultiplier
|
|
|
|
|
params[5].type = TYPE_DOUBLE;
|
|
|
|
|
params[5].double_value = m_indicatorTuner.adWFS.rangeClimax; // InpRangeClimaxMultiplier
|
|
|
|
|
params[6].type = TYPE_DOUBLE;
|
|
|
|
|
params[6].double_value = m_indicatorTuner.adWFS.rangeSignificant; // InpRangeSignificantMult
|
|
|
|
|
params[7].type = TYPE_DOUBLE;
|
|
|
|
|
params[7].double_value = m_indicatorTuner.adWFS.stVolRatio; // InpSTVolumeRatio
|
|
|
|
|
params[8].type = TYPE_DOUBLE;
|
|
|
|
|
params[8].double_value = m_indicatorTuner.adWFS.atrMult; // InpATRMultiplier
|
|
|
|
|
params[9].type = TYPE_INT;
|
|
|
|
|
params[9].integer_value = 0; // InpContextMode - DO NOT tune
|
|
|
|
|
params[10].type = TYPE_INT;
|
|
|
|
|
params[10].integer_value = 5; // InpSessionType - DO NOT tune
|
|
|
|
|
params[11].type = TYPE_INT;
|
|
|
|
|
params[11].integer_value = 1; // InpSessionCount - DO NOT tune
|
|
|
|
|
if(!m_ADWyckoffFailedStructure.Create(m_symbol.Name(), m_period, IND_CUSTOM, 12, params))
|
|
|
|
|
{
|
|
|
|
|
printf(__FUNCTION__ + ": error initializing object");
|
|
|
|
|
return (false);
|
|
|
|
|
}
|
|
|
|
|
m_ADWyckoffFailedStructure.NumBuffers(5);
|
|
|
|
|
//--- ok
|
|
|
|
|
return (true);
|
|
|
|
|
}
|
|
|
|
|
//+------------------------------------------------------------------+
|
|
|
|
|
//| Initialize AD Wyckoff Significant Bar Inversion (CustomIndicators\ADWyckoffSignificantBarInversion.mq5) |
|
|
|
|
|
//+------------------------------------------------------------------+
|
|
|
|
|
bool CExpertSignalAIBase::InitADWyckoffSignificantBarInversion(CIndicators * indicators, bool addToCollection)
|
|
|
|
|
{
|
|
|
|
|
//--- check pointer
|
|
|
|
|
if(indicators == NULL)
|
|
|
|
|
return (false);
|
|
|
|
|
//--- add object to collection
|
|
|
|
|
if(addToCollection && !indicators.Add(GetPointer(m_ADWyckoffSignificantBarInversion)))
|
|
|
|
|
{
|
|
|
|
|
printf(__FUNCTION__ + ": error adding object");
|
|
|
|
|
return (false);
|
|
|
|
|
}
|
|
|
|
|
//--- initialize object; params[1..] mirror ADWyckoffSignificantBarInversion.mq5's own input order exactly
|
|
|
|
|
MqlParam params[8];
|
|
|
|
|
params[0].type = TYPE_STRING;
|
|
|
|
|
params[0].string_value = WARRIOR_CI("ADWyckoffSignificantBarInversion");
|
|
|
|
|
params[1].type = TYPE_INT;
|
|
|
|
|
params[1].integer_value = m_indicatorTuner.adWSBI.lookback; // InpLookback
|
|
|
|
|
params[2].type = TYPE_DOUBLE;
|
|
|
|
|
params[2].double_value = m_indicatorTuner.adWSBI.rangeSignificant; // InpRangeSignificant
|
|
|
|
|
params[3].type = TYPE_DOUBLE;
|
|
|
|
|
params[3].double_value = m_indicatorTuner.adWSBI.volumeHigh; // InpVolumeHigh
|
|
|
|
|
params[4].type = TYPE_DOUBLE;
|
|
|
|
|
params[4].double_value = m_indicatorTuner.adWSBI.atr; // InpATR
|
|
|
|
|
params[5].type = TYPE_INT;
|
|
|
|
|
params[5].integer_value = 0; // InpContextMode - DO NOT tune
|
|
|
|
|
params[6].type = TYPE_INT;
|
|
|
|
|
params[6].integer_value = 5; // InpSessionType - DO NOT tune
|
|
|
|
|
params[7].type = TYPE_INT;
|
|
|
|
|
params[7].integer_value = 1; // InpSessionCount - DO NOT tune
|
|
|
|
|
if(!m_ADWyckoffSignificantBarInversion.Create(m_symbol.Name(), m_period, IND_CUSTOM, 8, params))
|
|
|
|
|
{
|
|
|
|
|
printf(__FUNCTION__ + ": error initializing object");
|
|
|
|
|
return (false);
|
|
|
|
|
}
|
|
|
|
|
m_ADWyckoffSignificantBarInversion.NumBuffers(5);
|
|
|
|
|
//--- ok
|
|
|
|
|
return (true);
|
|
|
|
|
}
|
|
|
|
|
//+------------------------------------------------------------------+
|
|
|
|
|
//| Initialize AD ZigZag (CustomIndicators\ADZigZag.mq5) - the |
|
|
|
|
|
//| training-label source (see m_ADZigZag's declaration comment). |
|
|
|
|
|
//| Always run at its own stock defaults (Depth=12, Deviation=5, |
|
|
|
|
|
//| Backstep=3) - unlike the AD* feature indicators above, this has |
|
|
|
|
|
//| no tunable-param struct and is never touched by AutoTuneIndicators.|
|
|
|
|
|
//+------------------------------------------------------------------+
|
|
|
|
|
bool CExpertSignalAIBase::InitADZigZag(CIndicators * indicators, bool addToCollection)
|
|
|
|
|
{
|
|
|
|
|
//--- check pointer
|
|
|
|
|
if(indicators == NULL)
|
|
|
|
|
return (false);
|
|
|
|
|
//--- add object to collection
|
|
|
|
|
if(addToCollection && !indicators.Add(GetPointer(m_ADZigZag)))
|
|
|
|
|
{
|
|
|
|
|
printf(__FUNCTION__ + ": error adding object");
|
|
|
|
|
return (false);
|
|
|
|
|
}
|
|
|
|
|
//--- initialize object; params[1..] mirror ADZigZag.mq5's own input order exactly - stock defaults,
|
|
|
|
|
//--- intentionally not sourced from a tunable params struct (see this function's declaration comment)
|
|
|
|
|
MqlParam params[4];
|
|
|
|
|
params[0].type = TYPE_STRING;
|
|
|
|
|
params[0].string_value = WARRIOR_CI("ADZigZag");
|
|
|
|
|
params[1].type = TYPE_INT;
|
|
|
|
|
params[1].integer_value = 12; // InpDepth
|
|
|
|
|
params[2].type = TYPE_INT;
|
|
|
|
|
params[2].integer_value = 5; // InpDeviation
|
|
|
|
|
params[3].type = TYPE_INT;
|
|
|
|
|
params[3].integer_value = 3; // InpBackstep
|
|
|
|
|
if(!m_ADZigZag.Create(m_symbol.Name(), m_period, IND_CUSTOM, 4, params))
|
|
|
|
|
{
|
|
|
|
|
printf(__FUNCTION__ + ": error initializing object");
|
|
|
|
|
return (false);
|
|
|
|
|
}
|
|
|
|
|
// Must match ADZigZag.mq5's #property indicator_buffers exactly (3: main ZigZag buffer + 2
|
|
|
|
|
// internal INDICATOR_CALCULATIONS buffers), even though only buffer 0 is ever read via
|
|
|
|
|
// GetData() - see the working AD Wyckoff indicators' InitAD*() for the same pattern.
|
|
|
|
|
m_ADZigZag.NumBuffers(3);
|
|
|
|
|
//--- ok
|
|
|
|
|
return (true);
|
|
|
|
|
}
|
|
|
|
|
#endif // WARRIOR_AIBASE_FEATURES_MQH
|