refactor: split CExpertSignalAIBase implementation by responsibility
ExpertSignalAIBase.mqh was 8216 lines: the class declaration followed by 87
method bodies covering training, labelling, feature extraction, persistence,
chart drawing, online learning, the GA auto-tuner and inference, all in one
file. Train() alone is 1492 lines; a change to arrow drawing meant scrolling
past the era loop.
Moved the bodies into Expert\AIBase\, included at the bottom of the original
after the class declaration:
Training.mqh 1607 era loop, plateau ladder, checkpoint select, deploy
Features.mqh 1093 indicator creation + per-bar input feature vector
ChartUI.mqh 634 arrows, arrow persistence, status panel, cleanup
Persistence.mqh 492 .stats/.cfg sidecars, CPU-inference validation, copy
OnlineLearning.mqh 461 live continual learning, EMA shadow, OOS simulator
Labels.mqh 309 ZigZag pivot labels, async label-cache prebuild
AutoTune.mqh 275 genetic tuner (population, crossover, halving)
Inference.mqh 235 softmax, prior calibration, class priors
ExpertSignalAIBase.mqh 8216 -> 3131 (declaration + topology build only)
This is a pure relocation - verified mechanically, not by eye: HEAD's file
reconstructed from the eight partials plus the surviving remainder is
byte-identical to HEAD, span for span (scratchpad verify_split.py). No
declaration moved, no signature changed, no code rewritten, so behaviour is
unchanged by construction.
Compiles 0 errors, 0 warnings.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-29 00:42:45 -04:00
|
|
|
//+------------------------------------------------------------------+
|
|
|
|
|
//| Warrior_EA |
|
|
|
|
|
//| AnimateDread |
|
|
|
|
|
//| |
|
|
|
|
|
//| Indicator creation and the per-bar input feature vector. |
|
|
|
|
|
//| |
|
|
|
|
|
//| PARTIAL IMPLEMENTATION FILE - not standalone. |
|
|
|
|
|
//| This holds CExpertSignalAIBase method BODIES only. The class |
|
|
|
|
|
//| declaration lives in Expert\ExpertSignalAIBase.mqh, which |
|
|
|
|
|
//| #includes this file at the bottom, after the declaration. Do not |
|
|
|
|
|
//| include it anywhere else and do not compile it on its own. |
|
|
|
|
|
//| |
|
|
|
|
|
//| Split out purely to make the 8216-line original navigable; the |
|
|
|
|
|
//| code inside was moved verbatim, not rewritten. |
|
|
|
|
|
//+------------------------------------------------------------------+
|
|
|
|
|
#ifndef WARRIOR_AIBASE_FEATURES_MQH
|
|
|
|
|
#define WARRIOR_AIBASE_FEATURES_MQH
|
2026-08-02 01:09:18 -04:00
|
|
|
//--- Plausibility ceiling for any single input value, enforced once over the whole bar at the end of
|
|
|
|
|
//--- BufferTempDataCompute(). Deliberately far above every clamp used inside that function (the widest
|
|
|
|
|
//--- is +/-10) - this is not a normalization knob, it is the "no legitimate feature looks like this"
|
|
|
|
|
//--- line. See the sanitize loop at the end of BufferTempDataCompute() for what it protects.
|
|
|
|
|
#define FEATURE_ABS_MAX 1.0e4
|
2026-07-29 00:46:01 -04:00
|
|
|
//+------------------------------------------------------------------+
|
refactor: split CExpertSignalAIBase implementation by responsibility
ExpertSignalAIBase.mqh was 8216 lines: the class declaration followed by 87
method bodies covering training, labelling, feature extraction, persistence,
chart drawing, online learning, the GA auto-tuner and inference, all in one
file. Train() alone is 1492 lines; a change to arrow drawing meant scrolling
past the era loop.
Moved the bodies into Expert\AIBase\, included at the bottom of the original
after the class declaration:
Training.mqh 1607 era loop, plateau ladder, checkpoint select, deploy
Features.mqh 1093 indicator creation + per-bar input feature vector
ChartUI.mqh 634 arrows, arrow persistence, status panel, cleanup
Persistence.mqh 492 .stats/.cfg sidecars, CPU-inference validation, copy
OnlineLearning.mqh 461 live continual learning, EMA shadow, OOS simulator
Labels.mqh 309 ZigZag pivot labels, async label-cache prebuild
AutoTune.mqh 275 genetic tuner (population, crossover, halving)
Inference.mqh 235 softmax, prior calibration, class priors
ExpertSignalAIBase.mqh 8216 -> 3131 (declaration + topology build only)
This is a pure relocation - verified mechanically, not by eye: HEAD's file
reconstructed from the eight partials plus the surviving remainder is
byte-identical to HEAD, span for span (scratchpad verify_split.py). No
declaration moved, no signature changed, no code rewritten, so behaviour is
unchanged by construction.
Compiles 0 errors, 0 warnings.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-29 00:42:45 -04:00
|
|
|
//| Rebuilds only the enabled AD* CiCustom handles in place, so a new |
|
|
|
|
|
//| trial's member-struct param values take effect. Re-Create()-ing |
|
|
|
|
|
//| the existing CiCustom object (rather than removing/re-adding it |
|
|
|
|
|
//| to indicators) avoids adding the same pointer into the CIndicators|
|
|
|
|
|
//| collection twice, which would risk it being deleted twice on |
|
|
|
|
|
//| teardown - MQL5's CIndicators has no documented single-item |
|
fix: the indicator re-init leaked a terminal handle per candidate
This is what killed CONV and LSTM on 2026-08-07. Terminal journal:
19:19:40 6664 x "VirtualAlloc failed in large allocator"
19:19:40.829 expert Warrior_EA (SP500,H1) removed <- CONV
19:29:55 2048 x "VirtualAlloc failed in large allocator"
19:29:55.359 expert Warrior_EA (SP500,H1) removed <- LSTM
50ms and 71ms after each printed its "logit adjustment" line, i.e. the
instant era 0 tried to allocate its training queues. They did not hang -
MT5 shot them for running out of memory.
ReInitADIndicators() re-Create()s every enabled indicator and released
nothing. The comment above it asserted "CiCustom.Create() already
releases its old handle"; MQL5's CIndicator::Create is
m_handle = IndicatorCreate(symbol, period, type, num_params, params);
a plain overwrite, whose only success-path IndicatorRelease is in
~CIndicator. IndicatorRelease appeared nowhere in this codebase.
That function is the indicator tuner's inner loop. AutoTuneIndicators
scored 324 candidates per model on SP500 H1, so ~324 x 6 orphaned
terminal-side instances, each holding a full-history buffer set -
ADWyckoffEventStream is 14 buffers x ~38k bars x 8 bytes = ~4.3 MB each.
Gigabytes. Confirmed in the shutdown teardown, where a single surviving
expert still held 70 x ADWES, 53 x ADWyckoffEventStream, 23 x
WFS(48,3,1.80,1.10), 19 x ADMovingAverage, 18 x WYSB(162). Those
parameter sets are the tuner's candidate grids.
Release is UNCONDITIONAL, not gated on the handle having changed: MT5
refcounts instances by (symbol, period, params), so re-creating with
IDENTICAL params returns the SAME handle with the count incremented -
the "23 x WFS(48,3,1.80,1.10)" pattern. Either way Create() added one
reference and we hold one handle, so one release is owed.
Released AFTER the re-creates, never before: dropping the terminal's
last reference first would tear the instance down, so an identical-params
Create() would rebuild it from scratch instead of reusing the live one -
turning a refcount bump into a full recalculation over all history, 324
times over.
PAI was unaffected because it has no AD/Wyckoff indicators enabled (35
candidates, built-ins only). HYBRID survived on luck: CONV and LSTM died
2 and 12 minutes before its own sweep finished, freeing the memory.
Compiles clean: 0 errors, 0 warnings.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-08 15:42:03 -04:00
|
|
|
//| remove. |
|
|
|
|
|
//| |
|
|
|
|
|
//| This used to end "...and CiCustom.Create() already releases its |
|
|
|
|
|
//| old handle." IT DOES NOT, and that sentence cost two models. See |
|
|
|
|
|
//| the handle-release block in the definition below. |
|
refactor: split CExpertSignalAIBase implementation by responsibility
ExpertSignalAIBase.mqh was 8216 lines: the class declaration followed by 87
method bodies covering training, labelling, feature extraction, persistence,
chart drawing, online learning, the GA auto-tuner and inference, all in one
file. Train() alone is 1492 lines; a change to arrow drawing meant scrolling
past the era loop.
Moved the bodies into Expert\AIBase\, included at the bottom of the original
after the class declaration:
Training.mqh 1607 era loop, plateau ladder, checkpoint select, deploy
Features.mqh 1093 indicator creation + per-bar input feature vector
ChartUI.mqh 634 arrows, arrow persistence, status panel, cleanup
Persistence.mqh 492 .stats/.cfg sidecars, CPU-inference validation, copy
OnlineLearning.mqh 461 live continual learning, EMA shadow, OOS simulator
Labels.mqh 309 ZigZag pivot labels, async label-cache prebuild
AutoTune.mqh 275 genetic tuner (population, crossover, halving)
Inference.mqh 235 softmax, prior calibration, class priors
ExpertSignalAIBase.mqh 8216 -> 3131 (declaration + topology build only)
This is a pure relocation - verified mechanically, not by eye: HEAD's file
reconstructed from the eight partials plus the surviving remainder is
byte-identical to HEAD, span for span (scratchpad verify_split.py). No
declaration moved, no signature changed, no code rewritten, so behaviour is
unchanged by construction.
Compiles 0 errors, 0 warnings.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-29 00:42:45 -04:00
|
|
|
//+------------------------------------------------------------------+
|
fix: make the indicator tuner actually measure, and gate what it installs
ROOT CAUSE of the zero spread measured on SP500 H1 2026-08-07 (all 17 candidates
returned exactly 0.00359 nats): the tune loop re-inits the indicators and then
scores, with no RefreshData() between.
ReInitADIndicators() does its part - Create() builds a NEW handle carrying the
new parameters, and the feature cache is flagged stale so features really are
recomputed. But BufferTempDataCompute() reads the CIndicatorBuffer objects, and
only Refresh() copies data out of a handle into those. So every candidate was
scored on values still held from the PREVIOUS handle. My earlier guess in the
diagnostic ("suspect the feature cache") was wrong: the cache invalidation works.
Two things land together, because neither is safe alone:
1. RefreshData() after the re-init, so a candidate is scored on its own features.
2. A SELECTION GATE on the install. bestScore is a MAXIMUM over candidates, and
the maximum of N draws from a null beats its incumbent almost every time - so
"it beat the incumbent" installs noise. This selector is the highest-stakes of
the three found in this audit because it ACTS: it overwrites the user's
configured indicator settings and forces BuildFreshTopology(), so the network
then trains on whatever the noise picked. Fixing (1) without (2) would have
made a dormant bug actively harmful.
The gate draws the winner's own permutation null once, then corrects the p-value
for having chosen it out of N with Sidak: p_family = 1 - (1-p)^N. Sidak rather
than the max-of-N resample used by the geometry scan because each candidate here
has a DIFFERENT feature set, so their draws cannot be pooled; Sidak needs only
the one null. Exact under independence, mildly anti-conservative under positive
dependence - stated in the comment rather than hidden. A rejected winner restores
the configured settings, which best[] cannot do since the descent mutates it.
Also reports the least-ready tunable handle's BarsCalculated(). IndicatorCreate()
calculates asynchronously, so if the spread is STILL zero the handles simply are
not done and the tuner needs to yield between candidates rather than score them
back to back - a state machine like the label prebuild. That distinction is now
readable from the log instead of requiring another guess.
No input, topology or label change: no retrain.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-07 09:31:06 -04:00
|
|
|
//+------------------------------------------------------------------+
|
|
|
|
|
//| See the declaration. Minimum over the enabled tunable indicators, |
|
|
|
|
|
//| because the feature vector is only as ready as its least-ready |
|
|
|
|
|
//| component; -1 when nothing tunable is switched on. |
|
|
|
|
|
//+------------------------------------------------------------------+
|
fix(ensemble+depth): the barrier had no liveness escape, and the depth gate could not report the one state the evidence pointed at
Two charts (USDJPY 50,179 bars / XAUUSD 33,982) sat at era 0 for 38 minutes with
four of their eight members completely silent. Nothing in this commit guesses at
why the sweep fails - the last five guesses were all wrong. It makes the failure
say what it is, and stops one broken member taking its whole chart down with it.
WHAT THE LOG ACTUALLY SAYS, before any of this.
- The running build IS d9f834d (pulled 14:18, compiled 14:19:01, 0 errors), so
every depth instrument from 1dda479/7e63a8b/45c9e21 was live.
- It printed NOTHING. Zero "PRIMING", zero "CAPPED", zero "Per-indicator depth"
in 27 MB of journal. The instrument built to find the depth shortfall returned
"not this".
- On USDJPY at 14:24, CONV-cad8 completed eras 0 AND 1 across all 50,179 bars -
same chart, same 832-value window, same indicators, byte-identical fingerprint -
while LSTM-cad8 and HYB-cad8 reported ok=0 failed=50163. So it is not the
symbol, the history, the bar count or the indicator depth. It is per-member.
- ok=0 means the NEWEST anchors failed too, and a short indicator cannot do that.
The depth reading in project_silent_block_failures is therefore retired by its own
instrumentation. THE ROOT CAUSE IS STILL UNKNOWN and this commit does not claim one.
1. THE DEPTH GATE'S SILENT PATH WAS THE STATE IT WAS HUNTING.
ServableBars() read `if(servable <= 0 || servable >= want) return want;` - one
branch over three unrelated states, silent in all of them:
enabled == 0 -> nothing tunable is on. No cap. Healthy.
enabled > 0, servable == -1 -> a handle answered INVALID.
enabled > 0, servable == 0 -> created, never calculated.
BarsCalculated() returns -1 for a dead handle, so a dead MA is indistinguishable
from "no tunable indicators enabled" - and both returned `want` without printing a
character. That is exactly the state a per-member, every-index, depth-independent
failure produces, and it is the single reason a build carrying full depth
instrumentation logged nothing through the whole outage.
TunableBarsCalculated() now also reports HOW MANY indicators it consulted, and the
dead-handle case is reported (latched, with per-handle depths). The RETURN is
deliberately unchanged - what to do about a dead handle is not yet known, and
changing control flow on an unproven cause is how the last four fixes here went
wrong. SettledBars() routes its three pass-through states via ServableBars() so the
report is reachable from the training sweep, which is the only caller that hits it.
2. THE STALL REPORT NAMED A SLOT, NEVER A BLOCK.
"lookback slot 0 REJECTED (window had 24 of 832 values)" plus a guess ("an
indicator warm-up or a history-edge read"). Which guard fired was INFERRED by
counting 4+5+4+4+6+1 = 24 and concluding feature 25 must be the MA. The arithmetic
was right; every conclusion drawn from it was wrong, because a value count names a
POSITION and a position cannot tell cold from capped from invalid from off-the-end.
Every guard that can reject a bar now records itself - m_featureFailBlock - and the
report carries it, the series index, IndicatorDepthReport()'s per-handle depths,
and for each indicator whether the NEWEST bar reads. That last field is the whole
diagnosis in one word: newest-also-EMPTY means the buffer is unreadable everywhere
(cold or dead handle), newest-reads means a genuine history edge. Instrumented:
open, ATR, MA, RSI, MACD, Ichimoku, and all five AD blocks via ADIndicatorCold().
3. THE TOTAL-FAILURE BACKOFF WAS GATED ON THE WRONG QUESTION.
It armed only when m_featureFailTransient was set. Keeping that flag correct across
every guard is a list that has to stay right forever - the same shape of fix the
feature cache abandoned for the same reason - and the gate is pointless anyway: a
sweep where ZERO of 50,163 bars produced a window will produce zero again if it
restarts a millisecond later, transient or not. Doing that at full speed is what
starved six indicator threads on a six-core box. The backoff is now unconditional
on a total failure. The flag keeps its real job, deciding whether a MISS may be
cached, which is a per-bar question and not a scheduling one.
4. THE ERA BARRIER DEADLOCKED, AND SILENCED THE MEMBERS IT FROZE.
EnsembleMinTrainingEra() exempted deployed, stopped and paused members and its
comment concluded "so nothing deadlocks". Those three are all VOLUNTARY. A member
that simply CANNOT finish an era is none of them, so it pinned the minimum at its
own era with no time limit - and the hold branch's only action was
`m_lastEraCompleteTick = GetTickCount()`, which silences the stall watchdog. So on
USDJPY the two members that could not train reported, and the two healthy members
frozen behind them wrote nothing anywhere. The outage was visible only through the
members that were not suffering it.
- BarrierEraHeartbeat() stamps a clock on real era CHANGE, kept separate from
m_lastEraCompleteTick precisely because the barrier resets that one. Only a
member AT the minimum can be a blocker; a member ahead is idle by design and is
never counted as stuck.
- After ENSEMBLE_BARRIER_STUCK_MS (12 min) a non-advancing member is dropped from
the barrier minimum. It keeps training and rejoins the instant it completes an
era - at which point, being behind, it legitimately becomes the minimum again,
which is the documented resumed-laggard behaviour.
- Both transitions say so loudly, and the release states plainly that the
combined-vote score cannot be computed while the ensemble is desynchronised.
- A held member now writes a rate-limited journal line naming WHICH members it is
waiting on, so the blocker is read off one line.
5. THE PANEL FLICKER.
OnTickHandler gates its terse writer on !m_trainRunActive, and a barrier-held member
returns from Train() before ever setting it - so both writers thought they were the
only one updating the label and fought every tick. That is the reported "Getting
ready..." <-> "Waiting at era N for slower ensemble members" oscillation, and it hit
Perceptron but not Convolutional purely because Convolutional had a run active from
a completed era and Perceptron, resumed from disk, never did. Train()'s message is
the specific one, so it wins.
NEXT STEP once this is running: the stall line now ends in "REJECTED BY: ..." and
the per-handle depths. Read it. Do not reason around it.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-17 15:21:15 -04:00
|
|
|
int CExpertSignalAIBase::TunableBarsCalculated(int &enabled)
|
fix: make the indicator tuner actually measure, and gate what it installs
ROOT CAUSE of the zero spread measured on SP500 H1 2026-08-07 (all 17 candidates
returned exactly 0.00359 nats): the tune loop re-inits the indicators and then
scores, with no RefreshData() between.
ReInitADIndicators() does its part - Create() builds a NEW handle carrying the
new parameters, and the feature cache is flagged stale so features really are
recomputed. But BufferTempDataCompute() reads the CIndicatorBuffer objects, and
only Refresh() copies data out of a handle into those. So every candidate was
scored on values still held from the PREVIOUS handle. My earlier guess in the
diagnostic ("suspect the feature cache") was wrong: the cache invalidation works.
Two things land together, because neither is safe alone:
1. RefreshData() after the re-init, so a candidate is scored on its own features.
2. A SELECTION GATE on the install. bestScore is a MAXIMUM over candidates, and
the maximum of N draws from a null beats its incumbent almost every time - so
"it beat the incumbent" installs noise. This selector is the highest-stakes of
the three found in this audit because it ACTS: it overwrites the user's
configured indicator settings and forces BuildFreshTopology(), so the network
then trains on whatever the noise picked. Fixing (1) without (2) would have
made a dormant bug actively harmful.
The gate draws the winner's own permutation null once, then corrects the p-value
for having chosen it out of N with Sidak: p_family = 1 - (1-p)^N. Sidak rather
than the max-of-N resample used by the geometry scan because each candidate here
has a DIFFERENT feature set, so their draws cannot be pooled; Sidak needs only
the one null. Exact under independence, mildly anti-conservative under positive
dependence - stated in the comment rather than hidden. A rejected winner restores
the configured settings, which best[] cannot do since the descent mutates it.
Also reports the least-ready tunable handle's BarsCalculated(). IndicatorCreate()
calculates asynchronously, so if the spread is STILL zero the handles simply are
not done and the tuner needs to yield between candidates rather than score them
back to back - a state machine like the label prebuild. That distinction is now
readable from the log instead of requiring another guess.
No input, topology or label change: no retrain.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-07 09:31:06 -04:00
|
|
|
{
|
fix(ensemble+depth): the barrier had no liveness escape, and the depth gate could not report the one state the evidence pointed at
Two charts (USDJPY 50,179 bars / XAUUSD 33,982) sat at era 0 for 38 minutes with
four of their eight members completely silent. Nothing in this commit guesses at
why the sweep fails - the last five guesses were all wrong. It makes the failure
say what it is, and stops one broken member taking its whole chart down with it.
WHAT THE LOG ACTUALLY SAYS, before any of this.
- The running build IS d9f834d (pulled 14:18, compiled 14:19:01, 0 errors), so
every depth instrument from 1dda479/7e63a8b/45c9e21 was live.
- It printed NOTHING. Zero "PRIMING", zero "CAPPED", zero "Per-indicator depth"
in 27 MB of journal. The instrument built to find the depth shortfall returned
"not this".
- On USDJPY at 14:24, CONV-cad8 completed eras 0 AND 1 across all 50,179 bars -
same chart, same 832-value window, same indicators, byte-identical fingerprint -
while LSTM-cad8 and HYB-cad8 reported ok=0 failed=50163. So it is not the
symbol, the history, the bar count or the indicator depth. It is per-member.
- ok=0 means the NEWEST anchors failed too, and a short indicator cannot do that.
The depth reading in project_silent_block_failures is therefore retired by its own
instrumentation. THE ROOT CAUSE IS STILL UNKNOWN and this commit does not claim one.
1. THE DEPTH GATE'S SILENT PATH WAS THE STATE IT WAS HUNTING.
ServableBars() read `if(servable <= 0 || servable >= want) return want;` - one
branch over three unrelated states, silent in all of them:
enabled == 0 -> nothing tunable is on. No cap. Healthy.
enabled > 0, servable == -1 -> a handle answered INVALID.
enabled > 0, servable == 0 -> created, never calculated.
BarsCalculated() returns -1 for a dead handle, so a dead MA is indistinguishable
from "no tunable indicators enabled" - and both returned `want` without printing a
character. That is exactly the state a per-member, every-index, depth-independent
failure produces, and it is the single reason a build carrying full depth
instrumentation logged nothing through the whole outage.
TunableBarsCalculated() now also reports HOW MANY indicators it consulted, and the
dead-handle case is reported (latched, with per-handle depths). The RETURN is
deliberately unchanged - what to do about a dead handle is not yet known, and
changing control flow on an unproven cause is how the last four fixes here went
wrong. SettledBars() routes its three pass-through states via ServableBars() so the
report is reachable from the training sweep, which is the only caller that hits it.
2. THE STALL REPORT NAMED A SLOT, NEVER A BLOCK.
"lookback slot 0 REJECTED (window had 24 of 832 values)" plus a guess ("an
indicator warm-up or a history-edge read"). Which guard fired was INFERRED by
counting 4+5+4+4+6+1 = 24 and concluding feature 25 must be the MA. The arithmetic
was right; every conclusion drawn from it was wrong, because a value count names a
POSITION and a position cannot tell cold from capped from invalid from off-the-end.
Every guard that can reject a bar now records itself - m_featureFailBlock - and the
report carries it, the series index, IndicatorDepthReport()'s per-handle depths,
and for each indicator whether the NEWEST bar reads. That last field is the whole
diagnosis in one word: newest-also-EMPTY means the buffer is unreadable everywhere
(cold or dead handle), newest-reads means a genuine history edge. Instrumented:
open, ATR, MA, RSI, MACD, Ichimoku, and all five AD blocks via ADIndicatorCold().
3. THE TOTAL-FAILURE BACKOFF WAS GATED ON THE WRONG QUESTION.
It armed only when m_featureFailTransient was set. Keeping that flag correct across
every guard is a list that has to stay right forever - the same shape of fix the
feature cache abandoned for the same reason - and the gate is pointless anyway: a
sweep where ZERO of 50,163 bars produced a window will produce zero again if it
restarts a millisecond later, transient or not. Doing that at full speed is what
starved six indicator threads on a six-core box. The backoff is now unconditional
on a total failure. The flag keeps its real job, deciding whether a MISS may be
cached, which is a per-bar question and not a scheduling one.
4. THE ERA BARRIER DEADLOCKED, AND SILENCED THE MEMBERS IT FROZE.
EnsembleMinTrainingEra() exempted deployed, stopped and paused members and its
comment concluded "so nothing deadlocks". Those three are all VOLUNTARY. A member
that simply CANNOT finish an era is none of them, so it pinned the minimum at its
own era with no time limit - and the hold branch's only action was
`m_lastEraCompleteTick = GetTickCount()`, which silences the stall watchdog. So on
USDJPY the two members that could not train reported, and the two healthy members
frozen behind them wrote nothing anywhere. The outage was visible only through the
members that were not suffering it.
- BarrierEraHeartbeat() stamps a clock on real era CHANGE, kept separate from
m_lastEraCompleteTick precisely because the barrier resets that one. Only a
member AT the minimum can be a blocker; a member ahead is idle by design and is
never counted as stuck.
- After ENSEMBLE_BARRIER_STUCK_MS (12 min) a non-advancing member is dropped from
the barrier minimum. It keeps training and rejoins the instant it completes an
era - at which point, being behind, it legitimately becomes the minimum again,
which is the documented resumed-laggard behaviour.
- Both transitions say so loudly, and the release states plainly that the
combined-vote score cannot be computed while the ensemble is desynchronised.
- A held member now writes a rate-limited journal line naming WHICH members it is
waiting on, so the blocker is read off one line.
5. THE PANEL FLICKER.
OnTickHandler gates its terse writer on !m_trainRunActive, and a barrier-held member
returns from Train() before ever setting it - so both writers thought they were the
only one updating the label and fought every tick. That is the reported "Getting
ready..." <-> "Waiting at era N for slower ensemble members" oscillation, and it hit
Perceptron but not Convolutional purely because Convolutional had a run active from
a completed era and Perceptron, resumed from disk, never did. Train()'s message is
the specific one, so it wins.
NEXT STEP once this is running: the stall line now ends in "REJECTED BY: ..." and
the per-handle depths. Read it. Do not reason around it.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-17 15:21:15 -04:00
|
|
|
enabled = 0;
|
fix: make the indicator tuner actually measure, and gate what it installs
ROOT CAUSE of the zero spread measured on SP500 H1 2026-08-07 (all 17 candidates
returned exactly 0.00359 nats): the tune loop re-inits the indicators and then
scores, with no RefreshData() between.
ReInitADIndicators() does its part - Create() builds a NEW handle carrying the
new parameters, and the feature cache is flagged stale so features really are
recomputed. But BufferTempDataCompute() reads the CIndicatorBuffer objects, and
only Refresh() copies data out of a handle into those. So every candidate was
scored on values still held from the PREVIOUS handle. My earlier guess in the
diagnostic ("suspect the feature cache") was wrong: the cache invalidation works.
Two things land together, because neither is safe alone:
1. RefreshData() after the re-init, so a candidate is scored on its own features.
2. A SELECTION GATE on the install. bestScore is a MAXIMUM over candidates, and
the maximum of N draws from a null beats its incumbent almost every time - so
"it beat the incumbent" installs noise. This selector is the highest-stakes of
the three found in this audit because it ACTS: it overwrites the user's
configured indicator settings and forces BuildFreshTopology(), so the network
then trains on whatever the noise picked. Fixing (1) without (2) would have
made a dormant bug actively harmful.
The gate draws the winner's own permutation null once, then corrects the p-value
for having chosen it out of N with Sidak: p_family = 1 - (1-p)^N. Sidak rather
than the max-of-N resample used by the geometry scan because each candidate here
has a DIFFERENT feature set, so their draws cannot be pooled; Sidak needs only
the one null. Exact under independence, mildly anti-conservative under positive
dependence - stated in the comment rather than hidden. A rejected winner restores
the configured settings, which best[] cannot do since the descent mutates it.
Also reports the least-ready tunable handle's BarsCalculated(). IndicatorCreate()
calculates asynchronously, so if the spread is STILL zero the handles simply are
not done and the tuner needs to yield between candidates rather than score them
back to back - a state machine like the label prebuild. That distinction is now
readable from the log instead of requiring another guess.
No input, topology or label change: no retrain.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-07 09:31:06 -04:00
|
|
|
int worst = INT_MAX;
|
|
|
|
|
if(m_useMA)
|
fix(ensemble+depth): the barrier had no liveness escape, and the depth gate could not report the one state the evidence pointed at
Two charts (USDJPY 50,179 bars / XAUUSD 33,982) sat at era 0 for 38 minutes with
four of their eight members completely silent. Nothing in this commit guesses at
why the sweep fails - the last five guesses were all wrong. It makes the failure
say what it is, and stops one broken member taking its whole chart down with it.
WHAT THE LOG ACTUALLY SAYS, before any of this.
- The running build IS d9f834d (pulled 14:18, compiled 14:19:01, 0 errors), so
every depth instrument from 1dda479/7e63a8b/45c9e21 was live.
- It printed NOTHING. Zero "PRIMING", zero "CAPPED", zero "Per-indicator depth"
in 27 MB of journal. The instrument built to find the depth shortfall returned
"not this".
- On USDJPY at 14:24, CONV-cad8 completed eras 0 AND 1 across all 50,179 bars -
same chart, same 832-value window, same indicators, byte-identical fingerprint -
while LSTM-cad8 and HYB-cad8 reported ok=0 failed=50163. So it is not the
symbol, the history, the bar count or the indicator depth. It is per-member.
- ok=0 means the NEWEST anchors failed too, and a short indicator cannot do that.
The depth reading in project_silent_block_failures is therefore retired by its own
instrumentation. THE ROOT CAUSE IS STILL UNKNOWN and this commit does not claim one.
1. THE DEPTH GATE'S SILENT PATH WAS THE STATE IT WAS HUNTING.
ServableBars() read `if(servable <= 0 || servable >= want) return want;` - one
branch over three unrelated states, silent in all of them:
enabled == 0 -> nothing tunable is on. No cap. Healthy.
enabled > 0, servable == -1 -> a handle answered INVALID.
enabled > 0, servable == 0 -> created, never calculated.
BarsCalculated() returns -1 for a dead handle, so a dead MA is indistinguishable
from "no tunable indicators enabled" - and both returned `want` without printing a
character. That is exactly the state a per-member, every-index, depth-independent
failure produces, and it is the single reason a build carrying full depth
instrumentation logged nothing through the whole outage.
TunableBarsCalculated() now also reports HOW MANY indicators it consulted, and the
dead-handle case is reported (latched, with per-handle depths). The RETURN is
deliberately unchanged - what to do about a dead handle is not yet known, and
changing control flow on an unproven cause is how the last four fixes here went
wrong. SettledBars() routes its three pass-through states via ServableBars() so the
report is reachable from the training sweep, which is the only caller that hits it.
2. THE STALL REPORT NAMED A SLOT, NEVER A BLOCK.
"lookback slot 0 REJECTED (window had 24 of 832 values)" plus a guess ("an
indicator warm-up or a history-edge read"). Which guard fired was INFERRED by
counting 4+5+4+4+6+1 = 24 and concluding feature 25 must be the MA. The arithmetic
was right; every conclusion drawn from it was wrong, because a value count names a
POSITION and a position cannot tell cold from capped from invalid from off-the-end.
Every guard that can reject a bar now records itself - m_featureFailBlock - and the
report carries it, the series index, IndicatorDepthReport()'s per-handle depths,
and for each indicator whether the NEWEST bar reads. That last field is the whole
diagnosis in one word: newest-also-EMPTY means the buffer is unreadable everywhere
(cold or dead handle), newest-reads means a genuine history edge. Instrumented:
open, ATR, MA, RSI, MACD, Ichimoku, and all five AD blocks via ADIndicatorCold().
3. THE TOTAL-FAILURE BACKOFF WAS GATED ON THE WRONG QUESTION.
It armed only when m_featureFailTransient was set. Keeping that flag correct across
every guard is a list that has to stay right forever - the same shape of fix the
feature cache abandoned for the same reason - and the gate is pointless anyway: a
sweep where ZERO of 50,163 bars produced a window will produce zero again if it
restarts a millisecond later, transient or not. Doing that at full speed is what
starved six indicator threads on a six-core box. The backoff is now unconditional
on a total failure. The flag keeps its real job, deciding whether a MISS may be
cached, which is a per-bar question and not a scheduling one.
4. THE ERA BARRIER DEADLOCKED, AND SILENCED THE MEMBERS IT FROZE.
EnsembleMinTrainingEra() exempted deployed, stopped and paused members and its
comment concluded "so nothing deadlocks". Those three are all VOLUNTARY. A member
that simply CANNOT finish an era is none of them, so it pinned the minimum at its
own era with no time limit - and the hold branch's only action was
`m_lastEraCompleteTick = GetTickCount()`, which silences the stall watchdog. So on
USDJPY the two members that could not train reported, and the two healthy members
frozen behind them wrote nothing anywhere. The outage was visible only through the
members that were not suffering it.
- BarrierEraHeartbeat() stamps a clock on real era CHANGE, kept separate from
m_lastEraCompleteTick precisely because the barrier resets that one. Only a
member AT the minimum can be a blocker; a member ahead is idle by design and is
never counted as stuck.
- After ENSEMBLE_BARRIER_STUCK_MS (12 min) a non-advancing member is dropped from
the barrier minimum. It keeps training and rejoins the instant it completes an
era - at which point, being behind, it legitimately becomes the minimum again,
which is the documented resumed-laggard behaviour.
- Both transitions say so loudly, and the release states plainly that the
combined-vote score cannot be computed while the ensemble is desynchronised.
- A held member now writes a rate-limited journal line naming WHICH members it is
waiting on, so the blocker is read off one line.
5. THE PANEL FLICKER.
OnTickHandler gates its terse writer on !m_trainRunActive, and a barrier-held member
returns from Train() before ever setting it - so both writers thought they were the
only one updating the label and fought every tick. That is the reported "Getting
ready..." <-> "Waiting at era N for slower ensemble members" oscillation, and it hit
Perceptron but not Convolutional purely because Convolutional had a run active from
a completed era and Perceptron, resumed from disk, never did. Train()'s message is
the specific one, so it wins.
NEXT STEP once this is running: the stall line now ends in "REJECTED BY: ..." and
the per-handle depths. Read it. Do not reason around it.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-17 15:21:15 -04:00
|
|
|
{
|
|
|
|
|
enabled++;
|
fix: make the indicator tuner actually measure, and gate what it installs
ROOT CAUSE of the zero spread measured on SP500 H1 2026-08-07 (all 17 candidates
returned exactly 0.00359 nats): the tune loop re-inits the indicators and then
scores, with no RefreshData() between.
ReInitADIndicators() does its part - Create() builds a NEW handle carrying the
new parameters, and the feature cache is flagged stale so features really are
recomputed. But BufferTempDataCompute() reads the CIndicatorBuffer objects, and
only Refresh() copies data out of a handle into those. So every candidate was
scored on values still held from the PREVIOUS handle. My earlier guess in the
diagnostic ("suspect the feature cache") was wrong: the cache invalidation works.
Two things land together, because neither is safe alone:
1. RefreshData() after the re-init, so a candidate is scored on its own features.
2. A SELECTION GATE on the install. bestScore is a MAXIMUM over candidates, and
the maximum of N draws from a null beats its incumbent almost every time - so
"it beat the incumbent" installs noise. This selector is the highest-stakes of
the three found in this audit because it ACTS: it overwrites the user's
configured indicator settings and forces BuildFreshTopology(), so the network
then trains on whatever the noise picked. Fixing (1) without (2) would have
made a dormant bug actively harmful.
The gate draws the winner's own permutation null once, then corrects the p-value
for having chosen it out of N with Sidak: p_family = 1 - (1-p)^N. Sidak rather
than the max-of-N resample used by the geometry scan because each candidate here
has a DIFFERENT feature set, so their draws cannot be pooled; Sidak needs only
the one null. Exact under independence, mildly anti-conservative under positive
dependence - stated in the comment rather than hidden. A rejected winner restores
the configured settings, which best[] cannot do since the descent mutates it.
Also reports the least-ready tunable handle's BarsCalculated(). IndicatorCreate()
calculates asynchronously, so if the spread is STILL zero the handles simply are
not done and the tuner needs to yield between candidates rather than score them
back to back - a state machine like the label prebuild. That distinction is now
readable from the log instead of requiring another guess.
No input, topology or label change: no retrain.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-07 09:31:06 -04:00
|
|
|
worst = (int)MathMin(worst, m_MA.BarsCalculated());
|
fix(ensemble+depth): the barrier had no liveness escape, and the depth gate could not report the one state the evidence pointed at
Two charts (USDJPY 50,179 bars / XAUUSD 33,982) sat at era 0 for 38 minutes with
four of their eight members completely silent. Nothing in this commit guesses at
why the sweep fails - the last five guesses were all wrong. It makes the failure
say what it is, and stops one broken member taking its whole chart down with it.
WHAT THE LOG ACTUALLY SAYS, before any of this.
- The running build IS d9f834d (pulled 14:18, compiled 14:19:01, 0 errors), so
every depth instrument from 1dda479/7e63a8b/45c9e21 was live.
- It printed NOTHING. Zero "PRIMING", zero "CAPPED", zero "Per-indicator depth"
in 27 MB of journal. The instrument built to find the depth shortfall returned
"not this".
- On USDJPY at 14:24, CONV-cad8 completed eras 0 AND 1 across all 50,179 bars -
same chart, same 832-value window, same indicators, byte-identical fingerprint -
while LSTM-cad8 and HYB-cad8 reported ok=0 failed=50163. So it is not the
symbol, the history, the bar count or the indicator depth. It is per-member.
- ok=0 means the NEWEST anchors failed too, and a short indicator cannot do that.
The depth reading in project_silent_block_failures is therefore retired by its own
instrumentation. THE ROOT CAUSE IS STILL UNKNOWN and this commit does not claim one.
1. THE DEPTH GATE'S SILENT PATH WAS THE STATE IT WAS HUNTING.
ServableBars() read `if(servable <= 0 || servable >= want) return want;` - one
branch over three unrelated states, silent in all of them:
enabled == 0 -> nothing tunable is on. No cap. Healthy.
enabled > 0, servable == -1 -> a handle answered INVALID.
enabled > 0, servable == 0 -> created, never calculated.
BarsCalculated() returns -1 for a dead handle, so a dead MA is indistinguishable
from "no tunable indicators enabled" - and both returned `want` without printing a
character. That is exactly the state a per-member, every-index, depth-independent
failure produces, and it is the single reason a build carrying full depth
instrumentation logged nothing through the whole outage.
TunableBarsCalculated() now also reports HOW MANY indicators it consulted, and the
dead-handle case is reported (latched, with per-handle depths). The RETURN is
deliberately unchanged - what to do about a dead handle is not yet known, and
changing control flow on an unproven cause is how the last four fixes here went
wrong. SettledBars() routes its three pass-through states via ServableBars() so the
report is reachable from the training sweep, which is the only caller that hits it.
2. THE STALL REPORT NAMED A SLOT, NEVER A BLOCK.
"lookback slot 0 REJECTED (window had 24 of 832 values)" plus a guess ("an
indicator warm-up or a history-edge read"). Which guard fired was INFERRED by
counting 4+5+4+4+6+1 = 24 and concluding feature 25 must be the MA. The arithmetic
was right; every conclusion drawn from it was wrong, because a value count names a
POSITION and a position cannot tell cold from capped from invalid from off-the-end.
Every guard that can reject a bar now records itself - m_featureFailBlock - and the
report carries it, the series index, IndicatorDepthReport()'s per-handle depths,
and for each indicator whether the NEWEST bar reads. That last field is the whole
diagnosis in one word: newest-also-EMPTY means the buffer is unreadable everywhere
(cold or dead handle), newest-reads means a genuine history edge. Instrumented:
open, ATR, MA, RSI, MACD, Ichimoku, and all five AD blocks via ADIndicatorCold().
3. THE TOTAL-FAILURE BACKOFF WAS GATED ON THE WRONG QUESTION.
It armed only when m_featureFailTransient was set. Keeping that flag correct across
every guard is a list that has to stay right forever - the same shape of fix the
feature cache abandoned for the same reason - and the gate is pointless anyway: a
sweep where ZERO of 50,163 bars produced a window will produce zero again if it
restarts a millisecond later, transient or not. Doing that at full speed is what
starved six indicator threads on a six-core box. The backoff is now unconditional
on a total failure. The flag keeps its real job, deciding whether a MISS may be
cached, which is a per-bar question and not a scheduling one.
4. THE ERA BARRIER DEADLOCKED, AND SILENCED THE MEMBERS IT FROZE.
EnsembleMinTrainingEra() exempted deployed, stopped and paused members and its
comment concluded "so nothing deadlocks". Those three are all VOLUNTARY. A member
that simply CANNOT finish an era is none of them, so it pinned the minimum at its
own era with no time limit - and the hold branch's only action was
`m_lastEraCompleteTick = GetTickCount()`, which silences the stall watchdog. So on
USDJPY the two members that could not train reported, and the two healthy members
frozen behind them wrote nothing anywhere. The outage was visible only through the
members that were not suffering it.
- BarrierEraHeartbeat() stamps a clock on real era CHANGE, kept separate from
m_lastEraCompleteTick precisely because the barrier resets that one. Only a
member AT the minimum can be a blocker; a member ahead is idle by design and is
never counted as stuck.
- After ENSEMBLE_BARRIER_STUCK_MS (12 min) a non-advancing member is dropped from
the barrier minimum. It keeps training and rejoins the instant it completes an
era - at which point, being behind, it legitimately becomes the minimum again,
which is the documented resumed-laggard behaviour.
- Both transitions say so loudly, and the release states plainly that the
combined-vote score cannot be computed while the ensemble is desynchronised.
- A held member now writes a rate-limited journal line naming WHICH members it is
waiting on, so the blocker is read off one line.
5. THE PANEL FLICKER.
OnTickHandler gates its terse writer on !m_trainRunActive, and a barrier-held member
returns from Train() before ever setting it - so both writers thought they were the
only one updating the label and fought every tick. That is the reported "Getting
ready..." <-> "Waiting at era N for slower ensemble members" oscillation, and it hit
Perceptron but not Convolutional purely because Convolutional had a run active from
a completed era and Perceptron, resumed from disk, never did. Train()'s message is
the specific one, so it wins.
NEXT STEP once this is running: the stall line now ends in "REJECTED BY: ..." and
the per-handle depths. Read it. Do not reason around it.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-17 15:21:15 -04:00
|
|
|
}
|
fix: make the indicator tuner actually measure, and gate what it installs
ROOT CAUSE of the zero spread measured on SP500 H1 2026-08-07 (all 17 candidates
returned exactly 0.00359 nats): the tune loop re-inits the indicators and then
scores, with no RefreshData() between.
ReInitADIndicators() does its part - Create() builds a NEW handle carrying the
new parameters, and the feature cache is flagged stale so features really are
recomputed. But BufferTempDataCompute() reads the CIndicatorBuffer objects, and
only Refresh() copies data out of a handle into those. So every candidate was
scored on values still held from the PREVIOUS handle. My earlier guess in the
diagnostic ("suspect the feature cache") was wrong: the cache invalidation works.
Two things land together, because neither is safe alone:
1. RefreshData() after the re-init, so a candidate is scored on its own features.
2. A SELECTION GATE on the install. bestScore is a MAXIMUM over candidates, and
the maximum of N draws from a null beats its incumbent almost every time - so
"it beat the incumbent" installs noise. This selector is the highest-stakes of
the three found in this audit because it ACTS: it overwrites the user's
configured indicator settings and forces BuildFreshTopology(), so the network
then trains on whatever the noise picked. Fixing (1) without (2) would have
made a dormant bug actively harmful.
The gate draws the winner's own permutation null once, then corrects the p-value
for having chosen it out of N with Sidak: p_family = 1 - (1-p)^N. Sidak rather
than the max-of-N resample used by the geometry scan because each candidate here
has a DIFFERENT feature set, so their draws cannot be pooled; Sidak needs only
the one null. Exact under independence, mildly anti-conservative under positive
dependence - stated in the comment rather than hidden. A rejected winner restores
the configured settings, which best[] cannot do since the descent mutates it.
Also reports the least-ready tunable handle's BarsCalculated(). IndicatorCreate()
calculates asynchronously, so if the spread is STILL zero the handles simply are
not done and the tuner needs to yield between candidates rather than score them
back to back - a state machine like the label prebuild. That distinction is now
readable from the log instead of requiring another guess.
No input, topology or label change: no retrain.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-07 09:31:06 -04:00
|
|
|
if(m_useRSI)
|
fix(ensemble+depth): the barrier had no liveness escape, and the depth gate could not report the one state the evidence pointed at
Two charts (USDJPY 50,179 bars / XAUUSD 33,982) sat at era 0 for 38 minutes with
four of their eight members completely silent. Nothing in this commit guesses at
why the sweep fails - the last five guesses were all wrong. It makes the failure
say what it is, and stops one broken member taking its whole chart down with it.
WHAT THE LOG ACTUALLY SAYS, before any of this.
- The running build IS d9f834d (pulled 14:18, compiled 14:19:01, 0 errors), so
every depth instrument from 1dda479/7e63a8b/45c9e21 was live.
- It printed NOTHING. Zero "PRIMING", zero "CAPPED", zero "Per-indicator depth"
in 27 MB of journal. The instrument built to find the depth shortfall returned
"not this".
- On USDJPY at 14:24, CONV-cad8 completed eras 0 AND 1 across all 50,179 bars -
same chart, same 832-value window, same indicators, byte-identical fingerprint -
while LSTM-cad8 and HYB-cad8 reported ok=0 failed=50163. So it is not the
symbol, the history, the bar count or the indicator depth. It is per-member.
- ok=0 means the NEWEST anchors failed too, and a short indicator cannot do that.
The depth reading in project_silent_block_failures is therefore retired by its own
instrumentation. THE ROOT CAUSE IS STILL UNKNOWN and this commit does not claim one.
1. THE DEPTH GATE'S SILENT PATH WAS THE STATE IT WAS HUNTING.
ServableBars() read `if(servable <= 0 || servable >= want) return want;` - one
branch over three unrelated states, silent in all of them:
enabled == 0 -> nothing tunable is on. No cap. Healthy.
enabled > 0, servable == -1 -> a handle answered INVALID.
enabled > 0, servable == 0 -> created, never calculated.
BarsCalculated() returns -1 for a dead handle, so a dead MA is indistinguishable
from "no tunable indicators enabled" - and both returned `want` without printing a
character. That is exactly the state a per-member, every-index, depth-independent
failure produces, and it is the single reason a build carrying full depth
instrumentation logged nothing through the whole outage.
TunableBarsCalculated() now also reports HOW MANY indicators it consulted, and the
dead-handle case is reported (latched, with per-handle depths). The RETURN is
deliberately unchanged - what to do about a dead handle is not yet known, and
changing control flow on an unproven cause is how the last four fixes here went
wrong. SettledBars() routes its three pass-through states via ServableBars() so the
report is reachable from the training sweep, which is the only caller that hits it.
2. THE STALL REPORT NAMED A SLOT, NEVER A BLOCK.
"lookback slot 0 REJECTED (window had 24 of 832 values)" plus a guess ("an
indicator warm-up or a history-edge read"). Which guard fired was INFERRED by
counting 4+5+4+4+6+1 = 24 and concluding feature 25 must be the MA. The arithmetic
was right; every conclusion drawn from it was wrong, because a value count names a
POSITION and a position cannot tell cold from capped from invalid from off-the-end.
Every guard that can reject a bar now records itself - m_featureFailBlock - and the
report carries it, the series index, IndicatorDepthReport()'s per-handle depths,
and for each indicator whether the NEWEST bar reads. That last field is the whole
diagnosis in one word: newest-also-EMPTY means the buffer is unreadable everywhere
(cold or dead handle), newest-reads means a genuine history edge. Instrumented:
open, ATR, MA, RSI, MACD, Ichimoku, and all five AD blocks via ADIndicatorCold().
3. THE TOTAL-FAILURE BACKOFF WAS GATED ON THE WRONG QUESTION.
It armed only when m_featureFailTransient was set. Keeping that flag correct across
every guard is a list that has to stay right forever - the same shape of fix the
feature cache abandoned for the same reason - and the gate is pointless anyway: a
sweep where ZERO of 50,163 bars produced a window will produce zero again if it
restarts a millisecond later, transient or not. Doing that at full speed is what
starved six indicator threads on a six-core box. The backoff is now unconditional
on a total failure. The flag keeps its real job, deciding whether a MISS may be
cached, which is a per-bar question and not a scheduling one.
4. THE ERA BARRIER DEADLOCKED, AND SILENCED THE MEMBERS IT FROZE.
EnsembleMinTrainingEra() exempted deployed, stopped and paused members and its
comment concluded "so nothing deadlocks". Those three are all VOLUNTARY. A member
that simply CANNOT finish an era is none of them, so it pinned the minimum at its
own era with no time limit - and the hold branch's only action was
`m_lastEraCompleteTick = GetTickCount()`, which silences the stall watchdog. So on
USDJPY the two members that could not train reported, and the two healthy members
frozen behind them wrote nothing anywhere. The outage was visible only through the
members that were not suffering it.
- BarrierEraHeartbeat() stamps a clock on real era CHANGE, kept separate from
m_lastEraCompleteTick precisely because the barrier resets that one. Only a
member AT the minimum can be a blocker; a member ahead is idle by design and is
never counted as stuck.
- After ENSEMBLE_BARRIER_STUCK_MS (12 min) a non-advancing member is dropped from
the barrier minimum. It keeps training and rejoins the instant it completes an
era - at which point, being behind, it legitimately becomes the minimum again,
which is the documented resumed-laggard behaviour.
- Both transitions say so loudly, and the release states plainly that the
combined-vote score cannot be computed while the ensemble is desynchronised.
- A held member now writes a rate-limited journal line naming WHICH members it is
waiting on, so the blocker is read off one line.
5. THE PANEL FLICKER.
OnTickHandler gates its terse writer on !m_trainRunActive, and a barrier-held member
returns from Train() before ever setting it - so both writers thought they were the
only one updating the label and fought every tick. That is the reported "Getting
ready..." <-> "Waiting at era N for slower ensemble members" oscillation, and it hit
Perceptron but not Convolutional purely because Convolutional had a run active from
a completed era and Perceptron, resumed from disk, never did. Train()'s message is
the specific one, so it wins.
NEXT STEP once this is running: the stall line now ends in "REJECTED BY: ..." and
the per-handle depths. Read it. Do not reason around it.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-17 15:21:15 -04:00
|
|
|
{
|
|
|
|
|
enabled++;
|
fix: make the indicator tuner actually measure, and gate what it installs
ROOT CAUSE of the zero spread measured on SP500 H1 2026-08-07 (all 17 candidates
returned exactly 0.00359 nats): the tune loop re-inits the indicators and then
scores, with no RefreshData() between.
ReInitADIndicators() does its part - Create() builds a NEW handle carrying the
new parameters, and the feature cache is flagged stale so features really are
recomputed. But BufferTempDataCompute() reads the CIndicatorBuffer objects, and
only Refresh() copies data out of a handle into those. So every candidate was
scored on values still held from the PREVIOUS handle. My earlier guess in the
diagnostic ("suspect the feature cache") was wrong: the cache invalidation works.
Two things land together, because neither is safe alone:
1. RefreshData() after the re-init, so a candidate is scored on its own features.
2. A SELECTION GATE on the install. bestScore is a MAXIMUM over candidates, and
the maximum of N draws from a null beats its incumbent almost every time - so
"it beat the incumbent" installs noise. This selector is the highest-stakes of
the three found in this audit because it ACTS: it overwrites the user's
configured indicator settings and forces BuildFreshTopology(), so the network
then trains on whatever the noise picked. Fixing (1) without (2) would have
made a dormant bug actively harmful.
The gate draws the winner's own permutation null once, then corrects the p-value
for having chosen it out of N with Sidak: p_family = 1 - (1-p)^N. Sidak rather
than the max-of-N resample used by the geometry scan because each candidate here
has a DIFFERENT feature set, so their draws cannot be pooled; Sidak needs only
the one null. Exact under independence, mildly anti-conservative under positive
dependence - stated in the comment rather than hidden. A rejected winner restores
the configured settings, which best[] cannot do since the descent mutates it.
Also reports the least-ready tunable handle's BarsCalculated(). IndicatorCreate()
calculates asynchronously, so if the spread is STILL zero the handles simply are
not done and the tuner needs to yield between candidates rather than score them
back to back - a state machine like the label prebuild. That distinction is now
readable from the log instead of requiring another guess.
No input, topology or label change: no retrain.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-07 09:31:06 -04:00
|
|
|
worst = (int)MathMin(worst, m_RSI.BarsCalculated());
|
fix(ensemble+depth): the barrier had no liveness escape, and the depth gate could not report the one state the evidence pointed at
Two charts (USDJPY 50,179 bars / XAUUSD 33,982) sat at era 0 for 38 minutes with
four of their eight members completely silent. Nothing in this commit guesses at
why the sweep fails - the last five guesses were all wrong. It makes the failure
say what it is, and stops one broken member taking its whole chart down with it.
WHAT THE LOG ACTUALLY SAYS, before any of this.
- The running build IS d9f834d (pulled 14:18, compiled 14:19:01, 0 errors), so
every depth instrument from 1dda479/7e63a8b/45c9e21 was live.
- It printed NOTHING. Zero "PRIMING", zero "CAPPED", zero "Per-indicator depth"
in 27 MB of journal. The instrument built to find the depth shortfall returned
"not this".
- On USDJPY at 14:24, CONV-cad8 completed eras 0 AND 1 across all 50,179 bars -
same chart, same 832-value window, same indicators, byte-identical fingerprint -
while LSTM-cad8 and HYB-cad8 reported ok=0 failed=50163. So it is not the
symbol, the history, the bar count or the indicator depth. It is per-member.
- ok=0 means the NEWEST anchors failed too, and a short indicator cannot do that.
The depth reading in project_silent_block_failures is therefore retired by its own
instrumentation. THE ROOT CAUSE IS STILL UNKNOWN and this commit does not claim one.
1. THE DEPTH GATE'S SILENT PATH WAS THE STATE IT WAS HUNTING.
ServableBars() read `if(servable <= 0 || servable >= want) return want;` - one
branch over three unrelated states, silent in all of them:
enabled == 0 -> nothing tunable is on. No cap. Healthy.
enabled > 0, servable == -1 -> a handle answered INVALID.
enabled > 0, servable == 0 -> created, never calculated.
BarsCalculated() returns -1 for a dead handle, so a dead MA is indistinguishable
from "no tunable indicators enabled" - and both returned `want` without printing a
character. That is exactly the state a per-member, every-index, depth-independent
failure produces, and it is the single reason a build carrying full depth
instrumentation logged nothing through the whole outage.
TunableBarsCalculated() now also reports HOW MANY indicators it consulted, and the
dead-handle case is reported (latched, with per-handle depths). The RETURN is
deliberately unchanged - what to do about a dead handle is not yet known, and
changing control flow on an unproven cause is how the last four fixes here went
wrong. SettledBars() routes its three pass-through states via ServableBars() so the
report is reachable from the training sweep, which is the only caller that hits it.
2. THE STALL REPORT NAMED A SLOT, NEVER A BLOCK.
"lookback slot 0 REJECTED (window had 24 of 832 values)" plus a guess ("an
indicator warm-up or a history-edge read"). Which guard fired was INFERRED by
counting 4+5+4+4+6+1 = 24 and concluding feature 25 must be the MA. The arithmetic
was right; every conclusion drawn from it was wrong, because a value count names a
POSITION and a position cannot tell cold from capped from invalid from off-the-end.
Every guard that can reject a bar now records itself - m_featureFailBlock - and the
report carries it, the series index, IndicatorDepthReport()'s per-handle depths,
and for each indicator whether the NEWEST bar reads. That last field is the whole
diagnosis in one word: newest-also-EMPTY means the buffer is unreadable everywhere
(cold or dead handle), newest-reads means a genuine history edge. Instrumented:
open, ATR, MA, RSI, MACD, Ichimoku, and all five AD blocks via ADIndicatorCold().
3. THE TOTAL-FAILURE BACKOFF WAS GATED ON THE WRONG QUESTION.
It armed only when m_featureFailTransient was set. Keeping that flag correct across
every guard is a list that has to stay right forever - the same shape of fix the
feature cache abandoned for the same reason - and the gate is pointless anyway: a
sweep where ZERO of 50,163 bars produced a window will produce zero again if it
restarts a millisecond later, transient or not. Doing that at full speed is what
starved six indicator threads on a six-core box. The backoff is now unconditional
on a total failure. The flag keeps its real job, deciding whether a MISS may be
cached, which is a per-bar question and not a scheduling one.
4. THE ERA BARRIER DEADLOCKED, AND SILENCED THE MEMBERS IT FROZE.
EnsembleMinTrainingEra() exempted deployed, stopped and paused members and its
comment concluded "so nothing deadlocks". Those three are all VOLUNTARY. A member
that simply CANNOT finish an era is none of them, so it pinned the minimum at its
own era with no time limit - and the hold branch's only action was
`m_lastEraCompleteTick = GetTickCount()`, which silences the stall watchdog. So on
USDJPY the two members that could not train reported, and the two healthy members
frozen behind them wrote nothing anywhere. The outage was visible only through the
members that were not suffering it.
- BarrierEraHeartbeat() stamps a clock on real era CHANGE, kept separate from
m_lastEraCompleteTick precisely because the barrier resets that one. Only a
member AT the minimum can be a blocker; a member ahead is idle by design and is
never counted as stuck.
- After ENSEMBLE_BARRIER_STUCK_MS (12 min) a non-advancing member is dropped from
the barrier minimum. It keeps training and rejoins the instant it completes an
era - at which point, being behind, it legitimately becomes the minimum again,
which is the documented resumed-laggard behaviour.
- Both transitions say so loudly, and the release states plainly that the
combined-vote score cannot be computed while the ensemble is desynchronised.
- A held member now writes a rate-limited journal line naming WHICH members it is
waiting on, so the blocker is read off one line.
5. THE PANEL FLICKER.
OnTickHandler gates its terse writer on !m_trainRunActive, and a barrier-held member
returns from Train() before ever setting it - so both writers thought they were the
only one updating the label and fought every tick. That is the reported "Getting
ready..." <-> "Waiting at era N for slower ensemble members" oscillation, and it hit
Perceptron but not Convolutional purely because Convolutional had a run active from
a completed era and Perceptron, resumed from disk, never did. Train()'s message is
the specific one, so it wins.
NEXT STEP once this is running: the stall line now ends in "REJECTED BY: ..." and
the per-handle depths. Read it. Do not reason around it.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-17 15:21:15 -04:00
|
|
|
}
|
fix: make the indicator tuner actually measure, and gate what it installs
ROOT CAUSE of the zero spread measured on SP500 H1 2026-08-07 (all 17 candidates
returned exactly 0.00359 nats): the tune loop re-inits the indicators and then
scores, with no RefreshData() between.
ReInitADIndicators() does its part - Create() builds a NEW handle carrying the
new parameters, and the feature cache is flagged stale so features really are
recomputed. But BufferTempDataCompute() reads the CIndicatorBuffer objects, and
only Refresh() copies data out of a handle into those. So every candidate was
scored on values still held from the PREVIOUS handle. My earlier guess in the
diagnostic ("suspect the feature cache") was wrong: the cache invalidation works.
Two things land together, because neither is safe alone:
1. RefreshData() after the re-init, so a candidate is scored on its own features.
2. A SELECTION GATE on the install. bestScore is a MAXIMUM over candidates, and
the maximum of N draws from a null beats its incumbent almost every time - so
"it beat the incumbent" installs noise. This selector is the highest-stakes of
the three found in this audit because it ACTS: it overwrites the user's
configured indicator settings and forces BuildFreshTopology(), so the network
then trains on whatever the noise picked. Fixing (1) without (2) would have
made a dormant bug actively harmful.
The gate draws the winner's own permutation null once, then corrects the p-value
for having chosen it out of N with Sidak: p_family = 1 - (1-p)^N. Sidak rather
than the max-of-N resample used by the geometry scan because each candidate here
has a DIFFERENT feature set, so their draws cannot be pooled; Sidak needs only
the one null. Exact under independence, mildly anti-conservative under positive
dependence - stated in the comment rather than hidden. A rejected winner restores
the configured settings, which best[] cannot do since the descent mutates it.
Also reports the least-ready tunable handle's BarsCalculated(). IndicatorCreate()
calculates asynchronously, so if the spread is STILL zero the handles simply are
not done and the tuner needs to yield between candidates rather than score them
back to back - a state machine like the label prebuild. That distinction is now
readable from the log instead of requiring another guess.
No input, topology or label change: no retrain.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-07 09:31:06 -04:00
|
|
|
if(m_useMACD)
|
fix(ensemble+depth): the barrier had no liveness escape, and the depth gate could not report the one state the evidence pointed at
Two charts (USDJPY 50,179 bars / XAUUSD 33,982) sat at era 0 for 38 minutes with
four of their eight members completely silent. Nothing in this commit guesses at
why the sweep fails - the last five guesses were all wrong. It makes the failure
say what it is, and stops one broken member taking its whole chart down with it.
WHAT THE LOG ACTUALLY SAYS, before any of this.
- The running build IS d9f834d (pulled 14:18, compiled 14:19:01, 0 errors), so
every depth instrument from 1dda479/7e63a8b/45c9e21 was live.
- It printed NOTHING. Zero "PRIMING", zero "CAPPED", zero "Per-indicator depth"
in 27 MB of journal. The instrument built to find the depth shortfall returned
"not this".
- On USDJPY at 14:24, CONV-cad8 completed eras 0 AND 1 across all 50,179 bars -
same chart, same 832-value window, same indicators, byte-identical fingerprint -
while LSTM-cad8 and HYB-cad8 reported ok=0 failed=50163. So it is not the
symbol, the history, the bar count or the indicator depth. It is per-member.
- ok=0 means the NEWEST anchors failed too, and a short indicator cannot do that.
The depth reading in project_silent_block_failures is therefore retired by its own
instrumentation. THE ROOT CAUSE IS STILL UNKNOWN and this commit does not claim one.
1. THE DEPTH GATE'S SILENT PATH WAS THE STATE IT WAS HUNTING.
ServableBars() read `if(servable <= 0 || servable >= want) return want;` - one
branch over three unrelated states, silent in all of them:
enabled == 0 -> nothing tunable is on. No cap. Healthy.
enabled > 0, servable == -1 -> a handle answered INVALID.
enabled > 0, servable == 0 -> created, never calculated.
BarsCalculated() returns -1 for a dead handle, so a dead MA is indistinguishable
from "no tunable indicators enabled" - and both returned `want` without printing a
character. That is exactly the state a per-member, every-index, depth-independent
failure produces, and it is the single reason a build carrying full depth
instrumentation logged nothing through the whole outage.
TunableBarsCalculated() now also reports HOW MANY indicators it consulted, and the
dead-handle case is reported (latched, with per-handle depths). The RETURN is
deliberately unchanged - what to do about a dead handle is not yet known, and
changing control flow on an unproven cause is how the last four fixes here went
wrong. SettledBars() routes its three pass-through states via ServableBars() so the
report is reachable from the training sweep, which is the only caller that hits it.
2. THE STALL REPORT NAMED A SLOT, NEVER A BLOCK.
"lookback slot 0 REJECTED (window had 24 of 832 values)" plus a guess ("an
indicator warm-up or a history-edge read"). Which guard fired was INFERRED by
counting 4+5+4+4+6+1 = 24 and concluding feature 25 must be the MA. The arithmetic
was right; every conclusion drawn from it was wrong, because a value count names a
POSITION and a position cannot tell cold from capped from invalid from off-the-end.
Every guard that can reject a bar now records itself - m_featureFailBlock - and the
report carries it, the series index, IndicatorDepthReport()'s per-handle depths,
and for each indicator whether the NEWEST bar reads. That last field is the whole
diagnosis in one word: newest-also-EMPTY means the buffer is unreadable everywhere
(cold or dead handle), newest-reads means a genuine history edge. Instrumented:
open, ATR, MA, RSI, MACD, Ichimoku, and all five AD blocks via ADIndicatorCold().
3. THE TOTAL-FAILURE BACKOFF WAS GATED ON THE WRONG QUESTION.
It armed only when m_featureFailTransient was set. Keeping that flag correct across
every guard is a list that has to stay right forever - the same shape of fix the
feature cache abandoned for the same reason - and the gate is pointless anyway: a
sweep where ZERO of 50,163 bars produced a window will produce zero again if it
restarts a millisecond later, transient or not. Doing that at full speed is what
starved six indicator threads on a six-core box. The backoff is now unconditional
on a total failure. The flag keeps its real job, deciding whether a MISS may be
cached, which is a per-bar question and not a scheduling one.
4. THE ERA BARRIER DEADLOCKED, AND SILENCED THE MEMBERS IT FROZE.
EnsembleMinTrainingEra() exempted deployed, stopped and paused members and its
comment concluded "so nothing deadlocks". Those three are all VOLUNTARY. A member
that simply CANNOT finish an era is none of them, so it pinned the minimum at its
own era with no time limit - and the hold branch's only action was
`m_lastEraCompleteTick = GetTickCount()`, which silences the stall watchdog. So on
USDJPY the two members that could not train reported, and the two healthy members
frozen behind them wrote nothing anywhere. The outage was visible only through the
members that were not suffering it.
- BarrierEraHeartbeat() stamps a clock on real era CHANGE, kept separate from
m_lastEraCompleteTick precisely because the barrier resets that one. Only a
member AT the minimum can be a blocker; a member ahead is idle by design and is
never counted as stuck.
- After ENSEMBLE_BARRIER_STUCK_MS (12 min) a non-advancing member is dropped from
the barrier minimum. It keeps training and rejoins the instant it completes an
era - at which point, being behind, it legitimately becomes the minimum again,
which is the documented resumed-laggard behaviour.
- Both transitions say so loudly, and the release states plainly that the
combined-vote score cannot be computed while the ensemble is desynchronised.
- A held member now writes a rate-limited journal line naming WHICH members it is
waiting on, so the blocker is read off one line.
5. THE PANEL FLICKER.
OnTickHandler gates its terse writer on !m_trainRunActive, and a barrier-held member
returns from Train() before ever setting it - so both writers thought they were the
only one updating the label and fought every tick. That is the reported "Getting
ready..." <-> "Waiting at era N for slower ensemble members" oscillation, and it hit
Perceptron but not Convolutional purely because Convolutional had a run active from
a completed era and Perceptron, resumed from disk, never did. Train()'s message is
the specific one, so it wins.
NEXT STEP once this is running: the stall line now ends in "REJECTED BY: ..." and
the per-handle depths. Read it. Do not reason around it.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-17 15:21:15 -04:00
|
|
|
{
|
|
|
|
|
enabled++;
|
fix: make the indicator tuner actually measure, and gate what it installs
ROOT CAUSE of the zero spread measured on SP500 H1 2026-08-07 (all 17 candidates
returned exactly 0.00359 nats): the tune loop re-inits the indicators and then
scores, with no RefreshData() between.
ReInitADIndicators() does its part - Create() builds a NEW handle carrying the
new parameters, and the feature cache is flagged stale so features really are
recomputed. But BufferTempDataCompute() reads the CIndicatorBuffer objects, and
only Refresh() copies data out of a handle into those. So every candidate was
scored on values still held from the PREVIOUS handle. My earlier guess in the
diagnostic ("suspect the feature cache") was wrong: the cache invalidation works.
Two things land together, because neither is safe alone:
1. RefreshData() after the re-init, so a candidate is scored on its own features.
2. A SELECTION GATE on the install. bestScore is a MAXIMUM over candidates, and
the maximum of N draws from a null beats its incumbent almost every time - so
"it beat the incumbent" installs noise. This selector is the highest-stakes of
the three found in this audit because it ACTS: it overwrites the user's
configured indicator settings and forces BuildFreshTopology(), so the network
then trains on whatever the noise picked. Fixing (1) without (2) would have
made a dormant bug actively harmful.
The gate draws the winner's own permutation null once, then corrects the p-value
for having chosen it out of N with Sidak: p_family = 1 - (1-p)^N. Sidak rather
than the max-of-N resample used by the geometry scan because each candidate here
has a DIFFERENT feature set, so their draws cannot be pooled; Sidak needs only
the one null. Exact under independence, mildly anti-conservative under positive
dependence - stated in the comment rather than hidden. A rejected winner restores
the configured settings, which best[] cannot do since the descent mutates it.
Also reports the least-ready tunable handle's BarsCalculated(). IndicatorCreate()
calculates asynchronously, so if the spread is STILL zero the handles simply are
not done and the tuner needs to yield between candidates rather than score them
back to back - a state machine like the label prebuild. That distinction is now
readable from the log instead of requiring another guess.
No input, topology or label change: no retrain.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-07 09:31:06 -04:00
|
|
|
worst = (int)MathMin(worst, m_MACDFeature.BarsCalculated());
|
fix(ensemble+depth): the barrier had no liveness escape, and the depth gate could not report the one state the evidence pointed at
Two charts (USDJPY 50,179 bars / XAUUSD 33,982) sat at era 0 for 38 minutes with
four of their eight members completely silent. Nothing in this commit guesses at
why the sweep fails - the last five guesses were all wrong. It makes the failure
say what it is, and stops one broken member taking its whole chart down with it.
WHAT THE LOG ACTUALLY SAYS, before any of this.
- The running build IS d9f834d (pulled 14:18, compiled 14:19:01, 0 errors), so
every depth instrument from 1dda479/7e63a8b/45c9e21 was live.
- It printed NOTHING. Zero "PRIMING", zero "CAPPED", zero "Per-indicator depth"
in 27 MB of journal. The instrument built to find the depth shortfall returned
"not this".
- On USDJPY at 14:24, CONV-cad8 completed eras 0 AND 1 across all 50,179 bars -
same chart, same 832-value window, same indicators, byte-identical fingerprint -
while LSTM-cad8 and HYB-cad8 reported ok=0 failed=50163. So it is not the
symbol, the history, the bar count or the indicator depth. It is per-member.
- ok=0 means the NEWEST anchors failed too, and a short indicator cannot do that.
The depth reading in project_silent_block_failures is therefore retired by its own
instrumentation. THE ROOT CAUSE IS STILL UNKNOWN and this commit does not claim one.
1. THE DEPTH GATE'S SILENT PATH WAS THE STATE IT WAS HUNTING.
ServableBars() read `if(servable <= 0 || servable >= want) return want;` - one
branch over three unrelated states, silent in all of them:
enabled == 0 -> nothing tunable is on. No cap. Healthy.
enabled > 0, servable == -1 -> a handle answered INVALID.
enabled > 0, servable == 0 -> created, never calculated.
BarsCalculated() returns -1 for a dead handle, so a dead MA is indistinguishable
from "no tunable indicators enabled" - and both returned `want` without printing a
character. That is exactly the state a per-member, every-index, depth-independent
failure produces, and it is the single reason a build carrying full depth
instrumentation logged nothing through the whole outage.
TunableBarsCalculated() now also reports HOW MANY indicators it consulted, and the
dead-handle case is reported (latched, with per-handle depths). The RETURN is
deliberately unchanged - what to do about a dead handle is not yet known, and
changing control flow on an unproven cause is how the last four fixes here went
wrong. SettledBars() routes its three pass-through states via ServableBars() so the
report is reachable from the training sweep, which is the only caller that hits it.
2. THE STALL REPORT NAMED A SLOT, NEVER A BLOCK.
"lookback slot 0 REJECTED (window had 24 of 832 values)" plus a guess ("an
indicator warm-up or a history-edge read"). Which guard fired was INFERRED by
counting 4+5+4+4+6+1 = 24 and concluding feature 25 must be the MA. The arithmetic
was right; every conclusion drawn from it was wrong, because a value count names a
POSITION and a position cannot tell cold from capped from invalid from off-the-end.
Every guard that can reject a bar now records itself - m_featureFailBlock - and the
report carries it, the series index, IndicatorDepthReport()'s per-handle depths,
and for each indicator whether the NEWEST bar reads. That last field is the whole
diagnosis in one word: newest-also-EMPTY means the buffer is unreadable everywhere
(cold or dead handle), newest-reads means a genuine history edge. Instrumented:
open, ATR, MA, RSI, MACD, Ichimoku, and all five AD blocks via ADIndicatorCold().
3. THE TOTAL-FAILURE BACKOFF WAS GATED ON THE WRONG QUESTION.
It armed only when m_featureFailTransient was set. Keeping that flag correct across
every guard is a list that has to stay right forever - the same shape of fix the
feature cache abandoned for the same reason - and the gate is pointless anyway: a
sweep where ZERO of 50,163 bars produced a window will produce zero again if it
restarts a millisecond later, transient or not. Doing that at full speed is what
starved six indicator threads on a six-core box. The backoff is now unconditional
on a total failure. The flag keeps its real job, deciding whether a MISS may be
cached, which is a per-bar question and not a scheduling one.
4. THE ERA BARRIER DEADLOCKED, AND SILENCED THE MEMBERS IT FROZE.
EnsembleMinTrainingEra() exempted deployed, stopped and paused members and its
comment concluded "so nothing deadlocks". Those three are all VOLUNTARY. A member
that simply CANNOT finish an era is none of them, so it pinned the minimum at its
own era with no time limit - and the hold branch's only action was
`m_lastEraCompleteTick = GetTickCount()`, which silences the stall watchdog. So on
USDJPY the two members that could not train reported, and the two healthy members
frozen behind them wrote nothing anywhere. The outage was visible only through the
members that were not suffering it.
- BarrierEraHeartbeat() stamps a clock on real era CHANGE, kept separate from
m_lastEraCompleteTick precisely because the barrier resets that one. Only a
member AT the minimum can be a blocker; a member ahead is idle by design and is
never counted as stuck.
- After ENSEMBLE_BARRIER_STUCK_MS (12 min) a non-advancing member is dropped from
the barrier minimum. It keeps training and rejoins the instant it completes an
era - at which point, being behind, it legitimately becomes the minimum again,
which is the documented resumed-laggard behaviour.
- Both transitions say so loudly, and the release states plainly that the
combined-vote score cannot be computed while the ensemble is desynchronised.
- A held member now writes a rate-limited journal line naming WHICH members it is
waiting on, so the blocker is read off one line.
5. THE PANEL FLICKER.
OnTickHandler gates its terse writer on !m_trainRunActive, and a barrier-held member
returns from Train() before ever setting it - so both writers thought they were the
only one updating the label and fought every tick. That is the reported "Getting
ready..." <-> "Waiting at era N for slower ensemble members" oscillation, and it hit
Perceptron but not Convolutional purely because Convolutional had a run active from
a completed era and Perceptron, resumed from disk, never did. Train()'s message is
the specific one, so it wins.
NEXT STEP once this is running: the stall line now ends in "REJECTED BY: ..." and
the per-handle depths. Read it. Do not reason around it.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-17 15:21:15 -04:00
|
|
|
}
|
fix: make the indicator tuner actually measure, and gate what it installs
ROOT CAUSE of the zero spread measured on SP500 H1 2026-08-07 (all 17 candidates
returned exactly 0.00359 nats): the tune loop re-inits the indicators and then
scores, with no RefreshData() between.
ReInitADIndicators() does its part - Create() builds a NEW handle carrying the
new parameters, and the feature cache is flagged stale so features really are
recomputed. But BufferTempDataCompute() reads the CIndicatorBuffer objects, and
only Refresh() copies data out of a handle into those. So every candidate was
scored on values still held from the PREVIOUS handle. My earlier guess in the
diagnostic ("suspect the feature cache") was wrong: the cache invalidation works.
Two things land together, because neither is safe alone:
1. RefreshData() after the re-init, so a candidate is scored on its own features.
2. A SELECTION GATE on the install. bestScore is a MAXIMUM over candidates, and
the maximum of N draws from a null beats its incumbent almost every time - so
"it beat the incumbent" installs noise. This selector is the highest-stakes of
the three found in this audit because it ACTS: it overwrites the user's
configured indicator settings and forces BuildFreshTopology(), so the network
then trains on whatever the noise picked. Fixing (1) without (2) would have
made a dormant bug actively harmful.
The gate draws the winner's own permutation null once, then corrects the p-value
for having chosen it out of N with Sidak: p_family = 1 - (1-p)^N. Sidak rather
than the max-of-N resample used by the geometry scan because each candidate here
has a DIFFERENT feature set, so their draws cannot be pooled; Sidak needs only
the one null. Exact under independence, mildly anti-conservative under positive
dependence - stated in the comment rather than hidden. A rejected winner restores
the configured settings, which best[] cannot do since the descent mutates it.
Also reports the least-ready tunable handle's BarsCalculated(). IndicatorCreate()
calculates asynchronously, so if the spread is STILL zero the handles simply are
not done and the tuner needs to yield between candidates rather than score them
back to back - a state machine like the label prebuild. That distinction is now
readable from the log instead of requiring another guess.
No input, topology or label change: no retrain.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-07 09:31:06 -04:00
|
|
|
if(m_useIchimoku)
|
fix(ensemble+depth): the barrier had no liveness escape, and the depth gate could not report the one state the evidence pointed at
Two charts (USDJPY 50,179 bars / XAUUSD 33,982) sat at era 0 for 38 minutes with
four of their eight members completely silent. Nothing in this commit guesses at
why the sweep fails - the last five guesses were all wrong. It makes the failure
say what it is, and stops one broken member taking its whole chart down with it.
WHAT THE LOG ACTUALLY SAYS, before any of this.
- The running build IS d9f834d (pulled 14:18, compiled 14:19:01, 0 errors), so
every depth instrument from 1dda479/7e63a8b/45c9e21 was live.
- It printed NOTHING. Zero "PRIMING", zero "CAPPED", zero "Per-indicator depth"
in 27 MB of journal. The instrument built to find the depth shortfall returned
"not this".
- On USDJPY at 14:24, CONV-cad8 completed eras 0 AND 1 across all 50,179 bars -
same chart, same 832-value window, same indicators, byte-identical fingerprint -
while LSTM-cad8 and HYB-cad8 reported ok=0 failed=50163. So it is not the
symbol, the history, the bar count or the indicator depth. It is per-member.
- ok=0 means the NEWEST anchors failed too, and a short indicator cannot do that.
The depth reading in project_silent_block_failures is therefore retired by its own
instrumentation. THE ROOT CAUSE IS STILL UNKNOWN and this commit does not claim one.
1. THE DEPTH GATE'S SILENT PATH WAS THE STATE IT WAS HUNTING.
ServableBars() read `if(servable <= 0 || servable >= want) return want;` - one
branch over three unrelated states, silent in all of them:
enabled == 0 -> nothing tunable is on. No cap. Healthy.
enabled > 0, servable == -1 -> a handle answered INVALID.
enabled > 0, servable == 0 -> created, never calculated.
BarsCalculated() returns -1 for a dead handle, so a dead MA is indistinguishable
from "no tunable indicators enabled" - and both returned `want` without printing a
character. That is exactly the state a per-member, every-index, depth-independent
failure produces, and it is the single reason a build carrying full depth
instrumentation logged nothing through the whole outage.
TunableBarsCalculated() now also reports HOW MANY indicators it consulted, and the
dead-handle case is reported (latched, with per-handle depths). The RETURN is
deliberately unchanged - what to do about a dead handle is not yet known, and
changing control flow on an unproven cause is how the last four fixes here went
wrong. SettledBars() routes its three pass-through states via ServableBars() so the
report is reachable from the training sweep, which is the only caller that hits it.
2. THE STALL REPORT NAMED A SLOT, NEVER A BLOCK.
"lookback slot 0 REJECTED (window had 24 of 832 values)" plus a guess ("an
indicator warm-up or a history-edge read"). Which guard fired was INFERRED by
counting 4+5+4+4+6+1 = 24 and concluding feature 25 must be the MA. The arithmetic
was right; every conclusion drawn from it was wrong, because a value count names a
POSITION and a position cannot tell cold from capped from invalid from off-the-end.
Every guard that can reject a bar now records itself - m_featureFailBlock - and the
report carries it, the series index, IndicatorDepthReport()'s per-handle depths,
and for each indicator whether the NEWEST bar reads. That last field is the whole
diagnosis in one word: newest-also-EMPTY means the buffer is unreadable everywhere
(cold or dead handle), newest-reads means a genuine history edge. Instrumented:
open, ATR, MA, RSI, MACD, Ichimoku, and all five AD blocks via ADIndicatorCold().
3. THE TOTAL-FAILURE BACKOFF WAS GATED ON THE WRONG QUESTION.
It armed only when m_featureFailTransient was set. Keeping that flag correct across
every guard is a list that has to stay right forever - the same shape of fix the
feature cache abandoned for the same reason - and the gate is pointless anyway: a
sweep where ZERO of 50,163 bars produced a window will produce zero again if it
restarts a millisecond later, transient or not. Doing that at full speed is what
starved six indicator threads on a six-core box. The backoff is now unconditional
on a total failure. The flag keeps its real job, deciding whether a MISS may be
cached, which is a per-bar question and not a scheduling one.
4. THE ERA BARRIER DEADLOCKED, AND SILENCED THE MEMBERS IT FROZE.
EnsembleMinTrainingEra() exempted deployed, stopped and paused members and its
comment concluded "so nothing deadlocks". Those three are all VOLUNTARY. A member
that simply CANNOT finish an era is none of them, so it pinned the minimum at its
own era with no time limit - and the hold branch's only action was
`m_lastEraCompleteTick = GetTickCount()`, which silences the stall watchdog. So on
USDJPY the two members that could not train reported, and the two healthy members
frozen behind them wrote nothing anywhere. The outage was visible only through the
members that were not suffering it.
- BarrierEraHeartbeat() stamps a clock on real era CHANGE, kept separate from
m_lastEraCompleteTick precisely because the barrier resets that one. Only a
member AT the minimum can be a blocker; a member ahead is idle by design and is
never counted as stuck.
- After ENSEMBLE_BARRIER_STUCK_MS (12 min) a non-advancing member is dropped from
the barrier minimum. It keeps training and rejoins the instant it completes an
era - at which point, being behind, it legitimately becomes the minimum again,
which is the documented resumed-laggard behaviour.
- Both transitions say so loudly, and the release states plainly that the
combined-vote score cannot be computed while the ensemble is desynchronised.
- A held member now writes a rate-limited journal line naming WHICH members it is
waiting on, so the blocker is read off one line.
5. THE PANEL FLICKER.
OnTickHandler gates its terse writer on !m_trainRunActive, and a barrier-held member
returns from Train() before ever setting it - so both writers thought they were the
only one updating the label and fought every tick. That is the reported "Getting
ready..." <-> "Waiting at era N for slower ensemble members" oscillation, and it hit
Perceptron but not Convolutional purely because Convolutional had a run active from
a completed era and Perceptron, resumed from disk, never did. Train()'s message is
the specific one, so it wins.
NEXT STEP once this is running: the stall line now ends in "REJECTED BY: ..." and
the per-handle depths. Read it. Do not reason around it.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-17 15:21:15 -04:00
|
|
|
{
|
|
|
|
|
enabled++;
|
fix: make the indicator tuner actually measure, and gate what it installs
ROOT CAUSE of the zero spread measured on SP500 H1 2026-08-07 (all 17 candidates
returned exactly 0.00359 nats): the tune loop re-inits the indicators and then
scores, with no RefreshData() between.
ReInitADIndicators() does its part - Create() builds a NEW handle carrying the
new parameters, and the feature cache is flagged stale so features really are
recomputed. But BufferTempDataCompute() reads the CIndicatorBuffer objects, and
only Refresh() copies data out of a handle into those. So every candidate was
scored on values still held from the PREVIOUS handle. My earlier guess in the
diagnostic ("suspect the feature cache") was wrong: the cache invalidation works.
Two things land together, because neither is safe alone:
1. RefreshData() after the re-init, so a candidate is scored on its own features.
2. A SELECTION GATE on the install. bestScore is a MAXIMUM over candidates, and
the maximum of N draws from a null beats its incumbent almost every time - so
"it beat the incumbent" installs noise. This selector is the highest-stakes of
the three found in this audit because it ACTS: it overwrites the user's
configured indicator settings and forces BuildFreshTopology(), so the network
then trains on whatever the noise picked. Fixing (1) without (2) would have
made a dormant bug actively harmful.
The gate draws the winner's own permutation null once, then corrects the p-value
for having chosen it out of N with Sidak: p_family = 1 - (1-p)^N. Sidak rather
than the max-of-N resample used by the geometry scan because each candidate here
has a DIFFERENT feature set, so their draws cannot be pooled; Sidak needs only
the one null. Exact under independence, mildly anti-conservative under positive
dependence - stated in the comment rather than hidden. A rejected winner restores
the configured settings, which best[] cannot do since the descent mutates it.
Also reports the least-ready tunable handle's BarsCalculated(). IndicatorCreate()
calculates asynchronously, so if the spread is STILL zero the handles simply are
not done and the tuner needs to yield between candidates rather than score them
back to back - a state machine like the label prebuild. That distinction is now
readable from the log instead of requiring another guess.
No input, topology or label change: no retrain.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-07 09:31:06 -04:00
|
|
|
worst = (int)MathMin(worst, m_Ichimoku.BarsCalculated());
|
fix(ensemble+depth): the barrier had no liveness escape, and the depth gate could not report the one state the evidence pointed at
Two charts (USDJPY 50,179 bars / XAUUSD 33,982) sat at era 0 for 38 minutes with
four of their eight members completely silent. Nothing in this commit guesses at
why the sweep fails - the last five guesses were all wrong. It makes the failure
say what it is, and stops one broken member taking its whole chart down with it.
WHAT THE LOG ACTUALLY SAYS, before any of this.
- The running build IS d9f834d (pulled 14:18, compiled 14:19:01, 0 errors), so
every depth instrument from 1dda479/7e63a8b/45c9e21 was live.
- It printed NOTHING. Zero "PRIMING", zero "CAPPED", zero "Per-indicator depth"
in 27 MB of journal. The instrument built to find the depth shortfall returned
"not this".
- On USDJPY at 14:24, CONV-cad8 completed eras 0 AND 1 across all 50,179 bars -
same chart, same 832-value window, same indicators, byte-identical fingerprint -
while LSTM-cad8 and HYB-cad8 reported ok=0 failed=50163. So it is not the
symbol, the history, the bar count or the indicator depth. It is per-member.
- ok=0 means the NEWEST anchors failed too, and a short indicator cannot do that.
The depth reading in project_silent_block_failures is therefore retired by its own
instrumentation. THE ROOT CAUSE IS STILL UNKNOWN and this commit does not claim one.
1. THE DEPTH GATE'S SILENT PATH WAS THE STATE IT WAS HUNTING.
ServableBars() read `if(servable <= 0 || servable >= want) return want;` - one
branch over three unrelated states, silent in all of them:
enabled == 0 -> nothing tunable is on. No cap. Healthy.
enabled > 0, servable == -1 -> a handle answered INVALID.
enabled > 0, servable == 0 -> created, never calculated.
BarsCalculated() returns -1 for a dead handle, so a dead MA is indistinguishable
from "no tunable indicators enabled" - and both returned `want` without printing a
character. That is exactly the state a per-member, every-index, depth-independent
failure produces, and it is the single reason a build carrying full depth
instrumentation logged nothing through the whole outage.
TunableBarsCalculated() now also reports HOW MANY indicators it consulted, and the
dead-handle case is reported (latched, with per-handle depths). The RETURN is
deliberately unchanged - what to do about a dead handle is not yet known, and
changing control flow on an unproven cause is how the last four fixes here went
wrong. SettledBars() routes its three pass-through states via ServableBars() so the
report is reachable from the training sweep, which is the only caller that hits it.
2. THE STALL REPORT NAMED A SLOT, NEVER A BLOCK.
"lookback slot 0 REJECTED (window had 24 of 832 values)" plus a guess ("an
indicator warm-up or a history-edge read"). Which guard fired was INFERRED by
counting 4+5+4+4+6+1 = 24 and concluding feature 25 must be the MA. The arithmetic
was right; every conclusion drawn from it was wrong, because a value count names a
POSITION and a position cannot tell cold from capped from invalid from off-the-end.
Every guard that can reject a bar now records itself - m_featureFailBlock - and the
report carries it, the series index, IndicatorDepthReport()'s per-handle depths,
and for each indicator whether the NEWEST bar reads. That last field is the whole
diagnosis in one word: newest-also-EMPTY means the buffer is unreadable everywhere
(cold or dead handle), newest-reads means a genuine history edge. Instrumented:
open, ATR, MA, RSI, MACD, Ichimoku, and all five AD blocks via ADIndicatorCold().
3. THE TOTAL-FAILURE BACKOFF WAS GATED ON THE WRONG QUESTION.
It armed only when m_featureFailTransient was set. Keeping that flag correct across
every guard is a list that has to stay right forever - the same shape of fix the
feature cache abandoned for the same reason - and the gate is pointless anyway: a
sweep where ZERO of 50,163 bars produced a window will produce zero again if it
restarts a millisecond later, transient or not. Doing that at full speed is what
starved six indicator threads on a six-core box. The backoff is now unconditional
on a total failure. The flag keeps its real job, deciding whether a MISS may be
cached, which is a per-bar question and not a scheduling one.
4. THE ERA BARRIER DEADLOCKED, AND SILENCED THE MEMBERS IT FROZE.
EnsembleMinTrainingEra() exempted deployed, stopped and paused members and its
comment concluded "so nothing deadlocks". Those three are all VOLUNTARY. A member
that simply CANNOT finish an era is none of them, so it pinned the minimum at its
own era with no time limit - and the hold branch's only action was
`m_lastEraCompleteTick = GetTickCount()`, which silences the stall watchdog. So on
USDJPY the two members that could not train reported, and the two healthy members
frozen behind them wrote nothing anywhere. The outage was visible only through the
members that were not suffering it.
- BarrierEraHeartbeat() stamps a clock on real era CHANGE, kept separate from
m_lastEraCompleteTick precisely because the barrier resets that one. Only a
member AT the minimum can be a blocker; a member ahead is idle by design and is
never counted as stuck.
- After ENSEMBLE_BARRIER_STUCK_MS (12 min) a non-advancing member is dropped from
the barrier minimum. It keeps training and rejoins the instant it completes an
era - at which point, being behind, it legitimately becomes the minimum again,
which is the documented resumed-laggard behaviour.
- Both transitions say so loudly, and the release states plainly that the
combined-vote score cannot be computed while the ensemble is desynchronised.
- A held member now writes a rate-limited journal line naming WHICH members it is
waiting on, so the blocker is read off one line.
5. THE PANEL FLICKER.
OnTickHandler gates its terse writer on !m_trainRunActive, and a barrier-held member
returns from Train() before ever setting it - so both writers thought they were the
only one updating the label and fought every tick. That is the reported "Getting
ready..." <-> "Waiting at era N for slower ensemble members" oscillation, and it hit
Perceptron but not Convolutional purely because Convolutional had a run active from
a completed era and Perceptron, resumed from disk, never did. Train()'s message is
the specific one, so it wins.
NEXT STEP once this is running: the stall line now ends in "REJECTED BY: ..." and
the per-handle depths. Read it. Do not reason around it.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-17 15:21:15 -04:00
|
|
|
}
|
fix: make the indicator tuner actually measure, and gate what it installs
ROOT CAUSE of the zero spread measured on SP500 H1 2026-08-07 (all 17 candidates
returned exactly 0.00359 nats): the tune loop re-inits the indicators and then
scores, with no RefreshData() between.
ReInitADIndicators() does its part - Create() builds a NEW handle carrying the
new parameters, and the feature cache is flagged stale so features really are
recomputed. But BufferTempDataCompute() reads the CIndicatorBuffer objects, and
only Refresh() copies data out of a handle into those. So every candidate was
scored on values still held from the PREVIOUS handle. My earlier guess in the
diagnostic ("suspect the feature cache") was wrong: the cache invalidation works.
Two things land together, because neither is safe alone:
1. RefreshData() after the re-init, so a candidate is scored on its own features.
2. A SELECTION GATE on the install. bestScore is a MAXIMUM over candidates, and
the maximum of N draws from a null beats its incumbent almost every time - so
"it beat the incumbent" installs noise. This selector is the highest-stakes of
the three found in this audit because it ACTS: it overwrites the user's
configured indicator settings and forces BuildFreshTopology(), so the network
then trains on whatever the noise picked. Fixing (1) without (2) would have
made a dormant bug actively harmful.
The gate draws the winner's own permutation null once, then corrects the p-value
for having chosen it out of N with Sidak: p_family = 1 - (1-p)^N. Sidak rather
than the max-of-N resample used by the geometry scan because each candidate here
has a DIFFERENT feature set, so their draws cannot be pooled; Sidak needs only
the one null. Exact under independence, mildly anti-conservative under positive
dependence - stated in the comment rather than hidden. A rejected winner restores
the configured settings, which best[] cannot do since the descent mutates it.
Also reports the least-ready tunable handle's BarsCalculated(). IndicatorCreate()
calculates asynchronously, so if the spread is STILL zero the handles simply are
not done and the tuner needs to yield between candidates rather than score them
back to back - a state machine like the label prebuild. That distinction is now
readable from the log instead of requiring another guess.
No input, topology or label change: no retrain.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-07 09:31:06 -04:00
|
|
|
if(m_useADCumulativeDelta)
|
fix(ensemble+depth): the barrier had no liveness escape, and the depth gate could not report the one state the evidence pointed at
Two charts (USDJPY 50,179 bars / XAUUSD 33,982) sat at era 0 for 38 minutes with
four of their eight members completely silent. Nothing in this commit guesses at
why the sweep fails - the last five guesses were all wrong. It makes the failure
say what it is, and stops one broken member taking its whole chart down with it.
WHAT THE LOG ACTUALLY SAYS, before any of this.
- The running build IS d9f834d (pulled 14:18, compiled 14:19:01, 0 errors), so
every depth instrument from 1dda479/7e63a8b/45c9e21 was live.
- It printed NOTHING. Zero "PRIMING", zero "CAPPED", zero "Per-indicator depth"
in 27 MB of journal. The instrument built to find the depth shortfall returned
"not this".
- On USDJPY at 14:24, CONV-cad8 completed eras 0 AND 1 across all 50,179 bars -
same chart, same 832-value window, same indicators, byte-identical fingerprint -
while LSTM-cad8 and HYB-cad8 reported ok=0 failed=50163. So it is not the
symbol, the history, the bar count or the indicator depth. It is per-member.
- ok=0 means the NEWEST anchors failed too, and a short indicator cannot do that.
The depth reading in project_silent_block_failures is therefore retired by its own
instrumentation. THE ROOT CAUSE IS STILL UNKNOWN and this commit does not claim one.
1. THE DEPTH GATE'S SILENT PATH WAS THE STATE IT WAS HUNTING.
ServableBars() read `if(servable <= 0 || servable >= want) return want;` - one
branch over three unrelated states, silent in all of them:
enabled == 0 -> nothing tunable is on. No cap. Healthy.
enabled > 0, servable == -1 -> a handle answered INVALID.
enabled > 0, servable == 0 -> created, never calculated.
BarsCalculated() returns -1 for a dead handle, so a dead MA is indistinguishable
from "no tunable indicators enabled" - and both returned `want` without printing a
character. That is exactly the state a per-member, every-index, depth-independent
failure produces, and it is the single reason a build carrying full depth
instrumentation logged nothing through the whole outage.
TunableBarsCalculated() now also reports HOW MANY indicators it consulted, and the
dead-handle case is reported (latched, with per-handle depths). The RETURN is
deliberately unchanged - what to do about a dead handle is not yet known, and
changing control flow on an unproven cause is how the last four fixes here went
wrong. SettledBars() routes its three pass-through states via ServableBars() so the
report is reachable from the training sweep, which is the only caller that hits it.
2. THE STALL REPORT NAMED A SLOT, NEVER A BLOCK.
"lookback slot 0 REJECTED (window had 24 of 832 values)" plus a guess ("an
indicator warm-up or a history-edge read"). Which guard fired was INFERRED by
counting 4+5+4+4+6+1 = 24 and concluding feature 25 must be the MA. The arithmetic
was right; every conclusion drawn from it was wrong, because a value count names a
POSITION and a position cannot tell cold from capped from invalid from off-the-end.
Every guard that can reject a bar now records itself - m_featureFailBlock - and the
report carries it, the series index, IndicatorDepthReport()'s per-handle depths,
and for each indicator whether the NEWEST bar reads. That last field is the whole
diagnosis in one word: newest-also-EMPTY means the buffer is unreadable everywhere
(cold or dead handle), newest-reads means a genuine history edge. Instrumented:
open, ATR, MA, RSI, MACD, Ichimoku, and all five AD blocks via ADIndicatorCold().
3. THE TOTAL-FAILURE BACKOFF WAS GATED ON THE WRONG QUESTION.
It armed only when m_featureFailTransient was set. Keeping that flag correct across
every guard is a list that has to stay right forever - the same shape of fix the
feature cache abandoned for the same reason - and the gate is pointless anyway: a
sweep where ZERO of 50,163 bars produced a window will produce zero again if it
restarts a millisecond later, transient or not. Doing that at full speed is what
starved six indicator threads on a six-core box. The backoff is now unconditional
on a total failure. The flag keeps its real job, deciding whether a MISS may be
cached, which is a per-bar question and not a scheduling one.
4. THE ERA BARRIER DEADLOCKED, AND SILENCED THE MEMBERS IT FROZE.
EnsembleMinTrainingEra() exempted deployed, stopped and paused members and its
comment concluded "so nothing deadlocks". Those three are all VOLUNTARY. A member
that simply CANNOT finish an era is none of them, so it pinned the minimum at its
own era with no time limit - and the hold branch's only action was
`m_lastEraCompleteTick = GetTickCount()`, which silences the stall watchdog. So on
USDJPY the two members that could not train reported, and the two healthy members
frozen behind them wrote nothing anywhere. The outage was visible only through the
members that were not suffering it.
- BarrierEraHeartbeat() stamps a clock on real era CHANGE, kept separate from
m_lastEraCompleteTick precisely because the barrier resets that one. Only a
member AT the minimum can be a blocker; a member ahead is idle by design and is
never counted as stuck.
- After ENSEMBLE_BARRIER_STUCK_MS (12 min) a non-advancing member is dropped from
the barrier minimum. It keeps training and rejoins the instant it completes an
era - at which point, being behind, it legitimately becomes the minimum again,
which is the documented resumed-laggard behaviour.
- Both transitions say so loudly, and the release states plainly that the
combined-vote score cannot be computed while the ensemble is desynchronised.
- A held member now writes a rate-limited journal line naming WHICH members it is
waiting on, so the blocker is read off one line.
5. THE PANEL FLICKER.
OnTickHandler gates its terse writer on !m_trainRunActive, and a barrier-held member
returns from Train() before ever setting it - so both writers thought they were the
only one updating the label and fought every tick. That is the reported "Getting
ready..." <-> "Waiting at era N for slower ensemble members" oscillation, and it hit
Perceptron but not Convolutional purely because Convolutional had a run active from
a completed era and Perceptron, resumed from disk, never did. Train()'s message is
the specific one, so it wins.
NEXT STEP once this is running: the stall line now ends in "REJECTED BY: ..." and
the per-handle depths. Read it. Do not reason around it.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-17 15:21:15 -04:00
|
|
|
{
|
|
|
|
|
enabled++;
|
fix: make the indicator tuner actually measure, and gate what it installs
ROOT CAUSE of the zero spread measured on SP500 H1 2026-08-07 (all 17 candidates
returned exactly 0.00359 nats): the tune loop re-inits the indicators and then
scores, with no RefreshData() between.
ReInitADIndicators() does its part - Create() builds a NEW handle carrying the
new parameters, and the feature cache is flagged stale so features really are
recomputed. But BufferTempDataCompute() reads the CIndicatorBuffer objects, and
only Refresh() copies data out of a handle into those. So every candidate was
scored on values still held from the PREVIOUS handle. My earlier guess in the
diagnostic ("suspect the feature cache") was wrong: the cache invalidation works.
Two things land together, because neither is safe alone:
1. RefreshData() after the re-init, so a candidate is scored on its own features.
2. A SELECTION GATE on the install. bestScore is a MAXIMUM over candidates, and
the maximum of N draws from a null beats its incumbent almost every time - so
"it beat the incumbent" installs noise. This selector is the highest-stakes of
the three found in this audit because it ACTS: it overwrites the user's
configured indicator settings and forces BuildFreshTopology(), so the network
then trains on whatever the noise picked. Fixing (1) without (2) would have
made a dormant bug actively harmful.
The gate draws the winner's own permutation null once, then corrects the p-value
for having chosen it out of N with Sidak: p_family = 1 - (1-p)^N. Sidak rather
than the max-of-N resample used by the geometry scan because each candidate here
has a DIFFERENT feature set, so their draws cannot be pooled; Sidak needs only
the one null. Exact under independence, mildly anti-conservative under positive
dependence - stated in the comment rather than hidden. A rejected winner restores
the configured settings, which best[] cannot do since the descent mutates it.
Also reports the least-ready tunable handle's BarsCalculated(). IndicatorCreate()
calculates asynchronously, so if the spread is STILL zero the handles simply are
not done and the tuner needs to yield between candidates rather than score them
back to back - a state machine like the label prebuild. That distinction is now
readable from the log instead of requiring another guess.
No input, topology or label change: no retrain.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-07 09:31:06 -04:00
|
|
|
worst = (int)MathMin(worst, m_ADCumulativeDelta.BarsCalculated());
|
fix(ensemble+depth): the barrier had no liveness escape, and the depth gate could not report the one state the evidence pointed at
Two charts (USDJPY 50,179 bars / XAUUSD 33,982) sat at era 0 for 38 minutes with
four of their eight members completely silent. Nothing in this commit guesses at
why the sweep fails - the last five guesses were all wrong. It makes the failure
say what it is, and stops one broken member taking its whole chart down with it.
WHAT THE LOG ACTUALLY SAYS, before any of this.
- The running build IS d9f834d (pulled 14:18, compiled 14:19:01, 0 errors), so
every depth instrument from 1dda479/7e63a8b/45c9e21 was live.
- It printed NOTHING. Zero "PRIMING", zero "CAPPED", zero "Per-indicator depth"
in 27 MB of journal. The instrument built to find the depth shortfall returned
"not this".
- On USDJPY at 14:24, CONV-cad8 completed eras 0 AND 1 across all 50,179 bars -
same chart, same 832-value window, same indicators, byte-identical fingerprint -
while LSTM-cad8 and HYB-cad8 reported ok=0 failed=50163. So it is not the
symbol, the history, the bar count or the indicator depth. It is per-member.
- ok=0 means the NEWEST anchors failed too, and a short indicator cannot do that.
The depth reading in project_silent_block_failures is therefore retired by its own
instrumentation. THE ROOT CAUSE IS STILL UNKNOWN and this commit does not claim one.
1. THE DEPTH GATE'S SILENT PATH WAS THE STATE IT WAS HUNTING.
ServableBars() read `if(servable <= 0 || servable >= want) return want;` - one
branch over three unrelated states, silent in all of them:
enabled == 0 -> nothing tunable is on. No cap. Healthy.
enabled > 0, servable == -1 -> a handle answered INVALID.
enabled > 0, servable == 0 -> created, never calculated.
BarsCalculated() returns -1 for a dead handle, so a dead MA is indistinguishable
from "no tunable indicators enabled" - and both returned `want` without printing a
character. That is exactly the state a per-member, every-index, depth-independent
failure produces, and it is the single reason a build carrying full depth
instrumentation logged nothing through the whole outage.
TunableBarsCalculated() now also reports HOW MANY indicators it consulted, and the
dead-handle case is reported (latched, with per-handle depths). The RETURN is
deliberately unchanged - what to do about a dead handle is not yet known, and
changing control flow on an unproven cause is how the last four fixes here went
wrong. SettledBars() routes its three pass-through states via ServableBars() so the
report is reachable from the training sweep, which is the only caller that hits it.
2. THE STALL REPORT NAMED A SLOT, NEVER A BLOCK.
"lookback slot 0 REJECTED (window had 24 of 832 values)" plus a guess ("an
indicator warm-up or a history-edge read"). Which guard fired was INFERRED by
counting 4+5+4+4+6+1 = 24 and concluding feature 25 must be the MA. The arithmetic
was right; every conclusion drawn from it was wrong, because a value count names a
POSITION and a position cannot tell cold from capped from invalid from off-the-end.
Every guard that can reject a bar now records itself - m_featureFailBlock - and the
report carries it, the series index, IndicatorDepthReport()'s per-handle depths,
and for each indicator whether the NEWEST bar reads. That last field is the whole
diagnosis in one word: newest-also-EMPTY means the buffer is unreadable everywhere
(cold or dead handle), newest-reads means a genuine history edge. Instrumented:
open, ATR, MA, RSI, MACD, Ichimoku, and all five AD blocks via ADIndicatorCold().
3. THE TOTAL-FAILURE BACKOFF WAS GATED ON THE WRONG QUESTION.
It armed only when m_featureFailTransient was set. Keeping that flag correct across
every guard is a list that has to stay right forever - the same shape of fix the
feature cache abandoned for the same reason - and the gate is pointless anyway: a
sweep where ZERO of 50,163 bars produced a window will produce zero again if it
restarts a millisecond later, transient or not. Doing that at full speed is what
starved six indicator threads on a six-core box. The backoff is now unconditional
on a total failure. The flag keeps its real job, deciding whether a MISS may be
cached, which is a per-bar question and not a scheduling one.
4. THE ERA BARRIER DEADLOCKED, AND SILENCED THE MEMBERS IT FROZE.
EnsembleMinTrainingEra() exempted deployed, stopped and paused members and its
comment concluded "so nothing deadlocks". Those three are all VOLUNTARY. A member
that simply CANNOT finish an era is none of them, so it pinned the minimum at its
own era with no time limit - and the hold branch's only action was
`m_lastEraCompleteTick = GetTickCount()`, which silences the stall watchdog. So on
USDJPY the two members that could not train reported, and the two healthy members
frozen behind them wrote nothing anywhere. The outage was visible only through the
members that were not suffering it.
- BarrierEraHeartbeat() stamps a clock on real era CHANGE, kept separate from
m_lastEraCompleteTick precisely because the barrier resets that one. Only a
member AT the minimum can be a blocker; a member ahead is idle by design and is
never counted as stuck.
- After ENSEMBLE_BARRIER_STUCK_MS (12 min) a non-advancing member is dropped from
the barrier minimum. It keeps training and rejoins the instant it completes an
era - at which point, being behind, it legitimately becomes the minimum again,
which is the documented resumed-laggard behaviour.
- Both transitions say so loudly, and the release states plainly that the
combined-vote score cannot be computed while the ensemble is desynchronised.
- A held member now writes a rate-limited journal line naming WHICH members it is
waiting on, so the blocker is read off one line.
5. THE PANEL FLICKER.
OnTickHandler gates its terse writer on !m_trainRunActive, and a barrier-held member
returns from Train() before ever setting it - so both writers thought they were the
only one updating the label and fought every tick. That is the reported "Getting
ready..." <-> "Waiting at era N for slower ensemble members" oscillation, and it hit
Perceptron but not Convolutional purely because Convolutional had a run active from
a completed era and Perceptron, resumed from disk, never did. Train()'s message is
the specific one, so it wins.
NEXT STEP once this is running: the stall line now ends in "REJECTED BY: ..." and
the per-handle depths. Read it. Do not reason around it.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-17 15:21:15 -04:00
|
|
|
}
|
fix: make the indicator tuner actually measure, and gate what it installs
ROOT CAUSE of the zero spread measured on SP500 H1 2026-08-07 (all 17 candidates
returned exactly 0.00359 nats): the tune loop re-inits the indicators and then
scores, with no RefreshData() between.
ReInitADIndicators() does its part - Create() builds a NEW handle carrying the
new parameters, and the feature cache is flagged stale so features really are
recomputed. But BufferTempDataCompute() reads the CIndicatorBuffer objects, and
only Refresh() copies data out of a handle into those. So every candidate was
scored on values still held from the PREVIOUS handle. My earlier guess in the
diagnostic ("suspect the feature cache") was wrong: the cache invalidation works.
Two things land together, because neither is safe alone:
1. RefreshData() after the re-init, so a candidate is scored on its own features.
2. A SELECTION GATE on the install. bestScore is a MAXIMUM over candidates, and
the maximum of N draws from a null beats its incumbent almost every time - so
"it beat the incumbent" installs noise. This selector is the highest-stakes of
the three found in this audit because it ACTS: it overwrites the user's
configured indicator settings and forces BuildFreshTopology(), so the network
then trains on whatever the noise picked. Fixing (1) without (2) would have
made a dormant bug actively harmful.
The gate draws the winner's own permutation null once, then corrects the p-value
for having chosen it out of N with Sidak: p_family = 1 - (1-p)^N. Sidak rather
than the max-of-N resample used by the geometry scan because each candidate here
has a DIFFERENT feature set, so their draws cannot be pooled; Sidak needs only
the one null. Exact under independence, mildly anti-conservative under positive
dependence - stated in the comment rather than hidden. A rejected winner restores
the configured settings, which best[] cannot do since the descent mutates it.
Also reports the least-ready tunable handle's BarsCalculated(). IndicatorCreate()
calculates asynchronously, so if the spread is STILL zero the handles simply are
not done and the tuner needs to yield between candidates rather than score them
back to back - a state machine like the label prebuild. That distinction is now
readable from the log instead of requiring another guess.
No input, topology or label change: no retrain.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-07 09:31:06 -04:00
|
|
|
if(m_useADShorteningOfThrust)
|
fix(ensemble+depth): the barrier had no liveness escape, and the depth gate could not report the one state the evidence pointed at
Two charts (USDJPY 50,179 bars / XAUUSD 33,982) sat at era 0 for 38 minutes with
four of their eight members completely silent. Nothing in this commit guesses at
why the sweep fails - the last five guesses were all wrong. It makes the failure
say what it is, and stops one broken member taking its whole chart down with it.
WHAT THE LOG ACTUALLY SAYS, before any of this.
- The running build IS d9f834d (pulled 14:18, compiled 14:19:01, 0 errors), so
every depth instrument from 1dda479/7e63a8b/45c9e21 was live.
- It printed NOTHING. Zero "PRIMING", zero "CAPPED", zero "Per-indicator depth"
in 27 MB of journal. The instrument built to find the depth shortfall returned
"not this".
- On USDJPY at 14:24, CONV-cad8 completed eras 0 AND 1 across all 50,179 bars -
same chart, same 832-value window, same indicators, byte-identical fingerprint -
while LSTM-cad8 and HYB-cad8 reported ok=0 failed=50163. So it is not the
symbol, the history, the bar count or the indicator depth. It is per-member.
- ok=0 means the NEWEST anchors failed too, and a short indicator cannot do that.
The depth reading in project_silent_block_failures is therefore retired by its own
instrumentation. THE ROOT CAUSE IS STILL UNKNOWN and this commit does not claim one.
1. THE DEPTH GATE'S SILENT PATH WAS THE STATE IT WAS HUNTING.
ServableBars() read `if(servable <= 0 || servable >= want) return want;` - one
branch over three unrelated states, silent in all of them:
enabled == 0 -> nothing tunable is on. No cap. Healthy.
enabled > 0, servable == -1 -> a handle answered INVALID.
enabled > 0, servable == 0 -> created, never calculated.
BarsCalculated() returns -1 for a dead handle, so a dead MA is indistinguishable
from "no tunable indicators enabled" - and both returned `want` without printing a
character. That is exactly the state a per-member, every-index, depth-independent
failure produces, and it is the single reason a build carrying full depth
instrumentation logged nothing through the whole outage.
TunableBarsCalculated() now also reports HOW MANY indicators it consulted, and the
dead-handle case is reported (latched, with per-handle depths). The RETURN is
deliberately unchanged - what to do about a dead handle is not yet known, and
changing control flow on an unproven cause is how the last four fixes here went
wrong. SettledBars() routes its three pass-through states via ServableBars() so the
report is reachable from the training sweep, which is the only caller that hits it.
2. THE STALL REPORT NAMED A SLOT, NEVER A BLOCK.
"lookback slot 0 REJECTED (window had 24 of 832 values)" plus a guess ("an
indicator warm-up or a history-edge read"). Which guard fired was INFERRED by
counting 4+5+4+4+6+1 = 24 and concluding feature 25 must be the MA. The arithmetic
was right; every conclusion drawn from it was wrong, because a value count names a
POSITION and a position cannot tell cold from capped from invalid from off-the-end.
Every guard that can reject a bar now records itself - m_featureFailBlock - and the
report carries it, the series index, IndicatorDepthReport()'s per-handle depths,
and for each indicator whether the NEWEST bar reads. That last field is the whole
diagnosis in one word: newest-also-EMPTY means the buffer is unreadable everywhere
(cold or dead handle), newest-reads means a genuine history edge. Instrumented:
open, ATR, MA, RSI, MACD, Ichimoku, and all five AD blocks via ADIndicatorCold().
3. THE TOTAL-FAILURE BACKOFF WAS GATED ON THE WRONG QUESTION.
It armed only when m_featureFailTransient was set. Keeping that flag correct across
every guard is a list that has to stay right forever - the same shape of fix the
feature cache abandoned for the same reason - and the gate is pointless anyway: a
sweep where ZERO of 50,163 bars produced a window will produce zero again if it
restarts a millisecond later, transient or not. Doing that at full speed is what
starved six indicator threads on a six-core box. The backoff is now unconditional
on a total failure. The flag keeps its real job, deciding whether a MISS may be
cached, which is a per-bar question and not a scheduling one.
4. THE ERA BARRIER DEADLOCKED, AND SILENCED THE MEMBERS IT FROZE.
EnsembleMinTrainingEra() exempted deployed, stopped and paused members and its
comment concluded "so nothing deadlocks". Those three are all VOLUNTARY. A member
that simply CANNOT finish an era is none of them, so it pinned the minimum at its
own era with no time limit - and the hold branch's only action was
`m_lastEraCompleteTick = GetTickCount()`, which silences the stall watchdog. So on
USDJPY the two members that could not train reported, and the two healthy members
frozen behind them wrote nothing anywhere. The outage was visible only through the
members that were not suffering it.
- BarrierEraHeartbeat() stamps a clock on real era CHANGE, kept separate from
m_lastEraCompleteTick precisely because the barrier resets that one. Only a
member AT the minimum can be a blocker; a member ahead is idle by design and is
never counted as stuck.
- After ENSEMBLE_BARRIER_STUCK_MS (12 min) a non-advancing member is dropped from
the barrier minimum. It keeps training and rejoins the instant it completes an
era - at which point, being behind, it legitimately becomes the minimum again,
which is the documented resumed-laggard behaviour.
- Both transitions say so loudly, and the release states plainly that the
combined-vote score cannot be computed while the ensemble is desynchronised.
- A held member now writes a rate-limited journal line naming WHICH members it is
waiting on, so the blocker is read off one line.
5. THE PANEL FLICKER.
OnTickHandler gates its terse writer on !m_trainRunActive, and a barrier-held member
returns from Train() before ever setting it - so both writers thought they were the
only one updating the label and fought every tick. That is the reported "Getting
ready..." <-> "Waiting at era N for slower ensemble members" oscillation, and it hit
Perceptron but not Convolutional purely because Convolutional had a run active from
a completed era and Perceptron, resumed from disk, never did. Train()'s message is
the specific one, so it wins.
NEXT STEP once this is running: the stall line now ends in "REJECTED BY: ..." and
the per-handle depths. Read it. Do not reason around it.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-17 15:21:15 -04:00
|
|
|
{
|
|
|
|
|
enabled++;
|
fix: make the indicator tuner actually measure, and gate what it installs
ROOT CAUSE of the zero spread measured on SP500 H1 2026-08-07 (all 17 candidates
returned exactly 0.00359 nats): the tune loop re-inits the indicators and then
scores, with no RefreshData() between.
ReInitADIndicators() does its part - Create() builds a NEW handle carrying the
new parameters, and the feature cache is flagged stale so features really are
recomputed. But BufferTempDataCompute() reads the CIndicatorBuffer objects, and
only Refresh() copies data out of a handle into those. So every candidate was
scored on values still held from the PREVIOUS handle. My earlier guess in the
diagnostic ("suspect the feature cache") was wrong: the cache invalidation works.
Two things land together, because neither is safe alone:
1. RefreshData() after the re-init, so a candidate is scored on its own features.
2. A SELECTION GATE on the install. bestScore is a MAXIMUM over candidates, and
the maximum of N draws from a null beats its incumbent almost every time - so
"it beat the incumbent" installs noise. This selector is the highest-stakes of
the three found in this audit because it ACTS: it overwrites the user's
configured indicator settings and forces BuildFreshTopology(), so the network
then trains on whatever the noise picked. Fixing (1) without (2) would have
made a dormant bug actively harmful.
The gate draws the winner's own permutation null once, then corrects the p-value
for having chosen it out of N with Sidak: p_family = 1 - (1-p)^N. Sidak rather
than the max-of-N resample used by the geometry scan because each candidate here
has a DIFFERENT feature set, so their draws cannot be pooled; Sidak needs only
the one null. Exact under independence, mildly anti-conservative under positive
dependence - stated in the comment rather than hidden. A rejected winner restores
the configured settings, which best[] cannot do since the descent mutates it.
Also reports the least-ready tunable handle's BarsCalculated(). IndicatorCreate()
calculates asynchronously, so if the spread is STILL zero the handles simply are
not done and the tuner needs to yield between candidates rather than score them
back to back - a state machine like the label prebuild. That distinction is now
readable from the log instead of requiring another guess.
No input, topology or label change: no retrain.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-07 09:31:06 -04:00
|
|
|
worst = (int)MathMin(worst, m_ADShorteningOfThrust.BarsCalculated());
|
fix(ensemble+depth): the barrier had no liveness escape, and the depth gate could not report the one state the evidence pointed at
Two charts (USDJPY 50,179 bars / XAUUSD 33,982) sat at era 0 for 38 minutes with
four of their eight members completely silent. Nothing in this commit guesses at
why the sweep fails - the last five guesses were all wrong. It makes the failure
say what it is, and stops one broken member taking its whole chart down with it.
WHAT THE LOG ACTUALLY SAYS, before any of this.
- The running build IS d9f834d (pulled 14:18, compiled 14:19:01, 0 errors), so
every depth instrument from 1dda479/7e63a8b/45c9e21 was live.
- It printed NOTHING. Zero "PRIMING", zero "CAPPED", zero "Per-indicator depth"
in 27 MB of journal. The instrument built to find the depth shortfall returned
"not this".
- On USDJPY at 14:24, CONV-cad8 completed eras 0 AND 1 across all 50,179 bars -
same chart, same 832-value window, same indicators, byte-identical fingerprint -
while LSTM-cad8 and HYB-cad8 reported ok=0 failed=50163. So it is not the
symbol, the history, the bar count or the indicator depth. It is per-member.
- ok=0 means the NEWEST anchors failed too, and a short indicator cannot do that.
The depth reading in project_silent_block_failures is therefore retired by its own
instrumentation. THE ROOT CAUSE IS STILL UNKNOWN and this commit does not claim one.
1. THE DEPTH GATE'S SILENT PATH WAS THE STATE IT WAS HUNTING.
ServableBars() read `if(servable <= 0 || servable >= want) return want;` - one
branch over three unrelated states, silent in all of them:
enabled == 0 -> nothing tunable is on. No cap. Healthy.
enabled > 0, servable == -1 -> a handle answered INVALID.
enabled > 0, servable == 0 -> created, never calculated.
BarsCalculated() returns -1 for a dead handle, so a dead MA is indistinguishable
from "no tunable indicators enabled" - and both returned `want` without printing a
character. That is exactly the state a per-member, every-index, depth-independent
failure produces, and it is the single reason a build carrying full depth
instrumentation logged nothing through the whole outage.
TunableBarsCalculated() now also reports HOW MANY indicators it consulted, and the
dead-handle case is reported (latched, with per-handle depths). The RETURN is
deliberately unchanged - what to do about a dead handle is not yet known, and
changing control flow on an unproven cause is how the last four fixes here went
wrong. SettledBars() routes its three pass-through states via ServableBars() so the
report is reachable from the training sweep, which is the only caller that hits it.
2. THE STALL REPORT NAMED A SLOT, NEVER A BLOCK.
"lookback slot 0 REJECTED (window had 24 of 832 values)" plus a guess ("an
indicator warm-up or a history-edge read"). Which guard fired was INFERRED by
counting 4+5+4+4+6+1 = 24 and concluding feature 25 must be the MA. The arithmetic
was right; every conclusion drawn from it was wrong, because a value count names a
POSITION and a position cannot tell cold from capped from invalid from off-the-end.
Every guard that can reject a bar now records itself - m_featureFailBlock - and the
report carries it, the series index, IndicatorDepthReport()'s per-handle depths,
and for each indicator whether the NEWEST bar reads. That last field is the whole
diagnosis in one word: newest-also-EMPTY means the buffer is unreadable everywhere
(cold or dead handle), newest-reads means a genuine history edge. Instrumented:
open, ATR, MA, RSI, MACD, Ichimoku, and all five AD blocks via ADIndicatorCold().
3. THE TOTAL-FAILURE BACKOFF WAS GATED ON THE WRONG QUESTION.
It armed only when m_featureFailTransient was set. Keeping that flag correct across
every guard is a list that has to stay right forever - the same shape of fix the
feature cache abandoned for the same reason - and the gate is pointless anyway: a
sweep where ZERO of 50,163 bars produced a window will produce zero again if it
restarts a millisecond later, transient or not. Doing that at full speed is what
starved six indicator threads on a six-core box. The backoff is now unconditional
on a total failure. The flag keeps its real job, deciding whether a MISS may be
cached, which is a per-bar question and not a scheduling one.
4. THE ERA BARRIER DEADLOCKED, AND SILENCED THE MEMBERS IT FROZE.
EnsembleMinTrainingEra() exempted deployed, stopped and paused members and its
comment concluded "so nothing deadlocks". Those three are all VOLUNTARY. A member
that simply CANNOT finish an era is none of them, so it pinned the minimum at its
own era with no time limit - and the hold branch's only action was
`m_lastEraCompleteTick = GetTickCount()`, which silences the stall watchdog. So on
USDJPY the two members that could not train reported, and the two healthy members
frozen behind them wrote nothing anywhere. The outage was visible only through the
members that were not suffering it.
- BarrierEraHeartbeat() stamps a clock on real era CHANGE, kept separate from
m_lastEraCompleteTick precisely because the barrier resets that one. Only a
member AT the minimum can be a blocker; a member ahead is idle by design and is
never counted as stuck.
- After ENSEMBLE_BARRIER_STUCK_MS (12 min) a non-advancing member is dropped from
the barrier minimum. It keeps training and rejoins the instant it completes an
era - at which point, being behind, it legitimately becomes the minimum again,
which is the documented resumed-laggard behaviour.
- Both transitions say so loudly, and the release states plainly that the
combined-vote score cannot be computed while the ensemble is desynchronised.
- A held member now writes a rate-limited journal line naming WHICH members it is
waiting on, so the blocker is read off one line.
5. THE PANEL FLICKER.
OnTickHandler gates its terse writer on !m_trainRunActive, and a barrier-held member
returns from Train() before ever setting it - so both writers thought they were the
only one updating the label and fought every tick. That is the reported "Getting
ready..." <-> "Waiting at era N for slower ensemble members" oscillation, and it hit
Perceptron but not Convolutional purely because Convolutional had a run active from
a completed era and Perceptron, resumed from disk, never did. Train()'s message is
the specific one, so it wins.
NEXT STEP once this is running: the stall line now ends in "REJECTED BY: ..." and
the per-handle depths. Read it. Do not reason around it.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-17 15:21:15 -04:00
|
|
|
}
|
fix: make the indicator tuner actually measure, and gate what it installs
ROOT CAUSE of the zero spread measured on SP500 H1 2026-08-07 (all 17 candidates
returned exactly 0.00359 nats): the tune loop re-inits the indicators and then
scores, with no RefreshData() between.
ReInitADIndicators() does its part - Create() builds a NEW handle carrying the
new parameters, and the feature cache is flagged stale so features really are
recomputed. But BufferTempDataCompute() reads the CIndicatorBuffer objects, and
only Refresh() copies data out of a handle into those. So every candidate was
scored on values still held from the PREVIOUS handle. My earlier guess in the
diagnostic ("suspect the feature cache") was wrong: the cache invalidation works.
Two things land together, because neither is safe alone:
1. RefreshData() after the re-init, so a candidate is scored on its own features.
2. A SELECTION GATE on the install. bestScore is a MAXIMUM over candidates, and
the maximum of N draws from a null beats its incumbent almost every time - so
"it beat the incumbent" installs noise. This selector is the highest-stakes of
the three found in this audit because it ACTS: it overwrites the user's
configured indicator settings and forces BuildFreshTopology(), so the network
then trains on whatever the noise picked. Fixing (1) without (2) would have
made a dormant bug actively harmful.
The gate draws the winner's own permutation null once, then corrects the p-value
for having chosen it out of N with Sidak: p_family = 1 - (1-p)^N. Sidak rather
than the max-of-N resample used by the geometry scan because each candidate here
has a DIFFERENT feature set, so their draws cannot be pooled; Sidak needs only
the one null. Exact under independence, mildly anti-conservative under positive
dependence - stated in the comment rather than hidden. A rejected winner restores
the configured settings, which best[] cannot do since the descent mutates it.
Also reports the least-ready tunable handle's BarsCalculated(). IndicatorCreate()
calculates asynchronously, so if the spread is STILL zero the handles simply are
not done and the tuner needs to yield between candidates rather than score them
back to back - a state machine like the label prebuild. That distinction is now
readable from the log instead of requiring another guess.
No input, topology or label change: no retrain.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-07 09:31:06 -04:00
|
|
|
if(m_useADWyckoffEventStream)
|
fix(ensemble+depth): the barrier had no liveness escape, and the depth gate could not report the one state the evidence pointed at
Two charts (USDJPY 50,179 bars / XAUUSD 33,982) sat at era 0 for 38 minutes with
four of their eight members completely silent. Nothing in this commit guesses at
why the sweep fails - the last five guesses were all wrong. It makes the failure
say what it is, and stops one broken member taking its whole chart down with it.
WHAT THE LOG ACTUALLY SAYS, before any of this.
- The running build IS d9f834d (pulled 14:18, compiled 14:19:01, 0 errors), so
every depth instrument from 1dda479/7e63a8b/45c9e21 was live.
- It printed NOTHING. Zero "PRIMING", zero "CAPPED", zero "Per-indicator depth"
in 27 MB of journal. The instrument built to find the depth shortfall returned
"not this".
- On USDJPY at 14:24, CONV-cad8 completed eras 0 AND 1 across all 50,179 bars -
same chart, same 832-value window, same indicators, byte-identical fingerprint -
while LSTM-cad8 and HYB-cad8 reported ok=0 failed=50163. So it is not the
symbol, the history, the bar count or the indicator depth. It is per-member.
- ok=0 means the NEWEST anchors failed too, and a short indicator cannot do that.
The depth reading in project_silent_block_failures is therefore retired by its own
instrumentation. THE ROOT CAUSE IS STILL UNKNOWN and this commit does not claim one.
1. THE DEPTH GATE'S SILENT PATH WAS THE STATE IT WAS HUNTING.
ServableBars() read `if(servable <= 0 || servable >= want) return want;` - one
branch over three unrelated states, silent in all of them:
enabled == 0 -> nothing tunable is on. No cap. Healthy.
enabled > 0, servable == -1 -> a handle answered INVALID.
enabled > 0, servable == 0 -> created, never calculated.
BarsCalculated() returns -1 for a dead handle, so a dead MA is indistinguishable
from "no tunable indicators enabled" - and both returned `want` without printing a
character. That is exactly the state a per-member, every-index, depth-independent
failure produces, and it is the single reason a build carrying full depth
instrumentation logged nothing through the whole outage.
TunableBarsCalculated() now also reports HOW MANY indicators it consulted, and the
dead-handle case is reported (latched, with per-handle depths). The RETURN is
deliberately unchanged - what to do about a dead handle is not yet known, and
changing control flow on an unproven cause is how the last four fixes here went
wrong. SettledBars() routes its three pass-through states via ServableBars() so the
report is reachable from the training sweep, which is the only caller that hits it.
2. THE STALL REPORT NAMED A SLOT, NEVER A BLOCK.
"lookback slot 0 REJECTED (window had 24 of 832 values)" plus a guess ("an
indicator warm-up or a history-edge read"). Which guard fired was INFERRED by
counting 4+5+4+4+6+1 = 24 and concluding feature 25 must be the MA. The arithmetic
was right; every conclusion drawn from it was wrong, because a value count names a
POSITION and a position cannot tell cold from capped from invalid from off-the-end.
Every guard that can reject a bar now records itself - m_featureFailBlock - and the
report carries it, the series index, IndicatorDepthReport()'s per-handle depths,
and for each indicator whether the NEWEST bar reads. That last field is the whole
diagnosis in one word: newest-also-EMPTY means the buffer is unreadable everywhere
(cold or dead handle), newest-reads means a genuine history edge. Instrumented:
open, ATR, MA, RSI, MACD, Ichimoku, and all five AD blocks via ADIndicatorCold().
3. THE TOTAL-FAILURE BACKOFF WAS GATED ON THE WRONG QUESTION.
It armed only when m_featureFailTransient was set. Keeping that flag correct across
every guard is a list that has to stay right forever - the same shape of fix the
feature cache abandoned for the same reason - and the gate is pointless anyway: a
sweep where ZERO of 50,163 bars produced a window will produce zero again if it
restarts a millisecond later, transient or not. Doing that at full speed is what
starved six indicator threads on a six-core box. The backoff is now unconditional
on a total failure. The flag keeps its real job, deciding whether a MISS may be
cached, which is a per-bar question and not a scheduling one.
4. THE ERA BARRIER DEADLOCKED, AND SILENCED THE MEMBERS IT FROZE.
EnsembleMinTrainingEra() exempted deployed, stopped and paused members and its
comment concluded "so nothing deadlocks". Those three are all VOLUNTARY. A member
that simply CANNOT finish an era is none of them, so it pinned the minimum at its
own era with no time limit - and the hold branch's only action was
`m_lastEraCompleteTick = GetTickCount()`, which silences the stall watchdog. So on
USDJPY the two members that could not train reported, and the two healthy members
frozen behind them wrote nothing anywhere. The outage was visible only through the
members that were not suffering it.
- BarrierEraHeartbeat() stamps a clock on real era CHANGE, kept separate from
m_lastEraCompleteTick precisely because the barrier resets that one. Only a
member AT the minimum can be a blocker; a member ahead is idle by design and is
never counted as stuck.
- After ENSEMBLE_BARRIER_STUCK_MS (12 min) a non-advancing member is dropped from
the barrier minimum. It keeps training and rejoins the instant it completes an
era - at which point, being behind, it legitimately becomes the minimum again,
which is the documented resumed-laggard behaviour.
- Both transitions say so loudly, and the release states plainly that the
combined-vote score cannot be computed while the ensemble is desynchronised.
- A held member now writes a rate-limited journal line naming WHICH members it is
waiting on, so the blocker is read off one line.
5. THE PANEL FLICKER.
OnTickHandler gates its terse writer on !m_trainRunActive, and a barrier-held member
returns from Train() before ever setting it - so both writers thought they were the
only one updating the label and fought every tick. That is the reported "Getting
ready..." <-> "Waiting at era N for slower ensemble members" oscillation, and it hit
Perceptron but not Convolutional purely because Convolutional had a run active from
a completed era and Perceptron, resumed from disk, never did. Train()'s message is
the specific one, so it wins.
NEXT STEP once this is running: the stall line now ends in "REJECTED BY: ..." and
the per-handle depths. Read it. Do not reason around it.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-17 15:21:15 -04:00
|
|
|
{
|
|
|
|
|
enabled++;
|
fix: make the indicator tuner actually measure, and gate what it installs
ROOT CAUSE of the zero spread measured on SP500 H1 2026-08-07 (all 17 candidates
returned exactly 0.00359 nats): the tune loop re-inits the indicators and then
scores, with no RefreshData() between.
ReInitADIndicators() does its part - Create() builds a NEW handle carrying the
new parameters, and the feature cache is flagged stale so features really are
recomputed. But BufferTempDataCompute() reads the CIndicatorBuffer objects, and
only Refresh() copies data out of a handle into those. So every candidate was
scored on values still held from the PREVIOUS handle. My earlier guess in the
diagnostic ("suspect the feature cache") was wrong: the cache invalidation works.
Two things land together, because neither is safe alone:
1. RefreshData() after the re-init, so a candidate is scored on its own features.
2. A SELECTION GATE on the install. bestScore is a MAXIMUM over candidates, and
the maximum of N draws from a null beats its incumbent almost every time - so
"it beat the incumbent" installs noise. This selector is the highest-stakes of
the three found in this audit because it ACTS: it overwrites the user's
configured indicator settings and forces BuildFreshTopology(), so the network
then trains on whatever the noise picked. Fixing (1) without (2) would have
made a dormant bug actively harmful.
The gate draws the winner's own permutation null once, then corrects the p-value
for having chosen it out of N with Sidak: p_family = 1 - (1-p)^N. Sidak rather
than the max-of-N resample used by the geometry scan because each candidate here
has a DIFFERENT feature set, so their draws cannot be pooled; Sidak needs only
the one null. Exact under independence, mildly anti-conservative under positive
dependence - stated in the comment rather than hidden. A rejected winner restores
the configured settings, which best[] cannot do since the descent mutates it.
Also reports the least-ready tunable handle's BarsCalculated(). IndicatorCreate()
calculates asynchronously, so if the spread is STILL zero the handles simply are
not done and the tuner needs to yield between candidates rather than score them
back to back - a state machine like the label prebuild. That distinction is now
readable from the log instead of requiring another guess.
No input, topology or label change: no retrain.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-07 09:31:06 -04:00
|
|
|
worst = (int)MathMin(worst, m_ADWyckoffEventStream.BarsCalculated());
|
fix(ensemble+depth): the barrier had no liveness escape, and the depth gate could not report the one state the evidence pointed at
Two charts (USDJPY 50,179 bars / XAUUSD 33,982) sat at era 0 for 38 minutes with
four of their eight members completely silent. Nothing in this commit guesses at
why the sweep fails - the last five guesses were all wrong. It makes the failure
say what it is, and stops one broken member taking its whole chart down with it.
WHAT THE LOG ACTUALLY SAYS, before any of this.
- The running build IS d9f834d (pulled 14:18, compiled 14:19:01, 0 errors), so
every depth instrument from 1dda479/7e63a8b/45c9e21 was live.
- It printed NOTHING. Zero "PRIMING", zero "CAPPED", zero "Per-indicator depth"
in 27 MB of journal. The instrument built to find the depth shortfall returned
"not this".
- On USDJPY at 14:24, CONV-cad8 completed eras 0 AND 1 across all 50,179 bars -
same chart, same 832-value window, same indicators, byte-identical fingerprint -
while LSTM-cad8 and HYB-cad8 reported ok=0 failed=50163. So it is not the
symbol, the history, the bar count or the indicator depth. It is per-member.
- ok=0 means the NEWEST anchors failed too, and a short indicator cannot do that.
The depth reading in project_silent_block_failures is therefore retired by its own
instrumentation. THE ROOT CAUSE IS STILL UNKNOWN and this commit does not claim one.
1. THE DEPTH GATE'S SILENT PATH WAS THE STATE IT WAS HUNTING.
ServableBars() read `if(servable <= 0 || servable >= want) return want;` - one
branch over three unrelated states, silent in all of them:
enabled == 0 -> nothing tunable is on. No cap. Healthy.
enabled > 0, servable == -1 -> a handle answered INVALID.
enabled > 0, servable == 0 -> created, never calculated.
BarsCalculated() returns -1 for a dead handle, so a dead MA is indistinguishable
from "no tunable indicators enabled" - and both returned `want` without printing a
character. That is exactly the state a per-member, every-index, depth-independent
failure produces, and it is the single reason a build carrying full depth
instrumentation logged nothing through the whole outage.
TunableBarsCalculated() now also reports HOW MANY indicators it consulted, and the
dead-handle case is reported (latched, with per-handle depths). The RETURN is
deliberately unchanged - what to do about a dead handle is not yet known, and
changing control flow on an unproven cause is how the last four fixes here went
wrong. SettledBars() routes its three pass-through states via ServableBars() so the
report is reachable from the training sweep, which is the only caller that hits it.
2. THE STALL REPORT NAMED A SLOT, NEVER A BLOCK.
"lookback slot 0 REJECTED (window had 24 of 832 values)" plus a guess ("an
indicator warm-up or a history-edge read"). Which guard fired was INFERRED by
counting 4+5+4+4+6+1 = 24 and concluding feature 25 must be the MA. The arithmetic
was right; every conclusion drawn from it was wrong, because a value count names a
POSITION and a position cannot tell cold from capped from invalid from off-the-end.
Every guard that can reject a bar now records itself - m_featureFailBlock - and the
report carries it, the series index, IndicatorDepthReport()'s per-handle depths,
and for each indicator whether the NEWEST bar reads. That last field is the whole
diagnosis in one word: newest-also-EMPTY means the buffer is unreadable everywhere
(cold or dead handle), newest-reads means a genuine history edge. Instrumented:
open, ATR, MA, RSI, MACD, Ichimoku, and all five AD blocks via ADIndicatorCold().
3. THE TOTAL-FAILURE BACKOFF WAS GATED ON THE WRONG QUESTION.
It armed only when m_featureFailTransient was set. Keeping that flag correct across
every guard is a list that has to stay right forever - the same shape of fix the
feature cache abandoned for the same reason - and the gate is pointless anyway: a
sweep where ZERO of 50,163 bars produced a window will produce zero again if it
restarts a millisecond later, transient or not. Doing that at full speed is what
starved six indicator threads on a six-core box. The backoff is now unconditional
on a total failure. The flag keeps its real job, deciding whether a MISS may be
cached, which is a per-bar question and not a scheduling one.
4. THE ERA BARRIER DEADLOCKED, AND SILENCED THE MEMBERS IT FROZE.
EnsembleMinTrainingEra() exempted deployed, stopped and paused members and its
comment concluded "so nothing deadlocks". Those three are all VOLUNTARY. A member
that simply CANNOT finish an era is none of them, so it pinned the minimum at its
own era with no time limit - and the hold branch's only action was
`m_lastEraCompleteTick = GetTickCount()`, which silences the stall watchdog. So on
USDJPY the two members that could not train reported, and the two healthy members
frozen behind them wrote nothing anywhere. The outage was visible only through the
members that were not suffering it.
- BarrierEraHeartbeat() stamps a clock on real era CHANGE, kept separate from
m_lastEraCompleteTick precisely because the barrier resets that one. Only a
member AT the minimum can be a blocker; a member ahead is idle by design and is
never counted as stuck.
- After ENSEMBLE_BARRIER_STUCK_MS (12 min) a non-advancing member is dropped from
the barrier minimum. It keeps training and rejoins the instant it completes an
era - at which point, being behind, it legitimately becomes the minimum again,
which is the documented resumed-laggard behaviour.
- Both transitions say so loudly, and the release states plainly that the
combined-vote score cannot be computed while the ensemble is desynchronised.
- A held member now writes a rate-limited journal line naming WHICH members it is
waiting on, so the blocker is read off one line.
5. THE PANEL FLICKER.
OnTickHandler gates its terse writer on !m_trainRunActive, and a barrier-held member
returns from Train() before ever setting it - so both writers thought they were the
only one updating the label and fought every tick. That is the reported "Getting
ready..." <-> "Waiting at era N for slower ensemble members" oscillation, and it hit
Perceptron but not Convolutional purely because Convolutional had a run active from
a completed era and Perceptron, resumed from disk, never did. Train()'s message is
the specific one, so it wins.
NEXT STEP once this is running: the stall line now ends in "REJECTED BY: ..." and
the per-handle depths. Read it. Do not reason around it.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-17 15:21:15 -04:00
|
|
|
}
|
fix: make the indicator tuner actually measure, and gate what it installs
ROOT CAUSE of the zero spread measured on SP500 H1 2026-08-07 (all 17 candidates
returned exactly 0.00359 nats): the tune loop re-inits the indicators and then
scores, with no RefreshData() between.
ReInitADIndicators() does its part - Create() builds a NEW handle carrying the
new parameters, and the feature cache is flagged stale so features really are
recomputed. But BufferTempDataCompute() reads the CIndicatorBuffer objects, and
only Refresh() copies data out of a handle into those. So every candidate was
scored on values still held from the PREVIOUS handle. My earlier guess in the
diagnostic ("suspect the feature cache") was wrong: the cache invalidation works.
Two things land together, because neither is safe alone:
1. RefreshData() after the re-init, so a candidate is scored on its own features.
2. A SELECTION GATE on the install. bestScore is a MAXIMUM over candidates, and
the maximum of N draws from a null beats its incumbent almost every time - so
"it beat the incumbent" installs noise. This selector is the highest-stakes of
the three found in this audit because it ACTS: it overwrites the user's
configured indicator settings and forces BuildFreshTopology(), so the network
then trains on whatever the noise picked. Fixing (1) without (2) would have
made a dormant bug actively harmful.
The gate draws the winner's own permutation null once, then corrects the p-value
for having chosen it out of N with Sidak: p_family = 1 - (1-p)^N. Sidak rather
than the max-of-N resample used by the geometry scan because each candidate here
has a DIFFERENT feature set, so their draws cannot be pooled; Sidak needs only
the one null. Exact under independence, mildly anti-conservative under positive
dependence - stated in the comment rather than hidden. A rejected winner restores
the configured settings, which best[] cannot do since the descent mutates it.
Also reports the least-ready tunable handle's BarsCalculated(). IndicatorCreate()
calculates asynchronously, so if the spread is STILL zero the handles simply are
not done and the tuner needs to yield between candidates rather than score them
back to back - a state machine like the label prebuild. That distinction is now
readable from the log instead of requiring another guess.
No input, topology or label change: no retrain.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-07 09:31:06 -04:00
|
|
|
if(m_useADWyckoffFailedStructure)
|
fix(ensemble+depth): the barrier had no liveness escape, and the depth gate could not report the one state the evidence pointed at
Two charts (USDJPY 50,179 bars / XAUUSD 33,982) sat at era 0 for 38 minutes with
four of their eight members completely silent. Nothing in this commit guesses at
why the sweep fails - the last five guesses were all wrong. It makes the failure
say what it is, and stops one broken member taking its whole chart down with it.
WHAT THE LOG ACTUALLY SAYS, before any of this.
- The running build IS d9f834d (pulled 14:18, compiled 14:19:01, 0 errors), so
every depth instrument from 1dda479/7e63a8b/45c9e21 was live.
- It printed NOTHING. Zero "PRIMING", zero "CAPPED", zero "Per-indicator depth"
in 27 MB of journal. The instrument built to find the depth shortfall returned
"not this".
- On USDJPY at 14:24, CONV-cad8 completed eras 0 AND 1 across all 50,179 bars -
same chart, same 832-value window, same indicators, byte-identical fingerprint -
while LSTM-cad8 and HYB-cad8 reported ok=0 failed=50163. So it is not the
symbol, the history, the bar count or the indicator depth. It is per-member.
- ok=0 means the NEWEST anchors failed too, and a short indicator cannot do that.
The depth reading in project_silent_block_failures is therefore retired by its own
instrumentation. THE ROOT CAUSE IS STILL UNKNOWN and this commit does not claim one.
1. THE DEPTH GATE'S SILENT PATH WAS THE STATE IT WAS HUNTING.
ServableBars() read `if(servable <= 0 || servable >= want) return want;` - one
branch over three unrelated states, silent in all of them:
enabled == 0 -> nothing tunable is on. No cap. Healthy.
enabled > 0, servable == -1 -> a handle answered INVALID.
enabled > 0, servable == 0 -> created, never calculated.
BarsCalculated() returns -1 for a dead handle, so a dead MA is indistinguishable
from "no tunable indicators enabled" - and both returned `want` without printing a
character. That is exactly the state a per-member, every-index, depth-independent
failure produces, and it is the single reason a build carrying full depth
instrumentation logged nothing through the whole outage.
TunableBarsCalculated() now also reports HOW MANY indicators it consulted, and the
dead-handle case is reported (latched, with per-handle depths). The RETURN is
deliberately unchanged - what to do about a dead handle is not yet known, and
changing control flow on an unproven cause is how the last four fixes here went
wrong. SettledBars() routes its three pass-through states via ServableBars() so the
report is reachable from the training sweep, which is the only caller that hits it.
2. THE STALL REPORT NAMED A SLOT, NEVER A BLOCK.
"lookback slot 0 REJECTED (window had 24 of 832 values)" plus a guess ("an
indicator warm-up or a history-edge read"). Which guard fired was INFERRED by
counting 4+5+4+4+6+1 = 24 and concluding feature 25 must be the MA. The arithmetic
was right; every conclusion drawn from it was wrong, because a value count names a
POSITION and a position cannot tell cold from capped from invalid from off-the-end.
Every guard that can reject a bar now records itself - m_featureFailBlock - and the
report carries it, the series index, IndicatorDepthReport()'s per-handle depths,
and for each indicator whether the NEWEST bar reads. That last field is the whole
diagnosis in one word: newest-also-EMPTY means the buffer is unreadable everywhere
(cold or dead handle), newest-reads means a genuine history edge. Instrumented:
open, ATR, MA, RSI, MACD, Ichimoku, and all five AD blocks via ADIndicatorCold().
3. THE TOTAL-FAILURE BACKOFF WAS GATED ON THE WRONG QUESTION.
It armed only when m_featureFailTransient was set. Keeping that flag correct across
every guard is a list that has to stay right forever - the same shape of fix the
feature cache abandoned for the same reason - and the gate is pointless anyway: a
sweep where ZERO of 50,163 bars produced a window will produce zero again if it
restarts a millisecond later, transient or not. Doing that at full speed is what
starved six indicator threads on a six-core box. The backoff is now unconditional
on a total failure. The flag keeps its real job, deciding whether a MISS may be
cached, which is a per-bar question and not a scheduling one.
4. THE ERA BARRIER DEADLOCKED, AND SILENCED THE MEMBERS IT FROZE.
EnsembleMinTrainingEra() exempted deployed, stopped and paused members and its
comment concluded "so nothing deadlocks". Those three are all VOLUNTARY. A member
that simply CANNOT finish an era is none of them, so it pinned the minimum at its
own era with no time limit - and the hold branch's only action was
`m_lastEraCompleteTick = GetTickCount()`, which silences the stall watchdog. So on
USDJPY the two members that could not train reported, and the two healthy members
frozen behind them wrote nothing anywhere. The outage was visible only through the
members that were not suffering it.
- BarrierEraHeartbeat() stamps a clock on real era CHANGE, kept separate from
m_lastEraCompleteTick precisely because the barrier resets that one. Only a
member AT the minimum can be a blocker; a member ahead is idle by design and is
never counted as stuck.
- After ENSEMBLE_BARRIER_STUCK_MS (12 min) a non-advancing member is dropped from
the barrier minimum. It keeps training and rejoins the instant it completes an
era - at which point, being behind, it legitimately becomes the minimum again,
which is the documented resumed-laggard behaviour.
- Both transitions say so loudly, and the release states plainly that the
combined-vote score cannot be computed while the ensemble is desynchronised.
- A held member now writes a rate-limited journal line naming WHICH members it is
waiting on, so the blocker is read off one line.
5. THE PANEL FLICKER.
OnTickHandler gates its terse writer on !m_trainRunActive, and a barrier-held member
returns from Train() before ever setting it - so both writers thought they were the
only one updating the label and fought every tick. That is the reported "Getting
ready..." <-> "Waiting at era N for slower ensemble members" oscillation, and it hit
Perceptron but not Convolutional purely because Convolutional had a run active from
a completed era and Perceptron, resumed from disk, never did. Train()'s message is
the specific one, so it wins.
NEXT STEP once this is running: the stall line now ends in "REJECTED BY: ..." and
the per-handle depths. Read it. Do not reason around it.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-17 15:21:15 -04:00
|
|
|
{
|
|
|
|
|
enabled++;
|
fix: make the indicator tuner actually measure, and gate what it installs
ROOT CAUSE of the zero spread measured on SP500 H1 2026-08-07 (all 17 candidates
returned exactly 0.00359 nats): the tune loop re-inits the indicators and then
scores, with no RefreshData() between.
ReInitADIndicators() does its part - Create() builds a NEW handle carrying the
new parameters, and the feature cache is flagged stale so features really are
recomputed. But BufferTempDataCompute() reads the CIndicatorBuffer objects, and
only Refresh() copies data out of a handle into those. So every candidate was
scored on values still held from the PREVIOUS handle. My earlier guess in the
diagnostic ("suspect the feature cache") was wrong: the cache invalidation works.
Two things land together, because neither is safe alone:
1. RefreshData() after the re-init, so a candidate is scored on its own features.
2. A SELECTION GATE on the install. bestScore is a MAXIMUM over candidates, and
the maximum of N draws from a null beats its incumbent almost every time - so
"it beat the incumbent" installs noise. This selector is the highest-stakes of
the three found in this audit because it ACTS: it overwrites the user's
configured indicator settings and forces BuildFreshTopology(), so the network
then trains on whatever the noise picked. Fixing (1) without (2) would have
made a dormant bug actively harmful.
The gate draws the winner's own permutation null once, then corrects the p-value
for having chosen it out of N with Sidak: p_family = 1 - (1-p)^N. Sidak rather
than the max-of-N resample used by the geometry scan because each candidate here
has a DIFFERENT feature set, so their draws cannot be pooled; Sidak needs only
the one null. Exact under independence, mildly anti-conservative under positive
dependence - stated in the comment rather than hidden. A rejected winner restores
the configured settings, which best[] cannot do since the descent mutates it.
Also reports the least-ready tunable handle's BarsCalculated(). IndicatorCreate()
calculates asynchronously, so if the spread is STILL zero the handles simply are
not done and the tuner needs to yield between candidates rather than score them
back to back - a state machine like the label prebuild. That distinction is now
readable from the log instead of requiring another guess.
No input, topology or label change: no retrain.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-07 09:31:06 -04:00
|
|
|
worst = (int)MathMin(worst, m_ADWyckoffFailedStructure.BarsCalculated());
|
fix(ensemble+depth): the barrier had no liveness escape, and the depth gate could not report the one state the evidence pointed at
Two charts (USDJPY 50,179 bars / XAUUSD 33,982) sat at era 0 for 38 minutes with
four of their eight members completely silent. Nothing in this commit guesses at
why the sweep fails - the last five guesses were all wrong. It makes the failure
say what it is, and stops one broken member taking its whole chart down with it.
WHAT THE LOG ACTUALLY SAYS, before any of this.
- The running build IS d9f834d (pulled 14:18, compiled 14:19:01, 0 errors), so
every depth instrument from 1dda479/7e63a8b/45c9e21 was live.
- It printed NOTHING. Zero "PRIMING", zero "CAPPED", zero "Per-indicator depth"
in 27 MB of journal. The instrument built to find the depth shortfall returned
"not this".
- On USDJPY at 14:24, CONV-cad8 completed eras 0 AND 1 across all 50,179 bars -
same chart, same 832-value window, same indicators, byte-identical fingerprint -
while LSTM-cad8 and HYB-cad8 reported ok=0 failed=50163. So it is not the
symbol, the history, the bar count or the indicator depth. It is per-member.
- ok=0 means the NEWEST anchors failed too, and a short indicator cannot do that.
The depth reading in project_silent_block_failures is therefore retired by its own
instrumentation. THE ROOT CAUSE IS STILL UNKNOWN and this commit does not claim one.
1. THE DEPTH GATE'S SILENT PATH WAS THE STATE IT WAS HUNTING.
ServableBars() read `if(servable <= 0 || servable >= want) return want;` - one
branch over three unrelated states, silent in all of them:
enabled == 0 -> nothing tunable is on. No cap. Healthy.
enabled > 0, servable == -1 -> a handle answered INVALID.
enabled > 0, servable == 0 -> created, never calculated.
BarsCalculated() returns -1 for a dead handle, so a dead MA is indistinguishable
from "no tunable indicators enabled" - and both returned `want` without printing a
character. That is exactly the state a per-member, every-index, depth-independent
failure produces, and it is the single reason a build carrying full depth
instrumentation logged nothing through the whole outage.
TunableBarsCalculated() now also reports HOW MANY indicators it consulted, and the
dead-handle case is reported (latched, with per-handle depths). The RETURN is
deliberately unchanged - what to do about a dead handle is not yet known, and
changing control flow on an unproven cause is how the last four fixes here went
wrong. SettledBars() routes its three pass-through states via ServableBars() so the
report is reachable from the training sweep, which is the only caller that hits it.
2. THE STALL REPORT NAMED A SLOT, NEVER A BLOCK.
"lookback slot 0 REJECTED (window had 24 of 832 values)" plus a guess ("an
indicator warm-up or a history-edge read"). Which guard fired was INFERRED by
counting 4+5+4+4+6+1 = 24 and concluding feature 25 must be the MA. The arithmetic
was right; every conclusion drawn from it was wrong, because a value count names a
POSITION and a position cannot tell cold from capped from invalid from off-the-end.
Every guard that can reject a bar now records itself - m_featureFailBlock - and the
report carries it, the series index, IndicatorDepthReport()'s per-handle depths,
and for each indicator whether the NEWEST bar reads. That last field is the whole
diagnosis in one word: newest-also-EMPTY means the buffer is unreadable everywhere
(cold or dead handle), newest-reads means a genuine history edge. Instrumented:
open, ATR, MA, RSI, MACD, Ichimoku, and all five AD blocks via ADIndicatorCold().
3. THE TOTAL-FAILURE BACKOFF WAS GATED ON THE WRONG QUESTION.
It armed only when m_featureFailTransient was set. Keeping that flag correct across
every guard is a list that has to stay right forever - the same shape of fix the
feature cache abandoned for the same reason - and the gate is pointless anyway: a
sweep where ZERO of 50,163 bars produced a window will produce zero again if it
restarts a millisecond later, transient or not. Doing that at full speed is what
starved six indicator threads on a six-core box. The backoff is now unconditional
on a total failure. The flag keeps its real job, deciding whether a MISS may be
cached, which is a per-bar question and not a scheduling one.
4. THE ERA BARRIER DEADLOCKED, AND SILENCED THE MEMBERS IT FROZE.
EnsembleMinTrainingEra() exempted deployed, stopped and paused members and its
comment concluded "so nothing deadlocks". Those three are all VOLUNTARY. A member
that simply CANNOT finish an era is none of them, so it pinned the minimum at its
own era with no time limit - and the hold branch's only action was
`m_lastEraCompleteTick = GetTickCount()`, which silences the stall watchdog. So on
USDJPY the two members that could not train reported, and the two healthy members
frozen behind them wrote nothing anywhere. The outage was visible only through the
members that were not suffering it.
- BarrierEraHeartbeat() stamps a clock on real era CHANGE, kept separate from
m_lastEraCompleteTick precisely because the barrier resets that one. Only a
member AT the minimum can be a blocker; a member ahead is idle by design and is
never counted as stuck.
- After ENSEMBLE_BARRIER_STUCK_MS (12 min) a non-advancing member is dropped from
the barrier minimum. It keeps training and rejoins the instant it completes an
era - at which point, being behind, it legitimately becomes the minimum again,
which is the documented resumed-laggard behaviour.
- Both transitions say so loudly, and the release states plainly that the
combined-vote score cannot be computed while the ensemble is desynchronised.
- A held member now writes a rate-limited journal line naming WHICH members it is
waiting on, so the blocker is read off one line.
5. THE PANEL FLICKER.
OnTickHandler gates its terse writer on !m_trainRunActive, and a barrier-held member
returns from Train() before ever setting it - so both writers thought they were the
only one updating the label and fought every tick. That is the reported "Getting
ready..." <-> "Waiting at era N for slower ensemble members" oscillation, and it hit
Perceptron but not Convolutional purely because Convolutional had a run active from
a completed era and Perceptron, resumed from disk, never did. Train()'s message is
the specific one, so it wins.
NEXT STEP once this is running: the stall line now ends in "REJECTED BY: ..." and
the per-handle depths. Read it. Do not reason around it.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-17 15:21:15 -04:00
|
|
|
}
|
fix: make the indicator tuner actually measure, and gate what it installs
ROOT CAUSE of the zero spread measured on SP500 H1 2026-08-07 (all 17 candidates
returned exactly 0.00359 nats): the tune loop re-inits the indicators and then
scores, with no RefreshData() between.
ReInitADIndicators() does its part - Create() builds a NEW handle carrying the
new parameters, and the feature cache is flagged stale so features really are
recomputed. But BufferTempDataCompute() reads the CIndicatorBuffer objects, and
only Refresh() copies data out of a handle into those. So every candidate was
scored on values still held from the PREVIOUS handle. My earlier guess in the
diagnostic ("suspect the feature cache") was wrong: the cache invalidation works.
Two things land together, because neither is safe alone:
1. RefreshData() after the re-init, so a candidate is scored on its own features.
2. A SELECTION GATE on the install. bestScore is a MAXIMUM over candidates, and
the maximum of N draws from a null beats its incumbent almost every time - so
"it beat the incumbent" installs noise. This selector is the highest-stakes of
the three found in this audit because it ACTS: it overwrites the user's
configured indicator settings and forces BuildFreshTopology(), so the network
then trains on whatever the noise picked. Fixing (1) without (2) would have
made a dormant bug actively harmful.
The gate draws the winner's own permutation null once, then corrects the p-value
for having chosen it out of N with Sidak: p_family = 1 - (1-p)^N. Sidak rather
than the max-of-N resample used by the geometry scan because each candidate here
has a DIFFERENT feature set, so their draws cannot be pooled; Sidak needs only
the one null. Exact under independence, mildly anti-conservative under positive
dependence - stated in the comment rather than hidden. A rejected winner restores
the configured settings, which best[] cannot do since the descent mutates it.
Also reports the least-ready tunable handle's BarsCalculated(). IndicatorCreate()
calculates asynchronously, so if the spread is STILL zero the handles simply are
not done and the tuner needs to yield between candidates rather than score them
back to back - a state machine like the label prebuild. That distinction is now
readable from the log instead of requiring another guess.
No input, topology or label change: no retrain.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-07 09:31:06 -04:00
|
|
|
if(m_useADWyckoffSignificantBarInversion)
|
fix(ensemble+depth): the barrier had no liveness escape, and the depth gate could not report the one state the evidence pointed at
Two charts (USDJPY 50,179 bars / XAUUSD 33,982) sat at era 0 for 38 minutes with
four of their eight members completely silent. Nothing in this commit guesses at
why the sweep fails - the last five guesses were all wrong. It makes the failure
say what it is, and stops one broken member taking its whole chart down with it.
WHAT THE LOG ACTUALLY SAYS, before any of this.
- The running build IS d9f834d (pulled 14:18, compiled 14:19:01, 0 errors), so
every depth instrument from 1dda479/7e63a8b/45c9e21 was live.
- It printed NOTHING. Zero "PRIMING", zero "CAPPED", zero "Per-indicator depth"
in 27 MB of journal. The instrument built to find the depth shortfall returned
"not this".
- On USDJPY at 14:24, CONV-cad8 completed eras 0 AND 1 across all 50,179 bars -
same chart, same 832-value window, same indicators, byte-identical fingerprint -
while LSTM-cad8 and HYB-cad8 reported ok=0 failed=50163. So it is not the
symbol, the history, the bar count or the indicator depth. It is per-member.
- ok=0 means the NEWEST anchors failed too, and a short indicator cannot do that.
The depth reading in project_silent_block_failures is therefore retired by its own
instrumentation. THE ROOT CAUSE IS STILL UNKNOWN and this commit does not claim one.
1. THE DEPTH GATE'S SILENT PATH WAS THE STATE IT WAS HUNTING.
ServableBars() read `if(servable <= 0 || servable >= want) return want;` - one
branch over three unrelated states, silent in all of them:
enabled == 0 -> nothing tunable is on. No cap. Healthy.
enabled > 0, servable == -1 -> a handle answered INVALID.
enabled > 0, servable == 0 -> created, never calculated.
BarsCalculated() returns -1 for a dead handle, so a dead MA is indistinguishable
from "no tunable indicators enabled" - and both returned `want` without printing a
character. That is exactly the state a per-member, every-index, depth-independent
failure produces, and it is the single reason a build carrying full depth
instrumentation logged nothing through the whole outage.
TunableBarsCalculated() now also reports HOW MANY indicators it consulted, and the
dead-handle case is reported (latched, with per-handle depths). The RETURN is
deliberately unchanged - what to do about a dead handle is not yet known, and
changing control flow on an unproven cause is how the last four fixes here went
wrong. SettledBars() routes its three pass-through states via ServableBars() so the
report is reachable from the training sweep, which is the only caller that hits it.
2. THE STALL REPORT NAMED A SLOT, NEVER A BLOCK.
"lookback slot 0 REJECTED (window had 24 of 832 values)" plus a guess ("an
indicator warm-up or a history-edge read"). Which guard fired was INFERRED by
counting 4+5+4+4+6+1 = 24 and concluding feature 25 must be the MA. The arithmetic
was right; every conclusion drawn from it was wrong, because a value count names a
POSITION and a position cannot tell cold from capped from invalid from off-the-end.
Every guard that can reject a bar now records itself - m_featureFailBlock - and the
report carries it, the series index, IndicatorDepthReport()'s per-handle depths,
and for each indicator whether the NEWEST bar reads. That last field is the whole
diagnosis in one word: newest-also-EMPTY means the buffer is unreadable everywhere
(cold or dead handle), newest-reads means a genuine history edge. Instrumented:
open, ATR, MA, RSI, MACD, Ichimoku, and all five AD blocks via ADIndicatorCold().
3. THE TOTAL-FAILURE BACKOFF WAS GATED ON THE WRONG QUESTION.
It armed only when m_featureFailTransient was set. Keeping that flag correct across
every guard is a list that has to stay right forever - the same shape of fix the
feature cache abandoned for the same reason - and the gate is pointless anyway: a
sweep where ZERO of 50,163 bars produced a window will produce zero again if it
restarts a millisecond later, transient or not. Doing that at full speed is what
starved six indicator threads on a six-core box. The backoff is now unconditional
on a total failure. The flag keeps its real job, deciding whether a MISS may be
cached, which is a per-bar question and not a scheduling one.
4. THE ERA BARRIER DEADLOCKED, AND SILENCED THE MEMBERS IT FROZE.
EnsembleMinTrainingEra() exempted deployed, stopped and paused members and its
comment concluded "so nothing deadlocks". Those three are all VOLUNTARY. A member
that simply CANNOT finish an era is none of them, so it pinned the minimum at its
own era with no time limit - and the hold branch's only action was
`m_lastEraCompleteTick = GetTickCount()`, which silences the stall watchdog. So on
USDJPY the two members that could not train reported, and the two healthy members
frozen behind them wrote nothing anywhere. The outage was visible only through the
members that were not suffering it.
- BarrierEraHeartbeat() stamps a clock on real era CHANGE, kept separate from
m_lastEraCompleteTick precisely because the barrier resets that one. Only a
member AT the minimum can be a blocker; a member ahead is idle by design and is
never counted as stuck.
- After ENSEMBLE_BARRIER_STUCK_MS (12 min) a non-advancing member is dropped from
the barrier minimum. It keeps training and rejoins the instant it completes an
era - at which point, being behind, it legitimately becomes the minimum again,
which is the documented resumed-laggard behaviour.
- Both transitions say so loudly, and the release states plainly that the
combined-vote score cannot be computed while the ensemble is desynchronised.
- A held member now writes a rate-limited journal line naming WHICH members it is
waiting on, so the blocker is read off one line.
5. THE PANEL FLICKER.
OnTickHandler gates its terse writer on !m_trainRunActive, and a barrier-held member
returns from Train() before ever setting it - so both writers thought they were the
only one updating the label and fought every tick. That is the reported "Getting
ready..." <-> "Waiting at era N for slower ensemble members" oscillation, and it hit
Perceptron but not Convolutional purely because Convolutional had a run active from
a completed era and Perceptron, resumed from disk, never did. Train()'s message is
the specific one, so it wins.
NEXT STEP once this is running: the stall line now ends in "REJECTED BY: ..." and
the per-handle depths. Read it. Do not reason around it.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-17 15:21:15 -04:00
|
|
|
{
|
|
|
|
|
enabled++;
|
fix: make the indicator tuner actually measure, and gate what it installs
ROOT CAUSE of the zero spread measured on SP500 H1 2026-08-07 (all 17 candidates
returned exactly 0.00359 nats): the tune loop re-inits the indicators and then
scores, with no RefreshData() between.
ReInitADIndicators() does its part - Create() builds a NEW handle carrying the
new parameters, and the feature cache is flagged stale so features really are
recomputed. But BufferTempDataCompute() reads the CIndicatorBuffer objects, and
only Refresh() copies data out of a handle into those. So every candidate was
scored on values still held from the PREVIOUS handle. My earlier guess in the
diagnostic ("suspect the feature cache") was wrong: the cache invalidation works.
Two things land together, because neither is safe alone:
1. RefreshData() after the re-init, so a candidate is scored on its own features.
2. A SELECTION GATE on the install. bestScore is a MAXIMUM over candidates, and
the maximum of N draws from a null beats its incumbent almost every time - so
"it beat the incumbent" installs noise. This selector is the highest-stakes of
the three found in this audit because it ACTS: it overwrites the user's
configured indicator settings and forces BuildFreshTopology(), so the network
then trains on whatever the noise picked. Fixing (1) without (2) would have
made a dormant bug actively harmful.
The gate draws the winner's own permutation null once, then corrects the p-value
for having chosen it out of N with Sidak: p_family = 1 - (1-p)^N. Sidak rather
than the max-of-N resample used by the geometry scan because each candidate here
has a DIFFERENT feature set, so their draws cannot be pooled; Sidak needs only
the one null. Exact under independence, mildly anti-conservative under positive
dependence - stated in the comment rather than hidden. A rejected winner restores
the configured settings, which best[] cannot do since the descent mutates it.
Also reports the least-ready tunable handle's BarsCalculated(). IndicatorCreate()
calculates asynchronously, so if the spread is STILL zero the handles simply are
not done and the tuner needs to yield between candidates rather than score them
back to back - a state machine like the label prebuild. That distinction is now
readable from the log instead of requiring another guess.
No input, topology or label change: no retrain.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-07 09:31:06 -04:00
|
|
|
worst = (int)MathMin(worst, m_ADWyckoffSignificantBarInversion.BarsCalculated());
|
fix(ensemble+depth): the barrier had no liveness escape, and the depth gate could not report the one state the evidence pointed at
Two charts (USDJPY 50,179 bars / XAUUSD 33,982) sat at era 0 for 38 minutes with
four of their eight members completely silent. Nothing in this commit guesses at
why the sweep fails - the last five guesses were all wrong. It makes the failure
say what it is, and stops one broken member taking its whole chart down with it.
WHAT THE LOG ACTUALLY SAYS, before any of this.
- The running build IS d9f834d (pulled 14:18, compiled 14:19:01, 0 errors), so
every depth instrument from 1dda479/7e63a8b/45c9e21 was live.
- It printed NOTHING. Zero "PRIMING", zero "CAPPED", zero "Per-indicator depth"
in 27 MB of journal. The instrument built to find the depth shortfall returned
"not this".
- On USDJPY at 14:24, CONV-cad8 completed eras 0 AND 1 across all 50,179 bars -
same chart, same 832-value window, same indicators, byte-identical fingerprint -
while LSTM-cad8 and HYB-cad8 reported ok=0 failed=50163. So it is not the
symbol, the history, the bar count or the indicator depth. It is per-member.
- ok=0 means the NEWEST anchors failed too, and a short indicator cannot do that.
The depth reading in project_silent_block_failures is therefore retired by its own
instrumentation. THE ROOT CAUSE IS STILL UNKNOWN and this commit does not claim one.
1. THE DEPTH GATE'S SILENT PATH WAS THE STATE IT WAS HUNTING.
ServableBars() read `if(servable <= 0 || servable >= want) return want;` - one
branch over three unrelated states, silent in all of them:
enabled == 0 -> nothing tunable is on. No cap. Healthy.
enabled > 0, servable == -1 -> a handle answered INVALID.
enabled > 0, servable == 0 -> created, never calculated.
BarsCalculated() returns -1 for a dead handle, so a dead MA is indistinguishable
from "no tunable indicators enabled" - and both returned `want` without printing a
character. That is exactly the state a per-member, every-index, depth-independent
failure produces, and it is the single reason a build carrying full depth
instrumentation logged nothing through the whole outage.
TunableBarsCalculated() now also reports HOW MANY indicators it consulted, and the
dead-handle case is reported (latched, with per-handle depths). The RETURN is
deliberately unchanged - what to do about a dead handle is not yet known, and
changing control flow on an unproven cause is how the last four fixes here went
wrong. SettledBars() routes its three pass-through states via ServableBars() so the
report is reachable from the training sweep, which is the only caller that hits it.
2. THE STALL REPORT NAMED A SLOT, NEVER A BLOCK.
"lookback slot 0 REJECTED (window had 24 of 832 values)" plus a guess ("an
indicator warm-up or a history-edge read"). Which guard fired was INFERRED by
counting 4+5+4+4+6+1 = 24 and concluding feature 25 must be the MA. The arithmetic
was right; every conclusion drawn from it was wrong, because a value count names a
POSITION and a position cannot tell cold from capped from invalid from off-the-end.
Every guard that can reject a bar now records itself - m_featureFailBlock - and the
report carries it, the series index, IndicatorDepthReport()'s per-handle depths,
and for each indicator whether the NEWEST bar reads. That last field is the whole
diagnosis in one word: newest-also-EMPTY means the buffer is unreadable everywhere
(cold or dead handle), newest-reads means a genuine history edge. Instrumented:
open, ATR, MA, RSI, MACD, Ichimoku, and all five AD blocks via ADIndicatorCold().
3. THE TOTAL-FAILURE BACKOFF WAS GATED ON THE WRONG QUESTION.
It armed only when m_featureFailTransient was set. Keeping that flag correct across
every guard is a list that has to stay right forever - the same shape of fix the
feature cache abandoned for the same reason - and the gate is pointless anyway: a
sweep where ZERO of 50,163 bars produced a window will produce zero again if it
restarts a millisecond later, transient or not. Doing that at full speed is what
starved six indicator threads on a six-core box. The backoff is now unconditional
on a total failure. The flag keeps its real job, deciding whether a MISS may be
cached, which is a per-bar question and not a scheduling one.
4. THE ERA BARRIER DEADLOCKED, AND SILENCED THE MEMBERS IT FROZE.
EnsembleMinTrainingEra() exempted deployed, stopped and paused members and its
comment concluded "so nothing deadlocks". Those three are all VOLUNTARY. A member
that simply CANNOT finish an era is none of them, so it pinned the minimum at its
own era with no time limit - and the hold branch's only action was
`m_lastEraCompleteTick = GetTickCount()`, which silences the stall watchdog. So on
USDJPY the two members that could not train reported, and the two healthy members
frozen behind them wrote nothing anywhere. The outage was visible only through the
members that were not suffering it.
- BarrierEraHeartbeat() stamps a clock on real era CHANGE, kept separate from
m_lastEraCompleteTick precisely because the barrier resets that one. Only a
member AT the minimum can be a blocker; a member ahead is idle by design and is
never counted as stuck.
- After ENSEMBLE_BARRIER_STUCK_MS (12 min) a non-advancing member is dropped from
the barrier minimum. It keeps training and rejoins the instant it completes an
era - at which point, being behind, it legitimately becomes the minimum again,
which is the documented resumed-laggard behaviour.
- Both transitions say so loudly, and the release states plainly that the
combined-vote score cannot be computed while the ensemble is desynchronised.
- A held member now writes a rate-limited journal line naming WHICH members it is
waiting on, so the blocker is read off one line.
5. THE PANEL FLICKER.
OnTickHandler gates its terse writer on !m_trainRunActive, and a barrier-held member
returns from Train() before ever setting it - so both writers thought they were the
only one updating the label and fought every tick. That is the reported "Getting
ready..." <-> "Waiting at era N for slower ensemble members" oscillation, and it hit
Perceptron but not Convolutional purely because Convolutional had a run active from
a completed era and Perceptron, resumed from disk, never did. Train()'s message is
the specific one, so it wins.
NEXT STEP once this is running: the stall line now ends in "REJECTED BY: ..." and
the per-handle depths. Read it. Do not reason around it.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-17 15:21:15 -04:00
|
|
|
}
|
fix: make the indicator tuner actually measure, and gate what it installs
ROOT CAUSE of the zero spread measured on SP500 H1 2026-08-07 (all 17 candidates
returned exactly 0.00359 nats): the tune loop re-inits the indicators and then
scores, with no RefreshData() between.
ReInitADIndicators() does its part - Create() builds a NEW handle carrying the
new parameters, and the feature cache is flagged stale so features really are
recomputed. But BufferTempDataCompute() reads the CIndicatorBuffer objects, and
only Refresh() copies data out of a handle into those. So every candidate was
scored on values still held from the PREVIOUS handle. My earlier guess in the
diagnostic ("suspect the feature cache") was wrong: the cache invalidation works.
Two things land together, because neither is safe alone:
1. RefreshData() after the re-init, so a candidate is scored on its own features.
2. A SELECTION GATE on the install. bestScore is a MAXIMUM over candidates, and
the maximum of N draws from a null beats its incumbent almost every time - so
"it beat the incumbent" installs noise. This selector is the highest-stakes of
the three found in this audit because it ACTS: it overwrites the user's
configured indicator settings and forces BuildFreshTopology(), so the network
then trains on whatever the noise picked. Fixing (1) without (2) would have
made a dormant bug actively harmful.
The gate draws the winner's own permutation null once, then corrects the p-value
for having chosen it out of N with Sidak: p_family = 1 - (1-p)^N. Sidak rather
than the max-of-N resample used by the geometry scan because each candidate here
has a DIFFERENT feature set, so their draws cannot be pooled; Sidak needs only
the one null. Exact under independence, mildly anti-conservative under positive
dependence - stated in the comment rather than hidden. A rejected winner restores
the configured settings, which best[] cannot do since the descent mutates it.
Also reports the least-ready tunable handle's BarsCalculated(). IndicatorCreate()
calculates asynchronously, so if the spread is STILL zero the handles simply are
not done and the tuner needs to yield between candidates rather than score them
back to back - a state machine like the label prebuild. That distinction is now
readable from the log instead of requiring another guess.
No input, topology or label change: no retrain.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-07 09:31:06 -04:00
|
|
|
return (worst == INT_MAX) ? -1 : worst;
|
|
|
|
|
}
|
|
|
|
|
//+------------------------------------------------------------------+
|
fix(ensemble+depth): the barrier had no liveness escape, and the depth gate could not report the one state the evidence pointed at
Two charts (USDJPY 50,179 bars / XAUUSD 33,982) sat at era 0 for 38 minutes with
four of their eight members completely silent. Nothing in this commit guesses at
why the sweep fails - the last five guesses were all wrong. It makes the failure
say what it is, and stops one broken member taking its whole chart down with it.
WHAT THE LOG ACTUALLY SAYS, before any of this.
- The running build IS d9f834d (pulled 14:18, compiled 14:19:01, 0 errors), so
every depth instrument from 1dda479/7e63a8b/45c9e21 was live.
- It printed NOTHING. Zero "PRIMING", zero "CAPPED", zero "Per-indicator depth"
in 27 MB of journal. The instrument built to find the depth shortfall returned
"not this".
- On USDJPY at 14:24, CONV-cad8 completed eras 0 AND 1 across all 50,179 bars -
same chart, same 832-value window, same indicators, byte-identical fingerprint -
while LSTM-cad8 and HYB-cad8 reported ok=0 failed=50163. So it is not the
symbol, the history, the bar count or the indicator depth. It is per-member.
- ok=0 means the NEWEST anchors failed too, and a short indicator cannot do that.
The depth reading in project_silent_block_failures is therefore retired by its own
instrumentation. THE ROOT CAUSE IS STILL UNKNOWN and this commit does not claim one.
1. THE DEPTH GATE'S SILENT PATH WAS THE STATE IT WAS HUNTING.
ServableBars() read `if(servable <= 0 || servable >= want) return want;` - one
branch over three unrelated states, silent in all of them:
enabled == 0 -> nothing tunable is on. No cap. Healthy.
enabled > 0, servable == -1 -> a handle answered INVALID.
enabled > 0, servable == 0 -> created, never calculated.
BarsCalculated() returns -1 for a dead handle, so a dead MA is indistinguishable
from "no tunable indicators enabled" - and both returned `want` without printing a
character. That is exactly the state a per-member, every-index, depth-independent
failure produces, and it is the single reason a build carrying full depth
instrumentation logged nothing through the whole outage.
TunableBarsCalculated() now also reports HOW MANY indicators it consulted, and the
dead-handle case is reported (latched, with per-handle depths). The RETURN is
deliberately unchanged - what to do about a dead handle is not yet known, and
changing control flow on an unproven cause is how the last four fixes here went
wrong. SettledBars() routes its three pass-through states via ServableBars() so the
report is reachable from the training sweep, which is the only caller that hits it.
2. THE STALL REPORT NAMED A SLOT, NEVER A BLOCK.
"lookback slot 0 REJECTED (window had 24 of 832 values)" plus a guess ("an
indicator warm-up or a history-edge read"). Which guard fired was INFERRED by
counting 4+5+4+4+6+1 = 24 and concluding feature 25 must be the MA. The arithmetic
was right; every conclusion drawn from it was wrong, because a value count names a
POSITION and a position cannot tell cold from capped from invalid from off-the-end.
Every guard that can reject a bar now records itself - m_featureFailBlock - and the
report carries it, the series index, IndicatorDepthReport()'s per-handle depths,
and for each indicator whether the NEWEST bar reads. That last field is the whole
diagnosis in one word: newest-also-EMPTY means the buffer is unreadable everywhere
(cold or dead handle), newest-reads means a genuine history edge. Instrumented:
open, ATR, MA, RSI, MACD, Ichimoku, and all five AD blocks via ADIndicatorCold().
3. THE TOTAL-FAILURE BACKOFF WAS GATED ON THE WRONG QUESTION.
It armed only when m_featureFailTransient was set. Keeping that flag correct across
every guard is a list that has to stay right forever - the same shape of fix the
feature cache abandoned for the same reason - and the gate is pointless anyway: a
sweep where ZERO of 50,163 bars produced a window will produce zero again if it
restarts a millisecond later, transient or not. Doing that at full speed is what
starved six indicator threads on a six-core box. The backoff is now unconditional
on a total failure. The flag keeps its real job, deciding whether a MISS may be
cached, which is a per-bar question and not a scheduling one.
4. THE ERA BARRIER DEADLOCKED, AND SILENCED THE MEMBERS IT FROZE.
EnsembleMinTrainingEra() exempted deployed, stopped and paused members and its
comment concluded "so nothing deadlocks". Those three are all VOLUNTARY. A member
that simply CANNOT finish an era is none of them, so it pinned the minimum at its
own era with no time limit - and the hold branch's only action was
`m_lastEraCompleteTick = GetTickCount()`, which silences the stall watchdog. So on
USDJPY the two members that could not train reported, and the two healthy members
frozen behind them wrote nothing anywhere. The outage was visible only through the
members that were not suffering it.
- BarrierEraHeartbeat() stamps a clock on real era CHANGE, kept separate from
m_lastEraCompleteTick precisely because the barrier resets that one. Only a
member AT the minimum can be a blocker; a member ahead is idle by design and is
never counted as stuck.
- After ENSEMBLE_BARRIER_STUCK_MS (12 min) a non-advancing member is dropped from
the barrier minimum. It keeps training and rejoins the instant it completes an
era - at which point, being behind, it legitimately becomes the minimum again,
which is the documented resumed-laggard behaviour.
- Both transitions say so loudly, and the release states plainly that the
combined-vote score cannot be computed while the ensemble is desynchronised.
- A held member now writes a rate-limited journal line naming WHICH members it is
waiting on, so the blocker is read off one line.
5. THE PANEL FLICKER.
OnTickHandler gates its terse writer on !m_trainRunActive, and a barrier-held member
returns from Train() before ever setting it - so both writers thought they were the
only one updating the label and fought every tick. That is the reported "Getting
ready..." <-> "Waiting at era N for slower ensemble members" oscillation, and it hit
Perceptron but not Convolutional purely because Convolutional had a run active from
a completed era and Perceptron, resumed from disk, never did. Train()'s message is
the specific one, so it wins.
NEXT STEP once this is running: the stall line now ends in "REJECTED BY: ..." and
the per-handle depths. Read it. Do not reason around it.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-17 15:21:15 -04:00
|
|
|
//| Back-compatible form for the callers that only want the number. |
|
|
|
|
|
//+------------------------------------------------------------------+
|
|
|
|
|
int CExpertSignalAIBase::TunableBarsCalculated(void)
|
|
|
|
|
{
|
|
|
|
|
int enabled = 0;
|
|
|
|
|
return TunableBarsCalculated(enabled);
|
|
|
|
|
}
|
|
|
|
|
//+------------------------------------------------------------------+
|
fix(depth): route EVERY ResizeBuffers call site through one indicator-depth gate
1dda479 clamped the training sweep. It left five other paths asking the indicators
for a depth they cannot serve, and on a live account the quiet ones are worse than
the stall was - a stalled chart is visible, a chart trading on a degraded feature
window is not.
ServableBars(want, context) is now the single gate, and all six go through it:
training sweep clamp, floored at TRAIN_MIN_CLAMPED_BARS (below that a small
positive BarsCalculated is warm-up, which m_coldSweepTick owns)
label prebuild clamp - labels come from price/ADZigZag and would survive a
capped MA, but ResizeBuffers sizes EVERY buffer and a failed
CopyBuffer leaves m_MA EMPTY for the next reader, so this path
could silently re-break the block Train()'s clamp just fixed
live inference HOLD. Below `need` the swing block takes its degraded path and
inference runs on a different feature distribution than the model
was fitted on. This EA sizes real positions off that output, so
no signal beats a mismatched one
online learning HOLD, same reason and worse - this path WRITES to a live trading
model, so a mismatched (features, label) pair is not a wrong arrow,
it is a wrong weight update that compounds every bar
chart rescan clamp - SIGNAL_RESCAN_LOOKBACK_BARS is 5000 and MT5's smallest
"Max bars in chart" is also 5000, so this one is genuinely
reachable; uncapped it repaints the window all-Neutral
research export clamp before the emptiness test, so a capped symbol exports the
depth it has rather than writing a CSV with a dead feature block -
an artefact that looks complete and is silently wrong
Both HOLDs are insurance, not expected states: `need` tops out near 1,152 bars
(16 + 750 + 384 + 2) against a 5,000 floor on the terminal setting. They exist so
the failure mode is unreachable rather than merely unlikely.
Not changed: a genuinely SHORT price history still takes the old degraded path at
every site. That is pre-existing behaviour and narrowing it would mute charts that
trade today, so it stays a separate decision rather than a side effect of this fix.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-17 13:33:16 -04:00
|
|
|
//| See the declaration. THE one place that decides how much history |
|
|
|
|
|
//| may be asked of the indicators; every ResizeBuffers() call site |
|
|
|
|
|
//| goes through it. |
|
|
|
|
|
//+------------------------------------------------------------------+
|
|
|
|
|
int CExpertSignalAIBase::ServableBars(int want, string context)
|
|
|
|
|
{
|
|
|
|
|
if(want <= 0)
|
|
|
|
|
return want;
|
fix(ensemble+depth): the barrier had no liveness escape, and the depth gate could not report the one state the evidence pointed at
Two charts (USDJPY 50,179 bars / XAUUSD 33,982) sat at era 0 for 38 minutes with
four of their eight members completely silent. Nothing in this commit guesses at
why the sweep fails - the last five guesses were all wrong. It makes the failure
say what it is, and stops one broken member taking its whole chart down with it.
WHAT THE LOG ACTUALLY SAYS, before any of this.
- The running build IS d9f834d (pulled 14:18, compiled 14:19:01, 0 errors), so
every depth instrument from 1dda479/7e63a8b/45c9e21 was live.
- It printed NOTHING. Zero "PRIMING", zero "CAPPED", zero "Per-indicator depth"
in 27 MB of journal. The instrument built to find the depth shortfall returned
"not this".
- On USDJPY at 14:24, CONV-cad8 completed eras 0 AND 1 across all 50,179 bars -
same chart, same 832-value window, same indicators, byte-identical fingerprint -
while LSTM-cad8 and HYB-cad8 reported ok=0 failed=50163. So it is not the
symbol, the history, the bar count or the indicator depth. It is per-member.
- ok=0 means the NEWEST anchors failed too, and a short indicator cannot do that.
The depth reading in project_silent_block_failures is therefore retired by its own
instrumentation. THE ROOT CAUSE IS STILL UNKNOWN and this commit does not claim one.
1. THE DEPTH GATE'S SILENT PATH WAS THE STATE IT WAS HUNTING.
ServableBars() read `if(servable <= 0 || servable >= want) return want;` - one
branch over three unrelated states, silent in all of them:
enabled == 0 -> nothing tunable is on. No cap. Healthy.
enabled > 0, servable == -1 -> a handle answered INVALID.
enabled > 0, servable == 0 -> created, never calculated.
BarsCalculated() returns -1 for a dead handle, so a dead MA is indistinguishable
from "no tunable indicators enabled" - and both returned `want` without printing a
character. That is exactly the state a per-member, every-index, depth-independent
failure produces, and it is the single reason a build carrying full depth
instrumentation logged nothing through the whole outage.
TunableBarsCalculated() now also reports HOW MANY indicators it consulted, and the
dead-handle case is reported (latched, with per-handle depths). The RETURN is
deliberately unchanged - what to do about a dead handle is not yet known, and
changing control flow on an unproven cause is how the last four fixes here went
wrong. SettledBars() routes its three pass-through states via ServableBars() so the
report is reachable from the training sweep, which is the only caller that hits it.
2. THE STALL REPORT NAMED A SLOT, NEVER A BLOCK.
"lookback slot 0 REJECTED (window had 24 of 832 values)" plus a guess ("an
indicator warm-up or a history-edge read"). Which guard fired was INFERRED by
counting 4+5+4+4+6+1 = 24 and concluding feature 25 must be the MA. The arithmetic
was right; every conclusion drawn from it was wrong, because a value count names a
POSITION and a position cannot tell cold from capped from invalid from off-the-end.
Every guard that can reject a bar now records itself - m_featureFailBlock - and the
report carries it, the series index, IndicatorDepthReport()'s per-handle depths,
and for each indicator whether the NEWEST bar reads. That last field is the whole
diagnosis in one word: newest-also-EMPTY means the buffer is unreadable everywhere
(cold or dead handle), newest-reads means a genuine history edge. Instrumented:
open, ATR, MA, RSI, MACD, Ichimoku, and all five AD blocks via ADIndicatorCold().
3. THE TOTAL-FAILURE BACKOFF WAS GATED ON THE WRONG QUESTION.
It armed only when m_featureFailTransient was set. Keeping that flag correct across
every guard is a list that has to stay right forever - the same shape of fix the
feature cache abandoned for the same reason - and the gate is pointless anyway: a
sweep where ZERO of 50,163 bars produced a window will produce zero again if it
restarts a millisecond later, transient or not. Doing that at full speed is what
starved six indicator threads on a six-core box. The backoff is now unconditional
on a total failure. The flag keeps its real job, deciding whether a MISS may be
cached, which is a per-bar question and not a scheduling one.
4. THE ERA BARRIER DEADLOCKED, AND SILENCED THE MEMBERS IT FROZE.
EnsembleMinTrainingEra() exempted deployed, stopped and paused members and its
comment concluded "so nothing deadlocks". Those three are all VOLUNTARY. A member
that simply CANNOT finish an era is none of them, so it pinned the minimum at its
own era with no time limit - and the hold branch's only action was
`m_lastEraCompleteTick = GetTickCount()`, which silences the stall watchdog. So on
USDJPY the two members that could not train reported, and the two healthy members
frozen behind them wrote nothing anywhere. The outage was visible only through the
members that were not suffering it.
- BarrierEraHeartbeat() stamps a clock on real era CHANGE, kept separate from
m_lastEraCompleteTick precisely because the barrier resets that one. Only a
member AT the minimum can be a blocker; a member ahead is idle by design and is
never counted as stuck.
- After ENSEMBLE_BARRIER_STUCK_MS (12 min) a non-advancing member is dropped from
the barrier minimum. It keeps training and rejoins the instant it completes an
era - at which point, being behind, it legitimately becomes the minimum again,
which is the documented resumed-laggard behaviour.
- Both transitions say so loudly, and the release states plainly that the
combined-vote score cannot be computed while the ensemble is desynchronised.
- A held member now writes a rate-limited journal line naming WHICH members it is
waiting on, so the blocker is read off one line.
5. THE PANEL FLICKER.
OnTickHandler gates its terse writer on !m_trainRunActive, and a barrier-held member
returns from Train() before ever setting it - so both writers thought they were the
only one updating the label and fought every tick. That is the reported "Getting
ready..." <-> "Waiting at era N for slower ensemble members" oscillation, and it hit
Perceptron but not Convolutional purely because Convolutional had a run active from
a completed era and Perceptron, resumed from disk, never did. Train()'s message is
the specific one, so it wins.
NEXT STEP once this is running: the stall line now ends in "REJECTED BY: ..." and
the per-handle depths. Read it. Do not reason around it.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-17 15:21:15 -04:00
|
|
|
int enabled = 0;
|
|
|
|
|
int servable = TunableBarsCalculated(enabled);
|
|
|
|
|
//--- THE BLIND SPOT THAT COST 2026-08-17 (fixed the same day, after the fact). This used to read
|
|
|
|
|
//--- `if(servable <= 0 || servable >= want) return want;` - one branch covering three completely
|
|
|
|
|
//--- different states, and returning silently from all of them:
|
|
|
|
|
//--- enabled == 0 -> no tunable indicator is on, so there is genuinely no cap. Fine.
|
|
|
|
|
//--- enabled > 0, servable == -1 -> a handle answered INVALID. CopyBuffer will fail at every index.
|
|
|
|
|
//--- enabled > 0, servable == 0 -> a handle is created but has calculated nothing.
|
|
|
|
|
//--- The last two are the exact state the whole depth investigation was looking for, and the gate
|
|
|
|
|
//--- built to find it returned `want` without printing a character. That is why 27 MB of journal
|
|
|
|
|
//--- from a build that HAD this instrumentation contained zero depth lines while two charts sat at
|
|
|
|
|
//--- ok=0/failed=50163 for 38 minutes: the instrument's one silent path was the state it was hunting.
|
fix(indicators+panel): the dead handle is MEASURED now - recreate it; and order the ensemble panel by member, not by who published first
THE ANSWER, off the instrumentation added in be39674, first run:
ConvLSTM [HYB-2484]: TUNABLE INDICATOR REPORTS NO CALCULATED BARS - 1 tunable
indicator(s) enabled and the least-ready answers BarsCalculated()=-1 ...
Per-indicator depth: price=33982 MA=-1 ZigZag=33982 ATR=33982
MA=-1 with price, ZigZag and ATR all at full depth. **The handle is INVALID, not
short.** Same line on USDJPY (price=50179 MA=-1). Depth was never the problem;
the previous session's five theories were all answering the wrong question.
And it is per-member, not per-chart: LSTM-2484 ran the 34-candidate auto-tune on
that same chart at 15:24:18 and went on to train normally (feature health, 51
features, excursion head) reading the same indicator. Only ConvLSTM's handle -
the last member constructed - was dead. WHY is still not established. All four
members request ADMovingAverage with identical params, so MT5 hands them the SAME
refcounted handle, and the tuner's inner loop is Create-then-IndicatorRelease over
exactly that shared handle; that is the obvious suspect and it is NOT yet proven,
so this commit does not act on it.
1. IndicatorDepthReport() NOW PRINTS HANDLE NUMBERS, not just depths.
"MA=-1" says the handle is dead. "MA=-1(h12)" against another member's "MA=33982
(h12)" says it is the SAME handle and someone released it; "(h-1)" says it was
never created. That is the difference between a refcount bug and a creation
failure and it is one field. This is the measurement the shared-handle suspicion
needs before anyone acts on it.
2. RECREATE A DEAD HANDLE INSTEAD OF SWEEPING AGAINST IT.
A member that cannot read its own indicator must rebuild it. RepairDeadIndicatorHandles()
re-Creates only the ENABLED tunables reporting BarsCalculated() < 0 - a merely COLD
indicator (valid handle, 0 bars) is left alone to warm up the normal way. It does
NOT release first: -1 means the terminal no longer knows the handle, so there is
nothing to give back, and MT5 recycles handle VALUES so releasing a stale one could
decrement whatever now owns that number. 30s cooldown, because every ServableBars()
consumer reaches it including live inference on every tick. The feature cache is
dropped with it, and the log names before/after depths.
Cause-agnostic on purpose. Whatever is killing the handle, sweeping 50,163 bars
against a buffer that answers EMPTY_VALUE at every index - then discarding the era
and doing it again - is not a recovery.
3. THE SWEEP NOW HOLDS ON A DEAD HANDLE.
ServableBars() keeps answering `want` (its contract; live inference and online
learning have their own refusal paths and a 0 there reads as "no history at all").
SettledBars() - the training sweep's entry, the one caller that can afford to wait -
returns 0 instead, so Train() holds and reports rather than burning a full-history
pass it is guaranteed to throw away. A recreated handle is cold, so it primes
through the existing settle path on the next call. If the repair fails the member
holds indefinitely and says so every minute, and be39674's barrier liveness escape
releases the rest of the ensemble after 12 minutes - which is the correct
degradation and is exactly what the log shows happening.
4. THE PANEL ROWS WERE ORDERED BY WHO PUBLISHED FIRST.
Reported on XAUUSD: LSTM, ConvLSTM, Perceptron, Convolutional instead of
Perceptron, Convolutional, LSTM, ConvLSTM. ClaimEnsemblePanelSlot() handed out the
next free row on each member's FIRST PublishStatus() call, so the order was a race -
the members busy sweeping published before the ones sitting idle at the era barrier,
and be39674 sharpened it by (correctly) making a held member stop writing the terse
line. Rows are now keyed to m_ensembleIndex, the registration/construction order,
which is fixed for the life of the chart. Claimed on every publish rather than once,
so it is idempotent and refreshes the tag for a member whose ID was not final when it
first published (the config-tag suffix is appended during InitIndicators, after
EnsembleMember() registers). Unclaimed rows are skipped by the render and excluded
from the model count, so a member that has not published yet leaves no gap and shifts
nobody.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-17 15:40:00 -04:00
|
|
|
//--- The dead case is now REPORTED and REPAIRED. This function still answers `want` either way, so
|
|
|
|
|
//--- its own contract ("want, clamped to what the indicators can serve") is unchanged for live
|
|
|
|
|
//--- inference and online learning, which have their own refusal paths. Only SettledBars() - the
|
|
|
|
|
//--- training sweep's entry point, the one caller that can afford to wait - converts the same state
|
|
|
|
|
//--- into a HOLD. See the note at its own dead-handle branch for why the two deliberately differ.
|
fix(ensemble+depth): the barrier had no liveness escape, and the depth gate could not report the one state the evidence pointed at
Two charts (USDJPY 50,179 bars / XAUUSD 33,982) sat at era 0 for 38 minutes with
four of their eight members completely silent. Nothing in this commit guesses at
why the sweep fails - the last five guesses were all wrong. It makes the failure
say what it is, and stops one broken member taking its whole chart down with it.
WHAT THE LOG ACTUALLY SAYS, before any of this.
- The running build IS d9f834d (pulled 14:18, compiled 14:19:01, 0 errors), so
every depth instrument from 1dda479/7e63a8b/45c9e21 was live.
- It printed NOTHING. Zero "PRIMING", zero "CAPPED", zero "Per-indicator depth"
in 27 MB of journal. The instrument built to find the depth shortfall returned
"not this".
- On USDJPY at 14:24, CONV-cad8 completed eras 0 AND 1 across all 50,179 bars -
same chart, same 832-value window, same indicators, byte-identical fingerprint -
while LSTM-cad8 and HYB-cad8 reported ok=0 failed=50163. So it is not the
symbol, the history, the bar count or the indicator depth. It is per-member.
- ok=0 means the NEWEST anchors failed too, and a short indicator cannot do that.
The depth reading in project_silent_block_failures is therefore retired by its own
instrumentation. THE ROOT CAUSE IS STILL UNKNOWN and this commit does not claim one.
1. THE DEPTH GATE'S SILENT PATH WAS THE STATE IT WAS HUNTING.
ServableBars() read `if(servable <= 0 || servable >= want) return want;` - one
branch over three unrelated states, silent in all of them:
enabled == 0 -> nothing tunable is on. No cap. Healthy.
enabled > 0, servable == -1 -> a handle answered INVALID.
enabled > 0, servable == 0 -> created, never calculated.
BarsCalculated() returns -1 for a dead handle, so a dead MA is indistinguishable
from "no tunable indicators enabled" - and both returned `want` without printing a
character. That is exactly the state a per-member, every-index, depth-independent
failure produces, and it is the single reason a build carrying full depth
instrumentation logged nothing through the whole outage.
TunableBarsCalculated() now also reports HOW MANY indicators it consulted, and the
dead-handle case is reported (latched, with per-handle depths). The RETURN is
deliberately unchanged - what to do about a dead handle is not yet known, and
changing control flow on an unproven cause is how the last four fixes here went
wrong. SettledBars() routes its three pass-through states via ServableBars() so the
report is reachable from the training sweep, which is the only caller that hits it.
2. THE STALL REPORT NAMED A SLOT, NEVER A BLOCK.
"lookback slot 0 REJECTED (window had 24 of 832 values)" plus a guess ("an
indicator warm-up or a history-edge read"). Which guard fired was INFERRED by
counting 4+5+4+4+6+1 = 24 and concluding feature 25 must be the MA. The arithmetic
was right; every conclusion drawn from it was wrong, because a value count names a
POSITION and a position cannot tell cold from capped from invalid from off-the-end.
Every guard that can reject a bar now records itself - m_featureFailBlock - and the
report carries it, the series index, IndicatorDepthReport()'s per-handle depths,
and for each indicator whether the NEWEST bar reads. That last field is the whole
diagnosis in one word: newest-also-EMPTY means the buffer is unreadable everywhere
(cold or dead handle), newest-reads means a genuine history edge. Instrumented:
open, ATR, MA, RSI, MACD, Ichimoku, and all five AD blocks via ADIndicatorCold().
3. THE TOTAL-FAILURE BACKOFF WAS GATED ON THE WRONG QUESTION.
It armed only when m_featureFailTransient was set. Keeping that flag correct across
every guard is a list that has to stay right forever - the same shape of fix the
feature cache abandoned for the same reason - and the gate is pointless anyway: a
sweep where ZERO of 50,163 bars produced a window will produce zero again if it
restarts a millisecond later, transient or not. Doing that at full speed is what
starved six indicator threads on a six-core box. The backoff is now unconditional
on a total failure. The flag keeps its real job, deciding whether a MISS may be
cached, which is a per-bar question and not a scheduling one.
4. THE ERA BARRIER DEADLOCKED, AND SILENCED THE MEMBERS IT FROZE.
EnsembleMinTrainingEra() exempted deployed, stopped and paused members and its
comment concluded "so nothing deadlocks". Those three are all VOLUNTARY. A member
that simply CANNOT finish an era is none of them, so it pinned the minimum at its
own era with no time limit - and the hold branch's only action was
`m_lastEraCompleteTick = GetTickCount()`, which silences the stall watchdog. So on
USDJPY the two members that could not train reported, and the two healthy members
frozen behind them wrote nothing anywhere. The outage was visible only through the
members that were not suffering it.
- BarrierEraHeartbeat() stamps a clock on real era CHANGE, kept separate from
m_lastEraCompleteTick precisely because the barrier resets that one. Only a
member AT the minimum can be a blocker; a member ahead is idle by design and is
never counted as stuck.
- After ENSEMBLE_BARRIER_STUCK_MS (12 min) a non-advancing member is dropped from
the barrier minimum. It keeps training and rejoins the instant it completes an
era - at which point, being behind, it legitimately becomes the minimum again,
which is the documented resumed-laggard behaviour.
- Both transitions say so loudly, and the release states plainly that the
combined-vote score cannot be computed while the ensemble is desynchronised.
- A held member now writes a rate-limited journal line naming WHICH members it is
waiting on, so the blocker is read off one line.
5. THE PANEL FLICKER.
OnTickHandler gates its terse writer on !m_trainRunActive, and a barrier-held member
returns from Train() before ever setting it - so both writers thought they were the
only one updating the label and fought every tick. That is the reported "Getting
ready..." <-> "Waiting at era N for slower ensemble members" oscillation, and it hit
Perceptron but not Convolutional purely because Convolutional had a run active from
a completed era and Perceptron, resumed from disk, never did. Train()'s message is
the specific one, so it wins.
NEXT STEP once this is running: the stall line now ends in "REJECTED BY: ..." and
the per-handle depths. Read it. Do not reason around it.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-17 15:21:15 -04:00
|
|
|
if(enabled == 0 || servable >= want)
|
|
|
|
|
{
|
|
|
|
|
//--- Cleared on the healthy path too, not only on the clamp path below: a handle that recovers
|
|
|
|
|
//--- all the way to full depth would otherwise leave the latch set and a LATER outage would be
|
|
|
|
|
//--- swallowed - which is the failure mode this whole function is being fixed for.
|
|
|
|
|
m_indicatorDepthDeadWarned = false;
|
|
|
|
|
return want;
|
|
|
|
|
}
|
|
|
|
|
if(servable <= 0)
|
|
|
|
|
{
|
fix(indicators+panel): the dead handle is MEASURED now - recreate it; and order the ensemble panel by member, not by who published first
THE ANSWER, off the instrumentation added in be39674, first run:
ConvLSTM [HYB-2484]: TUNABLE INDICATOR REPORTS NO CALCULATED BARS - 1 tunable
indicator(s) enabled and the least-ready answers BarsCalculated()=-1 ...
Per-indicator depth: price=33982 MA=-1 ZigZag=33982 ATR=33982
MA=-1 with price, ZigZag and ATR all at full depth. **The handle is INVALID, not
short.** Same line on USDJPY (price=50179 MA=-1). Depth was never the problem;
the previous session's five theories were all answering the wrong question.
And it is per-member, not per-chart: LSTM-2484 ran the 34-candidate auto-tune on
that same chart at 15:24:18 and went on to train normally (feature health, 51
features, excursion head) reading the same indicator. Only ConvLSTM's handle -
the last member constructed - was dead. WHY is still not established. All four
members request ADMovingAverage with identical params, so MT5 hands them the SAME
refcounted handle, and the tuner's inner loop is Create-then-IndicatorRelease over
exactly that shared handle; that is the obvious suspect and it is NOT yet proven,
so this commit does not act on it.
1. IndicatorDepthReport() NOW PRINTS HANDLE NUMBERS, not just depths.
"MA=-1" says the handle is dead. "MA=-1(h12)" against another member's "MA=33982
(h12)" says it is the SAME handle and someone released it; "(h-1)" says it was
never created. That is the difference between a refcount bug and a creation
failure and it is one field. This is the measurement the shared-handle suspicion
needs before anyone acts on it.
2. RECREATE A DEAD HANDLE INSTEAD OF SWEEPING AGAINST IT.
A member that cannot read its own indicator must rebuild it. RepairDeadIndicatorHandles()
re-Creates only the ENABLED tunables reporting BarsCalculated() < 0 - a merely COLD
indicator (valid handle, 0 bars) is left alone to warm up the normal way. It does
NOT release first: -1 means the terminal no longer knows the handle, so there is
nothing to give back, and MT5 recycles handle VALUES so releasing a stale one could
decrement whatever now owns that number. 30s cooldown, because every ServableBars()
consumer reaches it including live inference on every tick. The feature cache is
dropped with it, and the log names before/after depths.
Cause-agnostic on purpose. Whatever is killing the handle, sweeping 50,163 bars
against a buffer that answers EMPTY_VALUE at every index - then discarding the era
and doing it again - is not a recovery.
3. THE SWEEP NOW HOLDS ON A DEAD HANDLE.
ServableBars() keeps answering `want` (its contract; live inference and online
learning have their own refusal paths and a 0 there reads as "no history at all").
SettledBars() - the training sweep's entry, the one caller that can afford to wait -
returns 0 instead, so Train() holds and reports rather than burning a full-history
pass it is guaranteed to throw away. A recreated handle is cold, so it primes
through the existing settle path on the next call. If the repair fails the member
holds indefinitely and says so every minute, and be39674's barrier liveness escape
releases the rest of the ensemble after 12 minutes - which is the correct
degradation and is exactly what the log shows happening.
4. THE PANEL ROWS WERE ORDERED BY WHO PUBLISHED FIRST.
Reported on XAUUSD: LSTM, ConvLSTM, Perceptron, Convolutional instead of
Perceptron, Convolutional, LSTM, ConvLSTM. ClaimEnsemblePanelSlot() handed out the
next free row on each member's FIRST PublishStatus() call, so the order was a race -
the members busy sweeping published before the ones sitting idle at the era barrier,
and be39674 sharpened it by (correctly) making a held member stop writing the terse
line. Rows are now keyed to m_ensembleIndex, the registration/construction order,
which is fixed for the life of the chart. Claimed on every publish rather than once,
so it is idempotent and refreshes the tag for a member whose ID was not final when it
first published (the config-tag suffix is appended during InitIndicators, after
EnsembleMember() registers). Unclaimed rows are skipped by the render and excluded
from the model count, so a member that has not published yet leaves no gap and shifts
nobody.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-17 15:40:00 -04:00
|
|
|
//--- REPORT FIRST, THEN REPAIR - in that order, so the depths in this line are the ones that
|
|
|
|
|
//--- caused it. Repairing first would print the freshly-created handle's numbers next to a
|
|
|
|
|
//--- message about a dead one, which is exactly the kind of self-contradicting log line that
|
|
|
|
|
//--- makes the next reader distrust the whole file.
|
fix(ensemble+depth): the barrier had no liveness escape, and the depth gate could not report the one state the evidence pointed at
Two charts (USDJPY 50,179 bars / XAUUSD 33,982) sat at era 0 for 38 minutes with
four of their eight members completely silent. Nothing in this commit guesses at
why the sweep fails - the last five guesses were all wrong. It makes the failure
say what it is, and stops one broken member taking its whole chart down with it.
WHAT THE LOG ACTUALLY SAYS, before any of this.
- The running build IS d9f834d (pulled 14:18, compiled 14:19:01, 0 errors), so
every depth instrument from 1dda479/7e63a8b/45c9e21 was live.
- It printed NOTHING. Zero "PRIMING", zero "CAPPED", zero "Per-indicator depth"
in 27 MB of journal. The instrument built to find the depth shortfall returned
"not this".
- On USDJPY at 14:24, CONV-cad8 completed eras 0 AND 1 across all 50,179 bars -
same chart, same 832-value window, same indicators, byte-identical fingerprint -
while LSTM-cad8 and HYB-cad8 reported ok=0 failed=50163. So it is not the
symbol, the history, the bar count or the indicator depth. It is per-member.
- ok=0 means the NEWEST anchors failed too, and a short indicator cannot do that.
The depth reading in project_silent_block_failures is therefore retired by its own
instrumentation. THE ROOT CAUSE IS STILL UNKNOWN and this commit does not claim one.
1. THE DEPTH GATE'S SILENT PATH WAS THE STATE IT WAS HUNTING.
ServableBars() read `if(servable <= 0 || servable >= want) return want;` - one
branch over three unrelated states, silent in all of them:
enabled == 0 -> nothing tunable is on. No cap. Healthy.
enabled > 0, servable == -1 -> a handle answered INVALID.
enabled > 0, servable == 0 -> created, never calculated.
BarsCalculated() returns -1 for a dead handle, so a dead MA is indistinguishable
from "no tunable indicators enabled" - and both returned `want` without printing a
character. That is exactly the state a per-member, every-index, depth-independent
failure produces, and it is the single reason a build carrying full depth
instrumentation logged nothing through the whole outage.
TunableBarsCalculated() now also reports HOW MANY indicators it consulted, and the
dead-handle case is reported (latched, with per-handle depths). The RETURN is
deliberately unchanged - what to do about a dead handle is not yet known, and
changing control flow on an unproven cause is how the last four fixes here went
wrong. SettledBars() routes its three pass-through states via ServableBars() so the
report is reachable from the training sweep, which is the only caller that hits it.
2. THE STALL REPORT NAMED A SLOT, NEVER A BLOCK.
"lookback slot 0 REJECTED (window had 24 of 832 values)" plus a guess ("an
indicator warm-up or a history-edge read"). Which guard fired was INFERRED by
counting 4+5+4+4+6+1 = 24 and concluding feature 25 must be the MA. The arithmetic
was right; every conclusion drawn from it was wrong, because a value count names a
POSITION and a position cannot tell cold from capped from invalid from off-the-end.
Every guard that can reject a bar now records itself - m_featureFailBlock - and the
report carries it, the series index, IndicatorDepthReport()'s per-handle depths,
and for each indicator whether the NEWEST bar reads. That last field is the whole
diagnosis in one word: newest-also-EMPTY means the buffer is unreadable everywhere
(cold or dead handle), newest-reads means a genuine history edge. Instrumented:
open, ATR, MA, RSI, MACD, Ichimoku, and all five AD blocks via ADIndicatorCold().
3. THE TOTAL-FAILURE BACKOFF WAS GATED ON THE WRONG QUESTION.
It armed only when m_featureFailTransient was set. Keeping that flag correct across
every guard is a list that has to stay right forever - the same shape of fix the
feature cache abandoned for the same reason - and the gate is pointless anyway: a
sweep where ZERO of 50,163 bars produced a window will produce zero again if it
restarts a millisecond later, transient or not. Doing that at full speed is what
starved six indicator threads on a six-core box. The backoff is now unconditional
on a total failure. The flag keeps its real job, deciding whether a MISS may be
cached, which is a per-bar question and not a scheduling one.
4. THE ERA BARRIER DEADLOCKED, AND SILENCED THE MEMBERS IT FROZE.
EnsembleMinTrainingEra() exempted deployed, stopped and paused members and its
comment concluded "so nothing deadlocks". Those three are all VOLUNTARY. A member
that simply CANNOT finish an era is none of them, so it pinned the minimum at its
own era with no time limit - and the hold branch's only action was
`m_lastEraCompleteTick = GetTickCount()`, which silences the stall watchdog. So on
USDJPY the two members that could not train reported, and the two healthy members
frozen behind them wrote nothing anywhere. The outage was visible only through the
members that were not suffering it.
- BarrierEraHeartbeat() stamps a clock on real era CHANGE, kept separate from
m_lastEraCompleteTick precisely because the barrier resets that one. Only a
member AT the minimum can be a blocker; a member ahead is idle by design and is
never counted as stuck.
- After ENSEMBLE_BARRIER_STUCK_MS (12 min) a non-advancing member is dropped from
the barrier minimum. It keeps training and rejoins the instant it completes an
era - at which point, being behind, it legitimately becomes the minimum again,
which is the documented resumed-laggard behaviour.
- Both transitions say so loudly, and the release states plainly that the
combined-vote score cannot be computed while the ensemble is desynchronised.
- A held member now writes a rate-limited journal line naming WHICH members it is
waiting on, so the blocker is read off one line.
5. THE PANEL FLICKER.
OnTickHandler gates its terse writer on !m_trainRunActive, and a barrier-held member
returns from Train() before ever setting it - so both writers thought they were the
only one updating the label and fought every tick. That is the reported "Getting
ready..." <-> "Waiting at era N for slower ensemble members" oscillation, and it hit
Perceptron but not Convolutional purely because Convolutional had a run active from
a completed era and Perceptron, resumed from disk, never did. Train()'s message is
the specific one, so it wins.
NEXT STEP once this is running: the stall line now ends in "REJECTED BY: ..." and
the per-handle depths. Read it. Do not reason around it.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-17 15:21:15 -04:00
|
|
|
if(!m_indicatorDepthDeadWarned)
|
|
|
|
|
{
|
|
|
|
|
m_indicatorDepthDeadWarned = true;
|
|
|
|
|
PrintFormat("%s: TUNABLE INDICATOR REPORTS NO CALCULATED BARS - %d tunable indicator(s) enabled"
|
|
|
|
|
" and the least-ready answers BarsCalculated()=%d while %s asked for %d. -1 means an"
|
|
|
|
|
" INVALID HANDLE, 0 means created-but-never-calculated; either way CopyBuffer fails"
|
|
|
|
|
" at EVERY index, the buffer holds nothing, and every feature block that reads it"
|
|
|
|
|
" rejects every bar. This is NOT the depth cap below (that one clamps and trains on"
|
|
|
|
|
" what is servable) - there is nothing to clamp to. Per-indicator depth:%s",
|
|
|
|
|
ID, enabled, servable, context, want, IndicatorDepthReport());
|
|
|
|
|
}
|
fix(indicators+panel): the dead handle is MEASURED now - recreate it; and order the ensemble panel by member, not by who published first
THE ANSWER, off the instrumentation added in be39674, first run:
ConvLSTM [HYB-2484]: TUNABLE INDICATOR REPORTS NO CALCULATED BARS - 1 tunable
indicator(s) enabled and the least-ready answers BarsCalculated()=-1 ...
Per-indicator depth: price=33982 MA=-1 ZigZag=33982 ATR=33982
MA=-1 with price, ZigZag and ATR all at full depth. **The handle is INVALID, not
short.** Same line on USDJPY (price=50179 MA=-1). Depth was never the problem;
the previous session's five theories were all answering the wrong question.
And it is per-member, not per-chart: LSTM-2484 ran the 34-candidate auto-tune on
that same chart at 15:24:18 and went on to train normally (feature health, 51
features, excursion head) reading the same indicator. Only ConvLSTM's handle -
the last member constructed - was dead. WHY is still not established. All four
members request ADMovingAverage with identical params, so MT5 hands them the SAME
refcounted handle, and the tuner's inner loop is Create-then-IndicatorRelease over
exactly that shared handle; that is the obvious suspect and it is NOT yet proven,
so this commit does not act on it.
1. IndicatorDepthReport() NOW PRINTS HANDLE NUMBERS, not just depths.
"MA=-1" says the handle is dead. "MA=-1(h12)" against another member's "MA=33982
(h12)" says it is the SAME handle and someone released it; "(h-1)" says it was
never created. That is the difference between a refcount bug and a creation
failure and it is one field. This is the measurement the shared-handle suspicion
needs before anyone acts on it.
2. RECREATE A DEAD HANDLE INSTEAD OF SWEEPING AGAINST IT.
A member that cannot read its own indicator must rebuild it. RepairDeadIndicatorHandles()
re-Creates only the ENABLED tunables reporting BarsCalculated() < 0 - a merely COLD
indicator (valid handle, 0 bars) is left alone to warm up the normal way. It does
NOT release first: -1 means the terminal no longer knows the handle, so there is
nothing to give back, and MT5 recycles handle VALUES so releasing a stale one could
decrement whatever now owns that number. 30s cooldown, because every ServableBars()
consumer reaches it including live inference on every tick. The feature cache is
dropped with it, and the log names before/after depths.
Cause-agnostic on purpose. Whatever is killing the handle, sweeping 50,163 bars
against a buffer that answers EMPTY_VALUE at every index - then discarding the era
and doing it again - is not a recovery.
3. THE SWEEP NOW HOLDS ON A DEAD HANDLE.
ServableBars() keeps answering `want` (its contract; live inference and online
learning have their own refusal paths and a 0 there reads as "no history at all").
SettledBars() - the training sweep's entry, the one caller that can afford to wait -
returns 0 instead, so Train() holds and reports rather than burning a full-history
pass it is guaranteed to throw away. A recreated handle is cold, so it primes
through the existing settle path on the next call. If the repair fails the member
holds indefinitely and says so every minute, and be39674's barrier liveness escape
releases the rest of the ensemble after 12 minutes - which is the correct
degradation and is exactly what the log shows happening.
4. THE PANEL ROWS WERE ORDERED BY WHO PUBLISHED FIRST.
Reported on XAUUSD: LSTM, ConvLSTM, Perceptron, Convolutional instead of
Perceptron, Convolutional, LSTM, ConvLSTM. ClaimEnsemblePanelSlot() handed out the
next free row on each member's FIRST PublishStatus() call, so the order was a race -
the members busy sweeping published before the ones sitting idle at the era barrier,
and be39674 sharpened it by (correctly) making a held member stop writing the terse
line. Rows are now keyed to m_ensembleIndex, the registration/construction order,
which is fixed for the life of the chart. Claimed on every publish rather than once,
so it is idempotent and refreshes the tag for a member whose ID was not final when it
first published (the config-tag suffix is appended during InitIndicators, after
EnsembleMember() registers). Unclaimed rows are skipped by the render and excluded
from the model count, so a member that has not published yet leaves no gap and shifts
nobody.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-17 15:40:00 -04:00
|
|
|
//--- A dead handle answers EMPTY_VALUE at every index, so a 50k-bar pass over it is 50k guaranteed
|
|
|
|
|
//--- rejections followed by a discarded era, forever - the exact loop that froze USDJPY and
|
|
|
|
|
//--- XAUUSD. Rate-limited inside, and it only touches handles reporting < 0, so a merely COLD
|
|
|
|
|
//--- indicator (valid handle, 0 bars calculated) is left alone to warm up normally.
|
|
|
|
|
RepairDeadIndicatorHandles();
|
fix(depth): route EVERY ResizeBuffers call site through one indicator-depth gate
1dda479 clamped the training sweep. It left five other paths asking the indicators
for a depth they cannot serve, and on a live account the quiet ones are worse than
the stall was - a stalled chart is visible, a chart trading on a degraded feature
window is not.
ServableBars(want, context) is now the single gate, and all six go through it:
training sweep clamp, floored at TRAIN_MIN_CLAMPED_BARS (below that a small
positive BarsCalculated is warm-up, which m_coldSweepTick owns)
label prebuild clamp - labels come from price/ADZigZag and would survive a
capped MA, but ResizeBuffers sizes EVERY buffer and a failed
CopyBuffer leaves m_MA EMPTY for the next reader, so this path
could silently re-break the block Train()'s clamp just fixed
live inference HOLD. Below `need` the swing block takes its degraded path and
inference runs on a different feature distribution than the model
was fitted on. This EA sizes real positions off that output, so
no signal beats a mismatched one
online learning HOLD, same reason and worse - this path WRITES to a live trading
model, so a mismatched (features, label) pair is not a wrong arrow,
it is a wrong weight update that compounds every bar
chart rescan clamp - SIGNAL_RESCAN_LOOKBACK_BARS is 5000 and MT5's smallest
"Max bars in chart" is also 5000, so this one is genuinely
reachable; uncapped it repaints the window all-Neutral
research export clamp before the emptiness test, so a capped symbol exports the
depth it has rather than writing a CSV with a dead feature block -
an artefact that looks complete and is silently wrong
Both HOLDs are insurance, not expected states: `need` tops out near 1,152 bars
(16 + 750 + 384 + 2) against a 5,000 floor on the terminal setting. They exist so
the failure mode is unreachable rather than merely unlikely.
Not changed: a genuinely SHORT price history still takes the old degraded path at
every site. That is pre-existing behaviour and narrowing it would mute charts that
trade today, so it stays a separate decision rather than a side effect of this fix.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-17 13:33:16 -04:00
|
|
|
return want;
|
fix(ensemble+depth): the barrier had no liveness escape, and the depth gate could not report the one state the evidence pointed at
Two charts (USDJPY 50,179 bars / XAUUSD 33,982) sat at era 0 for 38 minutes with
four of their eight members completely silent. Nothing in this commit guesses at
why the sweep fails - the last five guesses were all wrong. It makes the failure
say what it is, and stops one broken member taking its whole chart down with it.
WHAT THE LOG ACTUALLY SAYS, before any of this.
- The running build IS d9f834d (pulled 14:18, compiled 14:19:01, 0 errors), so
every depth instrument from 1dda479/7e63a8b/45c9e21 was live.
- It printed NOTHING. Zero "PRIMING", zero "CAPPED", zero "Per-indicator depth"
in 27 MB of journal. The instrument built to find the depth shortfall returned
"not this".
- On USDJPY at 14:24, CONV-cad8 completed eras 0 AND 1 across all 50,179 bars -
same chart, same 832-value window, same indicators, byte-identical fingerprint -
while LSTM-cad8 and HYB-cad8 reported ok=0 failed=50163. So it is not the
symbol, the history, the bar count or the indicator depth. It is per-member.
- ok=0 means the NEWEST anchors failed too, and a short indicator cannot do that.
The depth reading in project_silent_block_failures is therefore retired by its own
instrumentation. THE ROOT CAUSE IS STILL UNKNOWN and this commit does not claim one.
1. THE DEPTH GATE'S SILENT PATH WAS THE STATE IT WAS HUNTING.
ServableBars() read `if(servable <= 0 || servable >= want) return want;` - one
branch over three unrelated states, silent in all of them:
enabled == 0 -> nothing tunable is on. No cap. Healthy.
enabled > 0, servable == -1 -> a handle answered INVALID.
enabled > 0, servable == 0 -> created, never calculated.
BarsCalculated() returns -1 for a dead handle, so a dead MA is indistinguishable
from "no tunable indicators enabled" - and both returned `want` without printing a
character. That is exactly the state a per-member, every-index, depth-independent
failure produces, and it is the single reason a build carrying full depth
instrumentation logged nothing through the whole outage.
TunableBarsCalculated() now also reports HOW MANY indicators it consulted, and the
dead-handle case is reported (latched, with per-handle depths). The RETURN is
deliberately unchanged - what to do about a dead handle is not yet known, and
changing control flow on an unproven cause is how the last four fixes here went
wrong. SettledBars() routes its three pass-through states via ServableBars() so the
report is reachable from the training sweep, which is the only caller that hits it.
2. THE STALL REPORT NAMED A SLOT, NEVER A BLOCK.
"lookback slot 0 REJECTED (window had 24 of 832 values)" plus a guess ("an
indicator warm-up or a history-edge read"). Which guard fired was INFERRED by
counting 4+5+4+4+6+1 = 24 and concluding feature 25 must be the MA. The arithmetic
was right; every conclusion drawn from it was wrong, because a value count names a
POSITION and a position cannot tell cold from capped from invalid from off-the-end.
Every guard that can reject a bar now records itself - m_featureFailBlock - and the
report carries it, the series index, IndicatorDepthReport()'s per-handle depths,
and for each indicator whether the NEWEST bar reads. That last field is the whole
diagnosis in one word: newest-also-EMPTY means the buffer is unreadable everywhere
(cold or dead handle), newest-reads means a genuine history edge. Instrumented:
open, ATR, MA, RSI, MACD, Ichimoku, and all five AD blocks via ADIndicatorCold().
3. THE TOTAL-FAILURE BACKOFF WAS GATED ON THE WRONG QUESTION.
It armed only when m_featureFailTransient was set. Keeping that flag correct across
every guard is a list that has to stay right forever - the same shape of fix the
feature cache abandoned for the same reason - and the gate is pointless anyway: a
sweep where ZERO of 50,163 bars produced a window will produce zero again if it
restarts a millisecond later, transient or not. Doing that at full speed is what
starved six indicator threads on a six-core box. The backoff is now unconditional
on a total failure. The flag keeps its real job, deciding whether a MISS may be
cached, which is a per-bar question and not a scheduling one.
4. THE ERA BARRIER DEADLOCKED, AND SILENCED THE MEMBERS IT FROZE.
EnsembleMinTrainingEra() exempted deployed, stopped and paused members and its
comment concluded "so nothing deadlocks". Those three are all VOLUNTARY. A member
that simply CANNOT finish an era is none of them, so it pinned the minimum at its
own era with no time limit - and the hold branch's only action was
`m_lastEraCompleteTick = GetTickCount()`, which silences the stall watchdog. So on
USDJPY the two members that could not train reported, and the two healthy members
frozen behind them wrote nothing anywhere. The outage was visible only through the
members that were not suffering it.
- BarrierEraHeartbeat() stamps a clock on real era CHANGE, kept separate from
m_lastEraCompleteTick precisely because the barrier resets that one. Only a
member AT the minimum can be a blocker; a member ahead is idle by design and is
never counted as stuck.
- After ENSEMBLE_BARRIER_STUCK_MS (12 min) a non-advancing member is dropped from
the barrier minimum. It keeps training and rejoins the instant it completes an
era - at which point, being behind, it legitimately becomes the minimum again,
which is the documented resumed-laggard behaviour.
- Both transitions say so loudly, and the release states plainly that the
combined-vote score cannot be computed while the ensemble is desynchronised.
- A held member now writes a rate-limited journal line naming WHICH members it is
waiting on, so the blocker is read off one line.
5. THE PANEL FLICKER.
OnTickHandler gates its terse writer on !m_trainRunActive, and a barrier-held member
returns from Train() before ever setting it - so both writers thought they were the
only one updating the label and fought every tick. That is the reported "Getting
ready..." <-> "Waiting at era N for slower ensemble members" oscillation, and it hit
Perceptron but not Convolutional purely because Convolutional had a run active from
a completed era and Perceptron, resumed from disk, never did. Train()'s message is
the specific one, so it wins.
NEXT STEP once this is running: the stall line now ends in "REJECTED BY: ..." and
the per-handle depths. Read it. Do not reason around it.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-17 15:21:15 -04:00
|
|
|
}
|
|
|
|
|
m_indicatorDepthDeadWarned = false;
|
fix(depth): route EVERY ResizeBuffers call site through one indicator-depth gate
1dda479 clamped the training sweep. It left five other paths asking the indicators
for a depth they cannot serve, and on a live account the quiet ones are worse than
the stall was - a stalled chart is visible, a chart trading on a degraded feature
window is not.
ServableBars(want, context) is now the single gate, and all six go through it:
training sweep clamp, floored at TRAIN_MIN_CLAMPED_BARS (below that a small
positive BarsCalculated is warm-up, which m_coldSweepTick owns)
label prebuild clamp - labels come from price/ADZigZag and would survive a
capped MA, but ResizeBuffers sizes EVERY buffer and a failed
CopyBuffer leaves m_MA EMPTY for the next reader, so this path
could silently re-break the block Train()'s clamp just fixed
live inference HOLD. Below `need` the swing block takes its degraded path and
inference runs on a different feature distribution than the model
was fitted on. This EA sizes real positions off that output, so
no signal beats a mismatched one
online learning HOLD, same reason and worse - this path WRITES to a live trading
model, so a mismatched (features, label) pair is not a wrong arrow,
it is a wrong weight update that compounds every bar
chart rescan clamp - SIGNAL_RESCAN_LOOKBACK_BARS is 5000 and MT5's smallest
"Max bars in chart" is also 5000, so this one is genuinely
reachable; uncapped it repaints the window all-Neutral
research export clamp before the emptiness test, so a capped symbol exports the
depth it has rather than writing a CSV with a dead feature block -
an artefact that looks complete and is silently wrong
Both HOLDs are insurance, not expected states: `need` tops out near 1,152 bars
(16 + 750 + 384 + 2) against a 5,000 floor on the terminal setting. They exist so
the failure mode is unreachable rather than merely unlikely.
Not changed: a genuinely SHORT price history still takes the old degraded path at
every site. That is pre-existing behaviour and narrowing it would mute charts that
trade today, so it stays a separate decision rather than a side effect of this fix.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-17 13:33:16 -04:00
|
|
|
//--- -(m_historyBars + 2): the deepest window slot reads (r + m_historyBars - 1), and the MA block
|
|
|
|
|
//--- one further back again for its bar-over-bar change, so the last usable anchor sits that far
|
|
|
|
|
//--- inside the buffer. Without the margin the clamp would hand back a depth whose own deepest
|
|
|
|
|
//--- window still reads off the end - the exact failure it exists to prevent.
|
|
|
|
|
int capped = servable - ((int)m_historyBars + 2);
|
|
|
|
|
if(capped < 0)
|
|
|
|
|
capped = 0;
|
|
|
|
|
//--- Depends only on `servable` and m_historyBars, never on `want`, so it is stable across call
|
|
|
|
|
//--- sites and this logs once per real change rather than once per era per context.
|
|
|
|
|
if(m_indicatorDepthCapBars != capped)
|
|
|
|
|
{
|
|
|
|
|
m_indicatorDepthCapBars = capped;
|
|
|
|
|
PrintFormat("%s: indicator history CAPPED to %d bars (%s asked for %d) - the price series has"
|
feat(depth): prime -> settle -> sweep, and name which handle is short
"Max bars in chart" is set to Unlimited, so the static-terminal-limit reading in
1dda479 was wrong. Two other candidate causes are falsified too: the price series
is fully downloaded and flat (USDJPY 50,162 -> 50,163 over 80 minutes, i.e. one new
H4 bar), and the handles have been stable since 13:07 with zero windows for the 17
minutes after, so it is not download-in-progress and not handle churn. The MA period
tops out at 200 (ADIndicatorTuner MA_PERIOD_PRESETS) against ~50k bars, so it is not
indicator cost either.
What IS verified stays verified: CopyBuffer past the calculated depth fails outright
rather than short-reading, so the buffer holds nothing and every index reads
EMPTY_VALUE; m_MA is the only CiCustom whose block REJECTS on that (m_ADZigZag
neutral-fills, RSI/MACD/Ichimoku/ATR are built-ins); the wall is therefore feature 25
of every bar, exactly as the "24 of 832" stall lines said. And it is depth-correlated:
16k-bar charts train, 34k/50k get zero windows forever.
So the WHY is still open, and this fix does not depend on it. Per the user's protocol:
the request itself is the primer, so prime at full depth, then poll TunableBarsCalculated()
every 3s and hold the sweep until it stops changing (3 steady probes), then use whatever
it settled at. Bounded at 10 min, and a give-up is logged as a give-up so an abandoned
depth is never mistaken for a settled one. This supersedes 1dda479's clamp on the two
training paths, which snapshotted a value that may still have been climbing; the clamp
remains for the paths that cannot wait (inference/online/rescan/export, see 7e63a8b).
The load-bearing part is what does NOT happen while waiting: no sweep. A 50k-bar feature
scan starves the indicator threads the request just woke, which is how the failure
sustained itself for 40 minutes at a time - discard era, re-sweep, discard, which is the
0->100% oscillation on the panel.
Also adds IndicatorDepthReport(): per-handle BarsCalculated() on the priming, cap and
stall lines. The logs proved WHICH FEATURE died but never WHICH HANDLE was short, so the
cause had to be inferred - and was guessed wrong twice. The next occurrence reads it off.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-17 13:44:27 -04:00
|
|
|
" that much, but the least-ready tunable indicator has only calculated %d. Past what"
|
|
|
|
|
" an indicator has calculated CopyBuffer does not short-read, it FAILS, so the buffer"
|
|
|
|
|
" holds NOTHING and EVERY index reads EMPTY_VALUE - indistinguishable from a cold"
|
|
|
|
|
" indicator. Per-indicator depth:%s",
|
|
|
|
|
ID, capped, context, want, servable, IndicatorDepthReport());
|
fix(depth): route EVERY ResizeBuffers call site through one indicator-depth gate
1dda479 clamped the training sweep. It left five other paths asking the indicators
for a depth they cannot serve, and on a live account the quiet ones are worse than
the stall was - a stalled chart is visible, a chart trading on a degraded feature
window is not.
ServableBars(want, context) is now the single gate, and all six go through it:
training sweep clamp, floored at TRAIN_MIN_CLAMPED_BARS (below that a small
positive BarsCalculated is warm-up, which m_coldSweepTick owns)
label prebuild clamp - labels come from price/ADZigZag and would survive a
capped MA, but ResizeBuffers sizes EVERY buffer and a failed
CopyBuffer leaves m_MA EMPTY for the next reader, so this path
could silently re-break the block Train()'s clamp just fixed
live inference HOLD. Below `need` the swing block takes its degraded path and
inference runs on a different feature distribution than the model
was fitted on. This EA sizes real positions off that output, so
no signal beats a mismatched one
online learning HOLD, same reason and worse - this path WRITES to a live trading
model, so a mismatched (features, label) pair is not a wrong arrow,
it is a wrong weight update that compounds every bar
chart rescan clamp - SIGNAL_RESCAN_LOOKBACK_BARS is 5000 and MT5's smallest
"Max bars in chart" is also 5000, so this one is genuinely
reachable; uncapped it repaints the window all-Neutral
research export clamp before the emptiness test, so a capped symbol exports the
depth it has rather than writing a CSV with a dead feature block -
an artefact that looks complete and is silently wrong
Both HOLDs are insurance, not expected states: `need` tops out near 1,152 bars
(16 + 750 + 384 + 2) against a 5,000 floor on the terminal setting. They exist so
the failure mode is unreachable rather than merely unlikely.
Not changed: a genuinely SHORT price history still takes the old degraded path at
every site. That is pre-existing behaviour and narrowing it would mute charts that
trade today, so it stays a separate decision rather than a side effect of this fix.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-17 13:33:16 -04:00
|
|
|
}
|
|
|
|
|
return capped;
|
|
|
|
|
}
|
|
|
|
|
//+------------------------------------------------------------------+
|
feat(depth): prime -> settle -> sweep, and name which handle is short
"Max bars in chart" is set to Unlimited, so the static-terminal-limit reading in
1dda479 was wrong. Two other candidate causes are falsified too: the price series
is fully downloaded and flat (USDJPY 50,162 -> 50,163 over 80 minutes, i.e. one new
H4 bar), and the handles have been stable since 13:07 with zero windows for the 17
minutes after, so it is not download-in-progress and not handle churn. The MA period
tops out at 200 (ADIndicatorTuner MA_PERIOD_PRESETS) against ~50k bars, so it is not
indicator cost either.
What IS verified stays verified: CopyBuffer past the calculated depth fails outright
rather than short-reading, so the buffer holds nothing and every index reads
EMPTY_VALUE; m_MA is the only CiCustom whose block REJECTS on that (m_ADZigZag
neutral-fills, RSI/MACD/Ichimoku/ATR are built-ins); the wall is therefore feature 25
of every bar, exactly as the "24 of 832" stall lines said. And it is depth-correlated:
16k-bar charts train, 34k/50k get zero windows forever.
So the WHY is still open, and this fix does not depend on it. Per the user's protocol:
the request itself is the primer, so prime at full depth, then poll TunableBarsCalculated()
every 3s and hold the sweep until it stops changing (3 steady probes), then use whatever
it settled at. Bounded at 10 min, and a give-up is logged as a give-up so an abandoned
depth is never mistaken for a settled one. This supersedes 1dda479's clamp on the two
training paths, which snapshotted a value that may still have been climbing; the clamp
remains for the paths that cannot wait (inference/online/rescan/export, see 7e63a8b).
The load-bearing part is what does NOT happen while waiting: no sweep. A 50k-bar feature
scan starves the indicator threads the request just woke, which is how the failure
sustained itself for 40 minutes at a time - discard era, re-sweep, discard, which is the
0->100% oscillation on the panel.
Also adds IndicatorDepthReport(): per-handle BarsCalculated() on the priming, cap and
stall lines. The logs proved WHICH FEATURE died but never WHICH HANDLE was short, so the
cause had to be inferred - and was guessed wrong twice. The next occurrence reads it off.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-17 13:44:27 -04:00
|
|
|
//| See the declaration. ServableBars() with the WAIT in front of it. |
|
|
|
|
|
//+------------------------------------------------------------------+
|
|
|
|
|
int CExpertSignalAIBase::SettledBars(int want, string context)
|
|
|
|
|
{
|
|
|
|
|
if(want <= 0)
|
|
|
|
|
return want;
|
fix(ensemble+depth): the barrier had no liveness escape, and the depth gate could not report the one state the evidence pointed at
Two charts (USDJPY 50,179 bars / XAUUSD 33,982) sat at era 0 for 38 minutes with
four of their eight members completely silent. Nothing in this commit guesses at
why the sweep fails - the last five guesses were all wrong. It makes the failure
say what it is, and stops one broken member taking its whole chart down with it.
WHAT THE LOG ACTUALLY SAYS, before any of this.
- The running build IS d9f834d (pulled 14:18, compiled 14:19:01, 0 errors), so
every depth instrument from 1dda479/7e63a8b/45c9e21 was live.
- It printed NOTHING. Zero "PRIMING", zero "CAPPED", zero "Per-indicator depth"
in 27 MB of journal. The instrument built to find the depth shortfall returned
"not this".
- On USDJPY at 14:24, CONV-cad8 completed eras 0 AND 1 across all 50,179 bars -
same chart, same 832-value window, same indicators, byte-identical fingerprint -
while LSTM-cad8 and HYB-cad8 reported ok=0 failed=50163. So it is not the
symbol, the history, the bar count or the indicator depth. It is per-member.
- ok=0 means the NEWEST anchors failed too, and a short indicator cannot do that.
The depth reading in project_silent_block_failures is therefore retired by its own
instrumentation. THE ROOT CAUSE IS STILL UNKNOWN and this commit does not claim one.
1. THE DEPTH GATE'S SILENT PATH WAS THE STATE IT WAS HUNTING.
ServableBars() read `if(servable <= 0 || servable >= want) return want;` - one
branch over three unrelated states, silent in all of them:
enabled == 0 -> nothing tunable is on. No cap. Healthy.
enabled > 0, servable == -1 -> a handle answered INVALID.
enabled > 0, servable == 0 -> created, never calculated.
BarsCalculated() returns -1 for a dead handle, so a dead MA is indistinguishable
from "no tunable indicators enabled" - and both returned `want` without printing a
character. That is exactly the state a per-member, every-index, depth-independent
failure produces, and it is the single reason a build carrying full depth
instrumentation logged nothing through the whole outage.
TunableBarsCalculated() now also reports HOW MANY indicators it consulted, and the
dead-handle case is reported (latched, with per-handle depths). The RETURN is
deliberately unchanged - what to do about a dead handle is not yet known, and
changing control flow on an unproven cause is how the last four fixes here went
wrong. SettledBars() routes its three pass-through states via ServableBars() so the
report is reachable from the training sweep, which is the only caller that hits it.
2. THE STALL REPORT NAMED A SLOT, NEVER A BLOCK.
"lookback slot 0 REJECTED (window had 24 of 832 values)" plus a guess ("an
indicator warm-up or a history-edge read"). Which guard fired was INFERRED by
counting 4+5+4+4+6+1 = 24 and concluding feature 25 must be the MA. The arithmetic
was right; every conclusion drawn from it was wrong, because a value count names a
POSITION and a position cannot tell cold from capped from invalid from off-the-end.
Every guard that can reject a bar now records itself - m_featureFailBlock - and the
report carries it, the series index, IndicatorDepthReport()'s per-handle depths,
and for each indicator whether the NEWEST bar reads. That last field is the whole
diagnosis in one word: newest-also-EMPTY means the buffer is unreadable everywhere
(cold or dead handle), newest-reads means a genuine history edge. Instrumented:
open, ATR, MA, RSI, MACD, Ichimoku, and all five AD blocks via ADIndicatorCold().
3. THE TOTAL-FAILURE BACKOFF WAS GATED ON THE WRONG QUESTION.
It armed only when m_featureFailTransient was set. Keeping that flag correct across
every guard is a list that has to stay right forever - the same shape of fix the
feature cache abandoned for the same reason - and the gate is pointless anyway: a
sweep where ZERO of 50,163 bars produced a window will produce zero again if it
restarts a millisecond later, transient or not. Doing that at full speed is what
starved six indicator threads on a six-core box. The backoff is now unconditional
on a total failure. The flag keeps its real job, deciding whether a MISS may be
cached, which is a per-bar question and not a scheduling one.
4. THE ERA BARRIER DEADLOCKED, AND SILENCED THE MEMBERS IT FROZE.
EnsembleMinTrainingEra() exempted deployed, stopped and paused members and its
comment concluded "so nothing deadlocks". Those three are all VOLUNTARY. A member
that simply CANNOT finish an era is none of them, so it pinned the minimum at its
own era with no time limit - and the hold branch's only action was
`m_lastEraCompleteTick = GetTickCount()`, which silences the stall watchdog. So on
USDJPY the two members that could not train reported, and the two healthy members
frozen behind them wrote nothing anywhere. The outage was visible only through the
members that were not suffering it.
- BarrierEraHeartbeat() stamps a clock on real era CHANGE, kept separate from
m_lastEraCompleteTick precisely because the barrier resets that one. Only a
member AT the minimum can be a blocker; a member ahead is idle by design and is
never counted as stuck.
- After ENSEMBLE_BARRIER_STUCK_MS (12 min) a non-advancing member is dropped from
the barrier minimum. It keeps training and rejoins the instant it completes an
era - at which point, being behind, it legitimately becomes the minimum again,
which is the documented resumed-laggard behaviour.
- Both transitions say so loudly, and the release states plainly that the
combined-vote score cannot be computed while the ensemble is desynchronised.
- A held member now writes a rate-limited journal line naming WHICH members it is
waiting on, so the blocker is read off one line.
5. THE PANEL FLICKER.
OnTickHandler gates its terse writer on !m_trainRunActive, and a barrier-held member
returns from Train() before ever setting it - so both writers thought they were the
only one updating the label and fought every tick. That is the reported "Getting
ready..." <-> "Waiting at era N for slower ensemble members" oscillation, and it hit
Perceptron but not Convolutional purely because Convolutional had a run active from
a completed era and Perceptron, resumed from disk, never did. Train()'s message is
the specific one, so it wins.
NEXT STEP once this is running: the stall line now ends in "REJECTED BY: ..." and
the per-handle depths. Read it. Do not reason around it.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-17 15:21:15 -04:00
|
|
|
int enabled = 0;
|
|
|
|
|
int servable = TunableBarsCalculated(enabled);
|
|
|
|
|
//--- Same three-states-one-branch defect ServableBars() carried (see the long note there): `servable
|
|
|
|
|
//--- < 0` was read as "nothing tunable is on", but it is ALSO what a dead handle answers. Settling a
|
|
|
|
|
//--- dead handle is pointless - it will never climb - so this still falls straight through, but only
|
|
|
|
|
//--- ServableBars() gets to decide that now, and it reports the case on the way past.
|
fix(indicators+panel): the dead handle is MEASURED now - recreate it; and order the ensemble panel by member, not by who published first
THE ANSWER, off the instrumentation added in be39674, first run:
ConvLSTM [HYB-2484]: TUNABLE INDICATOR REPORTS NO CALCULATED BARS - 1 tunable
indicator(s) enabled and the least-ready answers BarsCalculated()=-1 ...
Per-indicator depth: price=33982 MA=-1 ZigZag=33982 ATR=33982
MA=-1 with price, ZigZag and ATR all at full depth. **The handle is INVALID, not
short.** Same line on USDJPY (price=50179 MA=-1). Depth was never the problem;
the previous session's five theories were all answering the wrong question.
And it is per-member, not per-chart: LSTM-2484 ran the 34-candidate auto-tune on
that same chart at 15:24:18 and went on to train normally (feature health, 51
features, excursion head) reading the same indicator. Only ConvLSTM's handle -
the last member constructed - was dead. WHY is still not established. All four
members request ADMovingAverage with identical params, so MT5 hands them the SAME
refcounted handle, and the tuner's inner loop is Create-then-IndicatorRelease over
exactly that shared handle; that is the obvious suspect and it is NOT yet proven,
so this commit does not act on it.
1. IndicatorDepthReport() NOW PRINTS HANDLE NUMBERS, not just depths.
"MA=-1" says the handle is dead. "MA=-1(h12)" against another member's "MA=33982
(h12)" says it is the SAME handle and someone released it; "(h-1)" says it was
never created. That is the difference between a refcount bug and a creation
failure and it is one field. This is the measurement the shared-handle suspicion
needs before anyone acts on it.
2. RECREATE A DEAD HANDLE INSTEAD OF SWEEPING AGAINST IT.
A member that cannot read its own indicator must rebuild it. RepairDeadIndicatorHandles()
re-Creates only the ENABLED tunables reporting BarsCalculated() < 0 - a merely COLD
indicator (valid handle, 0 bars) is left alone to warm up the normal way. It does
NOT release first: -1 means the terminal no longer knows the handle, so there is
nothing to give back, and MT5 recycles handle VALUES so releasing a stale one could
decrement whatever now owns that number. 30s cooldown, because every ServableBars()
consumer reaches it including live inference on every tick. The feature cache is
dropped with it, and the log names before/after depths.
Cause-agnostic on purpose. Whatever is killing the handle, sweeping 50,163 bars
against a buffer that answers EMPTY_VALUE at every index - then discarding the era
and doing it again - is not a recovery.
3. THE SWEEP NOW HOLDS ON A DEAD HANDLE.
ServableBars() keeps answering `want` (its contract; live inference and online
learning have their own refusal paths and a 0 there reads as "no history at all").
SettledBars() - the training sweep's entry, the one caller that can afford to wait -
returns 0 instead, so Train() holds and reports rather than burning a full-history
pass it is guaranteed to throw away. A recreated handle is cold, so it primes
through the existing settle path on the next call. If the repair fails the member
holds indefinitely and says so every minute, and be39674's barrier liveness escape
releases the rest of the ensemble after 12 minutes - which is the correct
degradation and is exactly what the log shows happening.
4. THE PANEL ROWS WERE ORDERED BY WHO PUBLISHED FIRST.
Reported on XAUUSD: LSTM, ConvLSTM, Perceptron, Convolutional instead of
Perceptron, Convolutional, LSTM, ConvLSTM. ClaimEnsemblePanelSlot() handed out the
next free row on each member's FIRST PublishStatus() call, so the order was a race -
the members busy sweeping published before the ones sitting idle at the era barrier,
and be39674 sharpened it by (correctly) making a held member stop writing the terse
line. Rows are now keyed to m_ensembleIndex, the registration/construction order,
which is fixed for the life of the chart. Claimed on every publish rather than once,
so it is idempotent and refreshes the tag for a member whose ID was not final when it
first published (the config-tag suffix is appended during InitIndicators, after
EnsembleMember() registers). Unclaimed rows are skipped by the render and excluded
from the model count, so a member that has not published yet leaves no gap and shifts
nobody.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-17 15:40:00 -04:00
|
|
|
if(enabled == 0 || servable >= want)
|
|
|
|
|
{
|
|
|
|
|
m_depthSettleStart = 0;
|
|
|
|
|
m_depthProbeStable = 0;
|
|
|
|
|
m_depthProbeLast = 0;
|
|
|
|
|
return want;
|
|
|
|
|
}
|
|
|
|
|
if(servable <= 0)
|
feat(depth): prime -> settle -> sweep, and name which handle is short
"Max bars in chart" is set to Unlimited, so the static-terminal-limit reading in
1dda479 was wrong. Two other candidate causes are falsified too: the price series
is fully downloaded and flat (USDJPY 50,162 -> 50,163 over 80 minutes, i.e. one new
H4 bar), and the handles have been stable since 13:07 with zero windows for the 17
minutes after, so it is not download-in-progress and not handle churn. The MA period
tops out at 200 (ADIndicatorTuner MA_PERIOD_PRESETS) against ~50k bars, so it is not
indicator cost either.
What IS verified stays verified: CopyBuffer past the calculated depth fails outright
rather than short-reading, so the buffer holds nothing and every index reads
EMPTY_VALUE; m_MA is the only CiCustom whose block REJECTS on that (m_ADZigZag
neutral-fills, RSI/MACD/Ichimoku/ATR are built-ins); the wall is therefore feature 25
of every bar, exactly as the "24 of 832" stall lines said. And it is depth-correlated:
16k-bar charts train, 34k/50k get zero windows forever.
So the WHY is still open, and this fix does not depend on it. Per the user's protocol:
the request itself is the primer, so prime at full depth, then poll TunableBarsCalculated()
every 3s and hold the sweep until it stops changing (3 steady probes), then use whatever
it settled at. Bounded at 10 min, and a give-up is logged as a give-up so an abandoned
depth is never mistaken for a settled one. This supersedes 1dda479's clamp on the two
training paths, which snapshotted a value that may still have been climbing; the clamp
remains for the paths that cannot wait (inference/online/rescan/export, see 7e63a8b).
The load-bearing part is what does NOT happen while waiting: no sweep. A 50k-bar feature
scan starves the indicator threads the request just woke, which is how the failure
sustained itself for 40 minutes at a time - discard era, re-sweep, discard, which is the
0->100% oscillation on the panel.
Also adds IndicatorDepthReport(): per-handle BarsCalculated() on the priming, cap and
stall lines. The logs proved WHICH FEATURE died but never WHICH HANDLE was short, so the
cause had to be inferred - and was guessed wrong twice. The next occurrence reads it off.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-17 13:44:27 -04:00
|
|
|
{
|
|
|
|
|
m_depthSettleStart = 0;
|
|
|
|
|
m_depthProbeStable = 0;
|
|
|
|
|
m_depthProbeLast = 0;
|
fix(indicators+panel): the dead handle is MEASURED now - recreate it; and order the ensemble panel by member, not by who published first
THE ANSWER, off the instrumentation added in be39674, first run:
ConvLSTM [HYB-2484]: TUNABLE INDICATOR REPORTS NO CALCULATED BARS - 1 tunable
indicator(s) enabled and the least-ready answers BarsCalculated()=-1 ...
Per-indicator depth: price=33982 MA=-1 ZigZag=33982 ATR=33982
MA=-1 with price, ZigZag and ATR all at full depth. **The handle is INVALID, not
short.** Same line on USDJPY (price=50179 MA=-1). Depth was never the problem;
the previous session's five theories were all answering the wrong question.
And it is per-member, not per-chart: LSTM-2484 ran the 34-candidate auto-tune on
that same chart at 15:24:18 and went on to train normally (feature health, 51
features, excursion head) reading the same indicator. Only ConvLSTM's handle -
the last member constructed - was dead. WHY is still not established. All four
members request ADMovingAverage with identical params, so MT5 hands them the SAME
refcounted handle, and the tuner's inner loop is Create-then-IndicatorRelease over
exactly that shared handle; that is the obvious suspect and it is NOT yet proven,
so this commit does not act on it.
1. IndicatorDepthReport() NOW PRINTS HANDLE NUMBERS, not just depths.
"MA=-1" says the handle is dead. "MA=-1(h12)" against another member's "MA=33982
(h12)" says it is the SAME handle and someone released it; "(h-1)" says it was
never created. That is the difference between a refcount bug and a creation
failure and it is one field. This is the measurement the shared-handle suspicion
needs before anyone acts on it.
2. RECREATE A DEAD HANDLE INSTEAD OF SWEEPING AGAINST IT.
A member that cannot read its own indicator must rebuild it. RepairDeadIndicatorHandles()
re-Creates only the ENABLED tunables reporting BarsCalculated() < 0 - a merely COLD
indicator (valid handle, 0 bars) is left alone to warm up the normal way. It does
NOT release first: -1 means the terminal no longer knows the handle, so there is
nothing to give back, and MT5 recycles handle VALUES so releasing a stale one could
decrement whatever now owns that number. 30s cooldown, because every ServableBars()
consumer reaches it including live inference on every tick. The feature cache is
dropped with it, and the log names before/after depths.
Cause-agnostic on purpose. Whatever is killing the handle, sweeping 50,163 bars
against a buffer that answers EMPTY_VALUE at every index - then discarding the era
and doing it again - is not a recovery.
3. THE SWEEP NOW HOLDS ON A DEAD HANDLE.
ServableBars() keeps answering `want` (its contract; live inference and online
learning have their own refusal paths and a 0 there reads as "no history at all").
SettledBars() - the training sweep's entry, the one caller that can afford to wait -
returns 0 instead, so Train() holds and reports rather than burning a full-history
pass it is guaranteed to throw away. A recreated handle is cold, so it primes
through the existing settle path on the next call. If the repair fails the member
holds indefinitely and says so every minute, and be39674's barrier liveness escape
releases the rest of the ensemble after 12 minutes - which is the correct
degradation and is exactly what the log shows happening.
4. THE PANEL ROWS WERE ORDERED BY WHO PUBLISHED FIRST.
Reported on XAUUSD: LSTM, ConvLSTM, Perceptron, Convolutional instead of
Perceptron, Convolutional, LSTM, ConvLSTM. ClaimEnsemblePanelSlot() handed out the
next free row on each member's FIRST PublishStatus() call, so the order was a race -
the members busy sweeping published before the ones sitting idle at the era barrier,
and be39674 sharpened it by (correctly) making a held member stop writing the terse
line. Rows are now keyed to m_ensembleIndex, the registration/construction order,
which is fixed for the life of the chart. Claimed on every publish rather than once,
so it is idempotent and refreshes the tag for a member whose ID was not final when it
first published (the config-tag suffix is appended during InitIndicators, after
EnsembleMember() registers). Unclaimed rows are skipped by the render and excluded
from the model count, so a member that has not published yet leaves no gap and shifts
nobody.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-17 15:40:00 -04:00
|
|
|
//--- Routed through ServableBars() rather than answering here, and that detour is the whole point:
|
|
|
|
|
//--- the training sweep - the ONLY caller that reaches the dead-handle state in practice - calls
|
|
|
|
|
//--- SettledBars, not ServableBars. Returning directly is what kept the report (and now the
|
|
|
|
|
//--- repair) unreachable from the one path that needed them.
|
|
|
|
|
ServableBars(want, context);
|
|
|
|
|
//--- 0 = HOLD, and this is the one place the two functions deliberately disagree. ServableBars
|
|
|
|
|
//--- must answer `want` for its own callers (live inference and online learning have their own
|
|
|
|
|
//--- refusal paths and a 0 there would read as "no history at all"), but the training sweep can
|
|
|
|
|
//--- afford to wait and must: nothing is readable at any index right now, so a full-history pass
|
|
|
|
|
//--- would reject every bar, discard the era and start over - the exact 38-minute loop this is
|
|
|
|
|
//--- being fixed for. Whether the handle was just recreated (cold, will climb) or is still dead
|
|
|
|
|
//--- (repair failed), holding is right; Train() reports the hold every minute and the era-barrier
|
|
|
|
|
//--- liveness escape releases the rest of the ensemble if it never resolves.
|
|
|
|
|
return 0;
|
feat(depth): prime -> settle -> sweep, and name which handle is short
"Max bars in chart" is set to Unlimited, so the static-terminal-limit reading in
1dda479 was wrong. Two other candidate causes are falsified too: the price series
is fully downloaded and flat (USDJPY 50,162 -> 50,163 over 80 minutes, i.e. one new
H4 bar), and the handles have been stable since 13:07 with zero windows for the 17
minutes after, so it is not download-in-progress and not handle churn. The MA period
tops out at 200 (ADIndicatorTuner MA_PERIOD_PRESETS) against ~50k bars, so it is not
indicator cost either.
What IS verified stays verified: CopyBuffer past the calculated depth fails outright
rather than short-reading, so the buffer holds nothing and every index reads
EMPTY_VALUE; m_MA is the only CiCustom whose block REJECTS on that (m_ADZigZag
neutral-fills, RSI/MACD/Ichimoku/ATR are built-ins); the wall is therefore feature 25
of every bar, exactly as the "24 of 832" stall lines said. And it is depth-correlated:
16k-bar charts train, 34k/50k get zero windows forever.
So the WHY is still open, and this fix does not depend on it. Per the user's protocol:
the request itself is the primer, so prime at full depth, then poll TunableBarsCalculated()
every 3s and hold the sweep until it stops changing (3 steady probes), then use whatever
it settled at. Bounded at 10 min, and a give-up is logged as a give-up so an abandoned
depth is never mistaken for a settled one. This supersedes 1dda479's clamp on the two
training paths, which snapshotted a value that may still have been climbing; the clamp
remains for the paths that cannot wait (inference/online/rescan/export, see 7e63a8b).
The load-bearing part is what does NOT happen while waiting: no sweep. A 50k-bar feature
scan starves the indicator threads the request just woke, which is how the failure
sustained itself for 40 minutes at a time - discard era, re-sweep, discard, which is the
0->100% oscillation on the panel.
Also adds IndicatorDepthReport(): per-handle BarsCalculated() on the priming, cap and
stall lines. The logs proved WHICH FEATURE died but never WHICH HANDLE was short, so the
cause had to be inferred - and was guessed wrong twice. The next occurrence reads it off.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-17 13:44:27 -04:00
|
|
|
}
|
|
|
|
|
uint now = GetTickCount();
|
|
|
|
|
//--- First shortfall: start the clock and let the priming request above do its work. Deliberately
|
|
|
|
|
//--- no sweep this call - a 50k-bar feature sweep is exactly what starves the indicator threads we
|
|
|
|
|
//--- are waiting on, which is how the old loop sustained itself for 40 minutes at a time.
|
|
|
|
|
if(m_depthSettleStart == 0)
|
|
|
|
|
{
|
|
|
|
|
m_depthSettleStart = now;
|
|
|
|
|
m_depthProbeTick = now;
|
|
|
|
|
m_depthProbeLast = servable;
|
|
|
|
|
m_depthProbeStable = 0;
|
|
|
|
|
PrintFormat("%s: PRIMING indicator history for the %s - %d of %d bars calculated so far. Holding"
|
|
|
|
|
" the sweep until the count stops rising (probe every %ds, needs %d steady probes,"
|
|
|
|
|
" gives up after %ds and uses whatever is there). Per-indicator depth:%s",
|
|
|
|
|
ID, context, servable, want, DEPTH_SETTLE_PROBE_MS / 1000,
|
|
|
|
|
DEPTH_SETTLE_STABLE_PROBES, DEPTH_SETTLE_TIMEOUT_MS / 1000, IndicatorDepthReport());
|
|
|
|
|
return 0;
|
|
|
|
|
}
|
|
|
|
|
//--- Unsigned subtraction, so this is correct across GetTickCount()'s 49-day wrap (same idiom as
|
|
|
|
|
//--- m_coldSweepTick's backoff).
|
|
|
|
|
if(now - m_depthProbeTick < DEPTH_SETTLE_PROBE_MS)
|
|
|
|
|
return 0;
|
|
|
|
|
m_depthProbeTick = now;
|
|
|
|
|
if(servable != m_depthProbeLast)
|
|
|
|
|
{
|
|
|
|
|
//--- STILL MOVING. Growing is the terminal working through the history; shrinking happens when a
|
|
|
|
|
//--- handle is rebuilt under us and starts over. Either way it is not settled, so the streak
|
|
|
|
|
//--- restarts rather than counting a change as a steady observation.
|
|
|
|
|
PrintFormat("%s: priming %s - %d of %d bars (was %d), still moving", ID, context, servable, want,
|
|
|
|
|
m_depthProbeLast);
|
|
|
|
|
m_depthProbeLast = servable;
|
|
|
|
|
m_depthProbeStable = 0;
|
|
|
|
|
return 0;
|
|
|
|
|
}
|
|
|
|
|
m_depthProbeStable++;
|
|
|
|
|
bool steady = (m_depthProbeStable >= DEPTH_SETTLE_STABLE_PROBES);
|
|
|
|
|
bool expired = ((now - m_depthSettleStart) >= DEPTH_SETTLE_TIMEOUT_MS);
|
|
|
|
|
if(!steady && !expired)
|
|
|
|
|
return 0;
|
|
|
|
|
//--- Settled (or waited long enough) BELOW what was asked. This is the real depth, not a snapshot of
|
|
|
|
|
//--- a value still climbing, so it is now safe to clamp to it and get on with training.
|
|
|
|
|
PrintFormat("%s: priming %s DONE - depth settled at %d of %d bars after %ds%s. Training proceeds on"
|
|
|
|
|
" the %d bars the indicators can actually serve.",
|
|
|
|
|
ID, context, servable, want, (int)((now - m_depthSettleStart) / 1000),
|
|
|
|
|
expired && !steady ? " (gave up waiting - it never went steady)" : "", servable);
|
|
|
|
|
m_depthSettleStart = 0;
|
|
|
|
|
m_depthProbeStable = 0;
|
|
|
|
|
m_depthProbeLast = 0;
|
|
|
|
|
return ServableBars(want, context);
|
|
|
|
|
}
|
|
|
|
|
//+------------------------------------------------------------------+
|
feat(gates): derive the recall floor instead of configuring it, publish what the configuration can PROVE, and stop an arbitrary member driving live exits
Three changes, all from the same principle: measure what is there before aiming
at it, and never certify a number you do not trade.
1. THE RECALL FLOOR IS DERIVED, AND IT MOVES BELOW CHANCE.
MinRecall=40 was a constant doing a statistical job. Its reference point is the
33.3% recall a zero-skill 3-class model gets on EVERY class, and against that the
constant was accidentally calibrated for exactly one sample size: on USDJPY CONV
(n_eff 195) 40% is chance + 2.0 SE; on SP500 PAI (n_eff 42) the same 40% is chance
+ 0.9 SE. One chart was being held to a bar twice as strict as the other, for no
reason anyone chose.
CollapseRecallFloorPct() computes it per class from that class's own effective
sample - EffectiveSampleSize(), so the overlap deflation the rest of the gates use
applies here too - as chance - EDGE_MIN_SIGMAS x SE. 26.5% at n_eff 195, 18.8% at
n_eff 42.
BELOW chance, deliberately, and this is the substantive change rather than the
arithmetic. This gate's only job is refusing to call a COLLAPSED model converged.
It is not a quality bar; the deploy gate is the quality bar and it is already
rigorous (chance + 2 SE on the deflated sample, Sidak over candidate eras, then the
cross-instrument pooled certificate). A convergence gate that ALSO demands
provably-above-chance recall on all three classes double-counts that job, and it
has failed that way twice here: MinRecall=60 blocked every SP500 H1 run in 2026-07,
and the 40 that replaced it made Neutral structurally unreachable once first-touch
resolution cut Neutral to a 0.65% residue. A floor nothing can reach does not make
a funded account safer, it stops the run converging at all.
Testing significantly BELOW chance instead catches what a fixed 40 was actually
catching - a model that has stopped emitting a class - and cannot become
unreachable by construction. It also fixes the direction the old constant scaled:
it now widens on a thin OOS window, where low recall genuinely cannot be told from
noise, and tightens on a rich one. Today's SP500 PAI (Buy 51 / Sell 18 / Neutral 30)
is still correctly blocked on Sell.
This also resolves a standing contradiction the code half-admitted at the
isBetterEra comment: selection ranks on coverage-weighted PRECISION while
convergence gated on RECALL, so a sparse high-precision abstainer - precisely the
model that could clear the deploy bar - was blocked by the floor.
The era line now PRINTS the derived floor. Anyone comparing these recalls against a
remembered "40" is reading the wrong bar.
2. DETECTABILITY: WHAT THIS CONFIGURATION COULD PROVE, BEFORE IT TRAINS.
The DEPLOY BAR line states the bar. It never said what reaching it would take, and
that is the actionable direction. ReportDetectability() inverts the same identity -
the gate passes when edge >= z x sqrt(p(1-p)/n_eff), so certifying an edge d needs
n_eff >= z^2 p(1-p)/d^2 independent calls, hence L times as many raw ones - and
prints a +2 / +5 / +10pp ladder as required independent calls, raw calls, and share
of the OOS window, marking any rung that needs more than the window holds IMPOSSIBLE.
Every term is a property of the CONFIGURATION - geometry via break-even, horizon via
mean label lifespan, window via oosCutoff - so no amount of training moves any of
them. It fires once, at the first healthy sweep, beside ReportFeatureHealth, for the
same reason: that is the first moment the bar grid, the measured geometry and the
lifespan are real numbers rather than defaults. It gates nothing.
This is the EdgeFinder discipline applied to our own gate: establish what the market
and the measurement design have to offer, then point the net at it - rather than
spending a thousand eras chasing something this OOS window could never certify.
3. AN ARBITRARY MEMBER WAS DRIVING LIVE EXITS AND TRAILING (user-identified).
Every ensemble member ran
g_LiveAISignedConfidence = SignedAIConfidence();
unconditionally, every tick. Last writer wins. Its consumers are the AI early-exit
route (CExpertSignalCustom::LiveSignedConfidence) and TrailingIntelligent - so on a
four-model chart an LSTM entry could be closed, and its stop moved, on the
Perceptron's opinion alone, decided by scheduling order. Not the vote, not a
weighted blend.
Now the mean across registered members, matching how the ensemble actually trades:
the open decision is the weighted-average vote, and an abstaining member contributes
0 and dilutes exactly as it does there. Members still training read 0, so a
half-trained ensemble reads WEAKER rather than louder - the safe direction for an
exit trigger. Deployed and paused members are included, which is the opposite of the
era barrier's exemption rule and correct for the opposite reason: that one asks who
must be waited for, this asks who has an opinion.
Latent today and staying that way for now by choice - Min_Vote_Close ships Disabled
(101, unreachable on both scales it drives) and TrailingStrategy is off, so live
exits are SL/TP only and the certified hold-to-barrier win rate is what actually
gets traded. Fixed now precisely because the plan is to enable vote exits once the
models are accurate, at which point a scheduling-order exit would be both harmful
and very hard to see.
STILL OPEN, and needs a decision before vote exits go on: the member gate and the
ensemble vote gate both grade hold-to-barrier, so enabling vote exits makes the
certified number stop describing the traded one. Warrior_EA.mq5 currently argues
barrier models may keep vote exits because "their label IS the vote's own horizon" -
that does not hold, since a vote flip at bar 5 of a 64-bar horizon is not the
target-before-stop outcome the gate measured. Either grade the OOS call on the real
exit rule (first of SL / TP / vote-flip / horizon) through the fill engine, or set
HoldToBarrier for ensemble members so the policy cannot drift from the certificate.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-17 16:15:08 -04:00
|
|
|
//| See the declaration. What this configuration would have to FIRE |
|
|
|
|
|
//| before any edge of a given size becomes certifiable. |
|
|
|
|
|
//+------------------------------------------------------------------+
|
|
|
|
|
void CExpertSignalAIBase::ReportDetectability(int oosBars)
|
|
|
|
|
{
|
|
|
|
|
if(m_detectabilityReported || oosBars <= 0)
|
|
|
|
|
return;
|
|
|
|
|
m_detectabilityReported = true;
|
|
|
|
|
double p = CostAdjustedBreakEvenPct() / 100.0;
|
|
|
|
|
if(p <= 0.0 || p >= 1.0)
|
|
|
|
|
return;
|
|
|
|
|
double L = MeanLabelLifespan();
|
|
|
|
|
//--- Invert the deploy gate. It passes when edge >= z * sqrt(p(1-p)/n_eff), so for a hypothesised
|
|
|
|
|
//--- true edge d it needs n_eff >= z^2 p(1-p) / d^2 independent calls - and since overlapping
|
|
|
|
|
//--- labels are worth ~1/L each, that is L times as many RAW calls. Everything on the right-hand
|
|
|
|
|
//--- side is a property of the CONFIGURATION (geometry via p, horizon via L, window via oosBars),
|
|
|
|
|
//--- not of the model, which is the whole point: no amount of training moves it.
|
|
|
|
|
string ladder = "";
|
|
|
|
|
double edges[3] = {2.0, 5.0, 10.0};
|
|
|
|
|
for(int i = 0; i < 3; i++)
|
|
|
|
|
{
|
|
|
|
|
double d = edges[i] / 100.0;
|
|
|
|
|
double needEff = (EDGE_MIN_SIGMAS * EDGE_MIN_SIGMAS) * p * (1.0 - p) / (d * d);
|
|
|
|
|
double needRaw = needEff * L;
|
|
|
|
|
double needCoverage = 100.0 * needRaw / (double)oosBars;
|
|
|
|
|
ladder += StringFormat(" %+.0fpp:%.0f indep=%.0f calls=%.0f%% of window%s |",
|
|
|
|
|
edges[i], needEff, needRaw, needCoverage,
|
|
|
|
|
needCoverage > 100.0 ? " IMPOSSIBLE" : "");
|
|
|
|
|
}
|
|
|
|
|
PrintFormat("%s: DETECTABILITY of this configuration (break-even %.1f%%, mean label lifespan %.1f"
|
|
|
|
|
" bars, OOS window %d bars) - to certify an edge of X the gate needs:%s"
|
|
|
|
|
" Read it as a budget, not a target: these are properties of the GEOMETRY, the HORIZON"
|
|
|
|
|
" and the WINDOW, so a better model cannot change any of them. Where a rung says"
|
|
|
|
|
" IMPOSSIBLE, no win rate this model could ever produce would clear the deploy bar on"
|
|
|
|
|
" this window - the answer there is more instruments, a lower timeframe or a narrower"
|
|
|
|
|
" barrier, never more eras. Coverage is also not free in the other direction: firing on"
|
|
|
|
|
" more bars buys independent calls at the cost of precision, so the reachable band is"
|
|
|
|
|
" bounded at both ends.",
|
|
|
|
|
ID, 100.0 * p, L, oosBars, ladder);
|
|
|
|
|
}
|
|
|
|
|
//+------------------------------------------------------------------+
|
|
|
|
|
//| See the declaration. The per-class COLLAPSE floor, derived rather |
|
|
|
|
|
//| than configured. |
|
|
|
|
|
//+------------------------------------------------------------------+
|
|
|
|
|
double CExpertSignalAIBase::CollapseRecallFloorPct(int classTrueCount)
|
|
|
|
|
{
|
|
|
|
|
//--- Chance recall is 1/K for K classes and does not depend on the class priors: a zero-skill model
|
|
|
|
|
//--- that emits class c with probability q gets recall q on EVERY true class, and the uniform
|
|
|
|
|
//--- zero-skill model has q = 1/K. That, not a configured constant, is the reference this floor is
|
|
|
|
|
//--- measured against.
|
|
|
|
|
double chance = 100.0 / 3.0;
|
|
|
|
|
double effN = EffectiveSampleSize((double)classTrueCount);
|
|
|
|
|
if(effN <= 0.0)
|
|
|
|
|
return (double)m_minDirectionalRecallPct;
|
|
|
|
|
double q = 1.0 / 3.0;
|
|
|
|
|
double se = 100.0 * MathSqrt(q * (1.0 - q) / effN);
|
|
|
|
|
double floorPct = chance - EDGE_MIN_SIGMAS * se;
|
|
|
|
|
//--- Never negative, and never so high it becomes the unreachable bar this replaced.
|
|
|
|
|
if(floorPct < 0.0)
|
|
|
|
|
floorPct = 0.0;
|
|
|
|
|
return floorPct;
|
|
|
|
|
}
|
|
|
|
|
//+------------------------------------------------------------------+
|
fix(indicators+panel): the dead handle is MEASURED now - recreate it; and order the ensemble panel by member, not by who published first
THE ANSWER, off the instrumentation added in be39674, first run:
ConvLSTM [HYB-2484]: TUNABLE INDICATOR REPORTS NO CALCULATED BARS - 1 tunable
indicator(s) enabled and the least-ready answers BarsCalculated()=-1 ...
Per-indicator depth: price=33982 MA=-1 ZigZag=33982 ATR=33982
MA=-1 with price, ZigZag and ATR all at full depth. **The handle is INVALID, not
short.** Same line on USDJPY (price=50179 MA=-1). Depth was never the problem;
the previous session's five theories were all answering the wrong question.
And it is per-member, not per-chart: LSTM-2484 ran the 34-candidate auto-tune on
that same chart at 15:24:18 and went on to train normally (feature health, 51
features, excursion head) reading the same indicator. Only ConvLSTM's handle -
the last member constructed - was dead. WHY is still not established. All four
members request ADMovingAverage with identical params, so MT5 hands them the SAME
refcounted handle, and the tuner's inner loop is Create-then-IndicatorRelease over
exactly that shared handle; that is the obvious suspect and it is NOT yet proven,
so this commit does not act on it.
1. IndicatorDepthReport() NOW PRINTS HANDLE NUMBERS, not just depths.
"MA=-1" says the handle is dead. "MA=-1(h12)" against another member's "MA=33982
(h12)" says it is the SAME handle and someone released it; "(h-1)" says it was
never created. That is the difference between a refcount bug and a creation
failure and it is one field. This is the measurement the shared-handle suspicion
needs before anyone acts on it.
2. RECREATE A DEAD HANDLE INSTEAD OF SWEEPING AGAINST IT.
A member that cannot read its own indicator must rebuild it. RepairDeadIndicatorHandles()
re-Creates only the ENABLED tunables reporting BarsCalculated() < 0 - a merely COLD
indicator (valid handle, 0 bars) is left alone to warm up the normal way. It does
NOT release first: -1 means the terminal no longer knows the handle, so there is
nothing to give back, and MT5 recycles handle VALUES so releasing a stale one could
decrement whatever now owns that number. 30s cooldown, because every ServableBars()
consumer reaches it including live inference on every tick. The feature cache is
dropped with it, and the log names before/after depths.
Cause-agnostic on purpose. Whatever is killing the handle, sweeping 50,163 bars
against a buffer that answers EMPTY_VALUE at every index - then discarding the era
and doing it again - is not a recovery.
3. THE SWEEP NOW HOLDS ON A DEAD HANDLE.
ServableBars() keeps answering `want` (its contract; live inference and online
learning have their own refusal paths and a 0 there reads as "no history at all").
SettledBars() - the training sweep's entry, the one caller that can afford to wait -
returns 0 instead, so Train() holds and reports rather than burning a full-history
pass it is guaranteed to throw away. A recreated handle is cold, so it primes
through the existing settle path on the next call. If the repair fails the member
holds indefinitely and says so every minute, and be39674's barrier liveness escape
releases the rest of the ensemble after 12 minutes - which is the correct
degradation and is exactly what the log shows happening.
4. THE PANEL ROWS WERE ORDERED BY WHO PUBLISHED FIRST.
Reported on XAUUSD: LSTM, ConvLSTM, Perceptron, Convolutional instead of
Perceptron, Convolutional, LSTM, ConvLSTM. ClaimEnsemblePanelSlot() handed out the
next free row on each member's FIRST PublishStatus() call, so the order was a race -
the members busy sweeping published before the ones sitting idle at the era barrier,
and be39674 sharpened it by (correctly) making a held member stop writing the terse
line. Rows are now keyed to m_ensembleIndex, the registration/construction order,
which is fixed for the life of the chart. Claimed on every publish rather than once,
so it is idempotent and refreshes the tag for a member whose ID was not final when it
first published (the config-tag suffix is appended during InitIndicators, after
EnsembleMember() registers). Unclaimed rows are skipped by the render and excluded
from the model count, so a member that has not published yet leaves no gap and shifts
nobody.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-17 15:40:00 -04:00
|
|
|
//| RE-CREATE any enabled tunable indicator whose handle has gone |
|
|
|
|
|
//| INVALID underneath us. See the declaration for the evidence. |
|
|
|
|
|
//+------------------------------------------------------------------+
|
|
|
|
|
bool CExpertSignalAIBase::RepairDeadIndicatorHandles(void)
|
|
|
|
|
{
|
|
|
|
|
if(m_indicatorsPtr == NULL)
|
|
|
|
|
return false;
|
|
|
|
|
uint now = GetTickCount();
|
|
|
|
|
//--- Cooldown, because every consumer of ServableBars() can reach this - the training sweep, live
|
|
|
|
|
//--- inference on every tick, online learning - and a repair storm against a terminal that is
|
|
|
|
|
//--- genuinely refusing to create the indicator would be worse than the outage it is fixing.
|
|
|
|
|
if(m_handleRepairTick != 0 && now - m_handleRepairTick < HANDLE_REPAIR_COOLDOWN_MS)
|
|
|
|
|
return false;
|
|
|
|
|
m_handleRepairTick = now;
|
|
|
|
|
//--- NOT released first, deliberately. BarsCalculated() answering -1 means the terminal no longer
|
|
|
|
|
//--- knows this handle, so there is nothing to give back - and MT5 recycles handle VALUES, so
|
|
|
|
|
//--- releasing a stale one risks decrementing whatever now owns that number. Re-Create only.
|
|
|
|
|
//--- addToCollection is false throughout: the CiCustom/CiMA objects are already in m_indicatorsPtr
|
|
|
|
|
//--- from InitIndicators(), and adding them twice would let the collection delete them twice.
|
|
|
|
|
int repaired = 0;
|
|
|
|
|
string before = IndicatorDepthReport();
|
|
|
|
|
if(m_useMA && m_MA.BarsCalculated() < 0)
|
|
|
|
|
repaired += (InitMA(m_indicatorsPtr, false) ? 1 : 0);
|
|
|
|
|
if(m_useRSI && m_RSI.BarsCalculated() < 0)
|
|
|
|
|
repaired += (InitRSI(m_indicatorsPtr, false) ? 1 : 0);
|
|
|
|
|
if(m_useMACD && m_MACDFeature.BarsCalculated() < 0)
|
|
|
|
|
repaired += (InitMACDFeature(m_indicatorsPtr, false) ? 1 : 0);
|
|
|
|
|
if(m_useIchimoku && m_Ichimoku.BarsCalculated() < 0)
|
|
|
|
|
repaired += (InitIchimoku(m_indicatorsPtr, false) ? 1 : 0);
|
|
|
|
|
if(m_useADCumulativeDelta && m_ADCumulativeDelta.BarsCalculated() < 0)
|
|
|
|
|
repaired += (InitADCumulativeDelta(m_indicatorsPtr, false) ? 1 : 0);
|
|
|
|
|
if(m_useADShorteningOfThrust && m_ADShorteningOfThrust.BarsCalculated() < 0)
|
|
|
|
|
repaired += (InitADShorteningOfThrust(m_indicatorsPtr, false) ? 1 : 0);
|
|
|
|
|
if(m_useADWyckoffEventStream && m_ADWyckoffEventStream.BarsCalculated() < 0)
|
|
|
|
|
repaired += (InitADWyckoffEventStream(m_indicatorsPtr, false) ? 1 : 0);
|
|
|
|
|
if(m_useADWyckoffFailedStructure && m_ADWyckoffFailedStructure.BarsCalculated() < 0)
|
|
|
|
|
repaired += (InitADWyckoffFailedStructure(m_indicatorsPtr, false) ? 1 : 0);
|
|
|
|
|
if(m_useADWyckoffSignificantBarInversion && m_ADWyckoffSignificantBarInversion.BarsCalculated() < 0)
|
|
|
|
|
repaired += (InitADWyckoffSignificantBarInversion(m_indicatorsPtr, false) ? 1 : 0);
|
|
|
|
|
if(repaired == 0)
|
|
|
|
|
return false;
|
|
|
|
|
//--- Every cached feature row was computed against the handle that just got replaced. The values
|
|
|
|
|
//--- themselves are identical (same params, same series) so this is belt-and-braces, but a cache
|
|
|
|
|
//--- keyed to a handle that no longer exists is exactly the kind of thing that survives a fix and
|
|
|
|
|
//--- resurfaces a week later.
|
|
|
|
|
ArrayInitialize(m_featureCacheHasValue, false);
|
|
|
|
|
PrintFormat("%s: RECREATED %d dead indicator handle(s) - the terminal had stopped recognising them,"
|
|
|
|
|
" so CopyBuffer failed at every index and every bar of the sweep was rejected. Depth"
|
|
|
|
|
" BEFORE:%s | AFTER:%s. A freshly created handle calculates asynchronously, so the next"
|
|
|
|
|
" few calls may still report a short depth - that is the normal priming path, not this"
|
|
|
|
|
" fault. If this line repeats on a cycle, something is releasing the handle out from"
|
|
|
|
|
" under this member and the recreate is only papering over it.",
|
|
|
|
|
ID, repaired, before, IndicatorDepthReport());
|
|
|
|
|
return true;
|
|
|
|
|
}
|
|
|
|
|
//+------------------------------------------------------------------+
|
feat(depth): prime -> settle -> sweep, and name which handle is short
"Max bars in chart" is set to Unlimited, so the static-terminal-limit reading in
1dda479 was wrong. Two other candidate causes are falsified too: the price series
is fully downloaded and flat (USDJPY 50,162 -> 50,163 over 80 minutes, i.e. one new
H4 bar), and the handles have been stable since 13:07 with zero windows for the 17
minutes after, so it is not download-in-progress and not handle churn. The MA period
tops out at 200 (ADIndicatorTuner MA_PERIOD_PRESETS) against ~50k bars, so it is not
indicator cost either.
What IS verified stays verified: CopyBuffer past the calculated depth fails outright
rather than short-reading, so the buffer holds nothing and every index reads
EMPTY_VALUE; m_MA is the only CiCustom whose block REJECTS on that (m_ADZigZag
neutral-fills, RSI/MACD/Ichimoku/ATR are built-ins); the wall is therefore feature 25
of every bar, exactly as the "24 of 832" stall lines said. And it is depth-correlated:
16k-bar charts train, 34k/50k get zero windows forever.
So the WHY is still open, and this fix does not depend on it. Per the user's protocol:
the request itself is the primer, so prime at full depth, then poll TunableBarsCalculated()
every 3s and hold the sweep until it stops changing (3 steady probes), then use whatever
it settled at. Bounded at 10 min, and a give-up is logged as a give-up so an abandoned
depth is never mistaken for a settled one. This supersedes 1dda479's clamp on the two
training paths, which snapshotted a value that may still have been climbing; the clamp
remains for the paths that cannot wait (inference/online/rescan/export, see 7e63a8b).
The load-bearing part is what does NOT happen while waiting: no sweep. A 50k-bar feature
scan starves the indicator threads the request just woke, which is how the failure
sustained itself for 40 minutes at a time - discard era, re-sweep, discard, which is the
0->100% oscillation on the panel.
Also adds IndicatorDepthReport(): per-handle BarsCalculated() on the priming, cap and
stall lines. The logs proved WHICH FEATURE died but never WHICH HANDLE was short, so the
cause had to be inferred - and was guessed wrong twice. The next occurrence reads it off.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-17 13:44:27 -04:00
|
|
|
//| See the declaration. Names WHICH indicator is short, so the next |
|
|
|
|
|
//| occurrence is read off the log instead of inferred. |
|
|
|
|
|
//+------------------------------------------------------------------+
|
|
|
|
|
string CExpertSignalAIBase::IndicatorDepthReport(void)
|
|
|
|
|
{
|
|
|
|
|
string s = StringFormat(" price=%d", Bars(m_symbol.Name(), PERIOD_CURRENT));
|
fix(indicators+panel): the dead handle is MEASURED now - recreate it; and order the ensemble panel by member, not by who published first
THE ANSWER, off the instrumentation added in be39674, first run:
ConvLSTM [HYB-2484]: TUNABLE INDICATOR REPORTS NO CALCULATED BARS - 1 tunable
indicator(s) enabled and the least-ready answers BarsCalculated()=-1 ...
Per-indicator depth: price=33982 MA=-1 ZigZag=33982 ATR=33982
MA=-1 with price, ZigZag and ATR all at full depth. **The handle is INVALID, not
short.** Same line on USDJPY (price=50179 MA=-1). Depth was never the problem;
the previous session's five theories were all answering the wrong question.
And it is per-member, not per-chart: LSTM-2484 ran the 34-candidate auto-tune on
that same chart at 15:24:18 and went on to train normally (feature health, 51
features, excursion head) reading the same indicator. Only ConvLSTM's handle -
the last member constructed - was dead. WHY is still not established. All four
members request ADMovingAverage with identical params, so MT5 hands them the SAME
refcounted handle, and the tuner's inner loop is Create-then-IndicatorRelease over
exactly that shared handle; that is the obvious suspect and it is NOT yet proven,
so this commit does not act on it.
1. IndicatorDepthReport() NOW PRINTS HANDLE NUMBERS, not just depths.
"MA=-1" says the handle is dead. "MA=-1(h12)" against another member's "MA=33982
(h12)" says it is the SAME handle and someone released it; "(h-1)" says it was
never created. That is the difference between a refcount bug and a creation
failure and it is one field. This is the measurement the shared-handle suspicion
needs before anyone acts on it.
2. RECREATE A DEAD HANDLE INSTEAD OF SWEEPING AGAINST IT.
A member that cannot read its own indicator must rebuild it. RepairDeadIndicatorHandles()
re-Creates only the ENABLED tunables reporting BarsCalculated() < 0 - a merely COLD
indicator (valid handle, 0 bars) is left alone to warm up the normal way. It does
NOT release first: -1 means the terminal no longer knows the handle, so there is
nothing to give back, and MT5 recycles handle VALUES so releasing a stale one could
decrement whatever now owns that number. 30s cooldown, because every ServableBars()
consumer reaches it including live inference on every tick. The feature cache is
dropped with it, and the log names before/after depths.
Cause-agnostic on purpose. Whatever is killing the handle, sweeping 50,163 bars
against a buffer that answers EMPTY_VALUE at every index - then discarding the era
and doing it again - is not a recovery.
3. THE SWEEP NOW HOLDS ON A DEAD HANDLE.
ServableBars() keeps answering `want` (its contract; live inference and online
learning have their own refusal paths and a 0 there reads as "no history at all").
SettledBars() - the training sweep's entry, the one caller that can afford to wait -
returns 0 instead, so Train() holds and reports rather than burning a full-history
pass it is guaranteed to throw away. A recreated handle is cold, so it primes
through the existing settle path on the next call. If the repair fails the member
holds indefinitely and says so every minute, and be39674's barrier liveness escape
releases the rest of the ensemble after 12 minutes - which is the correct
degradation and is exactly what the log shows happening.
4. THE PANEL ROWS WERE ORDERED BY WHO PUBLISHED FIRST.
Reported on XAUUSD: LSTM, ConvLSTM, Perceptron, Convolutional instead of
Perceptron, Convolutional, LSTM, ConvLSTM. ClaimEnsemblePanelSlot() handed out the
next free row on each member's FIRST PublishStatus() call, so the order was a race -
the members busy sweeping published before the ones sitting idle at the era barrier,
and be39674 sharpened it by (correctly) making a held member stop writing the terse
line. Rows are now keyed to m_ensembleIndex, the registration/construction order,
which is fixed for the life of the chart. Claimed on every publish rather than once,
so it is idempotent and refreshes the tag for a member whose ID was not final when it
first published (the config-tag suffix is appended during InitIndicators, after
EnsembleMember() registers). Unclaimed rows are skipped by the render and excluded
from the model count, so a member that has not published yet leaves no gap and shifts
nobody.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-17 15:40:00 -04:00
|
|
|
//--- HANDLE VALUES, not just depths. A depth of -1 says "this handle is dead"; the handle NUMBER is
|
|
|
|
|
//--- what says whether it was never created, or was created and later released out from under this
|
|
|
|
|
//--- member - and on an ensemble chart all four members request identical params, so MT5 hands them
|
|
|
|
|
//--- the SAME refcounted handle and one member's release is felt by the others. Comparing the number
|
|
|
|
|
//--- across members' log lines is what distinguishes those cases. Costs nothing to print.
|
feat(depth): prime -> settle -> sweep, and name which handle is short
"Max bars in chart" is set to Unlimited, so the static-terminal-limit reading in
1dda479 was wrong. Two other candidate causes are falsified too: the price series
is fully downloaded and flat (USDJPY 50,162 -> 50,163 over 80 minutes, i.e. one new
H4 bar), and the handles have been stable since 13:07 with zero windows for the 17
minutes after, so it is not download-in-progress and not handle churn. The MA period
tops out at 200 (ADIndicatorTuner MA_PERIOD_PRESETS) against ~50k bars, so it is not
indicator cost either.
What IS verified stays verified: CopyBuffer past the calculated depth fails outright
rather than short-reading, so the buffer holds nothing and every index reads
EMPTY_VALUE; m_MA is the only CiCustom whose block REJECTS on that (m_ADZigZag
neutral-fills, RSI/MACD/Ichimoku/ATR are built-ins); the wall is therefore feature 25
of every bar, exactly as the "24 of 832" stall lines said. And it is depth-correlated:
16k-bar charts train, 34k/50k get zero windows forever.
So the WHY is still open, and this fix does not depend on it. Per the user's protocol:
the request itself is the primer, so prime at full depth, then poll TunableBarsCalculated()
every 3s and hold the sweep until it stops changing (3 steady probes), then use whatever
it settled at. Bounded at 10 min, and a give-up is logged as a give-up so an abandoned
depth is never mistaken for a settled one. This supersedes 1dda479's clamp on the two
training paths, which snapshotted a value that may still have been climbing; the clamp
remains for the paths that cannot wait (inference/online/rescan/export, see 7e63a8b).
The load-bearing part is what does NOT happen while waiting: no sweep. A 50k-bar feature
scan starves the indicator threads the request just woke, which is how the failure
sustained itself for 40 minutes at a time - discard era, re-sweep, discard, which is the
0->100% oscillation on the panel.
Also adds IndicatorDepthReport(): per-handle BarsCalculated() on the priming, cap and
stall lines. The logs proved WHICH FEATURE died but never WHICH HANDLE was short, so the
cause had to be inferred - and was guessed wrong twice. The next occurrence reads it off.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-17 13:44:27 -04:00
|
|
|
if(m_useMA)
|
fix(indicators+panel): the dead handle is MEASURED now - recreate it; and order the ensemble panel by member, not by who published first
THE ANSWER, off the instrumentation added in be39674, first run:
ConvLSTM [HYB-2484]: TUNABLE INDICATOR REPORTS NO CALCULATED BARS - 1 tunable
indicator(s) enabled and the least-ready answers BarsCalculated()=-1 ...
Per-indicator depth: price=33982 MA=-1 ZigZag=33982 ATR=33982
MA=-1 with price, ZigZag and ATR all at full depth. **The handle is INVALID, not
short.** Same line on USDJPY (price=50179 MA=-1). Depth was never the problem;
the previous session's five theories were all answering the wrong question.
And it is per-member, not per-chart: LSTM-2484 ran the 34-candidate auto-tune on
that same chart at 15:24:18 and went on to train normally (feature health, 51
features, excursion head) reading the same indicator. Only ConvLSTM's handle -
the last member constructed - was dead. WHY is still not established. All four
members request ADMovingAverage with identical params, so MT5 hands them the SAME
refcounted handle, and the tuner's inner loop is Create-then-IndicatorRelease over
exactly that shared handle; that is the obvious suspect and it is NOT yet proven,
so this commit does not act on it.
1. IndicatorDepthReport() NOW PRINTS HANDLE NUMBERS, not just depths.
"MA=-1" says the handle is dead. "MA=-1(h12)" against another member's "MA=33982
(h12)" says it is the SAME handle and someone released it; "(h-1)" says it was
never created. That is the difference between a refcount bug and a creation
failure and it is one field. This is the measurement the shared-handle suspicion
needs before anyone acts on it.
2. RECREATE A DEAD HANDLE INSTEAD OF SWEEPING AGAINST IT.
A member that cannot read its own indicator must rebuild it. RepairDeadIndicatorHandles()
re-Creates only the ENABLED tunables reporting BarsCalculated() < 0 - a merely COLD
indicator (valid handle, 0 bars) is left alone to warm up the normal way. It does
NOT release first: -1 means the terminal no longer knows the handle, so there is
nothing to give back, and MT5 recycles handle VALUES so releasing a stale one could
decrement whatever now owns that number. 30s cooldown, because every ServableBars()
consumer reaches it including live inference on every tick. The feature cache is
dropped with it, and the log names before/after depths.
Cause-agnostic on purpose. Whatever is killing the handle, sweeping 50,163 bars
against a buffer that answers EMPTY_VALUE at every index - then discarding the era
and doing it again - is not a recovery.
3. THE SWEEP NOW HOLDS ON A DEAD HANDLE.
ServableBars() keeps answering `want` (its contract; live inference and online
learning have their own refusal paths and a 0 there reads as "no history at all").
SettledBars() - the training sweep's entry, the one caller that can afford to wait -
returns 0 instead, so Train() holds and reports rather than burning a full-history
pass it is guaranteed to throw away. A recreated handle is cold, so it primes
through the existing settle path on the next call. If the repair fails the member
holds indefinitely and says so every minute, and be39674's barrier liveness escape
releases the rest of the ensemble after 12 minutes - which is the correct
degradation and is exactly what the log shows happening.
4. THE PANEL ROWS WERE ORDERED BY WHO PUBLISHED FIRST.
Reported on XAUUSD: LSTM, ConvLSTM, Perceptron, Convolutional instead of
Perceptron, Convolutional, LSTM, ConvLSTM. ClaimEnsemblePanelSlot() handed out the
next free row on each member's FIRST PublishStatus() call, so the order was a race -
the members busy sweeping published before the ones sitting idle at the era barrier,
and be39674 sharpened it by (correctly) making a held member stop writing the terse
line. Rows are now keyed to m_ensembleIndex, the registration/construction order,
which is fixed for the life of the chart. Claimed on every publish rather than once,
so it is idempotent and refreshes the tag for a member whose ID was not final when it
first published (the config-tag suffix is appended during InitIndicators, after
EnsembleMember() registers). Unclaimed rows are skipped by the render and excluded
from the model count, so a member that has not published yet leaves no gap and shifts
nobody.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-17 15:40:00 -04:00
|
|
|
s += StringFormat(" MA=%d(h%d)", m_MA.BarsCalculated(), m_MA.Handle());
|
feat(depth): prime -> settle -> sweep, and name which handle is short
"Max bars in chart" is set to Unlimited, so the static-terminal-limit reading in
1dda479 was wrong. Two other candidate causes are falsified too: the price series
is fully downloaded and flat (USDJPY 50,162 -> 50,163 over 80 minutes, i.e. one new
H4 bar), and the handles have been stable since 13:07 with zero windows for the 17
minutes after, so it is not download-in-progress and not handle churn. The MA period
tops out at 200 (ADIndicatorTuner MA_PERIOD_PRESETS) against ~50k bars, so it is not
indicator cost either.
What IS verified stays verified: CopyBuffer past the calculated depth fails outright
rather than short-reading, so the buffer holds nothing and every index reads
EMPTY_VALUE; m_MA is the only CiCustom whose block REJECTS on that (m_ADZigZag
neutral-fills, RSI/MACD/Ichimoku/ATR are built-ins); the wall is therefore feature 25
of every bar, exactly as the "24 of 832" stall lines said. And it is depth-correlated:
16k-bar charts train, 34k/50k get zero windows forever.
So the WHY is still open, and this fix does not depend on it. Per the user's protocol:
the request itself is the primer, so prime at full depth, then poll TunableBarsCalculated()
every 3s and hold the sweep until it stops changing (3 steady probes), then use whatever
it settled at. Bounded at 10 min, and a give-up is logged as a give-up so an abandoned
depth is never mistaken for a settled one. This supersedes 1dda479's clamp on the two
training paths, which snapshotted a value that may still have been climbing; the clamp
remains for the paths that cannot wait (inference/online/rescan/export, see 7e63a8b).
The load-bearing part is what does NOT happen while waiting: no sweep. A 50k-bar feature
scan starves the indicator threads the request just woke, which is how the failure
sustained itself for 40 minutes at a time - discard era, re-sweep, discard, which is the
0->100% oscillation on the panel.
Also adds IndicatorDepthReport(): per-handle BarsCalculated() on the priming, cap and
stall lines. The logs proved WHICH FEATURE died but never WHICH HANDLE was short, so the
cause had to be inferred - and was guessed wrong twice. The next occurrence reads it off.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-17 13:44:27 -04:00
|
|
|
if(m_useRSI)
|
|
|
|
|
s += StringFormat(" RSI=%d", m_RSI.BarsCalculated());
|
|
|
|
|
if(m_useMACD)
|
|
|
|
|
s += StringFormat(" MACD=%d", m_MACDFeature.BarsCalculated());
|
|
|
|
|
if(m_useIchimoku)
|
|
|
|
|
s += StringFormat(" Ichi=%d", m_Ichimoku.BarsCalculated());
|
|
|
|
|
if(m_useADCumulativeDelta)
|
|
|
|
|
s += StringFormat(" CumDelta=%d", m_ADCumulativeDelta.BarsCalculated());
|
|
|
|
|
if(m_useADShorteningOfThrust)
|
|
|
|
|
s += StringFormat(" SoT=%d", m_ADShorteningOfThrust.BarsCalculated());
|
|
|
|
|
if(m_useADWyckoffEventStream)
|
|
|
|
|
s += StringFormat(" WES=%d", m_ADWyckoffEventStream.BarsCalculated());
|
|
|
|
|
if(m_useADWyckoffFailedStructure)
|
|
|
|
|
s += StringFormat(" WFS=%d", m_ADWyckoffFailedStructure.BarsCalculated());
|
|
|
|
|
if(m_useADWyckoffSignificantBarInversion)
|
|
|
|
|
s += StringFormat(" WSBI=%d", m_ADWyckoffSignificantBarInversion.BarsCalculated());
|
|
|
|
|
//--- Not tunable, so absent from TunableBarsCalculated() - but the swing block reads it on every bar
|
|
|
|
|
//--- and neutral-fills when it is short, which is silent. Worth seeing next to the others.
|
|
|
|
|
s += StringFormat(" ZigZag=%d ATR=%d", m_ADZigZag.BarsCalculated(), m_ATR.BarsCalculated());
|
|
|
|
|
return s;
|
|
|
|
|
}
|
|
|
|
|
//+------------------------------------------------------------------+
|
2026-08-13 10:23:11 -04:00
|
|
|
//+------------------------------------------------------------------+
|
|
|
|
|
//| Adopt a saved indicator-param set, rebuilding handles only on a |
|
|
|
|
|
//| REAL change. |
|
|
|
|
|
//| |
|
|
|
|
|
//| The resume path restores the params a model was trained with and |
|
|
|
|
|
//| used to call ReInitADIndicators unconditionally. In the common |
|
|
|
|
|
//| case the saved set is byte-identical to the values the indicators |
|
|
|
|
|
//| were created with a few hundred milliseconds earlier (the MI |
|
|
|
|
|
//| tuner usually keeps the configured settings), so the "rebuild" |
|
|
|
|
|
//| destroyed five working, already-calculating indicator instances |
|
|
|
|
|
//| to recreate them with the same inputs - at process start, with |
|
|
|
|
|
//| history still syncing. On a memory-starved box (2026-08-13: |
|
|
|
|
|
//| 1 GB free of 31) the replacements stayed cold for 6+ minutes and |
|
|
|
|
|
//| the resumed model could not train a single era. A no-change adopt |
|
|
|
|
|
//| now only aligns the tuner state and leaves the live instances |
|
|
|
|
|
//| alone. |
|
|
|
|
|
//+------------------------------------------------------------------+
|
|
|
|
|
bool CExpertSignalAIBase::AdoptIndicatorParams(const double &loaded[], CIndicators *indicators)
|
|
|
|
|
{
|
|
|
|
|
double current[];
|
|
|
|
|
m_indicatorTuner.Flatten(current);
|
|
|
|
|
bool changed = (ArraySize(current) != ArraySize(loaded));
|
|
|
|
|
if(!changed)
|
|
|
|
|
for(int k = 0; k < ArraySize(loaded); k++)
|
|
|
|
|
if(current[k] != loaded[k])
|
|
|
|
|
{
|
|
|
|
|
changed = true;
|
|
|
|
|
break;
|
|
|
|
|
}
|
|
|
|
|
//--- the tuner mirrors the model's params either way - it feeds the .nnw save and the fingerprint
|
|
|
|
|
m_indicatorTuner.Unflatten(loaded);
|
|
|
|
|
if(!changed)
|
|
|
|
|
{
|
|
|
|
|
PrintVerbose(ID + ": saved indicator params match the live indicators - keeping the existing"
|
|
|
|
|
" instances (no handle rebuild).");
|
|
|
|
|
return true;
|
|
|
|
|
}
|
|
|
|
|
Print(ID + ": saved indicator params differ from the live defaults - rebuilding the tunable"
|
|
|
|
|
" indicator handles to match the model they trained.");
|
|
|
|
|
return ReInitADIndicators(indicators);
|
|
|
|
|
}
|
|
|
|
|
//+------------------------------------------------------------------+
|
refactor: split CExpertSignalAIBase implementation by responsibility
ExpertSignalAIBase.mqh was 8216 lines: the class declaration followed by 87
method bodies covering training, labelling, feature extraction, persistence,
chart drawing, online learning, the GA auto-tuner and inference, all in one
file. Train() alone is 1492 lines; a change to arrow drawing meant scrolling
past the era loop.
Moved the bodies into Expert\AIBase\, included at the bottom of the original
after the class declaration:
Training.mqh 1607 era loop, plateau ladder, checkpoint select, deploy
Features.mqh 1093 indicator creation + per-bar input feature vector
ChartUI.mqh 634 arrows, arrow persistence, status panel, cleanup
Persistence.mqh 492 .stats/.cfg sidecars, CPU-inference validation, copy
OnlineLearning.mqh 461 live continual learning, EMA shadow, OOS simulator
Labels.mqh 309 ZigZag pivot labels, async label-cache prebuild
AutoTune.mqh 275 genetic tuner (population, crossover, halving)
Inference.mqh 235 softmax, prior calibration, class priors
ExpertSignalAIBase.mqh 8216 -> 3131 (declaration + topology build only)
This is a pure relocation - verified mechanically, not by eye: HEAD's file
reconstructed from the eight partials plus the surviving remainder is
byte-identical to HEAD, span for span (scratchpad verify_split.py). No
declaration moved, no signature changed, no code rewritten, so behaviour is
unchanged by construction.
Compiles 0 errors, 0 warnings.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-29 00:42:45 -04:00
|
|
|
bool CExpertSignalAIBase::ReInitADIndicators(CIndicators *indicators)
|
|
|
|
|
{
|
|
|
|
|
bool result = true;
|
fix: the indicator re-init leaked a terminal handle per candidate
This is what killed CONV and LSTM on 2026-08-07. Terminal journal:
19:19:40 6664 x "VirtualAlloc failed in large allocator"
19:19:40.829 expert Warrior_EA (SP500,H1) removed <- CONV
19:29:55 2048 x "VirtualAlloc failed in large allocator"
19:29:55.359 expert Warrior_EA (SP500,H1) removed <- LSTM
50ms and 71ms after each printed its "logit adjustment" line, i.e. the
instant era 0 tried to allocate its training queues. They did not hang -
MT5 shot them for running out of memory.
ReInitADIndicators() re-Create()s every enabled indicator and released
nothing. The comment above it asserted "CiCustom.Create() already
releases its old handle"; MQL5's CIndicator::Create is
m_handle = IndicatorCreate(symbol, period, type, num_params, params);
a plain overwrite, whose only success-path IndicatorRelease is in
~CIndicator. IndicatorRelease appeared nowhere in this codebase.
That function is the indicator tuner's inner loop. AutoTuneIndicators
scored 324 candidates per model on SP500 H1, so ~324 x 6 orphaned
terminal-side instances, each holding a full-history buffer set -
ADWyckoffEventStream is 14 buffers x ~38k bars x 8 bytes = ~4.3 MB each.
Gigabytes. Confirmed in the shutdown teardown, where a single surviving
expert still held 70 x ADWES, 53 x ADWyckoffEventStream, 23 x
WFS(48,3,1.80,1.10), 19 x ADMovingAverage, 18 x WYSB(162). Those
parameter sets are the tuner's candidate grids.
Release is UNCONDITIONAL, not gated on the handle having changed: MT5
refcounts instances by (symbol, period, params), so re-creating with
IDENTICAL params returns the SAME handle with the count incremented -
the "23 x WFS(48,3,1.80,1.10)" pattern. Either way Create() added one
reference and we hold one handle, so one release is owed.
Released AFTER the re-creates, never before: dropping the terminal's
last reference first would tear the instance down, so an identical-params
Create() would rebuild it from scratch instead of reusing the live one -
turning a refcount bump into a full recalculation over all history, 324
times over.
PAI was unaffected because it has no AD/Wyckoff indicators enabled (35
candidates, built-ins only). HYBRID survived on luck: CONV and LSTM died
2 and 12 minutes before its own sweep finished, freeing the memory.
Compiles clean: 0 errors, 0 warnings.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-08 15:42:03 -04:00
|
|
|
//--- RELEASE THE HANDLE EACH Create() IS ABOUT TO REPLACE. Without this every call here leaks one
|
|
|
|
|
//--- terminal-side indicator instance per enabled indicator, and this function is the tuner's inner
|
|
|
|
|
//--- loop - AutoTuneIndicators scored 324 candidates on SP500 H1, so ~324 x 6 orphaned instances per
|
|
|
|
|
//--- model, each holding a full-history buffer set (ADWyckoffEventStream is 14 buffers x ~38k bars x
|
|
|
|
|
//--- 8 bytes = ~4.3 MB EACH). That is gigabytes, and it is what killed CONV and LSTM on 2026-08-07:
|
|
|
|
|
//--- 6664 and 2048 "VirtualAlloc failed in large allocator" lines in the terminal journal, then
|
|
|
|
|
//--- "expert Warrior_EA (SP500,H1) removed", 50ms and 71ms after each finished its sweep and era 0
|
|
|
|
|
//--- tried to allocate. HYBRID only survived because those two died first and freed the memory.
|
|
|
|
|
//---
|
|
|
|
|
//--- THE COMMENT THAT USED TO SIT HERE SAID Create() "already releases its old handle". It does not.
|
|
|
|
|
//--- MQL5's CIndicator::Create (Include\Indicators\Indicator.mqh) is:
|
|
|
|
|
//--- m_handle = IndicatorCreate(symbol, period, type, num_params, params);
|
|
|
|
|
//--- - a plain overwrite. Its only success-path IndicatorRelease is in ~CIndicator. Nothing else in
|
|
|
|
|
//--- this codebase called IndicatorRelease at all.
|
|
|
|
|
//---
|
|
|
|
|
//--- UNCONDITIONAL, not "only when the handle changed". MT5 refcounts indicator instances by
|
|
|
|
|
//--- (symbol, period, params): re-creating with IDENTICAL params hands back the SAME handle with the
|
|
|
|
|
//--- count incremented, so skipping the release there would leak a reference just as surely - which is
|
|
|
|
|
//--- the "23 x WFS(48,3,1.80,1.10)" pattern in the journal, next to the distinct-parameter leaks from
|
|
|
|
|
//--- the candidate grid. Either way Create() added exactly one reference and we still hold exactly one
|
|
|
|
|
//--- handle, so exactly one release is owed.
|
|
|
|
|
int hCD = m_useADCumulativeDelta ? m_ADCumulativeDelta.Handle() : INVALID_HANDLE;
|
|
|
|
|
int hSOT = m_useADShorteningOfThrust ? m_ADShorteningOfThrust.Handle() : INVALID_HANDLE;
|
|
|
|
|
int hWES = m_useADWyckoffEventStream ? m_ADWyckoffEventStream.Handle() : INVALID_HANDLE;
|
|
|
|
|
int hWFS = m_useADWyckoffFailedStructure ? m_ADWyckoffFailedStructure.Handle() : INVALID_HANDLE;
|
|
|
|
|
int hWSBI = m_useADWyckoffSignificantBarInversion ? m_ADWyckoffSignificantBarInversion.Handle() : INVALID_HANDLE;
|
|
|
|
|
int hMA = m_useMA ? m_MA.Handle() : INVALID_HANDLE;
|
|
|
|
|
int hRSI = m_useRSI ? m_RSI.Handle() : INVALID_HANDLE;
|
|
|
|
|
int hMACD = m_useMACD ? m_MACDFeature.Handle() : INVALID_HANDLE;
|
|
|
|
|
int hIchi = m_useIchimoku ? m_Ichimoku.Handle() : INVALID_HANDLE;
|
refactor: split CExpertSignalAIBase implementation by responsibility
ExpertSignalAIBase.mqh was 8216 lines: the class declaration followed by 87
method bodies covering training, labelling, feature extraction, persistence,
chart drawing, online learning, the GA auto-tuner and inference, all in one
file. Train() alone is 1492 lines; a change to arrow drawing meant scrolling
past the era loop.
Moved the bodies into Expert\AIBase\, included at the bottom of the original
after the class declaration:
Training.mqh 1607 era loop, plateau ladder, checkpoint select, deploy
Features.mqh 1093 indicator creation + per-bar input feature vector
ChartUI.mqh 634 arrows, arrow persistence, status panel, cleanup
Persistence.mqh 492 .stats/.cfg sidecars, CPU-inference validation, copy
OnlineLearning.mqh 461 live continual learning, EMA shadow, OOS simulator
Labels.mqh 309 ZigZag pivot labels, async label-cache prebuild
AutoTune.mqh 275 genetic tuner (population, crossover, halving)
Inference.mqh 235 softmax, prior calibration, class priors
ExpertSignalAIBase.mqh 8216 -> 3131 (declaration + topology build only)
This is a pure relocation - verified mechanically, not by eye: HEAD's file
reconstructed from the eight partials plus the surviving remainder is
byte-identical to HEAD, span for span (scratchpad verify_split.py). No
declaration moved, no signature changed, no code rewritten, so behaviour is
unchanged by construction.
Compiles 0 errors, 0 warnings.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-29 00:42:45 -04:00
|
|
|
if(m_useADCumulativeDelta)
|
|
|
|
|
result = InitADCumulativeDelta(indicators, false) && result;
|
|
|
|
|
if(m_useADShorteningOfThrust)
|
|
|
|
|
result = InitADShorteningOfThrust(indicators, false) && result;
|
|
|
|
|
if(m_useADWyckoffEventStream)
|
|
|
|
|
result = InitADWyckoffEventStream(indicators, false) && result;
|
|
|
|
|
if(m_useADWyckoffFailedStructure)
|
|
|
|
|
result = InitADWyckoffFailedStructure(indicators, false) && result;
|
|
|
|
|
if(m_useADWyckoffSignificantBarInversion)
|
|
|
|
|
result = InitADWyckoffSignificantBarInversion(indicators, false) && result;
|
|
|
|
|
if(m_useMA)
|
|
|
|
|
result = InitMA(indicators, false) && result;
|
|
|
|
|
if(m_useRSI)
|
|
|
|
|
result = InitRSI(indicators, false) && result;
|
|
|
|
|
if(m_useMACD)
|
|
|
|
|
result = InitMACDFeature(indicators, false) && result;
|
|
|
|
|
if(m_useIchimoku)
|
|
|
|
|
result = InitIchimoku(indicators, false) && result;
|
fix: the indicator re-init leaked a terminal handle per candidate
This is what killed CONV and LSTM on 2026-08-07. Terminal journal:
19:19:40 6664 x "VirtualAlloc failed in large allocator"
19:19:40.829 expert Warrior_EA (SP500,H1) removed <- CONV
19:29:55 2048 x "VirtualAlloc failed in large allocator"
19:29:55.359 expert Warrior_EA (SP500,H1) removed <- LSTM
50ms and 71ms after each printed its "logit adjustment" line, i.e. the
instant era 0 tried to allocate its training queues. They did not hang -
MT5 shot them for running out of memory.
ReInitADIndicators() re-Create()s every enabled indicator and released
nothing. The comment above it asserted "CiCustom.Create() already
releases its old handle"; MQL5's CIndicator::Create is
m_handle = IndicatorCreate(symbol, period, type, num_params, params);
a plain overwrite, whose only success-path IndicatorRelease is in
~CIndicator. IndicatorRelease appeared nowhere in this codebase.
That function is the indicator tuner's inner loop. AutoTuneIndicators
scored 324 candidates per model on SP500 H1, so ~324 x 6 orphaned
terminal-side instances, each holding a full-history buffer set -
ADWyckoffEventStream is 14 buffers x ~38k bars x 8 bytes = ~4.3 MB each.
Gigabytes. Confirmed in the shutdown teardown, where a single surviving
expert still held 70 x ADWES, 53 x ADWyckoffEventStream, 23 x
WFS(48,3,1.80,1.10), 19 x ADMovingAverage, 18 x WYSB(162). Those
parameter sets are the tuner's candidate grids.
Release is UNCONDITIONAL, not gated on the handle having changed: MT5
refcounts instances by (symbol, period, params), so re-creating with
IDENTICAL params returns the SAME handle with the count incremented -
the "23 x WFS(48,3,1.80,1.10)" pattern. Either way Create() added one
reference and we hold one handle, so one release is owed.
Released AFTER the re-creates, never before: dropping the terminal's
last reference first would tear the instance down, so an identical-params
Create() would rebuild it from scratch instead of reusing the live one -
turning a refcount bump into a full recalculation over all history, 324
times over.
PAI was unaffected because it has no AD/Wyckoff indicators enabled (35
candidates, built-ins only). HYBRID survived on luck: CONV and LSTM died
2 and 12 minutes before its own sweep finished, freeing the memory.
Compiles clean: 0 errors, 0 warnings.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-08 15:42:03 -04:00
|
|
|
//--- AFTER the re-creates, never before: releasing first can drop the terminal's last reference and
|
|
|
|
|
//--- make it tear the instance down, so an identical-params Create() would then rebuild it from
|
|
|
|
|
//--- scratch instead of re-using the live one - turning a refcount bump into a full recalculation over
|
|
|
|
|
//--- the whole history, 324 times over. Released here, the old instance survives until its
|
|
|
|
|
//--- replacement exists.
|
|
|
|
|
if(hCD != INVALID_HANDLE)
|
|
|
|
|
IndicatorRelease(hCD);
|
|
|
|
|
if(hSOT != INVALID_HANDLE)
|
|
|
|
|
IndicatorRelease(hSOT);
|
|
|
|
|
if(hWES != INVALID_HANDLE)
|
|
|
|
|
IndicatorRelease(hWES);
|
|
|
|
|
if(hWFS != INVALID_HANDLE)
|
|
|
|
|
IndicatorRelease(hWFS);
|
|
|
|
|
if(hWSBI != INVALID_HANDLE)
|
|
|
|
|
IndicatorRelease(hWSBI);
|
|
|
|
|
if(hMA != INVALID_HANDLE)
|
|
|
|
|
IndicatorRelease(hMA);
|
|
|
|
|
if(hRSI != INVALID_HANDLE)
|
|
|
|
|
IndicatorRelease(hRSI);
|
|
|
|
|
if(hMACD != INVALID_HANDLE)
|
|
|
|
|
IndicatorRelease(hMACD);
|
|
|
|
|
if(hIchi != INVALID_HANDLE)
|
|
|
|
|
IndicatorRelease(hIchi);
|
refactor: split CExpertSignalAIBase implementation by responsibility
ExpertSignalAIBase.mqh was 8216 lines: the class declaration followed by 87
method bodies covering training, labelling, feature extraction, persistence,
chart drawing, online learning, the GA auto-tuner and inference, all in one
file. Train() alone is 1492 lines; a change to arrow drawing meant scrolling
past the era loop.
Moved the bodies into Expert\AIBase\, included at the bottom of the original
after the class declaration:
Training.mqh 1607 era loop, plateau ladder, checkpoint select, deploy
Features.mqh 1093 indicator creation + per-bar input feature vector
ChartUI.mqh 634 arrows, arrow persistence, status panel, cleanup
Persistence.mqh 492 .stats/.cfg sidecars, CPU-inference validation, copy
OnlineLearning.mqh 461 live continual learning, EMA shadow, OOS simulator
Labels.mqh 309 ZigZag pivot labels, async label-cache prebuild
AutoTune.mqh 275 genetic tuner (population, crossover, halving)
Inference.mqh 235 softmax, prior calibration, class priors
ExpertSignalAIBase.mqh 8216 -> 3131 (declaration + topology build only)
This is a pure relocation - verified mechanically, not by eye: HEAD's file
reconstructed from the eight partials plus the surviving remainder is
byte-identical to HEAD, span for span (scratchpad verify_split.py). No
declaration moved, no signature changed, no code rewritten, so behaviour is
unchanged by construction.
Compiles 0 errors, 0 warnings.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-29 00:42:45 -04:00
|
|
|
//--- Indicator params just changed, so every cached feature row is now stale (the feature values
|
|
|
|
|
//--- depend on these indicators; the LABELS do not - they come from ADZigZag - so the label cache is
|
|
|
|
|
//--- deliberately left intact and reused). Without this, a tuner candidate would silently train and be
|
|
|
|
|
//--- scored on the PREVIOUS candidate's features. Cheap: just flags rows for lazy recompute on next read.
|
|
|
|
|
ArrayInitialize(m_featureCacheHasValue, false);
|
|
|
|
|
return result;
|
|
|
|
|
}
|
|
|
|
|
//+------------------------------------------------------------------+
|
|
|
|
|
//| |
|
|
|
|
|
//+------------------------------------------------------------------+
|
|
|
|
|
bool CExpertSignalAIBase::ResizeBuffers(int barIndex)
|
|
|
|
|
{
|
|
|
|
|
// The Ichimoku feature's Chikou term reads m_Close at idx + ichiKijun (see its block in
|
|
|
|
|
// BufferTempDataCompute() for why that direction, and only that direction, is lookahead-free), which
|
|
|
|
|
// is further back than any other consumer of the close series reaches. Grow the close buffer to match
|
|
|
|
|
// when that feature is on, so the oldest requested bars resolve from real data instead of tripping
|
|
|
|
|
// that block's EMPTY_VALUE guard and being rejected as unusable examples.
|
fix(buffers): revert the MA +1 - it asked for a bar that does not exist and stopped every chart
REGRESSION I INTRODUCED IN 1dda479, live for ~15 minutes.
CSeries::BufferResize -> CheckLoadHistory -> CheckTerminalHistory succeeds only when
Bars() >= size. Train() calls ResizeBuffers with barIndex == Bars(), so sizing the MA
buffer to barIndex + 1 asks for one bar more than the symbol has and fails the WHOLE
ResizeBuffers call. The log named it exactly:
failed to get 50180 bars for USDJPY,PERIOD_H4 (Bars() = 50,179)
failed to get 33983 bars for XAUUSD,PERIOD_H4 (Bars() = 33,982)
StartLabelCachePrebuild() then bailed on the false return and stayed silent, so the
only symptom was Train() reporting "arming the first label-cache prebuild" forever
with labelCacheBars=0 - the panel's "getting ready".
The premise was wrong, not just the arithmetic. The MA block reads GetData(idx) AND
GetData(idx + 1), and at the OLDEST bar that second read is SUPPOSED to fail - there
is no older bar to difference against. Rejecting that one bar is correct behaviour;
buying it cost the entire history.
Two more things, since the same defect had a second instance and no alarm:
- The Ichimoku pair (closeBars and m_Ichimoku, both barIndex + ichiKijun) is the same
bug with a far larger constant, latent only because the feature is off. Both are now
clamped to Bars(). The oldest ichiKijun bars then have no cloud, which that block's
EMPTY_VALUE guard already handles per-bar - the right outcome.
- The prebuild's bare `return` on a false ResizeBuffers now says so once, naming the
depth and Bars(). MQL5's own "failed to get N bars" was in the log the entire time,
from a stack frame nothing connected to the prebuild.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-17 14:12:47 -04:00
|
|
|
// CLAMPED TO WHAT EXISTS. CSeries::BufferResize -> CheckLoadHistory -> CheckTerminalHistory only
|
|
|
|
|
// succeeds when Bars() >= size, and Train() calls this with barIndex == Bars() - so ANY block that
|
|
|
|
|
// asks for "barIndex + something" fails the whole ResizeBuffers call rather than just its own deep
|
|
|
|
|
// reads. The MA block proved that on 2026-08-17 with a +1 (see its comment below); the Ichimoku
|
|
|
|
|
// pair here is the same defect with a much larger constant, latent only because that feature has
|
|
|
|
|
// been off. The oldest ichiKijun bars then have no cloud, which the block's EMPTY_VALUE guard
|
|
|
|
|
// already handles as a per-bar rejection - the correct outcome, and cheap next to losing all of it.
|
|
|
|
|
int maxBars = Bars(m_symbol.Name(), PERIOD_CURRENT);
|
|
|
|
|
int closeBars = m_useIchimoku ? (int)MathMin(barIndex + m_indicatorTuner.ichiKijun, maxBars) : barIndex;
|
refactor: split CExpertSignalAIBase implementation by responsibility
ExpertSignalAIBase.mqh was 8216 lines: the class declaration followed by 87
method bodies covering training, labelling, feature extraction, persistence,
chart drawing, online learning, the GA auto-tuner and inference, all in one
file. Train() alone is 1492 lines; a change to arrow drawing meant scrolling
past the era loop.
Moved the bodies into Expert\AIBase\, included at the bottom of the original
after the class declaration:
Training.mqh 1607 era loop, plateau ladder, checkpoint select, deploy
Features.mqh 1093 indicator creation + per-bar input feature vector
ChartUI.mqh 634 arrows, arrow persistence, status panel, cleanup
Persistence.mqh 492 .stats/.cfg sidecars, CPU-inference validation, copy
OnlineLearning.mqh 461 live continual learning, EMA shadow, OOS simulator
Labels.mqh 309 ZigZag pivot labels, async label-cache prebuild
AutoTune.mqh 275 genetic tuner (population, crossover, halving)
Inference.mqh 235 softmax, prior calibration, class priors
ExpertSignalAIBase.mqh 8216 -> 3131 (declaration + topology build only)
This is a pure relocation - verified mechanically, not by eye: HEAD's file
reconstructed from the eight partials plus the surviving remainder is
byte-identical to HEAD, span for span (scratchpad verify_split.py). No
declaration moved, no signature changed, no code rewritten, so behaviour is
unchanged by construction.
Compiles 0 errors, 0 warnings.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-29 00:42:45 -04:00
|
|
|
if(!m_Open.BufferResize(barIndex) || !m_Close.BufferResize(closeBars) || !m_High.BufferResize(barIndex) || !m_Low.BufferResize(barIndex))
|
|
|
|
|
return false;
|
|
|
|
|
if(m_useVolumes)
|
|
|
|
|
{
|
|
|
|
|
if(!m_Volumes.BufferResize(barIndex))
|
|
|
|
|
return false;
|
|
|
|
|
}
|
|
|
|
|
// Unconditional - see InitTime()'s call site in InitIndicators() for why m_Time must always be live.
|
|
|
|
|
if(!m_Time.BufferResize(barIndex))
|
|
|
|
|
return false;
|
|
|
|
|
if(m_useMA)
|
|
|
|
|
{
|
fix(buffers): revert the MA +1 - it asked for a bar that does not exist and stopped every chart
REGRESSION I INTRODUCED IN 1dda479, live for ~15 minutes.
CSeries::BufferResize -> CheckLoadHistory -> CheckTerminalHistory succeeds only when
Bars() >= size. Train() calls ResizeBuffers with barIndex == Bars(), so sizing the MA
buffer to barIndex + 1 asks for one bar more than the symbol has and fails the WHOLE
ResizeBuffers call. The log named it exactly:
failed to get 50180 bars for USDJPY,PERIOD_H4 (Bars() = 50,179)
failed to get 33983 bars for XAUUSD,PERIOD_H4 (Bars() = 33,982)
StartLabelCachePrebuild() then bailed on the false return and stayed silent, so the
only symptom was Train() reporting "arming the first label-cache prebuild" forever
with labelCacheBars=0 - the panel's "getting ready".
The premise was wrong, not just the arithmetic. The MA block reads GetData(idx) AND
GetData(idx + 1), and at the OLDEST bar that second read is SUPPOSED to fail - there
is no older bar to difference against. Rejecting that one bar is correct behaviour;
buying it cost the entire history.
Two more things, since the same defect had a second instance and no alarm:
- The Ichimoku pair (closeBars and m_Ichimoku, both barIndex + ichiKijun) is the same
bug with a far larger constant, latent only because the feature is off. Both are now
clamped to Bars(). The oldest ichiKijun bars then have no cloud, which that block's
EMPTY_VALUE guard already handles per-bar - the right outcome.
- The prebuild's bare `return` on a false ResizeBuffers now says so once, naming the
depth and Bars(). MQL5's own "failed to get N bars" was in the log the entire time,
from a stack frame nothing connected to the prebuild.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-17 14:12:47 -04:00
|
|
|
// NOT barIndex + 1, though the MA block does read GetData(idx) AND GetData(idx + 1) for its
|
|
|
|
|
// bar-over-bar change. That looks like an off-by-one and is not one: Train() calls this with
|
|
|
|
|
// barIndex == Bars(), and CSeries::BufferResize -> CheckLoadHistory -> CheckTerminalHistory
|
|
|
|
|
// only succeeds when Bars() >= size, so asking for one MORE bar than exists fails the whole
|
|
|
|
|
// ResizeBuffers call. Shipped as a "fix" on 2026-08-17 and it stopped every chart dead:
|
|
|
|
|
// "failed to get 50180 bars for USDJPY,PERIOD_H4" (Bars() was 50,179) and 33,983 vs 33,982 on
|
|
|
|
|
// XAUUSD, after which StartLabelCachePrebuild() bailed on the false return - silently, forever,
|
|
|
|
|
// showing only as "arming the first label-cache prebuild".
|
|
|
|
|
// The read at the OLDEST bar is SUPPOSED to fail: there is no older bar to difference against.
|
|
|
|
|
// Rejecting that single bar is correct; buying it costs the entire history.
|
|
|
|
|
if(!m_MA.BufferResize(barIndex))
|
refactor: split CExpertSignalAIBase implementation by responsibility
ExpertSignalAIBase.mqh was 8216 lines: the class declaration followed by 87
method bodies covering training, labelling, feature extraction, persistence,
chart drawing, online learning, the GA auto-tuner and inference, all in one
file. Train() alone is 1492 lines; a change to arrow drawing meant scrolling
past the era loop.
Moved the bodies into Expert\AIBase\, included at the bottom of the original
after the class declaration:
Training.mqh 1607 era loop, plateau ladder, checkpoint select, deploy
Features.mqh 1093 indicator creation + per-bar input feature vector
ChartUI.mqh 634 arrows, arrow persistence, status panel, cleanup
Persistence.mqh 492 .stats/.cfg sidecars, CPU-inference validation, copy
OnlineLearning.mqh 461 live continual learning, EMA shadow, OOS simulator
Labels.mqh 309 ZigZag pivot labels, async label-cache prebuild
AutoTune.mqh 275 genetic tuner (population, crossover, halving)
Inference.mqh 235 softmax, prior calibration, class priors
ExpertSignalAIBase.mqh 8216 -> 3131 (declaration + topology build only)
This is a pure relocation - verified mechanically, not by eye: HEAD's file
reconstructed from the eight partials plus the surviving remainder is
byte-identical to HEAD, span for span (scratchpad verify_split.py). No
declaration moved, no signature changed, no code rewritten, so behaviour is
unchanged by construction.
Compiles 0 errors, 0 warnings.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-29 00:42:45 -04:00
|
|
|
return false;
|
|
|
|
|
}
|
|
|
|
|
if(m_useRSI)
|
|
|
|
|
{
|
|
|
|
|
if(!m_RSI.BufferResize(barIndex))
|
|
|
|
|
return false;
|
|
|
|
|
}
|
|
|
|
|
if(m_useMACD)
|
|
|
|
|
{
|
|
|
|
|
if(!m_MACDFeature.BufferResize(barIndex))
|
|
|
|
|
return false;
|
|
|
|
|
}
|
|
|
|
|
if(m_useIchimoku)
|
|
|
|
|
{
|
|
|
|
|
// + m_indicatorTuner.ichiKijun: the cloud reads reach that many bars FURTHER back than every other
|
|
|
|
|
// indicator here does (see the m_useIchimoku feature block for why the offset exists), so sizing
|
|
|
|
|
// this buffer to barIndex alone would leave the oldest requested bars' cloud values unavailable.
|
fix(buffers): revert the MA +1 - it asked for a bar that does not exist and stopped every chart
REGRESSION I INTRODUCED IN 1dda479, live for ~15 minutes.
CSeries::BufferResize -> CheckLoadHistory -> CheckTerminalHistory succeeds only when
Bars() >= size. Train() calls ResizeBuffers with barIndex == Bars(), so sizing the MA
buffer to barIndex + 1 asks for one bar more than the symbol has and fails the WHOLE
ResizeBuffers call. The log named it exactly:
failed to get 50180 bars for USDJPY,PERIOD_H4 (Bars() = 50,179)
failed to get 33983 bars for XAUUSD,PERIOD_H4 (Bars() = 33,982)
StartLabelCachePrebuild() then bailed on the false return and stayed silent, so the
only symptom was Train() reporting "arming the first label-cache prebuild" forever
with labelCacheBars=0 - the panel's "getting ready".
The premise was wrong, not just the arithmetic. The MA block reads GetData(idx) AND
GetData(idx + 1), and at the OLDEST bar that second read is SUPPOSED to fail - there
is no older bar to difference against. Rejecting that one bar is correct behaviour;
buying it cost the entire history.
Two more things, since the same defect had a second instance and no alarm:
- The Ichimoku pair (closeBars and m_Ichimoku, both barIndex + ichiKijun) is the same
bug with a far larger constant, latent only because the feature is off. Both are now
clamped to Bars(). The oldest ichiKijun bars then have no cloud, which that block's
EMPTY_VALUE guard already handles per-bar - the right outcome.
- The prebuild's bare `return` on a false ResizeBuffers now says so once, naming the
depth and Bars(). MQL5's own "failed to get N bars" was in the log the entire time,
from a stack frame nothing connected to the prebuild.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-17 14:12:47 -04:00
|
|
|
if(!m_Ichimoku.BufferResize((int)MathMin(barIndex + m_indicatorTuner.ichiKijun, maxBars)))
|
refactor: split CExpertSignalAIBase implementation by responsibility
ExpertSignalAIBase.mqh was 8216 lines: the class declaration followed by 87
method bodies covering training, labelling, feature extraction, persistence,
chart drawing, online learning, the GA auto-tuner and inference, all in one
file. Train() alone is 1492 lines; a change to arrow drawing meant scrolling
past the era loop.
Moved the bodies into Expert\AIBase\, included at the bottom of the original
after the class declaration:
Training.mqh 1607 era loop, plateau ladder, checkpoint select, deploy
Features.mqh 1093 indicator creation + per-bar input feature vector
ChartUI.mqh 634 arrows, arrow persistence, status panel, cleanup
Persistence.mqh 492 .stats/.cfg sidecars, CPU-inference validation, copy
OnlineLearning.mqh 461 live continual learning, EMA shadow, OOS simulator
Labels.mqh 309 ZigZag pivot labels, async label-cache prebuild
AutoTune.mqh 275 genetic tuner (population, crossover, halving)
Inference.mqh 235 softmax, prior calibration, class priors
ExpertSignalAIBase.mqh 8216 -> 3131 (declaration + topology build only)
This is a pure relocation - verified mechanically, not by eye: HEAD's file
reconstructed from the eight partials plus the surviving remainder is
byte-identical to HEAD, span for span (scratchpad verify_split.py). No
declaration moved, no signature changed, no code rewritten, so behaviour is
unchanged by construction.
Compiles 0 errors, 0 warnings.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-29 00:42:45 -04:00
|
|
|
return false;
|
|
|
|
|
}
|
|
|
|
|
// Unconditional (not gated by m_useATR): the ATR-normalization in BufferTempData() reads
|
|
|
|
|
// m_ATR.Main() regardless of whether ATR is enabled as an explicit extra input feature -
|
|
|
|
|
// m_useATR only controls that feature-count opt-in (see InitIndicators()'s "already init in the
|
|
|
|
|
// base class" comment), not whether ATR data itself needs to be kept live.
|
|
|
|
|
if(!m_ATR.BufferResize(barIndex))
|
|
|
|
|
return false;
|
feat(ai): triple-barrier labels replace exact-pivot ZigZag targets
The 31:1 class imbalance was self-inflicted by the TARGET, not a property
of the market. Labelling only the exact bar where a ZigZag pivot confirms
gave Buy 1164 / Sell 1164 / Neutral 35841, and every correction mechanism
this codebase accumulated sits downstream of that one choice: the
logit-adjusted loss and its range cap, the prior EMA, the +-3.0 output-bias
seed, balanced-accuracy-then-precision selection with its coverage floor,
the recall floor and its catch-22, the alternation gate, NMS, and the four
oversampling designs that collapsed before them.
The reference this engine is built on (references/neuronetworksbook.pdf
ch. 3.1/3.3) also uses ZigZag, but targets the DIRECTION TO THE NEXT
EXTREMUM on every bar - ~50/50 by construction, with no imbalance to
correct at all. It never had this problem because it never asked "is this
the pivot bar".
Labels are now the triple barrier (Lopez de Prado ch. 3), using the EA's
OWN SL_Mode/TP_Mode: does a trade opened at this bar's close reach its
target before its stop, within a horizon. Buy = long resolves, Sell =
short resolves, Neutral = neither. Consequences:
- dir-precision in the era line stops being a proxy and becomes the win
rate of the strategy under its own exit rules.
- Expected balance ~25/25/50 at the shipped 1:3 (gambler's ruin), i.e.
~2:1 instead of 31:1. Measured and logged at the end of the prebuild.
- Spread is charged on both legs, so it is a NET win rate.
- Intrabar ambiguity resolves to the STOP. OHLC cannot order two touches
inside one bar and the optimistic reading is how a backtested edge
becomes a live loss.
ZigZag stays as input features (EnableSwingContext) and now also supplies
the vertical barrier: the horizon is the median confirmed leg length,
snapped to a coarse ladder. Derived, not configured, and deliberately kept
out of the filename fingerprint - a filename keyed on a measured quantity
orphans a trained model the moment the measurement moves.
Removed, because the premise died with the old target:
- the alternation gate. Correct for pivot labels (a ZigZag cannot emit two
same-type pivots in a row, so a repeat was provably a false fire), and
wrong for barrier labels, which answer each bar independently. It also
took its worst consequence with it: a one-sided model previously got ONE
trade per backtest, a hard blocker on marketplace validation.
- SignalClusterWindow now defaults off - it de-duplicated repeats that are
now real trades. Kept as an opt-in display control.
- LABEL_WINDOW_BARS, the pivot-widening pass, ConfirmedZigZagLabel.
- the era-0 output-bias seed now needs a genuinely dominant class (0.70)
rather than 0.40; at ~50% Neutral a +-3.0 seed is a distortion, not a
correction.
Also fixed, both found while wiring the above:
1. RefreshConvergedSignal sized its buffers from a date delta
(Bars(sym, period, dtStudied, TimeCurrent())). dtStudied is a training
watermark; in the tester it is loaded from a live-chart save AHEAD of
the simulated date, so the interval inverted, Bars() returned ~0, and
the buffer came out at exactly m_historyBars - deep enough for the OHLC
window and far too shallow for the Donchian-50 / 20-bar-return / SMA
extension behind it. Inference silently computed DIFFERENT features
from the ones training learned on, live as well as in the tester. Now
sized from what the feature builder actually needs.
2. The barrier horizon is resolved on the deployed path too. A deployed
model never enters Train(), so it never reached the prebuild, and
OnlineLearnStep reads the horizon as its confirmation delay - left at
the fallback it would have backpropped bars whose barriers had not
resolved. Silent lookahead in the one place that writes to a live model.
SL_Mode/TP_Mode join the weights fingerprint: they define the labels now,
so a model trained at 1:3 must never be silently reused at 1:1. This
re-keys every pre-existing model by design - none were trained on this task.
Inference census extended with the vote gate. LongCondition/ShortCondition
open with a readiness check the refresh counters never see; in the tester it
reduces to "the seeded _optcache.nnw must have LOADED", and if it did not,
every vote is hard-zeroed while the model still answers Buy. The old three
counters would have read that as "the model says Neutral" - false, and a
completely different fix. This is the leading candidate for the
zero-direction backtest and the census can now name it in one run.
Both builds compile 0 errors / 0 warnings. Forces a full retrain.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-31 20:39:49 -04:00
|
|
|
// Unconditional, same reasoning as m_ATR above - m_ADZigZag drives the swing-context features AND
|
|
|
|
|
// ComputeBarrierHorizonBars()'s measurement, not an opt-in feature, so it's never gated by an
|
|
|
|
|
// m_use* flag. (It was also the training-label source until the 2026-08-01 triple-barrier relabel.)
|
refactor: split CExpertSignalAIBase implementation by responsibility
ExpertSignalAIBase.mqh was 8216 lines: the class declaration followed by 87
method bodies covering training, labelling, feature extraction, persistence,
chart drawing, online learning, the GA auto-tuner and inference, all in one
file. Train() alone is 1492 lines; a change to arrow drawing meant scrolling
past the era loop.
Moved the bodies into Expert\AIBase\, included at the bottom of the original
after the class declaration:
Training.mqh 1607 era loop, plateau ladder, checkpoint select, deploy
Features.mqh 1093 indicator creation + per-bar input feature vector
ChartUI.mqh 634 arrows, arrow persistence, status panel, cleanup
Persistence.mqh 492 .stats/.cfg sidecars, CPU-inference validation, copy
OnlineLearning.mqh 461 live continual learning, EMA shadow, OOS simulator
Labels.mqh 309 ZigZag pivot labels, async label-cache prebuild
AutoTune.mqh 275 genetic tuner (population, crossover, halving)
Inference.mqh 235 softmax, prior calibration, class priors
ExpertSignalAIBase.mqh 8216 -> 3131 (declaration + topology build only)
This is a pure relocation - verified mechanically, not by eye: HEAD's file
reconstructed from the eight partials plus the surviving remainder is
byte-identical to HEAD, span for span (scratchpad verify_split.py). No
declaration moved, no signature changed, no code rewritten, so behaviour is
unchanged by construction.
Compiles 0 errors, 0 warnings.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-29 00:42:45 -04:00
|
|
|
if(!m_ADZigZag.BufferResize(barIndex))
|
|
|
|
|
return false;
|
|
|
|
|
if(m_useADCumulativeDelta)
|
|
|
|
|
{
|
|
|
|
|
if(!m_ADCumulativeDelta.BufferResize(barIndex))
|
|
|
|
|
return false;
|
|
|
|
|
}
|
|
|
|
|
if(m_useADShorteningOfThrust)
|
|
|
|
|
{
|
|
|
|
|
if(!m_ADShorteningOfThrust.BufferResize(barIndex))
|
|
|
|
|
return false;
|
|
|
|
|
}
|
|
|
|
|
if(m_useADWyckoffEventStream)
|
|
|
|
|
{
|
|
|
|
|
if(!m_ADWyckoffEventStream.BufferResize(barIndex))
|
|
|
|
|
return false;
|
|
|
|
|
}
|
|
|
|
|
if(m_useADWyckoffFailedStructure)
|
|
|
|
|
{
|
|
|
|
|
if(!m_ADWyckoffFailedStructure.BufferResize(barIndex))
|
|
|
|
|
return false;
|
|
|
|
|
}
|
|
|
|
|
if(m_useADWyckoffSignificantBarInversion)
|
|
|
|
|
{
|
|
|
|
|
if(!m_ADWyckoffSignificantBarInversion.BufferResize(barIndex))
|
|
|
|
|
return false;
|
|
|
|
|
}
|
|
|
|
|
return true;
|
|
|
|
|
}
|
|
|
|
|
//+------------------------------------------------------------------+
|
|
|
|
|
//| |
|
|
|
|
|
//+------------------------------------------------------------------+
|
|
|
|
|
bool CExpertSignalAIBase::RefreshData()
|
|
|
|
|
{
|
|
|
|
|
// CSeries/CIndicator::Refresh() is void - there is no per-call success/failure signal to
|
|
|
|
|
// propagate here. The real data-validity check happens downstream, per value, in
|
|
|
|
|
// BufferTempDataCompute() (EMPTY_VALUE / atr<=0 guards) - this function's job is only to ask
|
|
|
|
|
// every buffer to refresh, unconditionally, before that per-value check runs.
|
|
|
|
|
m_Open.Refresh(OBJ_ALL_PERIODS);
|
|
|
|
|
m_Close.Refresh(OBJ_ALL_PERIODS);
|
|
|
|
|
m_High.Refresh(OBJ_ALL_PERIODS);
|
|
|
|
|
m_Low.Refresh(OBJ_ALL_PERIODS);
|
|
|
|
|
if(m_useVolumes)
|
|
|
|
|
{
|
|
|
|
|
m_Volumes.Refresh(OBJ_ALL_PERIODS);
|
|
|
|
|
}
|
|
|
|
|
// Unconditional - see InitTime()'s call site in InitIndicators() for why m_Time must always be live.
|
|
|
|
|
m_Time.Refresh(OBJ_ALL_PERIODS);
|
|
|
|
|
if(m_useMA)
|
|
|
|
|
{
|
|
|
|
|
m_MA.Refresh(OBJ_ALL_PERIODS);
|
|
|
|
|
}
|
|
|
|
|
if(m_useRSI)
|
|
|
|
|
{
|
|
|
|
|
m_RSI.Refresh(OBJ_ALL_PERIODS);
|
|
|
|
|
}
|
|
|
|
|
if(m_useMACD)
|
|
|
|
|
{
|
|
|
|
|
m_MACDFeature.Refresh(OBJ_ALL_PERIODS);
|
|
|
|
|
}
|
|
|
|
|
if(m_useIchimoku)
|
|
|
|
|
{
|
|
|
|
|
m_Ichimoku.Refresh(OBJ_ALL_PERIODS);
|
|
|
|
|
}
|
|
|
|
|
// Unconditional - see the matching BufferResize() comment above.
|
|
|
|
|
m_ATR.Refresh(OBJ_ALL_PERIODS);
|
|
|
|
|
m_ADZigZag.Refresh(OBJ_ALL_PERIODS);
|
|
|
|
|
if(m_useADCumulativeDelta)
|
|
|
|
|
{
|
|
|
|
|
m_ADCumulativeDelta.Refresh(OBJ_ALL_PERIODS);
|
|
|
|
|
}
|
|
|
|
|
if(m_useADShorteningOfThrust)
|
|
|
|
|
{
|
|
|
|
|
m_ADShorteningOfThrust.Refresh(OBJ_ALL_PERIODS);
|
|
|
|
|
}
|
|
|
|
|
if(m_useADWyckoffEventStream)
|
|
|
|
|
{
|
|
|
|
|
m_ADWyckoffEventStream.Refresh(OBJ_ALL_PERIODS);
|
|
|
|
|
}
|
|
|
|
|
if(m_useADWyckoffFailedStructure)
|
|
|
|
|
{
|
|
|
|
|
m_ADWyckoffFailedStructure.Refresh(OBJ_ALL_PERIODS);
|
|
|
|
|
}
|
|
|
|
|
if(m_useADWyckoffSignificantBarInversion)
|
|
|
|
|
{
|
|
|
|
|
m_ADWyckoffSignificantBarInversion.Refresh(OBJ_ALL_PERIODS);
|
|
|
|
|
}
|
|
|
|
|
return true;
|
|
|
|
|
}
|
|
|
|
|
//+------------------------------------------------------------------+
|
|
|
|
|
//| |
|
|
|
|
|
//+------------------------------------------------------------------+
|
|
|
|
|
//+------------------------------------------------------------------+
|
|
|
|
|
//| Cache-or-compute wrapper around BufferTempDataCompute(): a given |
|
|
|
|
|
//| now-relative bar index's feature vector is invariant until the |
|
|
|
|
|
//| next candle close (see m_featureCache's declaration comment), so |
|
|
|
|
|
//| a cache hit just replays the m_neuronsCount values already |
|
|
|
|
|
//| computed for this idx straight into TempData instead of re- |
|
|
|
|
|
//| deriving them from price/ATR/AD-indicator buffers again. |
|
|
|
|
|
//+------------------------------------------------------------------+
|
|
|
|
|
bool CExpertSignalAIBase::BufferTempData(int idx)
|
|
|
|
|
{
|
|
|
|
|
int width = m_neuronsCount;
|
|
|
|
|
bool cacheable = (idx >= 0 && idx < ArraySize(m_featureCacheHasValue) && width > 0);
|
|
|
|
|
if(cacheable && m_featureCacheHasValue[idx])
|
|
|
|
|
{
|
|
|
|
|
if(!m_featureCacheValid[idx])
|
|
|
|
|
return false;
|
|
|
|
|
int base = idx * width;
|
|
|
|
|
for(int f = 0; f < width; f++)
|
|
|
|
|
if(!TempData.Add(m_featureCache[base + f]))
|
|
|
|
|
return false;
|
|
|
|
|
return true;
|
|
|
|
|
}
|
|
|
|
|
int startTotal = TempData.Total();
|
|
|
|
|
bool ok = BufferTempDataCompute(idx);
|
diag: name the cause when every feature window fails, and enforce the width contract
Era 0 stalls with "NOT ONE of 54681 scanned bars produced a usable
feature window, windows ok=0 failed=54681" and nothing else. That line
reads identically for a cold ATR, a conditionally-missing optional
feature block and an out-of-range index, so it cannot be diagnosed
without one restart per hypothesis.
Two changes:
1. WIDTH CONTRACT in BufferTempData. Every enabled block must emit
exactly m_neuronsCount values on EVERY bar. A block that emits its
values on some bars and skips them on others (indicator, panel or
series unavailable for that bar) does not merely shorten the window -
it SHIFTS every feature after it into the wrong slot, and the net
then trains on silently misaligned inputs that still look like a
valid window to everything downstream. Now rejected, rolled back and
reported once, naming the optional blocks (XA / SPR / swing context)
as the ones carrying an availability test. Worth having independently
of the current stall.
2. BuildFeatureWindow records WHICH lookback slot rejected and how much
of the window was assembled, and the pass-1 stall report renders it:
"slot 0 of 20 REJECTED (window had 0 of 760)" is an indicator warm-up
or history-edge read; "every lookback bar ACCEPTED and the window was
still short: 640 of 760" is a missing 6-value block.
No behaviour change on a healthy run: the width check is an equality
that already holds, and the diagnostics render only inside the
total-failure branch.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-11 10:31:56 -04:00
|
|
|
//--- WIDTH CONTRACT. Every enabled feature block must emit exactly the number of values
|
|
|
|
|
//--- m_neuronsCount was computed from, on every bar, unconditionally - a block that emits its
|
|
|
|
|
//--- values on some bars and skips them on others (because an indicator, panel or series was
|
|
|
|
|
//--- unavailable for THAT bar) does not merely shorten the window: it SHIFTS every feature after it
|
|
|
|
|
//--- into the wrong slot, and the net then trains on silently misaligned inputs that still look like
|
|
|
|
|
//--- a valid window to everything downstream. Caught here rather than left to surface as
|
|
|
|
|
//--- BuildFeatureWindow's length check, which cannot say which bar or which block was responsible.
|
|
|
|
|
if(ok)
|
|
|
|
|
{
|
|
|
|
|
int produced = TempData.Total() - startTotal;
|
|
|
|
|
if(produced != width)
|
|
|
|
|
{
|
|
|
|
|
ok = false;
|
|
|
|
|
m_featureFailTransient = false; // a width fault is structural, never "not ready yet"
|
|
|
|
|
if(!m_featureWidthWarned)
|
|
|
|
|
{
|
|
|
|
|
m_featureWidthWarned = true;
|
|
|
|
|
PrintFormat("%s: FEATURE WIDTH MISMATCH at bar %d - the enabled blocks produced %d values"
|
|
|
|
|
" but m_neuronsCount says %d. Every feature after the short block would have"
|
|
|
|
|
" landed in the wrong slot, so the bar is rejected rather than trained on."
|
|
|
|
|
" A block that can be conditionally unavailable must emit neutral values, not"
|
|
|
|
|
" nothing. Check the optional blocks first (cross-asset XA, spread SPR, swing"
|
|
|
|
|
" context) - those are the ones with an availability test.",
|
|
|
|
|
ID, idx, produced, width);
|
|
|
|
|
}
|
|
|
|
|
//--- Roll back the partial bar so the caller's window cannot contain half of it.
|
|
|
|
|
while(TempData.Total() > startTotal)
|
|
|
|
|
TempData.Delete(TempData.Total() - 1);
|
|
|
|
|
}
|
|
|
|
|
}
|
2026-08-11 11:30:02 -04:00
|
|
|
//--- ONLY SUCCESSES ARE CACHED. A miss is never stored, in any form.
|
|
|
|
|
//---
|
|
|
|
|
//--- The previous rule cached a miss whenever it was not flagged transient, and flagged exactly TWO
|
|
|
|
|
//--- guards - the EMPTY_VALUE open and the cold ATR. That was the half-fix: every OTHER rejection in
|
|
|
|
|
//--- BufferTempDataCompute (an indicator buffer not yet calculated, a panel not yet built, a series
|
|
|
|
|
//--- not yet loaded, a failed Add) still cached as PERMANENT, so one early sweep across cold
|
|
|
|
|
//--- indicators poisoned those bars for the rest of the process. Observed 2026-08-11: the MI
|
|
|
|
|
//--- pre-scan runs ~3 s after OnInit, touches all 54k bars while the indicators are still warming,
|
|
|
|
|
//--- and the run then reported "0 samples" and never trained again - the same failure the two-guard
|
|
|
|
|
//--- version was written to prevent, arriving through the guards it did not cover.
|
|
|
|
|
//---
|
|
|
|
|
//--- Enumerating which rejections are "really" permanent is the wrong shape of fix: it is a list that
|
|
|
|
|
//--- has to be re-audited every time a feature block is added, and being wrong once costs the whole
|
|
|
|
|
//--- run silently. Caching only successes needs no list and cannot be wrong. The cost is bounded and
|
|
|
|
|
//--- small: in steady state the only bars that still fail are the handful at the deep end of history
|
|
|
|
|
//--- inside the indicators' own warm-up, so an era recomputes ~ind_Periods bars rather than 54k.
|
|
|
|
|
//---
|
|
|
|
|
//--- m_featureCacheValid is now always true where m_featureCacheHasValue is true. Both are kept
|
|
|
|
|
//--- rather than collapsed into one array: the pair is written and read in several places, and a
|
|
|
|
|
//--- silent meaning change is exactly how the last version of this drifted.
|
|
|
|
|
if(cacheable && ok)
|
refactor: split CExpertSignalAIBase implementation by responsibility
ExpertSignalAIBase.mqh was 8216 lines: the class declaration followed by 87
method bodies covering training, labelling, feature extraction, persistence,
chart drawing, online learning, the GA auto-tuner and inference, all in one
file. Train() alone is 1492 lines; a change to arrow drawing meant scrolling
past the era loop.
Moved the bodies into Expert\AIBase\, included at the bottom of the original
after the class declaration:
Training.mqh 1607 era loop, plateau ladder, checkpoint select, deploy
Features.mqh 1093 indicator creation + per-bar input feature vector
ChartUI.mqh 634 arrows, arrow persistence, status panel, cleanup
Persistence.mqh 492 .stats/.cfg sidecars, CPU-inference validation, copy
OnlineLearning.mqh 461 live continual learning, EMA shadow, OOS simulator
Labels.mqh 309 ZigZag pivot labels, async label-cache prebuild
AutoTune.mqh 275 genetic tuner (population, crossover, halving)
Inference.mqh 235 softmax, prior calibration, class priors
ExpertSignalAIBase.mqh 8216 -> 3131 (declaration + topology build only)
This is a pure relocation - verified mechanically, not by eye: HEAD's file
reconstructed from the eight partials plus the surviving remainder is
byte-identical to HEAD, span for span (scratchpad verify_split.py). No
declaration moved, no signature changed, no code rewritten, so behaviour is
unchanged by construction.
Compiles 0 errors, 0 warnings.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-29 00:42:45 -04:00
|
|
|
{
|
|
|
|
|
m_featureCacheHasValue[idx] = true;
|
2026-08-11 11:30:02 -04:00
|
|
|
m_featureCacheValid[idx] = true;
|
|
|
|
|
int base = idx * width;
|
|
|
|
|
int count = TempData.Total() - startTotal;
|
|
|
|
|
for(int f = 0; f < count && f < width; f++)
|
|
|
|
|
m_featureCache[base + f] = TempData.At(startTotal + f);
|
refactor: split CExpertSignalAIBase implementation by responsibility
ExpertSignalAIBase.mqh was 8216 lines: the class declaration followed by 87
method bodies covering training, labelling, feature extraction, persistence,
chart drawing, online learning, the GA auto-tuner and inference, all in one
file. Train() alone is 1492 lines; a change to arrow drawing meant scrolling
past the era loop.
Moved the bodies into Expert\AIBase\, included at the bottom of the original
after the class declaration:
Training.mqh 1607 era loop, plateau ladder, checkpoint select, deploy
Features.mqh 1093 indicator creation + per-bar input feature vector
ChartUI.mqh 634 arrows, arrow persistence, status panel, cleanup
Persistence.mqh 492 .stats/.cfg sidecars, CPU-inference validation, copy
OnlineLearning.mqh 461 live continual learning, EMA shadow, OOS simulator
Labels.mqh 309 ZigZag pivot labels, async label-cache prebuild
AutoTune.mqh 275 genetic tuner (population, crossover, halving)
Inference.mqh 235 softmax, prior calibration, class priors
ExpertSignalAIBase.mqh 8216 -> 3131 (declaration + topology build only)
This is a pure relocation - verified mechanically, not by eye: HEAD's file
reconstructed from the eight partials plus the surviving remainder is
byte-identical to HEAD, span for span (scratchpad verify_split.py). No
declaration moved, no signature changed, no code rewritten, so behaviour is
unchanged by construction.
Compiles 0 errors, 0 warnings.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-29 00:42:45 -04:00
|
|
|
}
|
|
|
|
|
return ok;
|
|
|
|
|
}
|
|
|
|
|
//+------------------------------------------------------------------+
|
fix: the sequence models were reading the window backwards
BuildFeatureWindow() replaces eight hand-rolled copies of the same loop
and feeds the window OLDEST BAR FIRST. Every copy fed it newest-first,
because MQL5 timeseries indices run backwards and `r + b` with b ascending
walks into the past.
Harmless for PAI and CONV - a dense layer learns a weight per position
either way, a conv learns time-mirrored kernels. Not harmless for the
recurrent stacks:
- LSTM_SeqStepForward reads `inputs + t*Iw`, so step t is block t.
- It writes output[] only when t == steps-1: the visible output IS the
last hidden state.
- c_t = f*c_{t-1} + i*g decays toward the start of the sequence.
lstm_seq_flowcheck.cpp measured block 0's influence on the output at
1.2e-2 of block T-1's, at the shipped forget bias of 1.0.
So the bar being PREDICTED sat at the far end of the decay and the output
was handed to the OLDEST bar in the window - the exact inverse of what the
window is for. ~80x backwards on LSTM and HYBRID, on all three tiers
(OpenCL kernel, CPU DLL, pure-MQL5 inference), which is why it never
surfaced as a backend discrepancy.
This does not create edge - the MI diagnostics read at the noise floor
(p=0.4975) with a working positive control. It makes the one hypothesis
those diagnostics explicitly do NOT cover testable: they are marginal and
per-bar, and state they "cannot rule out one that only exists in
combination or across time". The sequence model is the instrument for
across-time structure and it has been crippled, so that hypothesis has
never been honestly tested.
Fingerprint gets an unconditional |WIN:2 - the vector keeps its shape and
its features, so a stale .nnw would load cleanly and run a model fitted to
one ordering against the other, silently. Re-keying every config is the
point, not collateral damage. FORCES A FULL RETRAIN.
Also: the now-relative bar caches are re-keyed on the two live paths.
EnsureBarCachesCapacity() was only ever called from training paths, but
once m_trainingComplete is set ScheduleTrainingIfNeeded() routes every bar
to RefreshConvergedSignal() and Train() is never re-entered - so nothing
cleared the feature cache again for the life of the process. A chart that
trained to convergence kept replaying the rows computed for the last
training era's bar grid: the live signal froze at its convergence-time
value, and OnlineLearnStep() backpropped those stale features against
freshly resolved labels. Backtests were never affected (an inference-only
process never allocates the arrays, so every read recomputes).
Compiles clean: 0 errors, 0 warnings.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-07 18:28:44 -04:00
|
|
|
//| THE ONE PLACE a feature WINDOW is assembled. Every consumer - |
|
|
|
|
|
//| training pass 1/2/3, live inference, online learning, the OOS |
|
|
|
|
|
//| continual simulation, the chart rescan and the CPU-inference |
|
|
|
|
|
//| self-check - goes through here, because the thing this function |
|
|
|
|
|
//| fixes is a contract that eight hand-rolled copies of the same |
|
|
|
|
|
//| loop cannot hold on their own. |
|
|
|
|
|
//| |
|
|
|
|
|
//| ORDER IS CHRONOLOGICAL: OLDEST BAR FIRST, bar `r` (the bar being |
|
|
|
|
|
//| predicted) LAST. That is the whole point of this function. |
|
|
|
|
|
//| |
|
|
|
|
|
//| MQL5 timeseries indices run BACKWARDS - index 0 is the newest bar |
|
|
|
|
|
//| and increasing index walks into the past. So the obvious loop, |
|
|
|
|
|
//| `for(b = 0..T-1) BufferTempData(r + b)`, appends the window in |
|
|
|
|
|
//| REVERSE chronological order: the newest bar lands in block 0 and |
|
|
|
|
|
//| the oldest in block T-1. That is what every call site used to do. |
|
|
|
|
|
//| |
|
|
|
|
|
//| For the dense (PAI) and convolutional stacks it is harmless - a |
|
|
|
|
|
//| dense layer learns a weight per position either way, and a conv |
|
|
|
|
|
//| just learns time-mirrored kernels. For the RECURRENT stacks it is |
|
|
|
|
|
//| not, and it is not a subtlety: |
|
|
|
|
|
//| - CNeuronLSTMOCL walks steps t = 0..T-1 reading `inputs + t*Iw` |
|
|
|
|
|
//| (AI\Network.cl, LSTM_SeqStepForward), so step t consumes the |
|
|
|
|
|
//| t-th block in buffer order. |
|
|
|
|
|
//| - Its visible output is the LAST hidden state only - the kernel |
|
|
|
|
|
//| writes `output[id]` solely when `t == steps - 1`. |
|
|
|
|
|
//| - The cell state decays toward the start of the sequence: |
|
|
|
|
|
//| c_t = f*c_{t-1} + i*g. DirectML\lstm_seq_flowcheck.cpp measured |
|
|
|
|
|
//| block 0's influence on the output, relative to block T-1, at |
|
|
|
|
|
//| 1.2e-2 for the shipped LSTM_FORGET_BIAS_INIT of 1.0 (see that |
|
|
|
|
|
//| constant's comment for the full sweep). |
|
|
|
|
|
//| Fed newest-first, that put the bar being PREDICTED at the far end |
|
|
|
|
|
//| of the decay and handed the output to the OLDEST bar in the window |
|
|
|
|
|
//| - roughly 80x backwards, and the exact inverse of what the window |
|
|
|
|
|
//| exists for ("everything known as of this bar's close", see Train() |
|
|
|
|
|
//| 's r comment). Reversing it here makes the final timestep the |
|
|
|
|
|
//| current bar, which is the standard arrangement and the one the |
|
|
|
|
|
//| forget-bias sweep was implicitly reasoning about. |
|
|
|
|
|
//| |
|
|
|
|
|
//| Nothing downstream reads a fixed block position, so this is safe |
|
|
|
|
|
//| for every topology; it re-keys the weight fingerprint (see |
|
|
|
|
|
//| ConfigFingerprint's WIN token) precisely BECAUSE the input vector |
|
|
|
|
|
//| now means something different, and models trained under the old |
|
|
|
|
|
//| order must never load into it. |
|
|
|
|
|
//| |
|
|
|
|
|
//| Returns true only when the COMPLETE, correctly-sized window is in |
|
|
|
|
|
//| TempData - callers must not feedForward on a partial one (a stale |
|
|
|
|
|
//| output layer would be scored against this bar's label). |
|
refactor: split CExpertSignalAIBase implementation by responsibility
ExpertSignalAIBase.mqh was 8216 lines: the class declaration followed by 87
method bodies covering training, labelling, feature extraction, persistence,
chart drawing, online learning, the GA auto-tuner and inference, all in one
file. Train() alone is 1492 lines; a change to arrow drawing meant scrolling
past the era loop.
Moved the bodies into Expert\AIBase\, included at the bottom of the original
after the class declaration:
Training.mqh 1607 era loop, plateau ladder, checkpoint select, deploy
Features.mqh 1093 indicator creation + per-bar input feature vector
ChartUI.mqh 634 arrows, arrow persistence, status panel, cleanup
Persistence.mqh 492 .stats/.cfg sidecars, CPU-inference validation, copy
OnlineLearning.mqh 461 live continual learning, EMA shadow, OOS simulator
Labels.mqh 309 ZigZag pivot labels, async label-cache prebuild
AutoTune.mqh 275 genetic tuner (population, crossover, halving)
Inference.mqh 235 softmax, prior calibration, class priors
ExpertSignalAIBase.mqh 8216 -> 3131 (declaration + topology build only)
This is a pure relocation - verified mechanically, not by eye: HEAD's file
reconstructed from the eight partials plus the surviving remainder is
byte-identical to HEAD, span for span (scratchpad verify_split.py). No
declaration moved, no signature changed, no code rewritten, so behaviour is
unchanged by construction.
Compiles 0 errors, 0 warnings.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-29 00:42:45 -04:00
|
|
|
//+------------------------------------------------------------------+
|
fix(altdata): median-fill instead of zero-fill, and a one-shot feature-vector autopsy
ALT-DATA AUDIT. The files themselves are healthy - all six symbols, 6,073 daily
rows, 2010-01-01 to 2026-08-17, no constant or degenerate columns, sane tails
(mac_cpi/mac_unemp flat ~47d is monthly data behaving correctly). The problem is
not the data, it is what happens where the data ISN'T.
CAltDataPanel::Features() returned an all-ZERO vector for any bar older than the
file's first row, and left blank cells at 0 too. Both were deliberate ('the block
is additive context and must degrade, never reject the bar') and that reasoning
holds for the CHANGE columns - but half these features are LEVELS: vix, ivol,
mac_y10, mac_cpi, mac_unemp, eia_util. For a level, 0 is not a missing reading,
it is an impossible one far outside the series' range. VIX does not visit zero.
And the spike lands in exactly the wrong place. Every alt file starts 2010-01-01
while the charts run far deeper - USDJPY H4 reaches ~1994, roughly HALF its
history - so 'alt block is all zeros' is precisely the predicate 'this bar is
older than 2010'. The IS/OOS split is chronological, so that predicate covers
~half of IS and none of OOS: an in-sample feature guaranteed to be useless
out-of-sample, and a bimodal input for the first BatchNorm to normalise. Not a
lookahead leak - a distribution corruption, which is quieter and was never
reported anywhere.
Now filled with the column MEDIAN over the covered range. A constant cannot leak
whatever its source - it takes the same value on every pre-coverage bar, so it
carries no information about which of those bars won - which is what makes a
median computed over later data legitimate here. Median not mean because the
series are skewed. Blank cells get the same treatment (eia_stk_idx1y alone has
181 blanks in 6,073 rows) and the count is now logged at load.
THE BACKOFF WAS ALREADY THERE AND WAS DEAD. Training.mqh arms m_coldSweepTick on
m_featureFailTransient, but only the open/ATR guards ever set that flag, so
f0cf659's cold ADMovingAverage looked PERMANENT and the sweep re-ran at full
speed forever. Setting the flag in the indicator guards revives the mechanism
that was already designed for this; no second backoff was needed and the one I
first wrote has been removed in favour of it.
SELF-HEALING, as asked. ReportFeatureHealth() runs once, the first time pass 1
produces usable windows, samples ~400 bars spread across the whole training range
and names every feature slot that is CONSTANT or mostly-zero, tagging alt-block
slots as alt[i]. Both of today's failures were the same shape - a block silently
produces nothing while every downstream number stays plausible - and neither an
accuracy figure nor a model can tell 'this feature is always 0' from 'this
feature is genuinely 0 here'. Evenly spaced sampling so a block that dies only in
deep history is caught as surely as one dead everywhere. A report, not a gate:
a rare-flag feature can be legitimately constant, and refusing to train would
turn a diagnostic into an outage.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-17 12:53:51 -04:00
|
|
|
//+------------------------------------------------------------------+
|
|
|
|
|
//| ONE-SHOT FEATURE-VECTOR AUTOPSY, run the first time pass 1 finds |
|
|
|
|
|
//| usable windows. Samples bars across the whole training range and |
|
|
|
|
|
//| reports any feature slot that is DEGENERATE - constant, mostly |
|
|
|
|
|
//| zero, or non-finite - naming the block it belongs to. |
|
|
|
|
|
//| |
|
|
|
|
|
//| WHY THIS EXISTS. Two separate 2026-08-17 failures were invisible |
|
|
|
|
|
//| for hours because nothing ever looked at the assembled vector: |
|
|
|
|
|
//| |
|
|
|
|
|
//| - a cold ADMovingAverage returned EMPTY_VALUE for every index, |
|
|
|
|
|
//| so every window was rejected and cached as a permanent miss. |
|
|
|
|
|
//| The only symptom was a scan that never finished. |
|
|
|
|
|
//| - the alt-data block 0-filled every bar older than its file's |
|
|
|
|
|
//| first row (2010-01-01), which on USDJPY is ~half the history. |
|
|
|
|
|
//| Training simply absorbed 15 impossible zeros per bar. |
|
|
|
|
|
//| |
|
|
|
|
|
//| Both are the same shape of bug: a block silently produces |
|
|
|
|
|
//| nothing, and every downstream number stays plausible. A model |
|
|
|
|
|
//| CANNOT tell "this feature is always 0" from "this feature is |
|
|
|
|
|
//| genuinely 0 here", and neither can an accuracy figure - so the |
|
|
|
|
|
//| check has to happen where the values are, once, and shout. |
|
|
|
|
|
//| |
|
|
|
|
|
//| Deliberately a REPORT, not a gate: a degenerate feature is |
|
|
|
|
|
//| sometimes legitimate (a flag that is rare in this window), and |
|
|
|
|
|
//| refusing to train would turn a diagnostic into an outage. It |
|
|
|
|
|
//| names the problem and lets the operator decide. |
|
|
|
|
|
//+------------------------------------------------------------------+
|
|
|
|
|
void CExpertSignalAIBase::ReportFeatureHealth(int bars)
|
|
|
|
|
{
|
|
|
|
|
if(m_featureHealthReported || m_neuronsCount <= 0)
|
|
|
|
|
return;
|
|
|
|
|
m_featureHealthReported = true;
|
|
|
|
|
int per = m_neuronsCount; // features per BAR
|
|
|
|
|
int lo = MathMax((int)m_historyBars + MathMax(m_barrierHorizonBars, 1) + 2, 2);
|
|
|
|
|
int hi = MathMax(bars - 2, lo);
|
|
|
|
|
if(hi <= lo)
|
|
|
|
|
return;
|
|
|
|
|
//--- Evenly spaced sample across the whole range, so a block that dies only in the deep history
|
|
|
|
|
//--- (the alt-coverage case) is caught as surely as one that is dead everywhere (the cold-indicator
|
|
|
|
|
//--- case). 400 bars is enough to call a feature constant and costs a fraction of one era.
|
|
|
|
|
int want = 400;
|
|
|
|
|
int step = MathMax((hi - lo) / want, 1);
|
|
|
|
|
double vmin[], vmax[];
|
|
|
|
|
int zeroCnt[], seen = 0;
|
|
|
|
|
ArrayResize(vmin, per);
|
|
|
|
|
ArrayResize(vmax, per);
|
|
|
|
|
ArrayResize(zeroCnt, per);
|
|
|
|
|
for(int j = 0; j < per; j++)
|
|
|
|
|
{
|
|
|
|
|
vmin[j] = DBL_MAX;
|
|
|
|
|
vmax[j] = -DBL_MAX;
|
|
|
|
|
zeroCnt[j] = 0;
|
|
|
|
|
}
|
|
|
|
|
for(int i = lo; i <= hi; i += step)
|
|
|
|
|
{
|
|
|
|
|
//--- Read ONE bar's block, not a whole window: the per-bar row is what the blocks produce, and
|
|
|
|
|
//--- BuildFeatureWindow would just replicate it historyBars times.
|
|
|
|
|
TempData.Clear();
|
|
|
|
|
if(!BufferTempData(i))
|
|
|
|
|
continue;
|
|
|
|
|
if(TempData.Total() < per)
|
|
|
|
|
continue;
|
|
|
|
|
//--- The bar's own row is the LAST `per` values (BufferTempData appends).
|
|
|
|
|
int base = TempData.Total() - per;
|
|
|
|
|
for(int j = 0; j < per; j++)
|
|
|
|
|
{
|
|
|
|
|
double v = TempData.At(base + j);
|
|
|
|
|
if(!MathIsValidNumber(v))
|
|
|
|
|
continue;
|
|
|
|
|
if(v < vmin[j]) vmin[j] = v;
|
|
|
|
|
if(v > vmax[j]) vmax[j] = v;
|
|
|
|
|
if(v == 0.0) zeroCnt[j]++;
|
|
|
|
|
}
|
|
|
|
|
seen++;
|
|
|
|
|
}
|
|
|
|
|
if(seen < 20)
|
|
|
|
|
{
|
|
|
|
|
Print(ID + StringFormat(": feature health - only %d of %d sampled bars produced a readable row;"
|
|
|
|
|
" too few to judge. This is itself a warning: if it persists the feature"
|
|
|
|
|
" path is rejecting nearly everything.", seen, want));
|
|
|
|
|
return;
|
|
|
|
|
}
|
|
|
|
|
//--- The alt block is appended LAST in every row (see BufferTempDataCompute), so its slots are the
|
|
|
|
|
//--- final FeatureCount() of the `per`. Naming it matters: "slot 47 is constant" is a puzzle,
|
|
|
|
|
//--- "the alt-data block is constant" is an answer.
|
|
|
|
|
int altN = (m_useAltData ? m_altData.FeatureCount() : 0);
|
|
|
|
|
int altFrom = (altN > 0 && altN <= per) ? per - altN : -1;
|
|
|
|
|
string deadList = "", zeroList = "";
|
|
|
|
|
int dead = 0, mostlyZero = 0;
|
|
|
|
|
for(int j = 0; j < per; j++)
|
|
|
|
|
{
|
|
|
|
|
if(vmin[j] > vmax[j])
|
|
|
|
|
continue; // never read
|
|
|
|
|
bool isConst = (vmax[j] - vmin[j]) <= 1e-12;
|
|
|
|
|
bool isZeroy = (zeroCnt[j] * 2 > seen);
|
|
|
|
|
string tag = (altFrom >= 0 && j >= altFrom)
|
|
|
|
|
? StringFormat("alt[%d]", j - altFrom) : StringFormat("f%d", j);
|
|
|
|
|
if(isConst)
|
|
|
|
|
{
|
|
|
|
|
dead++;
|
|
|
|
|
if(dead <= 12)
|
|
|
|
|
deadList += StringFormat("%s%s=%.4g", (deadList == "" ? "" : " "), tag, vmin[j]);
|
|
|
|
|
}
|
|
|
|
|
else
|
|
|
|
|
if(isZeroy)
|
|
|
|
|
{
|
|
|
|
|
mostlyZero++;
|
|
|
|
|
if(mostlyZero <= 12)
|
|
|
|
|
zeroList += StringFormat("%s%s(%.0f%%)", (zeroList == "" ? "" : " "), tag,
|
|
|
|
|
100.0 * zeroCnt[j] / seen);
|
|
|
|
|
}
|
|
|
|
|
}
|
|
|
|
|
Print(ID + StringFormat(": FEATURE HEALTH on %d sampled bars x %d features%s - %d CONSTANT%s%s |"
|
|
|
|
|
" %d mostly-zero (>50%%)%s%s. A constant feature contributes nothing but"
|
|
|
|
|
" still consumes a first-layer column and a BatchNorm slot; a block that is"
|
|
|
|
|
" constant AND zero is usually a source that failed silently rather than a"
|
|
|
|
|
" quiet market.",
|
|
|
|
|
seen, per,
|
|
|
|
|
(altFrom >= 0 ? StringFormat(" (alt block = slots %d..%d)", altFrom, per - 1)
|
|
|
|
|
: " (no alt block)"),
|
|
|
|
|
dead, (deadList == "" ? "" : ": "), deadList,
|
|
|
|
|
mostlyZero, (zeroList == "" ? "" : ": "), zeroList));
|
|
|
|
|
}
|
|
|
|
|
//+------------------------------------------------------------------+
|
fix: the sequence models were reading the window backwards
BuildFeatureWindow() replaces eight hand-rolled copies of the same loop
and feeds the window OLDEST BAR FIRST. Every copy fed it newest-first,
because MQL5 timeseries indices run backwards and `r + b` with b ascending
walks into the past.
Harmless for PAI and CONV - a dense layer learns a weight per position
either way, a conv learns time-mirrored kernels. Not harmless for the
recurrent stacks:
- LSTM_SeqStepForward reads `inputs + t*Iw`, so step t is block t.
- It writes output[] only when t == steps-1: the visible output IS the
last hidden state.
- c_t = f*c_{t-1} + i*g decays toward the start of the sequence.
lstm_seq_flowcheck.cpp measured block 0's influence on the output at
1.2e-2 of block T-1's, at the shipped forget bias of 1.0.
So the bar being PREDICTED sat at the far end of the decay and the output
was handed to the OLDEST bar in the window - the exact inverse of what the
window is for. ~80x backwards on LSTM and HYBRID, on all three tiers
(OpenCL kernel, CPU DLL, pure-MQL5 inference), which is why it never
surfaced as a backend discrepancy.
This does not create edge - the MI diagnostics read at the noise floor
(p=0.4975) with a working positive control. It makes the one hypothesis
those diagnostics explicitly do NOT cover testable: they are marginal and
per-bar, and state they "cannot rule out one that only exists in
combination or across time". The sequence model is the instrument for
across-time structure and it has been crippled, so that hypothesis has
never been honestly tested.
Fingerprint gets an unconditional |WIN:2 - the vector keeps its shape and
its features, so a stale .nnw would load cleanly and run a model fitted to
one ordering against the other, silently. Re-keying every config is the
point, not collateral damage. FORCES A FULL RETRAIN.
Also: the now-relative bar caches are re-keyed on the two live paths.
EnsureBarCachesCapacity() was only ever called from training paths, but
once m_trainingComplete is set ScheduleTrainingIfNeeded() routes every bar
to RefreshConvergedSignal() and Train() is never re-entered - so nothing
cleared the feature cache again for the life of the process. A chart that
trained to convergence kept replaying the rows computed for the last
training era's bar grid: the live signal froze at its convergence-time
value, and OnlineLearnStep() backpropped those stale features against
freshly resolved labels. Backtests were never affected (an inference-only
process never allocates the arrays, so every read recomputes).
Compiles clean: 0 errors, 0 warnings.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-07 18:28:44 -04:00
|
|
|
bool CExpertSignalAIBase::BuildFeatureWindow(int r)
|
|
|
|
|
{
|
|
|
|
|
int width = (int)m_historyBars * m_neuronsCount;
|
|
|
|
|
TempData.Clear();
|
|
|
|
|
TempData.Reserve(width);
|
|
|
|
|
if(r < 0 || m_historyBars <= 0 || m_neuronsCount <= 0)
|
|
|
|
|
return false;
|
2026-08-16 13:39:00 -04:00
|
|
|
//--- Live-only freshness probe for the external block: two comparisons when quiet, a reload at
|
|
|
|
|
//--- most hourly once the chart outruns the exported data. Never fires in the tester (the newest
|
|
|
|
|
//--- bar is historical there). Safe against the feature cache: new alt rows only exist for a new
|
|
|
|
|
//--- D1 bar, and a new bar resets the per-bar-frame cache anyway.
|
|
|
|
|
if(m_useAltData)
|
|
|
|
|
m_altData.EnsureFresh((datetime)m_Time.GetData(0));
|
fix: the sequence models were reading the window backwards
BuildFeatureWindow() replaces eight hand-rolled copies of the same loop
and feeds the window OLDEST BAR FIRST. Every copy fed it newest-first,
because MQL5 timeseries indices run backwards and `r + b` with b ascending
walks into the past.
Harmless for PAI and CONV - a dense layer learns a weight per position
either way, a conv learns time-mirrored kernels. Not harmless for the
recurrent stacks:
- LSTM_SeqStepForward reads `inputs + t*Iw`, so step t is block t.
- It writes output[] only when t == steps-1: the visible output IS the
last hidden state.
- c_t = f*c_{t-1} + i*g decays toward the start of the sequence.
lstm_seq_flowcheck.cpp measured block 0's influence on the output at
1.2e-2 of block T-1's, at the shipped forget bias of 1.0.
So the bar being PREDICTED sat at the far end of the decay and the output
was handed to the OLDEST bar in the window - the exact inverse of what the
window is for. ~80x backwards on LSTM and HYBRID, on all three tiers
(OpenCL kernel, CPU DLL, pure-MQL5 inference), which is why it never
surfaced as a backend discrepancy.
This does not create edge - the MI diagnostics read at the noise floor
(p=0.4975) with a working positive control. It makes the one hypothesis
those diagnostics explicitly do NOT cover testable: they are marginal and
per-bar, and state they "cannot rule out one that only exists in
combination or across time". The sequence model is the instrument for
across-time structure and it has been crippled, so that hypothesis has
never been honestly tested.
Fingerprint gets an unconditional |WIN:2 - the vector keeps its shape and
its features, so a stale .nnw would load cleanly and run a model fitted to
one ordering against the other, silently. Re-keying every config is the
point, not collateral damage. FORCES A FULL RETRAIN.
Also: the now-relative bar caches are re-keyed on the two live paths.
EnsureBarCachesCapacity() was only ever called from training paths, but
once m_trainingComplete is set ScheduleTrainingIfNeeded() routes every bar
to RefreshConvergedSignal() and Train() is never re-entered - so nothing
cleared the feature cache again for the life of the process. A chart that
trained to convergence kept replaying the rows computed for the last
training era's bar grid: the live signal froze at its convergence-time
value, and OnlineLearnStep() backpropped those stale features against
freshly resolved labels. Backtests were never affected (an inference-only
process never allocates the arrays, so every read recomputes).
Compiles clean: 0 errors, 0 warnings.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-07 18:28:44 -04:00
|
|
|
//--- b counts bars BACK from r, so (m_historyBars - 1 - b) emits the deepest lookback first and
|
|
|
|
|
//--- lands on r itself on the final iteration. Identical set of bars as before, opposite order.
|
|
|
|
|
for(int b = 0; b < (int)m_historyBars; b++)
|
|
|
|
|
if(!BufferTempData(r + ((int)m_historyBars - 1 - b)))
|
diag: name the cause when every feature window fails, and enforce the width contract
Era 0 stalls with "NOT ONE of 54681 scanned bars produced a usable
feature window, windows ok=0 failed=54681" and nothing else. That line
reads identically for a cold ATR, a conditionally-missing optional
feature block and an out-of-range index, so it cannot be diagnosed
without one restart per hypothesis.
Two changes:
1. WIDTH CONTRACT in BufferTempData. Every enabled block must emit
exactly m_neuronsCount values on EVERY bar. A block that emits its
values on some bars and skips them on others (indicator, panel or
series unavailable for that bar) does not merely shorten the window -
it SHIFTS every feature after it into the wrong slot, and the net
then trains on silently misaligned inputs that still look like a
valid window to everything downstream. Now rejected, rolled back and
reported once, naming the optional blocks (XA / SPR / swing context)
as the ones carrying an availability test. Worth having independently
of the current stall.
2. BuildFeatureWindow records WHICH lookback slot rejected and how much
of the window was assembled, and the pass-1 stall report renders it:
"slot 0 of 20 REJECTED (window had 0 of 760)" is an indicator warm-up
or history-edge read; "every lookback bar ACCEPTED and the window was
still short: 640 of 760" is a missing 6-value block.
No behaviour change on a healthy run: the width check is an equality
that already holds, and the diagnostics render only inside the
total-failure branch.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-11 10:31:56 -04:00
|
|
|
{
|
|
|
|
|
//--- Which lookback slot rejected, and how much of the window had been assembled. Without
|
|
|
|
|
//--- this the pass-1 stall report can only say "0 of 54681 usable", which is true of a cold
|
|
|
|
|
//--- ATR, a missing optional block and an out-of-range index alike.
|
|
|
|
|
m_windowFailSlot = b;
|
|
|
|
|
m_windowFailTotal = TempData.Total();
|
fix: the sequence models were reading the window backwards
BuildFeatureWindow() replaces eight hand-rolled copies of the same loop
and feeds the window OLDEST BAR FIRST. Every copy fed it newest-first,
because MQL5 timeseries indices run backwards and `r + b` with b ascending
walks into the past.
Harmless for PAI and CONV - a dense layer learns a weight per position
either way, a conv learns time-mirrored kernels. Not harmless for the
recurrent stacks:
- LSTM_SeqStepForward reads `inputs + t*Iw`, so step t is block t.
- It writes output[] only when t == steps-1: the visible output IS the
last hidden state.
- c_t = f*c_{t-1} + i*g decays toward the start of the sequence.
lstm_seq_flowcheck.cpp measured block 0's influence on the output at
1.2e-2 of block T-1's, at the shipped forget bias of 1.0.
So the bar being PREDICTED sat at the far end of the decay and the output
was handed to the OLDEST bar in the window - the exact inverse of what the
window is for. ~80x backwards on LSTM and HYBRID, on all three tiers
(OpenCL kernel, CPU DLL, pure-MQL5 inference), which is why it never
surfaced as a backend discrepancy.
This does not create edge - the MI diagnostics read at the noise floor
(p=0.4975) with a working positive control. It makes the one hypothesis
those diagnostics explicitly do NOT cover testable: they are marginal and
per-bar, and state they "cannot rule out one that only exists in
combination or across time". The sequence model is the instrument for
across-time structure and it has been crippled, so that hypothesis has
never been honestly tested.
Fingerprint gets an unconditional |WIN:2 - the vector keeps its shape and
its features, so a stale .nnw would load cleanly and run a model fitted to
one ordering against the other, silently. Re-keying every config is the
point, not collateral damage. FORCES A FULL RETRAIN.
Also: the now-relative bar caches are re-keyed on the two live paths.
EnsureBarCachesCapacity() was only ever called from training paths, but
once m_trainingComplete is set ScheduleTrainingIfNeeded() routes every bar
to RefreshConvergedSignal() and Train() is never re-entered - so nothing
cleared the feature cache again for the life of the process. A chart that
trained to convergence kept replaying the rows computed for the last
training era's bar grid: the live signal froze at its convergence-time
value, and OnlineLearnStep() backpropped those stale features against
freshly resolved labels. Backtests were never affected (an inference-only
process never allocates the arrays, so every read recomputes).
Compiles clean: 0 errors, 0 warnings.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-07 18:28:44 -04:00
|
|
|
return false;
|
diag: name the cause when every feature window fails, and enforce the width contract
Era 0 stalls with "NOT ONE of 54681 scanned bars produced a usable
feature window, windows ok=0 failed=54681" and nothing else. That line
reads identically for a cold ATR, a conditionally-missing optional
feature block and an out-of-range index, so it cannot be diagnosed
without one restart per hypothesis.
Two changes:
1. WIDTH CONTRACT in BufferTempData. Every enabled block must emit
exactly m_neuronsCount values on EVERY bar. A block that emits its
values on some bars and skips them on others (indicator, panel or
series unavailable for that bar) does not merely shorten the window -
it SHIFTS every feature after it into the wrong slot, and the net
then trains on silently misaligned inputs that still look like a
valid window to everything downstream. Now rejected, rolled back and
reported once, naming the optional blocks (XA / SPR / swing context)
as the ones carrying an availability test. Worth having independently
of the current stall.
2. BuildFeatureWindow records WHICH lookback slot rejected and how much
of the window was assembled, and the pass-1 stall report renders it:
"slot 0 of 20 REJECTED (window had 0 of 760)" is an indicator warm-up
or history-edge read; "every lookback bar ACCEPTED and the window was
still short: 640 of 760" is a missing 6-value block.
No behaviour change on a healthy run: the width check is an equality
that already holds, and the diagnostics render only inside the
total-failure branch.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-11 10:31:56 -04:00
|
|
|
}
|
|
|
|
|
if(TempData.Total() < width)
|
|
|
|
|
{
|
|
|
|
|
//--- Nothing rejected the bar and the window is still short. Distinct fault from a guard
|
|
|
|
|
//--- rejection and it used to be indistinguishable from one; the per-bar width contract in
|
|
|
|
|
//--- BufferTempData should now catch this first, so reaching here means the shortfall is in the
|
|
|
|
|
//--- window assembly itself rather than in one bar's blocks.
|
|
|
|
|
m_windowFailSlot = -1;
|
|
|
|
|
m_windowFailTotal = TempData.Total();
|
|
|
|
|
return false;
|
|
|
|
|
}
|
fix(features): collapse only the anchor's own run - leave lagged readings put
User's call before deploy: "I would rather avoid lagging so the NN finds
accurate patterns." Correct instinct, and it picks the conservative variant.
110b384 deduplicated the WHOLE window, so every distinct reading survived at one
slot. The flaw is which slot: it depends on where the calendar-day boundary falls
inside that particular window, and on H4 that boundary cycles through ~6 phases.
A dense layer holds a separate weight per (slot, feature), so a given lag would
have landed on a different coordinate from one window to the next - turning a
stable lagged input into a moving one.
Now it blanks only bars carrying a BYTE-IDENTICAL copy of the anchor's reading
and stops at the first bar that differs. An as-of lookup into a daily file is a
step function in time, so those copies are exactly the contiguous run of bars
sharing the anchor's calendar day. Everything older keeps its natural replicated
run, in the same slots it always occupied - whatever the net learned to read
there, it still reads there.
Why the anchor's reading is the right one to isolate: the window's newest slot IS
the bar being predicted (BuildFeatureWindow's final iteration lands on r, and
pass 3 grades that same index), so it is the reading contemporaneous with the
decision - and the only one the alt screens ever validated. They measured the
CURRENT reading's MI against forward range and never tested lags, so the lagged
content is unproven, which is a reason to leave it undisturbed rather than a
licence to rearrange it.
What is still fixed: the anchor's reading reaches the first layer on one
coordinate instead of once per bar of its day, removing the ~16x gradient
upweight for the validated signal. And this is IDENTICAL to full dedup exactly
where replication was worst - on M15/H1 the whole window sits inside one calendar
day, so the anchor's run is the whole window - and a no-op on D1, where the bar
before the anchor is already a different day and the loop breaks immediately.
The two differ only on middle timeframes, and there this is the safe side.
Fingerprint |ALTW:1 -> |ALTW:2 so nothing trained under the hour-old full-dedup
semantics can silently resume under these.
Compile-verified: 0 errors, 0 warnings.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-16 22:21:22 -04:00
|
|
|
//--- THE ANCHOR BAR'S EXTERNAL READING ENTERS THE WINDOW ONCE, NOT ONCE PER BAR OF ITS DAY (2026-08-16).
|
perf(features): the external block enters the window once, not once per bar
Measured on the live SP500 D1 export (6073 rows, 13 features, 5888 simulated
16-bar windows):
distinct values per feature per window : 1.7 - 2.7 of 16 slots
variance in the first 13 PCs : 96.5 - 97.0%
components for 95% / 99% : 12 / 17-19
effective rank (entropy) : ~11.5
208 inputs carrying about 12 dimensions. Only 6 of the 13 features move daily
(VIX complex, USD, the rates trio); 5 are weekly (COT, EIA, output gap) and 2
monthly (CPI, unemployment). The lookup is as-of by bar open time into a DAILY
file, so bars sharing a calendar day are byte-identical by construction.
The cost is NOT overfitting capacity - collinear copies span ~12 directions,
not 208, so an earlier claim that this wasted 26% of the model overstated it.
It is GRADIENT WEIGHTING. Batch norm standardizes each of the 208 coordinates
independently; that rescales the copies without decorrelating them, so one
factor arrives on 16 unit-variance coordinates, each weight takes a full-size
step, and the factor's aggregate coefficient moves ~16x faster than a per-bar
price feature's. The network was biased toward the external block by a factor
of the window length - and pointing the wrong way, since these features cleared
only a marginal incremental screen while price is the base signal.
Zeroed at WINDOW ASSEMBLY, not in BufferTempData: that output is cached PER BAR
and a bar sits at slot 15 of one window and slot 0 of the next, so a
slot-dependent value there would poison the cache or force a recompute per slot.
The cache keeps true values; only this window's copies are cleared. Width
contract untouched - same count, same positions - so conv/LSTM/HYBRID keep their
bar-major rectangle unchanged and the block arrives at the newest bar, which for
the LSTM is the final timestep. Zero-variance coordinates are safe through batch
norm (divisor is MathMax(MathSqrt(var + BN_EPSILON), BN_MIN_STD)).
Fingerprint gains |ALTW:1 when alt data is on. Same width and same .cfg, so
nothing else would have caught a model trained under the replicated layout
resuming under this one. Conditional append per the existing rule: configs
without alt data keep their fingerprints and their trained models.
NOT the concat branch. CNet is a strictly linear stack (CLayerDescription has no
input-source field; NetBuild wires i to i+1 and stores layer L's weights on
L-1), so a real two-tower model needs a new multi-input layer type across
WarriorCPU, WarriorDML and the OpenCL kernels plus an .nnw format change - the
highest-risk change in this repo, in the code that produced the transposed dense
gradient, the Adam second-moment bug and the reversed LSTM window. This captures
the part of that idea the measurement actually supports, at no engine risk.
Compile-verified: 0 errors, 0 warnings.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-16 22:05:10 -04:00
|
|
|
//---
|
|
|
|
|
//--- The alt block is appended LAST in every bar's row (see BufferTempDataCompute), and its lookup
|
|
|
|
|
//--- is as-of by bar open time into a DAILY file - so every bar of the window that falls on the same
|
|
|
|
|
//--- calendar day gets byte-identical values. Measured on the live SP500 D1 export (6073 rows, 13
|
|
|
|
|
//--- features, 5888 windows): each feature takes 1.7-2.7 distinct values across the 16 slots, 12
|
|
|
|
|
//--- principal components carry 95% of the block's variance and its effective rank is ~11.5. So 208
|
|
|
|
|
//--- inputs were carrying about 12 dimensions. Only 6 of the 13 features move daily at all; 5 are
|
|
|
|
|
//--- weekly (COT, EIA) and 2 monthly (CPI, unemployment).
|
|
|
|
|
//---
|
|
|
|
|
//--- The cost is NOT overfitting capacity - the copies are collinear, so they span ~12 directions,
|
|
|
|
|
//--- not 208. It is the GRADIENT WEIGHTING. Batch norm standardizes each of the 208 coordinates
|
|
|
|
|
//--- independently, which rescales the copies but does not decorrelate them, so the same factor
|
|
|
|
|
//--- arrives on 16 unit-variance coordinates. Each of their weights takes a full-size step, and the
|
|
|
|
|
//--- factor's aggregate coefficient therefore moves ~16x faster than a per-bar price feature's. The
|
|
|
|
|
//--- network was structurally biased toward the external block by a factor of the window length -
|
|
|
|
|
//--- an unintended prior, and pointing the wrong way, since these features cleared only a marginal
|
|
|
|
|
//--- incremental screen while price features are the base signal.
|
|
|
|
|
//---
|
|
|
|
|
//--- Zeroed here, at WINDOW ASSEMBLY, rather than in BufferTempData: that function's output is cached
|
|
|
|
|
//--- PER BAR (m_featureCache) and a bar sits at slot 15 of one window and slot 0 of the next, so a
|
|
|
|
|
//--- slot-dependent value there would either poison the cache or force a recompute on every slot.
|
|
|
|
|
//--- The cache keeps the true values; only the copies in this window are cleared. The width contract
|
|
|
|
|
//--- is untouched (same count, same positions) so conv/LSTM/HYBRID keep their bar-major rectangle and
|
fix(features): dedup the alt block BY VALUE - aba9bd2 broke D1 charts
aba9bd2 kept only the newest slot's copy of the external block. That is right on
every intraday timeframe and WRONG on D1: there the 16 window bars are 16
distinct calendar days, the daily alt file returns a different row for each, and
blanking 15 of them destroyed real information instead of a copy of it. Caught
while extending the measurement to the other instruments.
Now compares values instead of slot positions: walk newest -> oldest, keep the
last DISTINCT reading, blank a slot only when it repeats one a newer slot
already carries. Exact on every timeframe with no timeframe test, and it also
handles weekends, holidays and publication gaps, where a window spans fewer
distinct rows than calendar days. How much collapses falls out of the data:
M15 x 16 bars = 0.17 calendar days -> 1 distinct row -> 15 of 16 blanked
H1 x 16 bars = 0.67 calendar days -> 1 distinct row -> 15 of 16 blanked
H4 x 16 bars = 2.67 calendar days -> ~3 rows -> ~13 of 16 blanked
D1 x 16 bars = 16 calendar days -> 16 rows -> NOTHING blanked
OTHER INSTRUMENTS - the question that prompted this. Per-symbol exports for
USDJPY/XAUUSD/EURUSD are not on disk (written only when that chart is attached),
but they are not needed: CAltData reads a DAILY file for every symbol, so bars
sharing a calendar day are byte-identical by construction everywhere. What
varies per symbol is only WHICH sources, and the catalog (AltDataFetch.mqh
AddSpec rows) gives:
SP500 13 = risk(3) + cot_spec_net(1) + eia(3) + mac(6)
EURUSD 15 = cot(3) + risk(3) + eia(3) + mac(6)
USDJPY 15 = cot(3) + risk(3) + eia(3) + mac(6)
XAUUSD 14 = risk(3) + ivol(2, GVZCLS) + eia(3) + mac(6)
Measured observation-date gaps in the raw sources on disk: VIX, USD index,
DGS10, T10Y2Y, T5YIE, DFF, ECBDFR all 1 day; COT and EIA 7 days; CPI and UNRATE
31 days. NO per-bar source exists anywhere in the catalog - the tick-activity
survivors from the flow screen are an in-terminal feature block, not alt data,
and are untouched by any of this. So the redundancy is universal across
instruments; only its magnitude varies, and by timeframe rather than by symbol.
Compile-verified: 0 errors, 0 warnings.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-16 22:11:52 -04:00
|
|
|
//--- need no change - and on an intraday chart the block arrives at the newest bar, which for the
|
|
|
|
|
//--- LSTM is the final timestep, where a "current state of the world" reading belongs. Near-constant
|
|
|
|
|
//--- coordinates are safe through batch norm: its divisor is
|
|
|
|
|
//--- MathMax(MathSqrt(var + BN_EPSILON), BN_MIN_STD).
|
|
|
|
|
//---
|
fix(features): collapse only the anchor's own run - leave lagged readings put
User's call before deploy: "I would rather avoid lagging so the NN finds
accurate patterns." Correct instinct, and it picks the conservative variant.
110b384 deduplicated the WHOLE window, so every distinct reading survived at one
slot. The flaw is which slot: it depends on where the calendar-day boundary falls
inside that particular window, and on H4 that boundary cycles through ~6 phases.
A dense layer holds a separate weight per (slot, feature), so a given lag would
have landed on a different coordinate from one window to the next - turning a
stable lagged input into a moving one.
Now it blanks only bars carrying a BYTE-IDENTICAL copy of the anchor's reading
and stops at the first bar that differs. An as-of lookup into a daily file is a
step function in time, so those copies are exactly the contiguous run of bars
sharing the anchor's calendar day. Everything older keeps its natural replicated
run, in the same slots it always occupied - whatever the net learned to read
there, it still reads there.
Why the anchor's reading is the right one to isolate: the window's newest slot IS
the bar being predicted (BuildFeatureWindow's final iteration lands on r, and
pass 3 grades that same index), so it is the reading contemporaneous with the
decision - and the only one the alt screens ever validated. They measured the
CURRENT reading's MI against forward range and never tested lags, so the lagged
content is unproven, which is a reason to leave it undisturbed rather than a
licence to rearrange it.
What is still fixed: the anchor's reading reaches the first layer on one
coordinate instead of once per bar of its day, removing the ~16x gradient
upweight for the validated signal. And this is IDENTICAL to full dedup exactly
where replication was worst - on M15/H1 the whole window sits inside one calendar
day, so the anchor's run is the whole window - and a no-op on D1, where the bar
before the anchor is already a different day and the loop breaks immediately.
The two differ only on middle timeframes, and there this is the safe side.
Fingerprint |ALTW:1 -> |ALTW:2 so nothing trained under the hour-old full-dedup
semantics can silently resume under these.
Compile-verified: 0 errors, 0 warnings.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-16 22:21:22 -04:00
|
|
|
//--- ONLY THE ANCHOR'S OWN RUN IS COLLAPSED. Nothing older is touched, ever.
|
|
|
|
|
//---
|
|
|
|
|
//--- The window's newest slot IS the bar being predicted (see the loop above - the final iteration
|
|
|
|
|
//--- lands on r itself, and pass 3 grades that same index), so the reading in that slot is the one
|
|
|
|
|
//--- contemporaneous with the decision, and it is the only one every alt screen actually validated.
|
|
|
|
|
//--- This walks back from the anchor blanking bars that carry a BYTE-IDENTICAL copy of it, and stops
|
|
|
|
|
//--- at the first bar that differs. Because an as-of lookup into a daily file is a step function in
|
|
|
|
|
//--- time, those copies are exactly the contiguous run of bars sharing the anchor's calendar day.
|
|
|
|
|
//---
|
|
|
|
|
//--- WHY IT STOPS THERE rather than deduplicating the whole window (2026-08-16, user's call before
|
|
|
|
|
//--- deploy: "I would rather avoid lagging so the NN finds accurate patterns"). Collapsing every
|
|
|
|
|
//--- distinct reading would leave each surviving at ONE slot - but WHICH slot depends on where the
|
|
|
|
|
//--- day boundary falls inside that particular window, so a given lag would land on a different
|
|
|
|
|
//--- coordinate from one window to the next. A dense layer holds a separate weight per (slot,
|
|
|
|
|
//--- feature), so that turns a stable lagged input into a moving one. The older readings' natural
|
|
|
|
|
//--- replicated runs are therefore left exactly as they were: whatever the net had learned to read
|
|
|
|
|
//--- from those slots, it still reads from those same slots. The alt screens measured the CURRENT
|
|
|
|
|
//--- reading's MI against forward range and never tested lags, so the lagged content is unproven -
|
|
|
|
|
//--- and unproven is a reason to leave it undisturbed, not a licence to rearrange it.
|
|
|
|
|
//---
|
|
|
|
|
//--- What this still buys: the anchor's reading reaches the first layer on ONE coordinate instead of
|
|
|
|
|
//--- once per bar of its day, which removes the gradient upweight described above for the only
|
|
|
|
|
//--- reading that was ever validated. It is also IDENTICAL to full deduplication exactly where the
|
|
|
|
|
//--- replication was worst - on M15 and H1 the entire window sits inside one calendar day, so the
|
|
|
|
|
//--- anchor's run IS the whole window - and a no-op on D1, where the bar immediately older than the
|
|
|
|
|
//--- anchor is already a different day and the loop breaks on its first comparison. The only
|
|
|
|
|
//--- timeframes where the two differ are the middle ones, and there this is the conservative choice.
|
perf(features): the external block enters the window once, not once per bar
Measured on the live SP500 D1 export (6073 rows, 13 features, 5888 simulated
16-bar windows):
distinct values per feature per window : 1.7 - 2.7 of 16 slots
variance in the first 13 PCs : 96.5 - 97.0%
components for 95% / 99% : 12 / 17-19
effective rank (entropy) : ~11.5
208 inputs carrying about 12 dimensions. Only 6 of the 13 features move daily
(VIX complex, USD, the rates trio); 5 are weekly (COT, EIA, output gap) and 2
monthly (CPI, unemployment). The lookup is as-of by bar open time into a DAILY
file, so bars sharing a calendar day are byte-identical by construction.
The cost is NOT overfitting capacity - collinear copies span ~12 directions,
not 208, so an earlier claim that this wasted 26% of the model overstated it.
It is GRADIENT WEIGHTING. Batch norm standardizes each of the 208 coordinates
independently; that rescales the copies without decorrelating them, so one
factor arrives on 16 unit-variance coordinates, each weight takes a full-size
step, and the factor's aggregate coefficient moves ~16x faster than a per-bar
price feature's. The network was biased toward the external block by a factor
of the window length - and pointing the wrong way, since these features cleared
only a marginal incremental screen while price is the base signal.
Zeroed at WINDOW ASSEMBLY, not in BufferTempData: that output is cached PER BAR
and a bar sits at slot 15 of one window and slot 0 of the next, so a
slot-dependent value there would poison the cache or force a recompute per slot.
The cache keeps true values; only this window's copies are cleared. Width
contract untouched - same count, same positions - so conv/LSTM/HYBRID keep their
bar-major rectangle unchanged and the block arrives at the newest bar, which for
the LSTM is the final timestep. Zero-variance coordinates are safe through batch
norm (divisor is MathMax(MathSqrt(var + BN_EPSILON), BN_MIN_STD)).
Fingerprint gains |ALTW:1 when alt data is on. Same width and same .cfg, so
nothing else would have caught a model trained under the replicated layout
resuming under this one. Conditional append per the existing rule: configs
without alt data keep their fingerprints and their trained models.
NOT the concat branch. CNet is a strictly linear stack (CLayerDescription has no
input-source field; NetBuild wires i to i+1 and stores layer L's weights on
L-1), so a real two-tower model needs a new multi-input layer type across
WarriorCPU, WarriorDML and the OpenCL kernels plus an .nnw format change - the
highest-risk change in this repo, in the code that produced the transposed dense
gradient, the Adam second-moment bug and the reversed LSTM window. This captures
the part of that idea the measurement actually supports, at no engine risk.
Compile-verified: 0 errors, 0 warnings.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-16 22:05:10 -04:00
|
|
|
if(m_useAltData)
|
|
|
|
|
{
|
|
|
|
|
int an = m_altData.FeatureCount();
|
fix(features): dedup the alt block BY VALUE - aba9bd2 broke D1 charts
aba9bd2 kept only the newest slot's copy of the external block. That is right on
every intraday timeframe and WRONG on D1: there the 16 window bars are 16
distinct calendar days, the daily alt file returns a different row for each, and
blanking 15 of them destroyed real information instead of a copy of it. Caught
while extending the measurement to the other instruments.
Now compares values instead of slot positions: walk newest -> oldest, keep the
last DISTINCT reading, blank a slot only when it repeats one a newer slot
already carries. Exact on every timeframe with no timeframe test, and it also
handles weekends, holidays and publication gaps, where a window spans fewer
distinct rows than calendar days. How much collapses falls out of the data:
M15 x 16 bars = 0.17 calendar days -> 1 distinct row -> 15 of 16 blanked
H1 x 16 bars = 0.67 calendar days -> 1 distinct row -> 15 of 16 blanked
H4 x 16 bars = 2.67 calendar days -> ~3 rows -> ~13 of 16 blanked
D1 x 16 bars = 16 calendar days -> 16 rows -> NOTHING blanked
OTHER INSTRUMENTS - the question that prompted this. Per-symbol exports for
USDJPY/XAUUSD/EURUSD are not on disk (written only when that chart is attached),
but they are not needed: CAltData reads a DAILY file for every symbol, so bars
sharing a calendar day are byte-identical by construction everywhere. What
varies per symbol is only WHICH sources, and the catalog (AltDataFetch.mqh
AddSpec rows) gives:
SP500 13 = risk(3) + cot_spec_net(1) + eia(3) + mac(6)
EURUSD 15 = cot(3) + risk(3) + eia(3) + mac(6)
USDJPY 15 = cot(3) + risk(3) + eia(3) + mac(6)
XAUUSD 14 = risk(3) + ivol(2, GVZCLS) + eia(3) + mac(6)
Measured observation-date gaps in the raw sources on disk: VIX, USD index,
DGS10, T10Y2Y, T5YIE, DFF, ECBDFR all 1 day; COT and EIA 7 days; CPI and UNRATE
31 days. NO per-bar source exists anywhere in the catalog - the tick-activity
survivors from the flow screen are an in-terminal feature block, not alt data,
and are untouched by any of this. So the redundancy is universal across
instruments; only its magnitude varies, and by timeframe rather than by symbol.
Compile-verified: 0 errors, 0 warnings.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-16 22:11:52 -04:00
|
|
|
if(an > 0 && an <= m_neuronsCount && (int)m_historyBars > 1)
|
|
|
|
|
{
|
fix(features): collapse only the anchor's own run - leave lagged readings put
User's call before deploy: "I would rather avoid lagging so the NN finds
accurate patterns." Correct instinct, and it picks the conservative variant.
110b384 deduplicated the WHOLE window, so every distinct reading survived at one
slot. The flaw is which slot: it depends on where the calendar-day boundary falls
inside that particular window, and on H4 that boundary cycles through ~6 phases.
A dense layer holds a separate weight per (slot, feature), so a given lag would
have landed on a different coordinate from one window to the next - turning a
stable lagged input into a moving one.
Now it blanks only bars carrying a BYTE-IDENTICAL copy of the anchor's reading
and stops at the first bar that differs. An as-of lookup into a daily file is a
step function in time, so those copies are exactly the contiguous run of bars
sharing the anchor's calendar day. Everything older keeps its natural replicated
run, in the same slots it always occupied - whatever the net learned to read
there, it still reads there.
Why the anchor's reading is the right one to isolate: the window's newest slot IS
the bar being predicted (BuildFeatureWindow's final iteration lands on r, and
pass 3 grades that same index), so it is the reading contemporaneous with the
decision - and the only one the alt screens ever validated. They measured the
CURRENT reading's MI against forward range and never tested lags, so the lagged
content is unproven, which is a reason to leave it undisturbed rather than a
licence to rearrange it.
What is still fixed: the anchor's reading reaches the first layer on one
coordinate instead of once per bar of its day, removing the ~16x gradient
upweight for the validated signal. And this is IDENTICAL to full dedup exactly
where replication was worst - on M15/H1 the whole window sits inside one calendar
day, so the anchor's run is the whole window - and a no-op on D1, where the bar
before the anchor is already a different day and the loop breaks immediately.
The two differ only on middle timeframes, and there this is the safe side.
Fingerprint |ALTW:1 -> |ALTW:2 so nothing trained under the hour-old full-dedup
semantics can silently resume under these.
Compile-verified: 0 errors, 0 warnings.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-16 22:21:22 -04:00
|
|
|
double anchor[];
|
|
|
|
|
ArrayResize(anchor, an);
|
fix(features): dedup the alt block BY VALUE - aba9bd2 broke D1 charts
aba9bd2 kept only the newest slot's copy of the external block. That is right on
every intraday timeframe and WRONG on D1: there the 16 window bars are 16
distinct calendar days, the daily alt file returns a different row for each, and
blanking 15 of them destroyed real information instead of a copy of it. Caught
while extending the measurement to the other instruments.
Now compares values instead of slot positions: walk newest -> oldest, keep the
last DISTINCT reading, blank a slot only when it repeats one a newer slot
already carries. Exact on every timeframe with no timeframe test, and it also
handles weekends, holidays and publication gaps, where a window spans fewer
distinct rows than calendar days. How much collapses falls out of the data:
M15 x 16 bars = 0.17 calendar days -> 1 distinct row -> 15 of 16 blanked
H1 x 16 bars = 0.67 calendar days -> 1 distinct row -> 15 of 16 blanked
H4 x 16 bars = 2.67 calendar days -> ~3 rows -> ~13 of 16 blanked
D1 x 16 bars = 16 calendar days -> 16 rows -> NOTHING blanked
OTHER INSTRUMENTS - the question that prompted this. Per-symbol exports for
USDJPY/XAUUSD/EURUSD are not on disk (written only when that chart is attached),
but they are not needed: CAltData reads a DAILY file for every symbol, so bars
sharing a calendar day are byte-identical by construction everywhere. What
varies per symbol is only WHICH sources, and the catalog (AltDataFetch.mqh
AddSpec rows) gives:
SP500 13 = risk(3) + cot_spec_net(1) + eia(3) + mac(6)
EURUSD 15 = cot(3) + risk(3) + eia(3) + mac(6)
USDJPY 15 = cot(3) + risk(3) + eia(3) + mac(6)
XAUUSD 14 = risk(3) + ivol(2, GVZCLS) + eia(3) + mac(6)
Measured observation-date gaps in the raw sources on disk: VIX, USD index,
DGS10, T10Y2Y, T5YIE, DFF, ECBDFR all 1 day; COT and EIA 7 days; CPI and UNRATE
31 days. NO per-bar source exists anywhere in the catalog - the tick-activity
survivors from the flow screen are an in-terminal feature block, not alt data,
and are untouched by any of this. So the redundancy is universal across
instruments; only its magnitude varies, and by timeframe rather than by symbol.
Compile-verified: 0 errors, 0 warnings.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-16 22:11:52 -04:00
|
|
|
int newest = ((int)m_historyBars - 1) * m_neuronsCount + (m_neuronsCount - an);
|
|
|
|
|
for(int k = 0; k < an; k++)
|
fix(features): collapse only the anchor's own run - leave lagged readings put
User's call before deploy: "I would rather avoid lagging so the NN finds
accurate patterns." Correct instinct, and it picks the conservative variant.
110b384 deduplicated the WHOLE window, so every distinct reading survived at one
slot. The flaw is which slot: it depends on where the calendar-day boundary falls
inside that particular window, and on H4 that boundary cycles through ~6 phases.
A dense layer holds a separate weight per (slot, feature), so a given lag would
have landed on a different coordinate from one window to the next - turning a
stable lagged input into a moving one.
Now it blanks only bars carrying a BYTE-IDENTICAL copy of the anchor's reading
and stops at the first bar that differs. An as-of lookup into a daily file is a
step function in time, so those copies are exactly the contiguous run of bars
sharing the anchor's calendar day. Everything older keeps its natural replicated
run, in the same slots it always occupied - whatever the net learned to read
there, it still reads there.
Why the anchor's reading is the right one to isolate: the window's newest slot IS
the bar being predicted (BuildFeatureWindow's final iteration lands on r, and
pass 3 grades that same index), so it is the reading contemporaneous with the
decision - and the only one the alt screens ever validated. They measured the
CURRENT reading's MI against forward range and never tested lags, so the lagged
content is unproven, which is a reason to leave it undisturbed rather than a
licence to rearrange it.
What is still fixed: the anchor's reading reaches the first layer on one
coordinate instead of once per bar of its day, removing the ~16x gradient
upweight for the validated signal. And this is IDENTICAL to full dedup exactly
where replication was worst - on M15/H1 the whole window sits inside one calendar
day, so the anchor's run is the whole window - and a no-op on D1, where the bar
before the anchor is already a different day and the loop breaks immediately.
The two differ only on middle timeframes, and there this is the safe side.
Fingerprint |ALTW:1 -> |ALTW:2 so nothing trained under the hour-old full-dedup
semantics can silently resume under these.
Compile-verified: 0 errors, 0 warnings.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-16 22:21:22 -04:00
|
|
|
anchor[k] = TempData.At(newest + k);
|
fix(features): dedup the alt block BY VALUE - aba9bd2 broke D1 charts
aba9bd2 kept only the newest slot's copy of the external block. That is right on
every intraday timeframe and WRONG on D1: there the 16 window bars are 16
distinct calendar days, the daily alt file returns a different row for each, and
blanking 15 of them destroyed real information instead of a copy of it. Caught
while extending the measurement to the other instruments.
Now compares values instead of slot positions: walk newest -> oldest, keep the
last DISTINCT reading, blank a slot only when it repeats one a newer slot
already carries. Exact on every timeframe with no timeframe test, and it also
handles weekends, holidays and publication gaps, where a window spans fewer
distinct rows than calendar days. How much collapses falls out of the data:
M15 x 16 bars = 0.17 calendar days -> 1 distinct row -> 15 of 16 blanked
H1 x 16 bars = 0.67 calendar days -> 1 distinct row -> 15 of 16 blanked
H4 x 16 bars = 2.67 calendar days -> ~3 rows -> ~13 of 16 blanked
D1 x 16 bars = 16 calendar days -> 16 rows -> NOTHING blanked
OTHER INSTRUMENTS - the question that prompted this. Per-symbol exports for
USDJPY/XAUUSD/EURUSD are not on disk (written only when that chart is attached),
but they are not needed: CAltData reads a DAILY file for every symbol, so bars
sharing a calendar day are byte-identical by construction everywhere. What
varies per symbol is only WHICH sources, and the catalog (AltDataFetch.mqh
AddSpec rows) gives:
SP500 13 = risk(3) + cot_spec_net(1) + eia(3) + mac(6)
EURUSD 15 = cot(3) + risk(3) + eia(3) + mac(6)
USDJPY 15 = cot(3) + risk(3) + eia(3) + mac(6)
XAUUSD 14 = risk(3) + ivol(2, GVZCLS) + eia(3) + mac(6)
Measured observation-date gaps in the raw sources on disk: VIX, USD index,
DGS10, T10Y2Y, T5YIE, DFF, ECBDFR all 1 day; COT and EIA 7 days; CPI and UNRATE
31 days. NO per-bar source exists anywhere in the catalog - the tick-activity
survivors from the flow screen are an in-terminal feature block, not alt data,
and are untouched by any of this. So the redundancy is universal across
instruments; only its magnitude varies, and by timeframe rather than by symbol.
Compile-verified: 0 errors, 0 warnings.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-16 22:11:52 -04:00
|
|
|
for(int b = (int)m_historyBars - 2; b >= 0; b--)
|
perf(features): the external block enters the window once, not once per bar
Measured on the live SP500 D1 export (6073 rows, 13 features, 5888 simulated
16-bar windows):
distinct values per feature per window : 1.7 - 2.7 of 16 slots
variance in the first 13 PCs : 96.5 - 97.0%
components for 95% / 99% : 12 / 17-19
effective rank (entropy) : ~11.5
208 inputs carrying about 12 dimensions. Only 6 of the 13 features move daily
(VIX complex, USD, the rates trio); 5 are weekly (COT, EIA, output gap) and 2
monthly (CPI, unemployment). The lookup is as-of by bar open time into a DAILY
file, so bars sharing a calendar day are byte-identical by construction.
The cost is NOT overfitting capacity - collinear copies span ~12 directions,
not 208, so an earlier claim that this wasted 26% of the model overstated it.
It is GRADIENT WEIGHTING. Batch norm standardizes each of the 208 coordinates
independently; that rescales the copies without decorrelating them, so one
factor arrives on 16 unit-variance coordinates, each weight takes a full-size
step, and the factor's aggregate coefficient moves ~16x faster than a per-bar
price feature's. The network was biased toward the external block by a factor
of the window length - and pointing the wrong way, since these features cleared
only a marginal incremental screen while price is the base signal.
Zeroed at WINDOW ASSEMBLY, not in BufferTempData: that output is cached PER BAR
and a bar sits at slot 15 of one window and slot 0 of the next, so a
slot-dependent value there would poison the cache or force a recompute per slot.
The cache keeps true values; only this window's copies are cleared. Width
contract untouched - same count, same positions - so conv/LSTM/HYBRID keep their
bar-major rectangle unchanged and the block arrives at the newest bar, which for
the LSTM is the final timestep. Zero-variance coordinates are safe through batch
norm (divisor is MathMax(MathSqrt(var + BN_EPSILON), BN_MIN_STD)).
Fingerprint gains |ALTW:1 when alt data is on. Same width and same .cfg, so
nothing else would have caught a model trained under the replicated layout
resuming under this one. Conditional append per the existing rule: configs
without alt data keep their fingerprints and their trained models.
NOT the concat branch. CNet is a strictly linear stack (CLayerDescription has no
input-source field; NetBuild wires i to i+1 and stores layer L's weights on
L-1), so a real two-tower model needs a new multi-input layer type across
WarriorCPU, WarriorDML and the OpenCL kernels plus an .nnw format change - the
highest-risk change in this repo, in the code that produced the transposed dense
gradient, the Adam second-moment bug and the reversed LSTM window. This captures
the part of that idea the measurement actually supports, at no engine risk.
Compile-verified: 0 errors, 0 warnings.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-16 22:05:10 -04:00
|
|
|
{
|
|
|
|
|
int altBase = b * m_neuronsCount + (m_neuronsCount - an);
|
fix(features): dedup the alt block BY VALUE - aba9bd2 broke D1 charts
aba9bd2 kept only the newest slot's copy of the external block. That is right on
every intraday timeframe and WRONG on D1: there the 16 window bars are 16
distinct calendar days, the daily alt file returns a different row for each, and
blanking 15 of them destroyed real information instead of a copy of it. Caught
while extending the measurement to the other instruments.
Now compares values instead of slot positions: walk newest -> oldest, keep the
last DISTINCT reading, blank a slot only when it repeats one a newer slot
already carries. Exact on every timeframe with no timeframe test, and it also
handles weekends, holidays and publication gaps, where a window spans fewer
distinct rows than calendar days. How much collapses falls out of the data:
M15 x 16 bars = 0.17 calendar days -> 1 distinct row -> 15 of 16 blanked
H1 x 16 bars = 0.67 calendar days -> 1 distinct row -> 15 of 16 blanked
H4 x 16 bars = 2.67 calendar days -> ~3 rows -> ~13 of 16 blanked
D1 x 16 bars = 16 calendar days -> 16 rows -> NOTHING blanked
OTHER INSTRUMENTS - the question that prompted this. Per-symbol exports for
USDJPY/XAUUSD/EURUSD are not on disk (written only when that chart is attached),
but they are not needed: CAltData reads a DAILY file for every symbol, so bars
sharing a calendar day are byte-identical by construction everywhere. What
varies per symbol is only WHICH sources, and the catalog (AltDataFetch.mqh
AddSpec rows) gives:
SP500 13 = risk(3) + cot_spec_net(1) + eia(3) + mac(6)
EURUSD 15 = cot(3) + risk(3) + eia(3) + mac(6)
USDJPY 15 = cot(3) + risk(3) + eia(3) + mac(6)
XAUUSD 14 = risk(3) + ivol(2, GVZCLS) + eia(3) + mac(6)
Measured observation-date gaps in the raw sources on disk: VIX, USD index,
DGS10, T10Y2Y, T5YIE, DFF, ECBDFR all 1 day; COT and EIA 7 days; CPI and UNRATE
31 days. NO per-bar source exists anywhere in the catalog - the tick-activity
survivors from the flow screen are an in-terminal feature block, not alt data,
and are untouched by any of this. So the redundancy is universal across
instruments; only its magnitude varies, and by timeframe rather than by symbol.
Compile-verified: 0 errors, 0 warnings.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-16 22:11:52 -04:00
|
|
|
bool same = true;
|
|
|
|
|
for(int k = 0; k < an && same; k++)
|
fix(features): collapse only the anchor's own run - leave lagged readings put
User's call before deploy: "I would rather avoid lagging so the NN finds
accurate patterns." Correct instinct, and it picks the conservative variant.
110b384 deduplicated the WHOLE window, so every distinct reading survived at one
slot. The flaw is which slot: it depends on where the calendar-day boundary falls
inside that particular window, and on H4 that boundary cycles through ~6 phases.
A dense layer holds a separate weight per (slot, feature), so a given lag would
have landed on a different coordinate from one window to the next - turning a
stable lagged input into a moving one.
Now it blanks only bars carrying a BYTE-IDENTICAL copy of the anchor's reading
and stops at the first bar that differs. An as-of lookup into a daily file is a
step function in time, so those copies are exactly the contiguous run of bars
sharing the anchor's calendar day. Everything older keeps its natural replicated
run, in the same slots it always occupied - whatever the net learned to read
there, it still reads there.
Why the anchor's reading is the right one to isolate: the window's newest slot IS
the bar being predicted (BuildFeatureWindow's final iteration lands on r, and
pass 3 grades that same index), so it is the reading contemporaneous with the
decision - and the only one the alt screens ever validated. They measured the
CURRENT reading's MI against forward range and never tested lags, so the lagged
content is unproven, which is a reason to leave it undisturbed rather than a
licence to rearrange it.
What is still fixed: the anchor's reading reaches the first layer on one
coordinate instead of once per bar of its day, removing the ~16x gradient
upweight for the validated signal. And this is IDENTICAL to full dedup exactly
where replication was worst - on M15/H1 the whole window sits inside one calendar
day, so the anchor's run is the whole window - and a no-op on D1, where the bar
before the anchor is already a different day and the loop breaks immediately.
The two differ only on middle timeframes, and there this is the safe side.
Fingerprint |ALTW:1 -> |ALTW:2 so nothing trained under the hour-old full-dedup
semantics can silently resume under these.
Compile-verified: 0 errors, 0 warnings.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-16 22:21:22 -04:00
|
|
|
if(TempData.At(altBase + k) != anchor[k])
|
fix(features): dedup the alt block BY VALUE - aba9bd2 broke D1 charts
aba9bd2 kept only the newest slot's copy of the external block. That is right on
every intraday timeframe and WRONG on D1: there the 16 window bars are 16
distinct calendar days, the daily alt file returns a different row for each, and
blanking 15 of them destroyed real information instead of a copy of it. Caught
while extending the measurement to the other instruments.
Now compares values instead of slot positions: walk newest -> oldest, keep the
last DISTINCT reading, blank a slot only when it repeats one a newer slot
already carries. Exact on every timeframe with no timeframe test, and it also
handles weekends, holidays and publication gaps, where a window spans fewer
distinct rows than calendar days. How much collapses falls out of the data:
M15 x 16 bars = 0.17 calendar days -> 1 distinct row -> 15 of 16 blanked
H1 x 16 bars = 0.67 calendar days -> 1 distinct row -> 15 of 16 blanked
H4 x 16 bars = 2.67 calendar days -> ~3 rows -> ~13 of 16 blanked
D1 x 16 bars = 16 calendar days -> 16 rows -> NOTHING blanked
OTHER INSTRUMENTS - the question that prompted this. Per-symbol exports for
USDJPY/XAUUSD/EURUSD are not on disk (written only when that chart is attached),
but they are not needed: CAltData reads a DAILY file for every symbol, so bars
sharing a calendar day are byte-identical by construction everywhere. What
varies per symbol is only WHICH sources, and the catalog (AltDataFetch.mqh
AddSpec rows) gives:
SP500 13 = risk(3) + cot_spec_net(1) + eia(3) + mac(6)
EURUSD 15 = cot(3) + risk(3) + eia(3) + mac(6)
USDJPY 15 = cot(3) + risk(3) + eia(3) + mac(6)
XAUUSD 14 = risk(3) + ivol(2, GVZCLS) + eia(3) + mac(6)
Measured observation-date gaps in the raw sources on disk: VIX, USD index,
DGS10, T10Y2Y, T5YIE, DFF, ECBDFR all 1 day; COT and EIA 7 days; CPI and UNRATE
31 days. NO per-bar source exists anywhere in the catalog - the tick-activity
survivors from the flow screen are an in-terminal feature block, not alt data,
and are untouched by any of this. So the redundancy is universal across
instruments; only its magnitude varies, and by timeframe rather than by symbol.
Compile-verified: 0 errors, 0 warnings.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-16 22:11:52 -04:00
|
|
|
same = false;
|
fix(features): collapse only the anchor's own run - leave lagged readings put
User's call before deploy: "I would rather avoid lagging so the NN finds
accurate patterns." Correct instinct, and it picks the conservative variant.
110b384 deduplicated the WHOLE window, so every distinct reading survived at one
slot. The flaw is which slot: it depends on where the calendar-day boundary falls
inside that particular window, and on H4 that boundary cycles through ~6 phases.
A dense layer holds a separate weight per (slot, feature), so a given lag would
have landed on a different coordinate from one window to the next - turning a
stable lagged input into a moving one.
Now it blanks only bars carrying a BYTE-IDENTICAL copy of the anchor's reading
and stops at the first bar that differs. An as-of lookup into a daily file is a
step function in time, so those copies are exactly the contiguous run of bars
sharing the anchor's calendar day. Everything older keeps its natural replicated
run, in the same slots it always occupied - whatever the net learned to read
there, it still reads there.
Why the anchor's reading is the right one to isolate: the window's newest slot IS
the bar being predicted (BuildFeatureWindow's final iteration lands on r, and
pass 3 grades that same index), so it is the reading contemporaneous with the
decision - and the only one the alt screens ever validated. They measured the
CURRENT reading's MI against forward range and never tested lags, so the lagged
content is unproven, which is a reason to leave it undisturbed rather than a
licence to rearrange it.
What is still fixed: the anchor's reading reaches the first layer on one
coordinate instead of once per bar of its day, removing the ~16x gradient
upweight for the validated signal. And this is IDENTICAL to full dedup exactly
where replication was worst - on M15/H1 the whole window sits inside one calendar
day, so the anchor's run is the whole window - and a no-op on D1, where the bar
before the anchor is already a different day and the loop breaks immediately.
The two differ only on middle timeframes, and there this is the safe side.
Fingerprint |ALTW:1 -> |ALTW:2 so nothing trained under the hour-old full-dedup
semantics can silently resume under these.
Compile-verified: 0 errors, 0 warnings.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-16 22:21:22 -04:00
|
|
|
if(!same)
|
|
|
|
|
break; // a different reading: this bar and everything older keep their values as-is
|
|
|
|
|
for(int k = 0; k < an; k++)
|
|
|
|
|
TempData.Update(altBase + k, 0.0);
|
perf(features): the external block enters the window once, not once per bar
Measured on the live SP500 D1 export (6073 rows, 13 features, 5888 simulated
16-bar windows):
distinct values per feature per window : 1.7 - 2.7 of 16 slots
variance in the first 13 PCs : 96.5 - 97.0%
components for 95% / 99% : 12 / 17-19
effective rank (entropy) : ~11.5
208 inputs carrying about 12 dimensions. Only 6 of the 13 features move daily
(VIX complex, USD, the rates trio); 5 are weekly (COT, EIA, output gap) and 2
monthly (CPI, unemployment). The lookup is as-of by bar open time into a DAILY
file, so bars sharing a calendar day are byte-identical by construction.
The cost is NOT overfitting capacity - collinear copies span ~12 directions,
not 208, so an earlier claim that this wasted 26% of the model overstated it.
It is GRADIENT WEIGHTING. Batch norm standardizes each of the 208 coordinates
independently; that rescales the copies without decorrelating them, so one
factor arrives on 16 unit-variance coordinates, each weight takes a full-size
step, and the factor's aggregate coefficient moves ~16x faster than a per-bar
price feature's. The network was biased toward the external block by a factor
of the window length - and pointing the wrong way, since these features cleared
only a marginal incremental screen while price is the base signal.
Zeroed at WINDOW ASSEMBLY, not in BufferTempData: that output is cached PER BAR
and a bar sits at slot 15 of one window and slot 0 of the next, so a
slot-dependent value there would poison the cache or force a recompute per slot.
The cache keeps true values; only this window's copies are cleared. Width
contract untouched - same count, same positions - so conv/LSTM/HYBRID keep their
bar-major rectangle unchanged and the block arrives at the newest bar, which for
the LSTM is the final timestep. Zero-variance coordinates are safe through batch
norm (divisor is MathMax(MathSqrt(var + BN_EPSILON), BN_MIN_STD)).
Fingerprint gains |ALTW:1 when alt data is on. Same width and same .cfg, so
nothing else would have caught a model trained under the replicated layout
resuming under this one. Conditional append per the existing rule: configs
without alt data keep their fingerprints and their trained models.
NOT the concat branch. CNet is a strictly linear stack (CLayerDescription has no
input-source field; NetBuild wires i to i+1 and stores layer L's weights on
L-1), so a real two-tower model needs a new multi-input layer type across
WarriorCPU, WarriorDML and the OpenCL kernels plus an .nnw format change - the
highest-risk change in this repo, in the code that produced the transposed dense
gradient, the Adam second-moment bug and the reversed LSTM window. This captures
the part of that idea the measurement actually supports, at no engine risk.
Compile-verified: 0 errors, 0 warnings.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-16 22:05:10 -04:00
|
|
|
}
|
fix(features): dedup the alt block BY VALUE - aba9bd2 broke D1 charts
aba9bd2 kept only the newest slot's copy of the external block. That is right on
every intraday timeframe and WRONG on D1: there the 16 window bars are 16
distinct calendar days, the daily alt file returns a different row for each, and
blanking 15 of them destroyed real information instead of a copy of it. Caught
while extending the measurement to the other instruments.
Now compares values instead of slot positions: walk newest -> oldest, keep the
last DISTINCT reading, blank a slot only when it repeats one a newer slot
already carries. Exact on every timeframe with no timeframe test, and it also
handles weekends, holidays and publication gaps, where a window spans fewer
distinct rows than calendar days. How much collapses falls out of the data:
M15 x 16 bars = 0.17 calendar days -> 1 distinct row -> 15 of 16 blanked
H1 x 16 bars = 0.67 calendar days -> 1 distinct row -> 15 of 16 blanked
H4 x 16 bars = 2.67 calendar days -> ~3 rows -> ~13 of 16 blanked
D1 x 16 bars = 16 calendar days -> 16 rows -> NOTHING blanked
OTHER INSTRUMENTS - the question that prompted this. Per-symbol exports for
USDJPY/XAUUSD/EURUSD are not on disk (written only when that chart is attached),
but they are not needed: CAltData reads a DAILY file for every symbol, so bars
sharing a calendar day are byte-identical by construction everywhere. What
varies per symbol is only WHICH sources, and the catalog (AltDataFetch.mqh
AddSpec rows) gives:
SP500 13 = risk(3) + cot_spec_net(1) + eia(3) + mac(6)
EURUSD 15 = cot(3) + risk(3) + eia(3) + mac(6)
USDJPY 15 = cot(3) + risk(3) + eia(3) + mac(6)
XAUUSD 14 = risk(3) + ivol(2, GVZCLS) + eia(3) + mac(6)
Measured observation-date gaps in the raw sources on disk: VIX, USD index,
DGS10, T10Y2Y, T5YIE, DFF, ECBDFR all 1 day; COT and EIA 7 days; CPI and UNRATE
31 days. NO per-bar source exists anywhere in the catalog - the tick-activity
survivors from the flow screen are an in-terminal feature block, not alt data,
and are untouched by any of this. So the redundancy is universal across
instruments; only its magnitude varies, and by timeframe rather than by symbol.
Compile-verified: 0 errors, 0 warnings.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-16 22:11:52 -04:00
|
|
|
}
|
perf(features): the external block enters the window once, not once per bar
Measured on the live SP500 D1 export (6073 rows, 13 features, 5888 simulated
16-bar windows):
distinct values per feature per window : 1.7 - 2.7 of 16 slots
variance in the first 13 PCs : 96.5 - 97.0%
components for 95% / 99% : 12 / 17-19
effective rank (entropy) : ~11.5
208 inputs carrying about 12 dimensions. Only 6 of the 13 features move daily
(VIX complex, USD, the rates trio); 5 are weekly (COT, EIA, output gap) and 2
monthly (CPI, unemployment). The lookup is as-of by bar open time into a DAILY
file, so bars sharing a calendar day are byte-identical by construction.
The cost is NOT overfitting capacity - collinear copies span ~12 directions,
not 208, so an earlier claim that this wasted 26% of the model overstated it.
It is GRADIENT WEIGHTING. Batch norm standardizes each of the 208 coordinates
independently; that rescales the copies without decorrelating them, so one
factor arrives on 16 unit-variance coordinates, each weight takes a full-size
step, and the factor's aggregate coefficient moves ~16x faster than a per-bar
price feature's. The network was biased toward the external block by a factor
of the window length - and pointing the wrong way, since these features cleared
only a marginal incremental screen while price is the base signal.
Zeroed at WINDOW ASSEMBLY, not in BufferTempData: that output is cached PER BAR
and a bar sits at slot 15 of one window and slot 0 of the next, so a
slot-dependent value there would poison the cache or force a recompute per slot.
The cache keeps true values; only this window's copies are cleared. Width
contract untouched - same count, same positions - so conv/LSTM/HYBRID keep their
bar-major rectangle unchanged and the block arrives at the newest bar, which for
the LSTM is the final timestep. Zero-variance coordinates are safe through batch
norm (divisor is MathMax(MathSqrt(var + BN_EPSILON), BN_MIN_STD)).
Fingerprint gains |ALTW:1 when alt data is on. Same width and same .cfg, so
nothing else would have caught a model trained under the replicated layout
resuming under this one. Conditional append per the existing rule: configs
without alt data keep their fingerprints and their trained models.
NOT the concat branch. CNet is a strictly linear stack (CLayerDescription has no
input-source field; NetBuild wires i to i+1 and stores layer L's weights on
L-1), so a real two-tower model needs a new multi-input layer type across
WarriorCPU, WarriorDML and the OpenCL kernels plus an .nnw format change - the
highest-risk change in this repo, in the code that produced the transposed dense
gradient, the Adam second-moment bug and the reversed LSTM window. This captures
the part of that idea the measurement actually supports, at no engine risk.
Compile-verified: 0 errors, 0 warnings.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-16 22:05:10 -04:00
|
|
|
}
|
diag: name the cause when every feature window fails, and enforce the width contract
Era 0 stalls with "NOT ONE of 54681 scanned bars produced a usable
feature window, windows ok=0 failed=54681" and nothing else. That line
reads identically for a cold ATR, a conditionally-missing optional
feature block and an out-of-range index, so it cannot be diagnosed
without one restart per hypothesis.
Two changes:
1. WIDTH CONTRACT in BufferTempData. Every enabled block must emit
exactly m_neuronsCount values on EVERY bar. A block that emits its
values on some bars and skips them on others (indicator, panel or
series unavailable for that bar) does not merely shorten the window -
it SHIFTS every feature after it into the wrong slot, and the net
then trains on silently misaligned inputs that still look like a
valid window to everything downstream. Now rejected, rolled back and
reported once, naming the optional blocks (XA / SPR / swing context)
as the ones carrying an availability test. Worth having independently
of the current stall.
2. BuildFeatureWindow records WHICH lookback slot rejected and how much
of the window was assembled, and the pass-1 stall report renders it:
"slot 0 of 20 REJECTED (window had 0 of 760)" is an indicator warm-up
or history-edge read; "every lookback bar ACCEPTED and the window was
still short: 640 of 760" is a missing 6-value block.
No behaviour change on a healthy run: the width check is an equality
that already holds, and the diagnostics render only inside the
total-failure branch.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-11 10:31:56 -04:00
|
|
|
return true;
|
fix: the sequence models were reading the window backwards
BuildFeatureWindow() replaces eight hand-rolled copies of the same loop
and feeds the window OLDEST BAR FIRST. Every copy fed it newest-first,
because MQL5 timeseries indices run backwards and `r + b` with b ascending
walks into the past.
Harmless for PAI and CONV - a dense layer learns a weight per position
either way, a conv learns time-mirrored kernels. Not harmless for the
recurrent stacks:
- LSTM_SeqStepForward reads `inputs + t*Iw`, so step t is block t.
- It writes output[] only when t == steps-1: the visible output IS the
last hidden state.
- c_t = f*c_{t-1} + i*g decays toward the start of the sequence.
lstm_seq_flowcheck.cpp measured block 0's influence on the output at
1.2e-2 of block T-1's, at the shipped forget bias of 1.0.
So the bar being PREDICTED sat at the far end of the decay and the output
was handed to the OLDEST bar in the window - the exact inverse of what the
window is for. ~80x backwards on LSTM and HYBRID, on all three tiers
(OpenCL kernel, CPU DLL, pure-MQL5 inference), which is why it never
surfaced as a backend discrepancy.
This does not create edge - the MI diagnostics read at the noise floor
(p=0.4975) with a working positive control. It makes the one hypothesis
those diagnostics explicitly do NOT cover testable: they are marginal and
per-bar, and state they "cannot rule out one that only exists in
combination or across time". The sequence model is the instrument for
across-time structure and it has been crippled, so that hypothesis has
never been honestly tested.
Fingerprint gets an unconditional |WIN:2 - the vector keeps its shape and
its features, so a stale .nnw would load cleanly and run a model fitted to
one ordering against the other, silently. Re-keying every config is the
point, not collateral damage. FORCES A FULL RETRAIN.
Also: the now-relative bar caches are re-keyed on the two live paths.
EnsureBarCachesCapacity() was only ever called from training paths, but
once m_trainingComplete is set ScheduleTrainingIfNeeded() routes every bar
to RefreshConvergedSignal() and Train() is never re-entered - so nothing
cleared the feature cache again for the life of the process. A chart that
trained to convergence kept replaying the rows computed for the last
training era's bar grid: the live signal froze at its convergence-time
value, and OnlineLearnStep() backpropped those stale features against
freshly resolved labels. Backtests were never affected (an inference-only
process never allocates the arrays, so every read recomputes).
Compiles clean: 0 errors, 0 warnings.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-07 18:28:44 -04:00
|
|
|
}
|
fix(signals): revive a dead MA model, and demote Sanyaku from state to event
Two defects surfaced by research/test_classic.py, both verified fixed by re-running the
transcription against 178k bars of EURUSD H1.
CSignalMA model 1 could never fire. For any recursive average - and MA_TYPE_EMA is the
shipped default - MA(i) = a*Close(i) + (1-a)*MA(i+1), so
DiffMA(i) = a * (Close(i) - MA(i+1))
DiffCloseMA(i) = (1-a) * (Close(i) - MA(i+1))
are positive multiples of one quantity and always share a sign. Model 1 asks for a close
BELOW a RISING average, which is precisely the combination that identity forbids: 0.000%
of bars, either direction, any symbol. The MQL5 standard library this was ported from
defaults to MODE_SMA, where the two are merely correlated - the bug arrived with the EMA
default, not with the port. Reading the slope one bar back (DiffMAPrev) breaks the tie for
every MA type while keeping the model's stated meaning. Now fires on 7.92% of bars.
CSignalIchimoku model 11 fired on 27% of bars at weight 100. Sanyaku is three standing
STATES conjoined with no transition term, so it held across long stretches - and being
last in the if-chain at the top weight, the module's highest-conviction reading was also
its most common one, overwriting all eight event models below it on a quarter of all bars.
The old comment rejected an event form because "demanding all three flip on the same bar
would fire almost never" - true, but that is not the alternative. Kouten is the TURN: the
ALIGNMENT transitions, and only one role need change for it to. Testing !Sanyaku(idx+1)
fires once per aligned stretch. Now 2.17%, in line with Kumo breakout (2.4%) and the
strong TK cross (1.1%). DataReady() extended one bar deeper to cover the lookback.
Neither pattern showed edge before or after; this is about the models meaning what they
say and the vote not being dominated by a constant.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-01 17:14:34 -04:00
|
|
|
//+------------------------------------------------------------------+
|
|
|
|
|
//| (Re)build the cross-asset panel over `bars` bars. |
|
|
|
|
|
//| |
|
|
|
|
|
//| Called from the same places that size the price buffers, because |
|
|
|
|
|
//| the panel is aligned to exactly that bar grid and a stale panel |
|
|
|
|
|
//| would silently mis-index. Cheap to call redundantly: Build() is |
|
|
|
|
|
//| one CopyClose per reference pair, not per bar. |
|
|
|
|
|
//| |
|
|
|
|
|
//| A failure here is NOT fatal. The panel logs its own reason and |
|
|
|
|
|
//| every Features() call then 0-fills, so the run continues without |
|
|
|
|
|
//| the cross-asset block instead of refusing to train. |
|
|
|
|
|
//+------------------------------------------------------------------+
|
|
|
|
|
bool CExpertSignalAIBase::BuildCrossAssetPanel(int bars)
|
|
|
|
|
{
|
|
|
|
|
if(!m_useCrossAsset)
|
|
|
|
|
return true;
|
|
|
|
|
if(bars <= 0)
|
|
|
|
|
return false;
|
feat(ai): spread as a volatility-regime feature, and fix a stale-index cache in both new blocks
Adds spread/ATR and the spread change ratio as network inputs (EnableSpreadFeature,
default on). Spread is the one microstructure channel that is both FX-available and
genuinely historical in the Strategy Tester - "during testing, the spread is not modeled
but is taken from historical data" - so unlike swap, signed tick flow or depth of market it
is something a backtest can honestly validate.
What it encodes, stated precisely because the raw measurement overstates it.
research/test_spread.py found spr/atr the strongest single feature in this codebase, on 5
of 8 instrument/geometry cells at 2-4x any volume feature. But the barrier LABEL charges
the spread inside its own barriers, so a wide-spread bar is mechanically likelier to
resolve as a loss and the feature would partly be predicting its own cost model. Relabelling
at zero cost and re-measuring the identical feature showed 20-40% of it WAS that tautology
and the majority was not (XAUUSD retained 97%). What survives is a volatility-regime
reading: spread is near-fixed while ATR is not, so the ratio runs high exactly when
realised volatility is below its own ATR estimate, which genuinely predicts whether
ATR-scaled barriers get reached. It is UNSIGNED - Neutral-vs-directional only, never a side.
Also fixes a stale-index bug I introduced with the cross-asset panel and had just repeated
in the spread series. Both cached on length alone:
if(m_crossAsset.Bars() >= bars) return true;
MQL5 series indices are relative to NOW, so one new closed candle shifts every index by
one. Keyed only on length, the panel keeps serving its index 0 as a bar that is no longer
the newest, and every cross-asset value is read one bar out of step with the price features
sitting beside it in the same vector - silently, with no error and no shape change. This is
the same class of defect as the dtStudied watermark behind the zero-direction backtests.
Both now carry a datetime anchor on m_Time.GetData(0), the same invalidation key the
label/feature bar caches already use.
And a performance fix that fell out of it: with correct invalidation the panel rebuilds on
every new bar, and RefreshConvergedSignal runs per bar - which in the tester would mean one
full multi-symbol resample per simulated bar at training depth. Inference only reads bars
0..m_historyBars-1 plus the panel's own slow window, so it now requests exactly that. The
cache check is >=, so a deeper panel left from training still satisfies it.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-01 17:42:40 -04:00
|
|
|
//--- Deep enough AND anchored to the current newest bar. Depth alone would leave the panel's
|
|
|
|
|
//--- index 0 pointing at a bar that is no longer the newest as soon as one candle closes, so
|
|
|
|
|
//--- every cross-asset feature would be read one bar out of step with the price features beside
|
|
|
|
|
//--- it - see m_crossAssetAnchor's declaration comment.
|
|
|
|
|
datetime anchor = m_Time.GetData(0);
|
|
|
|
|
if(m_crossAsset.IsReady() && m_crossAsset.Bars() >= bars && m_crossAssetAnchor == anchor && anchor > 0)
|
fix(signals): revive a dead MA model, and demote Sanyaku from state to event
Two defects surfaced by research/test_classic.py, both verified fixed by re-running the
transcription against 178k bars of EURUSD H1.
CSignalMA model 1 could never fire. For any recursive average - and MA_TYPE_EMA is the
shipped default - MA(i) = a*Close(i) + (1-a)*MA(i+1), so
DiffMA(i) = a * (Close(i) - MA(i+1))
DiffCloseMA(i) = (1-a) * (Close(i) - MA(i+1))
are positive multiples of one quantity and always share a sign. Model 1 asks for a close
BELOW a RISING average, which is precisely the combination that identity forbids: 0.000%
of bars, either direction, any symbol. The MQL5 standard library this was ported from
defaults to MODE_SMA, where the two are merely correlated - the bug arrived with the EMA
default, not with the port. Reading the slope one bar back (DiffMAPrev) breaks the tie for
every MA type while keeping the model's stated meaning. Now fires on 7.92% of bars.
CSignalIchimoku model 11 fired on 27% of bars at weight 100. Sanyaku is three standing
STATES conjoined with no transition term, so it held across long stretches - and being
last in the if-chain at the top weight, the module's highest-conviction reading was also
its most common one, overwriting all eight event models below it on a quarter of all bars.
The old comment rejected an event form because "demanding all three flip on the same bar
would fire almost never" - true, but that is not the alternative. Kouten is the TURN: the
ALIGNMENT transitions, and only one role need change for it to. Testing !Sanyaku(idx+1)
fires once per aligned stretch. Now 2.17%, in line with Kumo breakout (2.4%) and the
strong TK cross (1.1%). DataReady() extended one bar deeper to cover the lookback.
Neither pattern showed edge before or after; this is about the models meaning what they
say and the vote not being dominated by a constant.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-01 17:14:34 -04:00
|
|
|
return true;
|
2026-08-11 21:29:14 -04:00
|
|
|
//--- A trained model builds from the pair set it was trained on (adopted from the .cfg), never
|
|
|
|
|
//--- from whatever Market Watch holds today - see m_crossAssetPairsPinned.
|
|
|
|
|
if(m_crossAssetPairsPinned != "" && !m_crossAsset.HasPinnedPairs())
|
|
|
|
|
m_crossAsset.SetPinnedPairs(m_crossAssetPairsPinned);
|
feat(ai): spread as a volatility-regime feature, and fix a stale-index cache in both new blocks
Adds spread/ATR and the spread change ratio as network inputs (EnableSpreadFeature,
default on). Spread is the one microstructure channel that is both FX-available and
genuinely historical in the Strategy Tester - "during testing, the spread is not modeled
but is taken from historical data" - so unlike swap, signed tick flow or depth of market it
is something a backtest can honestly validate.
What it encodes, stated precisely because the raw measurement overstates it.
research/test_spread.py found spr/atr the strongest single feature in this codebase, on 5
of 8 instrument/geometry cells at 2-4x any volume feature. But the barrier LABEL charges
the spread inside its own barriers, so a wide-spread bar is mechanically likelier to
resolve as a loss and the feature would partly be predicting its own cost model. Relabelling
at zero cost and re-measuring the identical feature showed 20-40% of it WAS that tautology
and the majority was not (XAUUSD retained 97%). What survives is a volatility-regime
reading: spread is near-fixed while ATR is not, so the ratio runs high exactly when
realised volatility is below its own ATR estimate, which genuinely predicts whether
ATR-scaled barriers get reached. It is UNSIGNED - Neutral-vs-directional only, never a side.
Also fixes a stale-index bug I introduced with the cross-asset panel and had just repeated
in the spread series. Both cached on length alone:
if(m_crossAsset.Bars() >= bars) return true;
MQL5 series indices are relative to NOW, so one new closed candle shifts every index by
one. Keyed only on length, the panel keeps serving its index 0 as a bar that is no longer
the newest, and every cross-asset value is read one bar out of step with the price features
sitting beside it in the same vector - silently, with no error and no shape change. This is
the same class of defect as the dtStudied watermark behind the zero-direction backtests.
Both now carry a datetime anchor on m_Time.GetData(0), the same invalidation key the
label/feature bar caches already use.
And a performance fix that fell out of it: with correct invalidation the panel rebuilds on
every new bar, and RefreshConvergedSignal runs per bar - which in the tester would mean one
full multi-symbol resample per simulated bar at training depth. Inference only reads bars
0..m_historyBars-1 plus the panel's own slow window, so it now requests exactly that. The
cache check is >=, so a deeper panel left from training still satisfies it.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-01 17:42:40 -04:00
|
|
|
if(!m_crossAsset.Build(m_symbol.Name(), (ENUM_TIMEFRAMES)m_period, bars))
|
|
|
|
|
{
|
|
|
|
|
m_crossAssetAnchor = 0;
|
|
|
|
|
return false;
|
|
|
|
|
}
|
|
|
|
|
m_crossAssetAnchor = anchor;
|
2026-08-11 21:29:14 -04:00
|
|
|
//--- FIRST successful build of a model with no pinned set yet: this pair set is now this model's
|
|
|
|
|
//--- pair set for life. Stamp it and pin it to the .cfg one-shot, exactly like the derived barrier
|
|
|
|
|
//--- pair (see Labels.mqh's m_geometryCfgSaved block) - the .cfg was written at model creation,
|
|
|
|
|
//--- BEFORE the panel could possibly have built, so without this re-save the pin would live only
|
|
|
|
|
//--- in memory and every restart would silently fall back to discovery.
|
|
|
|
|
if(m_crossAssetPairsPinned == "" && m_crossAsset.UsedPairsCsv() != "")
|
|
|
|
|
{
|
|
|
|
|
m_crossAssetPairsPinned = m_crossAsset.UsedPairsCsv();
|
|
|
|
|
m_crossAsset.SetPinnedPairs(m_crossAssetPairsPinned);
|
|
|
|
|
if(!m_crossAssetCfgSaved && m_activeFileName != "")
|
|
|
|
|
{
|
|
|
|
|
m_crossAssetCfgSaved = true;
|
|
|
|
|
if(SaveTopologyConfiguration(m_activeFileName, m_initialNeuronsCount, m_hiddenLayersCount,
|
|
|
|
|
m_neuronsReduction, m_minNeuronsCount, m_optimizationAlgo,
|
|
|
|
|
m_historyBars, m_outputNeuronsCount, m_neuronsCount,
|
|
|
|
|
LEGACY_STUDY_PERIOD_SLOT, m_minTrainYear, m_isInitialized,
|
|
|
|
|
LEGACY_CONVERGE_WR_SLOT, m_fractalPeriods, m_convFilterCount,
|
|
|
|
|
m_lstmHiddenSize, m_activeFileCommon))
|
|
|
|
|
Print(ID + ": cross-asset pair set PINNED to the .cfg - [" + m_crossAssetPairsPinned +
|
|
|
|
|
"]. Restarts and redeploys now build the panel from exactly this set; Market Watch "
|
|
|
|
|
"changes no longer alter what a trained model's features mean.");
|
|
|
|
|
else
|
|
|
|
|
Print(ID + ": WARNING - failed to pin the cross-asset pair set to the .cfg; a restart "
|
|
|
|
|
"will re-discover Market Watch instead of adopting the trained set.");
|
|
|
|
|
}
|
|
|
|
|
}
|
feat(ai): spread as a volatility-regime feature, and fix a stale-index cache in both new blocks
Adds spread/ATR and the spread change ratio as network inputs (EnableSpreadFeature,
default on). Spread is the one microstructure channel that is both FX-available and
genuinely historical in the Strategy Tester - "during testing, the spread is not modeled
but is taken from historical data" - so unlike swap, signed tick flow or depth of market it
is something a backtest can honestly validate.
What it encodes, stated precisely because the raw measurement overstates it.
research/test_spread.py found spr/atr the strongest single feature in this codebase, on 5
of 8 instrument/geometry cells at 2-4x any volume feature. But the barrier LABEL charges
the spread inside its own barriers, so a wide-spread bar is mechanically likelier to
resolve as a loss and the feature would partly be predicting its own cost model. Relabelling
at zero cost and re-measuring the identical feature showed 20-40% of it WAS that tautology
and the majority was not (XAUUSD retained 97%). What survives is a volatility-regime
reading: spread is near-fixed while ATR is not, so the ratio runs high exactly when
realised volatility is below its own ATR estimate, which genuinely predicts whether
ATR-scaled barriers get reached. It is UNSIGNED - Neutral-vs-directional only, never a side.
Also fixes a stale-index bug I introduced with the cross-asset panel and had just repeated
in the spread series. Both cached on length alone:
if(m_crossAsset.Bars() >= bars) return true;
MQL5 series indices are relative to NOW, so one new closed candle shifts every index by
one. Keyed only on length, the panel keeps serving its index 0 as a bar that is no longer
the newest, and every cross-asset value is read one bar out of step with the price features
sitting beside it in the same vector - silently, with no error and no shape change. This is
the same class of defect as the dtStudied watermark behind the zero-direction backtests.
Both now carry a datetime anchor on m_Time.GetData(0), the same invalidation key the
label/feature bar caches already use.
And a performance fix that fell out of it: with correct invalidation the panel rebuilds on
every new bar, and RefreshConvergedSignal runs per bar - which in the tester would mean one
full multi-symbol resample per simulated bar at training depth. Inference only reads bars
0..m_historyBars-1 plus the panel's own slow window, so it now requests exactly that. The
cache check is >=, so a deeper panel left from training still satisfies it.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-01 17:42:40 -04:00
|
|
|
return true;
|
|
|
|
|
}
|
|
|
|
|
//+------------------------------------------------------------------+
|
|
|
|
|
//| Copy the historical spread series onto the current bar grid. |
|
|
|
|
|
//| |
|
|
|
|
|
//| CopySpread is a RANGE call, so this runs once wherever the price |
|
|
|
|
|
//| buffers are sized - never per bar. Values are in POINTS (int); |
|
|
|
|
|
//| the feature block converts with m_symbol.Point(). |
|
|
|
|
|
//| |
|
|
|
|
|
//| Non-fatal: a short or failed copy leaves m_spreadSeriesBars at |
|
|
|
|
|
//| whatever was actually obtained and the feature block 0-fills past |
|
|
|
|
|
//| it, matching the degraded-but-usable convention used by the swing |
|
|
|
|
|
//| and cross-asset blocks. Refusing to train because one auxiliary |
|
|
|
|
|
//| series came up short would be a far worse failure. |
|
|
|
|
|
//+------------------------------------------------------------------+
|
|
|
|
|
bool CExpertSignalAIBase::EnsureSpreadSeries(int bars)
|
|
|
|
|
{
|
feat: S2 meta-labeling head - binary trade-quality model over the classic-candidate corpus
The NN now has a target that is not per-bar direction (closed, best-of-999
p=1.0000): P(win | this journaled candidate, at the EA's own SL/TP, net of
cost). One net for all 52 pattern-sides, AIType=AI_META.
- NetForward.mqh: the host-side softmax+CE gradient generalized total==3 ->
2||3 on both backprop paths; a 2-class softmax IS a logistic head, and no
compute backend changes.
- SignalMETA.mqh (new): corpus loaded read-only from the LARGEST signal DB on
disk (decoupled from the config fingerprint that burned four S1 runs); the
GMT->server offset is measured PER ROW against entryPrice vs bar open
(DST-immune, histogram logged); a window-span regime filter drops the
pre-2017 daily-backfill rows; 31-feature setup descriptor appended at the
input (26 one-hot + side + tanh netVote + SL/TP ATR + spread/ATR).
- Training.mqh: candidate-queued pass 1, binary-target pass 2, per-candidate
calibration (2.5) and OOS (3) walks. Counter mapping win->Buy / loss->Sell
lets checkpoint selection, the edge floor, the plateau ladder and the
family-wise deploy gate run UNCHANGED: precision reads as win rate among
traded candidates, chance as the base win rate, recalls as sensitivity/
specificity. Era-end META line: coverage x (p - break-even) vs the null.
- Labels are the side-conditional triple-barrier win caches - never the DB's
stop-and-reverse outcome. Logit adjustment deliberately skipped (~40% base
rate). Live inference + online learning guarded off until S3.
- Fingerprint: conditional |TGT:META1; State\META\ folder + 2-output filename
slot keep meta models fully separate from direction models.
Compiles clean (0 errors, 0 warnings). S2 run = attach a chart with
AIType=AI_META; S3 wires the votes via the per-side hooks.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-13 06:52:31 -04:00
|
|
|
//--- The meta target's setup descriptor reads spread/ATR at the candidate's fire bar regardless of
|
|
|
|
|
//--- whether spread is enabled as a per-bar WINDOW feature, so the series must exist for it.
|
|
|
|
|
if(!m_useSpreadFeature && !IsMetaTarget())
|
feat(ai): spread as a volatility-regime feature, and fix a stale-index cache in both new blocks
Adds spread/ATR and the spread change ratio as network inputs (EnableSpreadFeature,
default on). Spread is the one microstructure channel that is both FX-available and
genuinely historical in the Strategy Tester - "during testing, the spread is not modeled
but is taken from historical data" - so unlike swap, signed tick flow or depth of market it
is something a backtest can honestly validate.
What it encodes, stated precisely because the raw measurement overstates it.
research/test_spread.py found spr/atr the strongest single feature in this codebase, on 5
of 8 instrument/geometry cells at 2-4x any volume feature. But the barrier LABEL charges
the spread inside its own barriers, so a wide-spread bar is mechanically likelier to
resolve as a loss and the feature would partly be predicting its own cost model. Relabelling
at zero cost and re-measuring the identical feature showed 20-40% of it WAS that tautology
and the majority was not (XAUUSD retained 97%). What survives is a volatility-regime
reading: spread is near-fixed while ATR is not, so the ratio runs high exactly when
realised volatility is below its own ATR estimate, which genuinely predicts whether
ATR-scaled barriers get reached. It is UNSIGNED - Neutral-vs-directional only, never a side.
Also fixes a stale-index bug I introduced with the cross-asset panel and had just repeated
in the spread series. Both cached on length alone:
if(m_crossAsset.Bars() >= bars) return true;
MQL5 series indices are relative to NOW, so one new closed candle shifts every index by
one. Keyed only on length, the panel keeps serving its index 0 as a bar that is no longer
the newest, and every cross-asset value is read one bar out of step with the price features
sitting beside it in the same vector - silently, with no error and no shape change. This is
the same class of defect as the dtStudied watermark behind the zero-direction backtests.
Both now carry a datetime anchor on m_Time.GetData(0), the same invalidation key the
label/feature bar caches already use.
And a performance fix that fell out of it: with correct invalidation the panel rebuilds on
every new bar, and RefreshConvergedSignal runs per bar - which in the tester would mean one
full multi-symbol resample per simulated bar at training depth. Inference only reads bars
0..m_historyBars-1 plus the panel's own slow window, so it now requests exactly that. The
cache check is >=, so a deeper panel left from training still satisfies it.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-01 17:42:40 -04:00
|
|
|
return true;
|
|
|
|
|
if(bars <= 0)
|
|
|
|
|
return false;
|
|
|
|
|
//--- Length alone is NOT a sufficient cache key - see m_spreadSeriesAnchor's declaration comment.
|
|
|
|
|
datetime anchor = m_Time.GetData(0);
|
|
|
|
|
if(m_spreadSeriesBars >= bars && m_spreadSeriesAnchor == anchor && anchor > 0)
|
|
|
|
|
return true;
|
|
|
|
|
ArraySetAsSeries(m_spreadSeries, true); // index 0 = newest, matching every other buffer here
|
|
|
|
|
int got = CopySpread(m_symbol.Name(), (ENUM_TIMEFRAMES)m_period, 0, bars, m_spreadSeries);
|
|
|
|
|
if(got <= 0)
|
|
|
|
|
{
|
|
|
|
|
m_spreadSeriesBars = 0;
|
|
|
|
|
m_spreadSeriesAnchor = 0;
|
|
|
|
|
Print(__FUNCTION__ + ": CopySpread returned " + IntegerToString(got) + " for " + m_symbol.Name() +
|
|
|
|
|
" - spread features 0-filled this run.");
|
|
|
|
|
return false;
|
|
|
|
|
}
|
|
|
|
|
m_spreadSeriesBars = got;
|
|
|
|
|
m_spreadSeriesAnchor = anchor;
|
|
|
|
|
return true;
|
fix(signals): revive a dead MA model, and demote Sanyaku from state to event
Two defects surfaced by research/test_classic.py, both verified fixed by re-running the
transcription against 178k bars of EURUSD H1.
CSignalMA model 1 could never fire. For any recursive average - and MA_TYPE_EMA is the
shipped default - MA(i) = a*Close(i) + (1-a)*MA(i+1), so
DiffMA(i) = a * (Close(i) - MA(i+1))
DiffCloseMA(i) = (1-a) * (Close(i) - MA(i+1))
are positive multiples of one quantity and always share a sign. Model 1 asks for a close
BELOW a RISING average, which is precisely the combination that identity forbids: 0.000%
of bars, either direction, any symbol. The MQL5 standard library this was ported from
defaults to MODE_SMA, where the two are merely correlated - the bug arrived with the EMA
default, not with the port. Reading the slope one bar back (DiffMAPrev) breaks the tie for
every MA type while keeping the model's stated meaning. Now fires on 7.92% of bars.
CSignalIchimoku model 11 fired on 27% of bars at weight 100. Sanyaku is three standing
STATES conjoined with no transition term, so it held across long stretches - and being
last in the if-chain at the top weight, the module's highest-conviction reading was also
its most common one, overwriting all eight event models below it on a quarter of all bars.
The old comment rejected an event form because "demanding all three flip on the same bar
would fire almost never" - true, but that is not the alternative. Kouten is the TURN: the
ALIGNMENT transitions, and only one role need change for it to. Testing !Sanyaku(idx+1)
fires once per aligned stretch. Now 2.17%, in line with Kumo breakout (2.4%) and the
strong TK cross (1.1%). DataReady() extended one bar deeper to cover the lookback.
Neither pattern showed edge before or after; this is about the models meaning what they
say and the vote not being dominated by a constant.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-01 17:14:34 -04:00
|
|
|
}
|
|
|
|
|
//+------------------------------------------------------------------+
|
refactor: split CExpertSignalAIBase implementation by responsibility
ExpertSignalAIBase.mqh was 8216 lines: the class declaration followed by 87
method bodies covering training, labelling, feature extraction, persistence,
chart drawing, online learning, the GA auto-tuner and inference, all in one
file. Train() alone is 1492 lines; a change to arrow drawing meant scrolling
past the era loop.
Moved the bodies into Expert\AIBase\, included at the bottom of the original
after the class declaration:
Training.mqh 1607 era loop, plateau ladder, checkpoint select, deploy
Features.mqh 1093 indicator creation + per-bar input feature vector
ChartUI.mqh 634 arrows, arrow persistence, status panel, cleanup
Persistence.mqh 492 .stats/.cfg sidecars, CPU-inference validation, copy
OnlineLearning.mqh 461 live continual learning, EMA shadow, OOS simulator
Labels.mqh 309 ZigZag pivot labels, async label-cache prebuild
AutoTune.mqh 275 genetic tuner (population, crossover, halving)
Inference.mqh 235 softmax, prior calibration, class priors
ExpertSignalAIBase.mqh 8216 -> 3131 (declaration + topology build only)
This is a pure relocation - verified mechanically, not by eye: HEAD's file
reconstructed from the eight partials plus the surviving remainder is
byte-identical to HEAD, span for span (scratchpad verify_split.py). No
declaration moved, no signature changed, no code rewritten, so behaviour is
unchanged by construction.
Compiles 0 errors, 0 warnings.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-29 00:42:45 -04:00
|
|
|
bool CExpertSignalAIBase::BufferTempDataCompute(int idx)
|
|
|
|
|
{
|
2026-08-02 01:09:18 -04:00
|
|
|
//--- Where THIS bar's block starts. The function appends m_neuronsCount values below; remembering
|
|
|
|
|
//--- the offset lets the whole vector be validated in one place at the end instead of at each of
|
|
|
|
|
//--- the ~60 Add() call sites.
|
|
|
|
|
int featureStart = TempData.Total();
|
fix: a resumed model cached a cold ATR as permanent, so it never trained
BufferTempData cached EVERY failure - m_featureCacheHasValue[idx]=true
with m_featureCacheValid[idx]=false - and the cache never re-tries a
miss. So a single feature read taken before the terminal had finished
calculating the indicator buffers marked those bars unusable for the
rest of the process, even though the data arrived milliseconds later.
MT5 fills an indicator's buffers asynchronously after the handle is
created, and a cold ATR returns 0 for EVERY index, not just its warm-up
tail. BufferTempDataCompute rejects a bar with no ATR (correctly - the
price features would be meaningless), so the whole window failed, and
the whole cache was poisoned.
Only resumed models were hit, because only they read features that
early. Topology.mqh sets m_warmupPassesRemaining = netLoaded ? 0 : 3:
a fresh start sits through three separately-scheduled Train() calls
before anything touches a feature, which is exactly what those passes
are for. A resumed one skips them and TuneIndicatorsAndTrain drives
StartLabelCachePrebuild and the MI report from the first chart event.
Its rationale - "a restart already has a proven-synced history" - holds
for HISTORY and not for INDICATORS, which are recreated every process
start.
Downstream: BuildFeatureWindow failed on every bar of every era, so
add_loop never went true, so pass 2, pass 3, the era counter and the
checkpoint were all skipped and pass 1 swept 0->100% forever. The
"0 samples" MI report line at startup was the same failure, four
seconds earlier, already visible in the log.
- a miss is now cached only when it is PERMANENT; the two "not ready
yet" guards mark m_featureFailTransient and are recomputed on the
next visit. Steady-state cost is ~ind_Periods bars per era, not 54k.
- an era that discards itself now drops the feature cache before
restarting, so any remaining cause of this state self-heals instead
of looping.
Deleting the .nnw "fixed" this only by turning the model back into a
fresh one.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-10 11:41:31 -04:00
|
|
|
//--- Cleared here, set by the two NOT-READY-YET guards below. See BufferTempData() for what it
|
|
|
|
|
//--- controls: a rejection caused by data that has not arrived yet must not be cached, because the
|
|
|
|
|
//--- cache never re-tries a miss.
|
|
|
|
|
m_featureFailTransient = false;
|
fix(ensemble+depth): the barrier had no liveness escape, and the depth gate could not report the one state the evidence pointed at
Two charts (USDJPY 50,179 bars / XAUUSD 33,982) sat at era 0 for 38 minutes with
four of their eight members completely silent. Nothing in this commit guesses at
why the sweep fails - the last five guesses were all wrong. It makes the failure
say what it is, and stops one broken member taking its whole chart down with it.
WHAT THE LOG ACTUALLY SAYS, before any of this.
- The running build IS d9f834d (pulled 14:18, compiled 14:19:01, 0 errors), so
every depth instrument from 1dda479/7e63a8b/45c9e21 was live.
- It printed NOTHING. Zero "PRIMING", zero "CAPPED", zero "Per-indicator depth"
in 27 MB of journal. The instrument built to find the depth shortfall returned
"not this".
- On USDJPY at 14:24, CONV-cad8 completed eras 0 AND 1 across all 50,179 bars -
same chart, same 832-value window, same indicators, byte-identical fingerprint -
while LSTM-cad8 and HYB-cad8 reported ok=0 failed=50163. So it is not the
symbol, the history, the bar count or the indicator depth. It is per-member.
- ok=0 means the NEWEST anchors failed too, and a short indicator cannot do that.
The depth reading in project_silent_block_failures is therefore retired by its own
instrumentation. THE ROOT CAUSE IS STILL UNKNOWN and this commit does not claim one.
1. THE DEPTH GATE'S SILENT PATH WAS THE STATE IT WAS HUNTING.
ServableBars() read `if(servable <= 0 || servable >= want) return want;` - one
branch over three unrelated states, silent in all of them:
enabled == 0 -> nothing tunable is on. No cap. Healthy.
enabled > 0, servable == -1 -> a handle answered INVALID.
enabled > 0, servable == 0 -> created, never calculated.
BarsCalculated() returns -1 for a dead handle, so a dead MA is indistinguishable
from "no tunable indicators enabled" - and both returned `want` without printing a
character. That is exactly the state a per-member, every-index, depth-independent
failure produces, and it is the single reason a build carrying full depth
instrumentation logged nothing through the whole outage.
TunableBarsCalculated() now also reports HOW MANY indicators it consulted, and the
dead-handle case is reported (latched, with per-handle depths). The RETURN is
deliberately unchanged - what to do about a dead handle is not yet known, and
changing control flow on an unproven cause is how the last four fixes here went
wrong. SettledBars() routes its three pass-through states via ServableBars() so the
report is reachable from the training sweep, which is the only caller that hits it.
2. THE STALL REPORT NAMED A SLOT, NEVER A BLOCK.
"lookback slot 0 REJECTED (window had 24 of 832 values)" plus a guess ("an
indicator warm-up or a history-edge read"). Which guard fired was INFERRED by
counting 4+5+4+4+6+1 = 24 and concluding feature 25 must be the MA. The arithmetic
was right; every conclusion drawn from it was wrong, because a value count names a
POSITION and a position cannot tell cold from capped from invalid from off-the-end.
Every guard that can reject a bar now records itself - m_featureFailBlock - and the
report carries it, the series index, IndicatorDepthReport()'s per-handle depths,
and for each indicator whether the NEWEST bar reads. That last field is the whole
diagnosis in one word: newest-also-EMPTY means the buffer is unreadable everywhere
(cold or dead handle), newest-reads means a genuine history edge. Instrumented:
open, ATR, MA, RSI, MACD, Ichimoku, and all five AD blocks via ADIndicatorCold().
3. THE TOTAL-FAILURE BACKOFF WAS GATED ON THE WRONG QUESTION.
It armed only when m_featureFailTransient was set. Keeping that flag correct across
every guard is a list that has to stay right forever - the same shape of fix the
feature cache abandoned for the same reason - and the gate is pointless anyway: a
sweep where ZERO of 50,163 bars produced a window will produce zero again if it
restarts a millisecond later, transient or not. Doing that at full speed is what
starved six indicator threads on a six-core box. The backoff is now unconditional
on a total failure. The flag keeps its real job, deciding whether a MISS may be
cached, which is a per-bar question and not a scheduling one.
4. THE ERA BARRIER DEADLOCKED, AND SILENCED THE MEMBERS IT FROZE.
EnsembleMinTrainingEra() exempted deployed, stopped and paused members and its
comment concluded "so nothing deadlocks". Those three are all VOLUNTARY. A member
that simply CANNOT finish an era is none of them, so it pinned the minimum at its
own era with no time limit - and the hold branch's only action was
`m_lastEraCompleteTick = GetTickCount()`, which silences the stall watchdog. So on
USDJPY the two members that could not train reported, and the two healthy members
frozen behind them wrote nothing anywhere. The outage was visible only through the
members that were not suffering it.
- BarrierEraHeartbeat() stamps a clock on real era CHANGE, kept separate from
m_lastEraCompleteTick precisely because the barrier resets that one. Only a
member AT the minimum can be a blocker; a member ahead is idle by design and is
never counted as stuck.
- After ENSEMBLE_BARRIER_STUCK_MS (12 min) a non-advancing member is dropped from
the barrier minimum. It keeps training and rejoins the instant it completes an
era - at which point, being behind, it legitimately becomes the minimum again,
which is the documented resumed-laggard behaviour.
- Both transitions say so loudly, and the release states plainly that the
combined-vote score cannot be computed while the ensemble is desynchronised.
- A held member now writes a rate-limited journal line naming WHICH members it is
waiting on, so the blocker is read off one line.
5. THE PANEL FLICKER.
OnTickHandler gates its terse writer on !m_trainRunActive, and a barrier-held member
returns from Train() before ever setting it - so both writers thought they were the
only one updating the label and fought every tick. That is the reported "Getting
ready..." <-> "Waiting at era N for slower ensemble members" oscillation, and it hit
Perceptron but not Convolutional purely because Convolutional had a run active from
a completed era and Perceptron, resumed from disk, never did. Train()'s message is
the specific one, so it wins.
NEXT STEP once this is running: the stall line now ends in "REJECTED BY: ..." and
the per-handle depths. Read it. Do not reason around it.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-17 15:21:15 -04:00
|
|
|
//--- Cleared alongside it, and written by every guard below that can return false - see the
|
|
|
|
|
//--- declaration for why a value COUNT was never enough to identify the block.
|
|
|
|
|
m_featureFailBlock = "";
|
|
|
|
|
m_featureFailIdx = idx;
|
refactor: split CExpertSignalAIBase implementation by responsibility
ExpertSignalAIBase.mqh was 8216 lines: the class declaration followed by 87
method bodies covering training, labelling, feature extraction, persistence,
chart drawing, online learning, the GA auto-tuner and inference, all in one
file. Train() alone is 1492 lines; a change to arrow drawing meant scrolling
past the era loop.
Moved the bodies into Expert\AIBase\, included at the bottom of the original
after the class declaration:
Training.mqh 1607 era loop, plateau ladder, checkpoint select, deploy
Features.mqh 1093 indicator creation + per-bar input feature vector
ChartUI.mqh 634 arrows, arrow persistence, status panel, cleanup
Persistence.mqh 492 .stats/.cfg sidecars, CPU-inference validation, copy
OnlineLearning.mqh 461 live continual learning, EMA shadow, OOS simulator
Labels.mqh 309 ZigZag pivot labels, async label-cache prebuild
AutoTune.mqh 275 genetic tuner (population, crossover, halving)
Inference.mqh 235 softmax, prior calibration, class priors
ExpertSignalAIBase.mqh 8216 -> 3131 (declaration + topology build only)
This is a pure relocation - verified mechanically, not by eye: HEAD's file
reconstructed from the eight partials plus the surviving remainder is
byte-identical to HEAD, span for span (scratchpad verify_split.py). No
declaration moved, no signature changed, no code rewritten, so behaviour is
unchanged by construction.
Compiles 0 errors, 0 warnings.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-29 00:42:45 -04:00
|
|
|
double open = m_Open.GetData(idx);
|
|
|
|
|
double close = m_Close.GetData(idx);
|
|
|
|
|
double high = m_High.GetData(idx);
|
|
|
|
|
double low = m_Low.GetData(idx);
|
|
|
|
|
MqlDateTime sTime;
|
|
|
|
|
TimeToStruct(m_Time.GetData(idx), sTime);
|
|
|
|
|
if(open == EMPTY_VALUE)
|
fix: a resumed model cached a cold ATR as permanent, so it never trained
BufferTempData cached EVERY failure - m_featureCacheHasValue[idx]=true
with m_featureCacheValid[idx]=false - and the cache never re-tries a
miss. So a single feature read taken before the terminal had finished
calculating the indicator buffers marked those bars unusable for the
rest of the process, even though the data arrived milliseconds later.
MT5 fills an indicator's buffers asynchronously after the handle is
created, and a cold ATR returns 0 for EVERY index, not just its warm-up
tail. BufferTempDataCompute rejects a bar with no ATR (correctly - the
price features would be meaningless), so the whole window failed, and
the whole cache was poisoned.
Only resumed models were hit, because only they read features that
early. Topology.mqh sets m_warmupPassesRemaining = netLoaded ? 0 : 3:
a fresh start sits through three separately-scheduled Train() calls
before anything touches a feature, which is exactly what those passes
are for. A resumed one skips them and TuneIndicatorsAndTrain drives
StartLabelCachePrebuild and the MI report from the first chart event.
Its rationale - "a restart already has a proven-synced history" - holds
for HISTORY and not for INDICATORS, which are recreated every process
start.
Downstream: BuildFeatureWindow failed on every bar of every era, so
add_loop never went true, so pass 2, pass 3, the era counter and the
checkpoint were all skipped and pass 1 swept 0->100% forever. The
"0 samples" MI report line at startup was the same failure, four
seconds earlier, already visible in the log.
- a miss is now cached only when it is PERMANENT; the two "not ready
yet" guards mark m_featureFailTransient and are recomputed on the
next visit. Steady-state cost is ~ind_Periods bars per era, not 54k.
- an era that discards itself now drops the feature cache before
restarting, so any remaining cause of this state self-heals instead
of looping.
Deleting the .nnw "fixed" this only by turning the model back into a
fresh one.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-10 11:41:31 -04:00
|
|
|
{
|
|
|
|
|
m_featureFailTransient = true;
|
fix(ensemble+depth): the barrier had no liveness escape, and the depth gate could not report the one state the evidence pointed at
Two charts (USDJPY 50,179 bars / XAUUSD 33,982) sat at era 0 for 38 minutes with
four of their eight members completely silent. Nothing in this commit guesses at
why the sweep fails - the last five guesses were all wrong. It makes the failure
say what it is, and stops one broken member taking its whole chart down with it.
WHAT THE LOG ACTUALLY SAYS, before any of this.
- The running build IS d9f834d (pulled 14:18, compiled 14:19:01, 0 errors), so
every depth instrument from 1dda479/7e63a8b/45c9e21 was live.
- It printed NOTHING. Zero "PRIMING", zero "CAPPED", zero "Per-indicator depth"
in 27 MB of journal. The instrument built to find the depth shortfall returned
"not this".
- On USDJPY at 14:24, CONV-cad8 completed eras 0 AND 1 across all 50,179 bars -
same chart, same 832-value window, same indicators, byte-identical fingerprint -
while LSTM-cad8 and HYB-cad8 reported ok=0 failed=50163. So it is not the
symbol, the history, the bar count or the indicator depth. It is per-member.
- ok=0 means the NEWEST anchors failed too, and a short indicator cannot do that.
The depth reading in project_silent_block_failures is therefore retired by its own
instrumentation. THE ROOT CAUSE IS STILL UNKNOWN and this commit does not claim one.
1. THE DEPTH GATE'S SILENT PATH WAS THE STATE IT WAS HUNTING.
ServableBars() read `if(servable <= 0 || servable >= want) return want;` - one
branch over three unrelated states, silent in all of them:
enabled == 0 -> nothing tunable is on. No cap. Healthy.
enabled > 0, servable == -1 -> a handle answered INVALID.
enabled > 0, servable == 0 -> created, never calculated.
BarsCalculated() returns -1 for a dead handle, so a dead MA is indistinguishable
from "no tunable indicators enabled" - and both returned `want` without printing a
character. That is exactly the state a per-member, every-index, depth-independent
failure produces, and it is the single reason a build carrying full depth
instrumentation logged nothing through the whole outage.
TunableBarsCalculated() now also reports HOW MANY indicators it consulted, and the
dead-handle case is reported (latched, with per-handle depths). The RETURN is
deliberately unchanged - what to do about a dead handle is not yet known, and
changing control flow on an unproven cause is how the last four fixes here went
wrong. SettledBars() routes its three pass-through states via ServableBars() so the
report is reachable from the training sweep, which is the only caller that hits it.
2. THE STALL REPORT NAMED A SLOT, NEVER A BLOCK.
"lookback slot 0 REJECTED (window had 24 of 832 values)" plus a guess ("an
indicator warm-up or a history-edge read"). Which guard fired was INFERRED by
counting 4+5+4+4+6+1 = 24 and concluding feature 25 must be the MA. The arithmetic
was right; every conclusion drawn from it was wrong, because a value count names a
POSITION and a position cannot tell cold from capped from invalid from off-the-end.
Every guard that can reject a bar now records itself - m_featureFailBlock - and the
report carries it, the series index, IndicatorDepthReport()'s per-handle depths,
and for each indicator whether the NEWEST bar reads. That last field is the whole
diagnosis in one word: newest-also-EMPTY means the buffer is unreadable everywhere
(cold or dead handle), newest-reads means a genuine history edge. Instrumented:
open, ATR, MA, RSI, MACD, Ichimoku, and all five AD blocks via ADIndicatorCold().
3. THE TOTAL-FAILURE BACKOFF WAS GATED ON THE WRONG QUESTION.
It armed only when m_featureFailTransient was set. Keeping that flag correct across
every guard is a list that has to stay right forever - the same shape of fix the
feature cache abandoned for the same reason - and the gate is pointless anyway: a
sweep where ZERO of 50,163 bars produced a window will produce zero again if it
restarts a millisecond later, transient or not. Doing that at full speed is what
starved six indicator threads on a six-core box. The backoff is now unconditional
on a total failure. The flag keeps its real job, deciding whether a MISS may be
cached, which is a per-bar question and not a scheduling one.
4. THE ERA BARRIER DEADLOCKED, AND SILENCED THE MEMBERS IT FROZE.
EnsembleMinTrainingEra() exempted deployed, stopped and paused members and its
comment concluded "so nothing deadlocks". Those three are all VOLUNTARY. A member
that simply CANNOT finish an era is none of them, so it pinned the minimum at its
own era with no time limit - and the hold branch's only action was
`m_lastEraCompleteTick = GetTickCount()`, which silences the stall watchdog. So on
USDJPY the two members that could not train reported, and the two healthy members
frozen behind them wrote nothing anywhere. The outage was visible only through the
members that were not suffering it.
- BarrierEraHeartbeat() stamps a clock on real era CHANGE, kept separate from
m_lastEraCompleteTick precisely because the barrier resets that one. Only a
member AT the minimum can be a blocker; a member ahead is idle by design and is
never counted as stuck.
- After ENSEMBLE_BARRIER_STUCK_MS (12 min) a non-advancing member is dropped from
the barrier minimum. It keeps training and rejoins the instant it completes an
era - at which point, being behind, it legitimately becomes the minimum again,
which is the documented resumed-laggard behaviour.
- Both transitions say so loudly, and the release states plainly that the
combined-vote score cannot be computed while the ensemble is desynchronised.
- A held member now writes a rate-limited journal line naming WHICH members it is
waiting on, so the blocker is read off one line.
5. THE PANEL FLICKER.
OnTickHandler gates its terse writer on !m_trainRunActive, and a barrier-held member
returns from Train() before ever setting it - so both writers thought they were the
only one updating the label and fought every tick. That is the reported "Getting
ready..." <-> "Waiting at era N for slower ensemble members" oscillation, and it hit
Perceptron but not Convolutional purely because Convolutional had a run active from
a completed era and Perceptron, resumed from disk, never did. Train()'s message is
the specific one, so it wins.
NEXT STEP once this is running: the stall line now ends in "REJECTED BY: ..." and
the per-handle depths. Read it. Do not reason around it.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-17 15:21:15 -04:00
|
|
|
m_featureFailBlock = "price/open (m_Open.GetData == EMPTY_VALUE)";
|
refactor: split CExpertSignalAIBase implementation by responsibility
ExpertSignalAIBase.mqh was 8216 lines: the class declaration followed by 87
method bodies covering training, labelling, feature extraction, persistence,
chart drawing, online learning, the GA auto-tuner and inference, all in one
file. Train() alone is 1492 lines; a change to arrow drawing meant scrolling
past the era loop.
Moved the bodies into Expert\AIBase\, included at the bottom of the original
after the class declaration:
Training.mqh 1607 era loop, plateau ladder, checkpoint select, deploy
Features.mqh 1093 indicator creation + per-bar input feature vector
ChartUI.mqh 634 arrows, arrow persistence, status panel, cleanup
Persistence.mqh 492 .stats/.cfg sidecars, CPU-inference validation, copy
OnlineLearning.mqh 461 live continual learning, EMA shadow, OOS simulator
Labels.mqh 309 ZigZag pivot labels, async label-cache prebuild
AutoTune.mqh 275 genetic tuner (population, crossover, halving)
Inference.mqh 235 softmax, prior calibration, class priors
ExpertSignalAIBase.mqh 8216 -> 3131 (declaration + topology build only)
This is a pure relocation - verified mechanically, not by eye: HEAD's file
reconstructed from the eight partials plus the surviving remainder is
byte-identical to HEAD, span for span (scratchpad verify_split.py). No
declaration moved, no signature changed, no code rewritten, so behaviour is
unchanged by construction.
Compiles 0 errors, 0 warnings.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-29 00:42:45 -04:00
|
|
|
return false;
|
fix: a resumed model cached a cold ATR as permanent, so it never trained
BufferTempData cached EVERY failure - m_featureCacheHasValue[idx]=true
with m_featureCacheValid[idx]=false - and the cache never re-tries a
miss. So a single feature read taken before the terminal had finished
calculating the indicator buffers marked those bars unusable for the
rest of the process, even though the data arrived milliseconds later.
MT5 fills an indicator's buffers asynchronously after the handle is
created, and a cold ATR returns 0 for EVERY index, not just its warm-up
tail. BufferTempDataCompute rejects a bar with no ATR (correctly - the
price features would be meaningless), so the whole window failed, and
the whole cache was poisoned.
Only resumed models were hit, because only they read features that
early. Topology.mqh sets m_warmupPassesRemaining = netLoaded ? 0 : 3:
a fresh start sits through three separately-scheduled Train() calls
before anything touches a feature, which is exactly what those passes
are for. A resumed one skips them and TuneIndicatorsAndTrain drives
StartLabelCachePrebuild and the MI report from the first chart event.
Its rationale - "a restart already has a proven-synced history" - holds
for HISTORY and not for INDICATORS, which are recreated every process
start.
Downstream: BuildFeatureWindow failed on every bar of every era, so
add_loop never went true, so pass 2, pass 3, the era counter and the
checkpoint were all skipped and pass 1 swept 0->100% forever. The
"0 samples" MI report line at startup was the same failure, four
seconds earlier, already visible in the log.
- a miss is now cached only when it is PERMANENT; the two "not ready
yet" guards mark m_featureFailTransient and are recomputed on the
next visit. Steady-state cost is ~ind_Periods bars per era, not 54k.
- an era that discards itself now drops the feature cache before
restarting, so any remaining cause of this state self-heals instead
of looping.
Deleting the .nnw "fixed" this only by turning the model back into a
fresh one.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-10 11:41:31 -04:00
|
|
|
}
|
refactor: split CExpertSignalAIBase implementation by responsibility
ExpertSignalAIBase.mqh was 8216 lines: the class declaration followed by 87
method bodies covering training, labelling, feature extraction, persistence,
chart drawing, online learning, the GA auto-tuner and inference, all in one
file. Train() alone is 1492 lines; a change to arrow drawing meant scrolling
past the era loop.
Moved the bodies into Expert\AIBase\, included at the bottom of the original
after the class declaration:
Training.mqh 1607 era loop, plateau ladder, checkpoint select, deploy
Features.mqh 1093 indicator creation + per-bar input feature vector
ChartUI.mqh 634 arrows, arrow persistence, status panel, cleanup
Persistence.mqh 492 .stats/.cfg sidecars, CPU-inference validation, copy
OnlineLearning.mqh 461 live continual learning, EMA shadow, OOS simulator
Labels.mqh 309 ZigZag pivot labels, async label-cache prebuild
AutoTune.mqh 275 genetic tuner (population, crossover, halving)
Inference.mqh 235 softmax, prior calibration, class priors
ExpertSignalAIBase.mqh 8216 -> 3131 (declaration + topology build only)
This is a pure relocation - verified mechanically, not by eye: HEAD's file
reconstructed from the eight partials plus the surviving remainder is
byte-identical to HEAD, span for span (scratchpad verify_split.py). No
declaration moved, no signature changed, no code rewritten, so behaviour is
unchanged by construction.
Compiles 0 errors, 0 warnings.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-29 00:42:45 -04:00
|
|
|
// ATR-normalize every raw-price-unit feature below instead of feeding e.g. 0.0005 on EURUSD vs.
|
|
|
|
|
// 50.0 on a JPY pair or an index straight into the network - with Adam and hardcoded, scale-
|
|
|
|
|
// sensitive activations (TANH saturates, PRELU's 0.01 leak only means anything relative to the
|
|
|
|
|
// input's own scale), an unnormalized feature either vanishes into rounding noise or dominates
|
|
|
|
|
// the weighted sum depending on which symbol/timeframe happens to be loaded. Dividing by the
|
|
|
|
|
// bar's own ATR expresses every price-based feature as "fraction of typical volatility", which
|
|
|
|
|
// is comparable across symbols/timeframes and centered near zero. No ATR reading yet (e.g. the
|
|
|
|
|
// first few bars of history) means every price feature this bar would be meaningless - reject
|
|
|
|
|
// the bar via the same "return false" convention as the EMPTY_VALUE check above.
|
|
|
|
|
double atr = m_ATR.Main(idx);
|
|
|
|
|
if(atr <= 0.0 || atr == EMPTY_VALUE)
|
fix: a resumed model cached a cold ATR as permanent, so it never trained
BufferTempData cached EVERY failure - m_featureCacheHasValue[idx]=true
with m_featureCacheValid[idx]=false - and the cache never re-tries a
miss. So a single feature read taken before the terminal had finished
calculating the indicator buffers marked those bars unusable for the
rest of the process, even though the data arrived milliseconds later.
MT5 fills an indicator's buffers asynchronously after the handle is
created, and a cold ATR returns 0 for EVERY index, not just its warm-up
tail. BufferTempDataCompute rejects a bar with no ATR (correctly - the
price features would be meaningless), so the whole window failed, and
the whole cache was poisoned.
Only resumed models were hit, because only they read features that
early. Topology.mqh sets m_warmupPassesRemaining = netLoaded ? 0 : 3:
a fresh start sits through three separately-scheduled Train() calls
before anything touches a feature, which is exactly what those passes
are for. A resumed one skips them and TuneIndicatorsAndTrain drives
StartLabelCachePrebuild and the MI report from the first chart event.
Its rationale - "a restart already has a proven-synced history" - holds
for HISTORY and not for INDICATORS, which are recreated every process
start.
Downstream: BuildFeatureWindow failed on every bar of every era, so
add_loop never went true, so pass 2, pass 3, the era counter and the
checkpoint were all skipped and pass 1 swept 0->100% forever. The
"0 samples" MI report line at startup was the same failure, four
seconds earlier, already visible in the log.
- a miss is now cached only when it is PERMANENT; the two "not ready
yet" guards mark m_featureFailTransient and are recomputed on the
next visit. Steady-state cost is ~ind_Periods bars per era, not 54k.
- an era that discards itself now drops the feature cache before
restarting, so any remaining cause of this state self-heals instead
of looping.
Deleting the .nnw "fixed" this only by turning the model back into a
fresh one.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-10 11:41:31 -04:00
|
|
|
{
|
|
|
|
|
//--- TRANSIENT BY NATURE, and the reason resumed models could never train. MT5 calculates an
|
|
|
|
|
//--- indicator's buffers asynchronously after the handle is created, so a call made before ATR
|
|
|
|
|
//--- has filled returns 0 for EVERY index, not just the warm-up tail. A FRESH model never saw
|
|
|
|
|
//--- this: it sits through m_warmupPassesRemaining separately-scheduled Train() calls before
|
|
|
|
|
//--- anything touches a feature, which is exactly what those passes are for. A RESUMED model
|
|
|
|
|
//--- skips them - TuneIndicatorsAndTrain drives StartLabelCachePrebuild and the MI report from
|
|
|
|
|
//--- the very first chart event, milliseconds after OnInit - so it read a cold ATR, every bar
|
|
|
|
|
//--- was rejected, and BufferTempData cached all of it as permanent misses. From then on
|
|
|
|
|
//--- BuildFeatureWindow failed on every bar of every era, add_loop never went true, and pass 1
|
|
|
|
|
//--- swept 0->100% forever with nothing in the journal (2026-08-10; deleting the .nnw "fixed"
|
|
|
|
|
//--- it only by turning the model back into a fresh one).
|
|
|
|
|
m_featureFailTransient = true;
|
fix(ensemble+depth): the barrier had no liveness escape, and the depth gate could not report the one state the evidence pointed at
Two charts (USDJPY 50,179 bars / XAUUSD 33,982) sat at era 0 for 38 minutes with
four of their eight members completely silent. Nothing in this commit guesses at
why the sweep fails - the last five guesses were all wrong. It makes the failure
say what it is, and stops one broken member taking its whole chart down with it.
WHAT THE LOG ACTUALLY SAYS, before any of this.
- The running build IS d9f834d (pulled 14:18, compiled 14:19:01, 0 errors), so
every depth instrument from 1dda479/7e63a8b/45c9e21 was live.
- It printed NOTHING. Zero "PRIMING", zero "CAPPED", zero "Per-indicator depth"
in 27 MB of journal. The instrument built to find the depth shortfall returned
"not this".
- On USDJPY at 14:24, CONV-cad8 completed eras 0 AND 1 across all 50,179 bars -
same chart, same 832-value window, same indicators, byte-identical fingerprint -
while LSTM-cad8 and HYB-cad8 reported ok=0 failed=50163. So it is not the
symbol, the history, the bar count or the indicator depth. It is per-member.
- ok=0 means the NEWEST anchors failed too, and a short indicator cannot do that.
The depth reading in project_silent_block_failures is therefore retired by its own
instrumentation. THE ROOT CAUSE IS STILL UNKNOWN and this commit does not claim one.
1. THE DEPTH GATE'S SILENT PATH WAS THE STATE IT WAS HUNTING.
ServableBars() read `if(servable <= 0 || servable >= want) return want;` - one
branch over three unrelated states, silent in all of them:
enabled == 0 -> nothing tunable is on. No cap. Healthy.
enabled > 0, servable == -1 -> a handle answered INVALID.
enabled > 0, servable == 0 -> created, never calculated.
BarsCalculated() returns -1 for a dead handle, so a dead MA is indistinguishable
from "no tunable indicators enabled" - and both returned `want` without printing a
character. That is exactly the state a per-member, every-index, depth-independent
failure produces, and it is the single reason a build carrying full depth
instrumentation logged nothing through the whole outage.
TunableBarsCalculated() now also reports HOW MANY indicators it consulted, and the
dead-handle case is reported (latched, with per-handle depths). The RETURN is
deliberately unchanged - what to do about a dead handle is not yet known, and
changing control flow on an unproven cause is how the last four fixes here went
wrong. SettledBars() routes its three pass-through states via ServableBars() so the
report is reachable from the training sweep, which is the only caller that hits it.
2. THE STALL REPORT NAMED A SLOT, NEVER A BLOCK.
"lookback slot 0 REJECTED (window had 24 of 832 values)" plus a guess ("an
indicator warm-up or a history-edge read"). Which guard fired was INFERRED by
counting 4+5+4+4+6+1 = 24 and concluding feature 25 must be the MA. The arithmetic
was right; every conclusion drawn from it was wrong, because a value count names a
POSITION and a position cannot tell cold from capped from invalid from off-the-end.
Every guard that can reject a bar now records itself - m_featureFailBlock - and the
report carries it, the series index, IndicatorDepthReport()'s per-handle depths,
and for each indicator whether the NEWEST bar reads. That last field is the whole
diagnosis in one word: newest-also-EMPTY means the buffer is unreadable everywhere
(cold or dead handle), newest-reads means a genuine history edge. Instrumented:
open, ATR, MA, RSI, MACD, Ichimoku, and all five AD blocks via ADIndicatorCold().
3. THE TOTAL-FAILURE BACKOFF WAS GATED ON THE WRONG QUESTION.
It armed only when m_featureFailTransient was set. Keeping that flag correct across
every guard is a list that has to stay right forever - the same shape of fix the
feature cache abandoned for the same reason - and the gate is pointless anyway: a
sweep where ZERO of 50,163 bars produced a window will produce zero again if it
restarts a millisecond later, transient or not. Doing that at full speed is what
starved six indicator threads on a six-core box. The backoff is now unconditional
on a total failure. The flag keeps its real job, deciding whether a MISS may be
cached, which is a per-bar question and not a scheduling one.
4. THE ERA BARRIER DEADLOCKED, AND SILENCED THE MEMBERS IT FROZE.
EnsembleMinTrainingEra() exempted deployed, stopped and paused members and its
comment concluded "so nothing deadlocks". Those three are all VOLUNTARY. A member
that simply CANNOT finish an era is none of them, so it pinned the minimum at its
own era with no time limit - and the hold branch's only action was
`m_lastEraCompleteTick = GetTickCount()`, which silences the stall watchdog. So on
USDJPY the two members that could not train reported, and the two healthy members
frozen behind them wrote nothing anywhere. The outage was visible only through the
members that were not suffering it.
- BarrierEraHeartbeat() stamps a clock on real era CHANGE, kept separate from
m_lastEraCompleteTick precisely because the barrier resets that one. Only a
member AT the minimum can be a blocker; a member ahead is idle by design and is
never counted as stuck.
- After ENSEMBLE_BARRIER_STUCK_MS (12 min) a non-advancing member is dropped from
the barrier minimum. It keeps training and rejoins the instant it completes an
era - at which point, being behind, it legitimately becomes the minimum again,
which is the documented resumed-laggard behaviour.
- Both transitions say so loudly, and the release states plainly that the
combined-vote score cannot be computed while the ensemble is desynchronised.
- A held member now writes a rate-limited journal line naming WHICH members it is
waiting on, so the blocker is read off one line.
5. THE PANEL FLICKER.
OnTickHandler gates its terse writer on !m_trainRunActive, and a barrier-held member
returns from Train() before ever setting it - so both writers thought they were the
only one updating the label and fought every tick. That is the reported "Getting
ready..." <-> "Waiting at era N for slower ensemble members" oscillation, and it hit
Perceptron but not Convolutional purely because Convolutional had a run active from
a completed era and Perceptron, resumed from disk, never did. Train()'s message is
the specific one, so it wins.
NEXT STEP once this is running: the stall line now ends in "REJECTED BY: ..." and
the per-handle depths. Read it. Do not reason around it.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-17 15:21:15 -04:00
|
|
|
m_featureFailBlock = StringFormat("ATR (m_ATR.Main=%.10g, needs > 0)", atr);
|
refactor: split CExpertSignalAIBase implementation by responsibility
ExpertSignalAIBase.mqh was 8216 lines: the class declaration followed by 87
method bodies covering training, labelling, feature extraction, persistence,
chart drawing, online learning, the GA auto-tuner and inference, all in one
file. Train() alone is 1492 lines; a change to arrow drawing meant scrolling
past the era loop.
Moved the bodies into Expert\AIBase\, included at the bottom of the original
after the class declaration:
Training.mqh 1607 era loop, plateau ladder, checkpoint select, deploy
Features.mqh 1093 indicator creation + per-bar input feature vector
ChartUI.mqh 634 arrows, arrow persistence, status panel, cleanup
Persistence.mqh 492 .stats/.cfg sidecars, CPU-inference validation, copy
OnlineLearning.mqh 461 live continual learning, EMA shadow, OOS simulator
Labels.mqh 309 ZigZag pivot labels, async label-cache prebuild
AutoTune.mqh 275 genetic tuner (population, crossover, halving)
Inference.mqh 235 softmax, prior calibration, class priors
ExpertSignalAIBase.mqh 8216 -> 3131 (declaration + topology build only)
This is a pure relocation - verified mechanically, not by eye: HEAD's file
reconstructed from the eight partials plus the surviving remainder is
byte-identical to HEAD, span for span (scratchpad verify_split.py). No
declaration moved, no signature changed, no code rewritten, so behaviour is
unchanged by construction.
Compiles 0 errors, 0 warnings.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-29 00:42:45 -04:00
|
|
|
return false;
|
fix: a resumed model cached a cold ATR as permanent, so it never trained
BufferTempData cached EVERY failure - m_featureCacheHasValue[idx]=true
with m_featureCacheValid[idx]=false - and the cache never re-tries a
miss. So a single feature read taken before the terminal had finished
calculating the indicator buffers marked those bars unusable for the
rest of the process, even though the data arrived milliseconds later.
MT5 fills an indicator's buffers asynchronously after the handle is
created, and a cold ATR returns 0 for EVERY index, not just its warm-up
tail. BufferTempDataCompute rejects a bar with no ATR (correctly - the
price features would be meaningless), so the whole window failed, and
the whole cache was poisoned.
Only resumed models were hit, because only they read features that
early. Topology.mqh sets m_warmupPassesRemaining = netLoaded ? 0 : 3:
a fresh start sits through three separately-scheduled Train() calls
before anything touches a feature, which is exactly what those passes
are for. A resumed one skips them and TuneIndicatorsAndTrain drives
StartLabelCachePrebuild and the MI report from the first chart event.
Its rationale - "a restart already has a proven-synced history" - holds
for HISTORY and not for INDICATORS, which are recreated every process
start.
Downstream: BuildFeatureWindow failed on every bar of every era, so
add_loop never went true, so pass 2, pass 3, the era counter and the
checkpoint were all skipped and pass 1 swept 0->100% forever. The
"0 samples" MI report line at startup was the same failure, four
seconds earlier, already visible in the log.
- a miss is now cached only when it is PERMANENT; the two "not ready
yet" guards mark m_featureFailTransient and are recomputed on the
next visit. Steady-state cost is ~ind_Periods bars per era, not 54k.
- an era that discards itself now drops the feature cache before
restarting, so any remaining cause of this state self-heals instead
of looping.
Deleting the .nnw "fixed" this only by turning the model back into a
fresh one.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-10 11:41:31 -04:00
|
|
|
}
|
refactor: split CExpertSignalAIBase implementation by responsibility
ExpertSignalAIBase.mqh was 8216 lines: the class declaration followed by 87
method bodies covering training, labelling, feature extraction, persistence,
chart drawing, online learning, the GA auto-tuner and inference, all in one
file. Train() alone is 1492 lines; a change to arrow drawing meant scrolling
past the era loop.
Moved the bodies into Expert\AIBase\, included at the bottom of the original
after the class declaration:
Training.mqh 1607 era loop, plateau ladder, checkpoint select, deploy
Features.mqh 1093 indicator creation + per-bar input feature vector
ChartUI.mqh 634 arrows, arrow persistence, status panel, cleanup
Persistence.mqh 492 .stats/.cfg sidecars, CPU-inference validation, copy
OnlineLearning.mqh 461 live continual learning, EMA shadow, OOS simulator
Labels.mqh 309 ZigZag pivot labels, async label-cache prebuild
AutoTune.mqh 275 genetic tuner (population, crossover, halving)
Inference.mqh 235 softmax, prior calibration, class priors
ExpertSignalAIBase.mqh 8216 -> 3131 (declaration + topology build only)
This is a pure relocation - verified mechanically, not by eye: HEAD's file
reconstructed from the eight partials plus the surviving remainder is
byte-identical to HEAD, span for span (scratchpad verify_split.py). No
declaration moved, no signature changed, no code rewritten, so behaviour is
unchanged by construction.
Compiles 0 errors, 0 warnings.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-29 00:42:45 -04:00
|
|
|
if(!TempData.Add((close - open) / atr) ||
|
|
|
|
|
!TempData.Add((high - open) / atr) ||
|
|
|
|
|
!TempData.Add((low - open) / atr) ||
|
|
|
|
|
// Explicit bullish/bearish flag - (close-open)/atr already encodes direction *and* magnitude
|
|
|
|
|
// together, which asks the network to disentangle "which way" from "how much" out of a single
|
|
|
|
|
// continuous value. Giving direction its own clean +1/-1/0 signal removes that ambiguity.
|
|
|
|
|
!TempData.Add(close > open ? 1.0 : (close < open ? -1.0 : 0.0)))
|
|
|
|
|
{
|
|
|
|
|
return false;
|
|
|
|
|
}
|
|
|
|
|
if(m_useSwingContext)
|
|
|
|
|
{
|
|
|
|
|
// Most recent CONFIRMED swing pivot as of bar idx - "confirmed" meaning at least
|
feat(ai): triple-barrier labels replace exact-pivot ZigZag targets
The 31:1 class imbalance was self-inflicted by the TARGET, not a property
of the market. Labelling only the exact bar where a ZigZag pivot confirms
gave Buy 1164 / Sell 1164 / Neutral 35841, and every correction mechanism
this codebase accumulated sits downstream of that one choice: the
logit-adjusted loss and its range cap, the prior EMA, the +-3.0 output-bias
seed, balanced-accuracy-then-precision selection with its coverage floor,
the recall floor and its catch-22, the alternation gate, NMS, and the four
oversampling designs that collapsed before them.
The reference this engine is built on (references/neuronetworksbook.pdf
ch. 3.1/3.3) also uses ZigZag, but targets the DIRECTION TO THE NEXT
EXTREMUM on every bar - ~50/50 by construction, with no imbalance to
correct at all. It never had this problem because it never asked "is this
the pivot bar".
Labels are now the triple barrier (Lopez de Prado ch. 3), using the EA's
OWN SL_Mode/TP_Mode: does a trade opened at this bar's close reach its
target before its stop, within a horizon. Buy = long resolves, Sell =
short resolves, Neutral = neither. Consequences:
- dir-precision in the era line stops being a proxy and becomes the win
rate of the strategy under its own exit rules.
- Expected balance ~25/25/50 at the shipped 1:3 (gambler's ruin), i.e.
~2:1 instead of 31:1. Measured and logged at the end of the prebuild.
- Spread is charged on both legs, so it is a NET win rate.
- Intrabar ambiguity resolves to the STOP. OHLC cannot order two touches
inside one bar and the optimistic reading is how a backtested edge
becomes a live loss.
ZigZag stays as input features (EnableSwingContext) and now also supplies
the vertical barrier: the horizon is the median confirmed leg length,
snapped to a coarse ladder. Derived, not configured, and deliberately kept
out of the filename fingerprint - a filename keyed on a measured quantity
orphans a trained model the moment the measurement moves.
Removed, because the premise died with the old target:
- the alternation gate. Correct for pivot labels (a ZigZag cannot emit two
same-type pivots in a row, so a repeat was provably a false fire), and
wrong for barrier labels, which answer each bar independently. It also
took its worst consequence with it: a one-sided model previously got ONE
trade per backtest, a hard blocker on marketplace validation.
- SignalClusterWindow now defaults off - it de-duplicated repeats that are
now real trades. Kept as an opt-in display control.
- LABEL_WINDOW_BARS, the pivot-widening pass, ConfirmedZigZagLabel.
- the era-0 output-bias seed now needs a genuinely dominant class (0.70)
rather than 0.40; at ~50% Neutral a +-3.0 seed is a distortion, not a
correction.
Also fixed, both found while wiring the above:
1. RefreshConvergedSignal sized its buffers from a date delta
(Bars(sym, period, dtStudied, TimeCurrent())). dtStudied is a training
watermark; in the tester it is loaded from a live-chart save AHEAD of
the simulated date, so the interval inverted, Bars() returned ~0, and
the buffer came out at exactly m_historyBars - deep enough for the OHLC
window and far too shallow for the Donchian-50 / 20-bar-return / SMA
extension behind it. Inference silently computed DIFFERENT features
from the ones training learned on, live as well as in the tester. Now
sized from what the feature builder actually needs.
2. The barrier horizon is resolved on the deployed path too. A deployed
model never enters Train(), so it never reached the prebuild, and
OnlineLearnStep reads the horizon as its confirmation delay - left at
the fallback it would have backpropped bars whose barriers had not
resolved. Silent lookahead in the one place that writes to a live model.
SL_Mode/TP_Mode join the weights fingerprint: they define the labels now,
so a model trained at 1:3 must never be silently reused at 1:1. This
re-keys every pre-existing model by design - none were trained on this task.
Inference census extended with the vote gate. LongCondition/ShortCondition
open with a readiness check the refresh counters never see; in the tester it
reduces to "the seeded _optcache.nnw must have LOADED", and if it did not,
every vote is hard-zeroed while the model still answers Buy. The old three
counters would have read that as "the model says Neutral" - false, and a
completely different fix. This is the leading candidate for the
zero-direction backtest and the census can now name it in one run.
Both builds compile 0 errors / 0 warnings. Forces a full retrain.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-31 20:39:49 -04:00
|
|
|
// m_swingConfirmationBars MORE bars have closed after it (see m_swingConfirmationBars' and
|
|
|
|
|
// m_useSwingContext's declaration comments). This is now the ONLY consumer of that embargo: the
|
|
|
|
|
// label side stopped needing it when the target became the triple barrier, whose own lookahead is
|
|
|
|
|
// m_barrierHorizonBars. Skipping this embargo here - e.g. reading
|
refactor: split CExpertSignalAIBase implementation by responsibility
ExpertSignalAIBase.mqh was 8216 lines: the class declaration followed by 87
method bodies covering training, labelling, feature extraction, persistence,
chart drawing, online learning, the GA auto-tuner and inference, all in one
file. Train() alone is 1492 lines; a change to arrow drawing meant scrolling
past the era loop.
Moved the bodies into Expert\AIBase\, included at the bottom of the original
after the class declaration:
Training.mqh 1607 era loop, plateau ladder, checkpoint select, deploy
Features.mqh 1093 indicator creation + per-bar input feature vector
ChartUI.mqh 634 arrows, arrow persistence, status panel, cleanup
Persistence.mqh 492 .stats/.cfg sidecars, CPU-inference validation, copy
OnlineLearning.mqh 461 live continual learning, EMA shadow, OOS simulator
Labels.mqh 309 ZigZag pivot labels, async label-cache prebuild
AutoTune.mqh 275 genetic tuner (population, crossover, halving)
Inference.mqh 235 softmax, prior calibration, class priors
ExpertSignalAIBase.mqh 8216 -> 3131 (declaration + topology build only)
This is a pure relocation - verified mechanically, not by eye: HEAD's file
reconstructed from the eight partials plus the surviving remainder is
byte-identical to HEAD, span for span (scratchpad verify_split.py). No
declaration moved, no signature changed, no code rewritten, so behaviour is
unchanged by construction.
Compiles 0 errors, 0 warnings.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-29 00:42:45 -04:00
|
|
|
// m_ADZigZag's raw current buffer value instead - would leak information a live bar at idx
|
|
|
|
|
// could never actually have had yet, since ZigZag's most recent 1-3 legs are still provisional
|
|
|
|
|
// and can be revised as new bars arrive.
|
|
|
|
|
int pivotIdx = -1;
|
|
|
|
|
double pivotPrice = 0.0;
|
|
|
|
|
bool pivotIsLow = false;
|
|
|
|
|
if(!FindConfirmedZigZagPivot(idx + MathMax(m_swingConfirmationBars, 1), pivotIdx, pivotPrice, pivotIsLow))
|
|
|
|
|
{
|
|
|
|
|
// No confirmed pivot within the scan cap (e.g. right at the start of available history) -
|
|
|
|
|
// this is legitimately "no swing context yet", not bad/missing data, so a neutral 0-fill
|
|
|
|
|
// keeps the bar usable rather than rejecting it outright like the ATR/EMPTY_VALUE guards do.
|
|
|
|
|
if(!TempData.Add(0.0) || !TempData.Add(0.0) || !TempData.Add(0.0) || !TempData.Add(0.0) || !TempData.Add(0.0))
|
|
|
|
|
return false;
|
|
|
|
|
}
|
|
|
|
|
else
|
|
|
|
|
{
|
|
|
|
|
// Direction of the CURRENT leg: the last confirmed pivot being a bottom means price has been
|
|
|
|
|
// rising away from it (an up-leg) ever since, and vice versa - same +1/-1 convention as the
|
|
|
|
|
// bullish/bearish flag above, just at swing scale instead of single-bar scale.
|
|
|
|
|
double direction = pivotIsLow ? 1.0 : -1.0;
|
|
|
|
|
// How far price has travelled since that pivot, ATR-normalized and signed (+ve above the
|
|
|
|
|
// pivot price, -ve below) - clamped generously since an extended trending leg has no natural
|
|
|
|
|
// ceiling the way a single bar's range does.
|
|
|
|
|
double distSincePivot = MathMax(-10.0, MathMin(10.0, (close - pivotPrice) / atr));
|
|
|
|
|
// Magnitude of the PRIOR completed leg (the pivot immediately before pivotIdx) - a scale
|
|
|
|
|
// reference for "is the current move big or small relative to the last full swing". No
|
|
|
|
|
// additional embargo needed here (see FindConfirmedZigZagPivot()'s declaration comment) -
|
|
|
|
|
// anything at or before an already-confirmed pivot is necessarily even older.
|
|
|
|
|
int priorPivotIdx = -1;
|
|
|
|
|
double priorPivotPrice = 0.0;
|
|
|
|
|
bool priorPivotIsLow = false;
|
|
|
|
|
bool havePrior = FindConfirmedZigZagPivot(pivotIdx + 1, priorPivotIdx, priorPivotPrice, priorPivotIsLow);
|
|
|
|
|
double priorLegMagnitude = havePrior ? MathMax(0.0, MathMin(10.0, MathAbs(pivotPrice - priorPivotPrice) / atr)) : 0.0;
|
|
|
|
|
// Retracement/extension ratio (current distance relative to the prior leg's own size) -
|
|
|
|
|
// Fibonacci-style relative position, often more informative than either raw magnitude alone
|
|
|
|
|
// since it's comparable across both quiet and volatile regimes. 0 when there's no prior leg
|
|
|
|
|
// to compare against yet.
|
|
|
|
|
double retracementRatio = (havePrior && priorLegMagnitude > 0.0001) ?
|
|
|
|
|
MathMax(-5.0, MathMin(5.0, distSincePivot / priorLegMagnitude)) : 0.0;
|
|
|
|
|
// Swing age (bars since the pivot) - a maturity/exhaustion proxy, same +/- style clamp
|
|
|
|
|
// convention as the volume-ratio feature below.
|
|
|
|
|
double barsSincePivot = MathMax(0.0, MathMin(5.0, (double)(pivotIdx - idx) / 100.0));
|
|
|
|
|
if(!TempData.Add(direction) ||
|
|
|
|
|
!TempData.Add(distSincePivot) ||
|
|
|
|
|
!TempData.Add(priorLegMagnitude) ||
|
|
|
|
|
!TempData.Add(retracementRatio) ||
|
|
|
|
|
!TempData.Add(barsSincePivot))
|
|
|
|
|
return false;
|
|
|
|
|
}
|
|
|
|
|
// Recent price-action context (4 features), computed from CLOSED bars at idx or older only -
|
|
|
|
|
// no ZigZag confirmation, so no repainting and NO embargo, and never stale, unlike the five
|
|
|
|
|
// pivot-anchored features above whose confirmed anchor is always >= m_swingConfirmationBars
|
|
|
|
|
// (~100) bars old. Those describe the OLD structure well but say nothing about the recent leg
|
|
|
|
|
// the bar actually sits in - which is exactly what's needed to tell a genuine reversal at a
|
|
|
|
|
// range extreme from a mid-trend bar that merely looks like a bottom/top (the "clustered
|
|
|
|
|
// counter-trend signals" failure mode). These locate the bar within its recent range and
|
|
|
|
|
// trend so the network can learn that a directional call belongs at an extreme of an extended
|
|
|
|
|
// move, not anywhere the local candle shape resembles a pivot. All windows walk toward OLDER
|
|
|
|
|
// bars (increasing index), so nothing here can see the future.
|
|
|
|
|
double hi20 = high, lo20 = low, hi50 = high, lo50 = low;
|
|
|
|
|
double sum20 = close, oldestClose20 = close;
|
|
|
|
|
int cnt20 = 1;
|
|
|
|
|
for(int w = 1; w < 50; w++)
|
|
|
|
|
{
|
|
|
|
|
int j = idx + w;
|
|
|
|
|
double jc = m_Close.GetData(j);
|
|
|
|
|
double jh = m_High.GetData(j);
|
|
|
|
|
double jl = m_Low.GetData(j);
|
|
|
|
|
// ran off the oldest edge of loaded history (out-of-range reads back as 0/EMPTY_VALUE) -
|
|
|
|
|
// use whatever window we gathered so far rather than rejecting the bar; a shorter early-
|
|
|
|
|
// history window is degraded-but-usable, same spirit as the pivot 0-fill above.
|
|
|
|
|
if(jh == EMPTY_VALUE || jh <= 0.0 || jl <= 0.0)
|
|
|
|
|
break;
|
|
|
|
|
if(jh > hi50)
|
|
|
|
|
hi50 = jh;
|
|
|
|
|
if(jl < lo50)
|
|
|
|
|
lo50 = jl;
|
|
|
|
|
if(w < 20)
|
|
|
|
|
{
|
|
|
|
|
if(jh > hi20)
|
|
|
|
|
hi20 = jh;
|
|
|
|
|
if(jl < lo20)
|
|
|
|
|
lo20 = jl;
|
|
|
|
|
sum20 += jc;
|
|
|
|
|
oldestClose20 = jc;
|
|
|
|
|
cnt20++;
|
|
|
|
|
}
|
|
|
|
|
}
|
|
|
|
|
// Donchian position: where close sits inside the recent high/low range, rescaled to [-1,+1]
|
|
|
|
|
// (-1 = at the range low / bottom candidate, +1 = at the range high / top candidate, 0 = mid-
|
|
|
|
|
// range / mid-trend). Two scales - a short 20-bar and a medium 50-bar view - so the network
|
|
|
|
|
// sees both local and swing-scale extremity. 0 (mid) when the range is degenerate.
|
|
|
|
|
double range20 = hi20 - lo20;
|
|
|
|
|
double range50 = hi50 - lo50;
|
|
|
|
|
double donchPos20 = (range20 > 0.0) ? ((close - lo20) / range20 - 0.5) * 2.0 : 0.0;
|
|
|
|
|
double donchPos50 = (range50 > 0.0) ? ((close - lo50) / range50 - 0.5) * 2.0 : 0.0;
|
|
|
|
|
// Net directional displacement over the recent window, ATR-normalized and signed - the
|
|
|
|
|
// prevailing-trend strength/direction the counter-trend clusters were ignoring.
|
|
|
|
|
double recentReturn = MathMax(-10.0, MathMin(10.0, (close - oldestClose20) / atr));
|
|
|
|
|
// Distance from the recent mean (SMA), ATR-normalized - a stretch/exhaustion proxy distinct
|
|
|
|
|
// from the net return (a move can be far from its mean with little net displacement, or vice
|
|
|
|
|
// versa); genuine reversals tend to be over-extended from equilibrium.
|
|
|
|
|
double smaExtension = MathMax(-10.0, MathMin(10.0, (close - sum20 / cnt20) / atr));
|
|
|
|
|
if(!TempData.Add(donchPos20) ||
|
|
|
|
|
!TempData.Add(donchPos50) ||
|
|
|
|
|
!TempData.Add(recentReturn) ||
|
|
|
|
|
!TempData.Add(smaExtension))
|
|
|
|
|
return false;
|
|
|
|
|
}
|
|
|
|
|
if(m_useVolumes)
|
|
|
|
|
{
|
feat(ai): widen the volume feature block from 1 value to 4
The block fed exactly one number: (v[i] - v[i-1]) / v[i-1]. That is the first difference,
and it cannot express three things that matter - the LEVEL relative to a baseline (two
dead bars and two frantic bars both read ~0 change), and the two volume-vs-range
interactions, where heavy participation that went NOWHERE (absorption) and heavy
participation that travelled (continuation) mean opposite things and currently collapse
onto the same value.
research/test_volume.py measures each candidate's mutual information with the triple-
barrier label across 3 instruments x 2 geometries, against a BLOCK-permutation null -
blocks sized to the barrier horizon, because adjacent labels share almost their entire
outcome window and a free shuffle yields a null so tight that everything looks
significant. Finite-sample MI bias (~7/n here) is reported alongside rather than
subtracted, since the permutation null already absorbs it.
Result: volLevel beats the shipped change ratio outright on 4 of 6 cells (EURUSD 2:3
+0.000118 excess at p=0.006, USDJPY 1:2 +0.000284 at p=0.002); absorption is the single
strongest reading anywhere in the sweep at EURUSD 1:2 (+0.000404, p=0.002) though it is
null on XAUUSD; vol x range clears on 4 of 6. The shipped change ratio is itself
significant on 5 of 6, so it stays.
Kept OUT: a session-relative z-score against the same hour-of-day's own recent history.
It was the weakest candidate - null on both EURUSD cells - and it is the only one needing
per-hour rolling bookkeeping in MQL5. Not worth the state for a reading that did not
survive its own null on the primary instrument.
Magnitudes, stated plainly because they are the point: the excess MI is ~2e-4 nats against
a label entropy near 1.05. That is under a tenth of one percent of the label's
uncertainty. It is real, it repeats across instruments, and it is nowhere near an edge -
this is worth having because it costs one 50-bar loop, not because it changes the answer.
Prior work stands: the whole single-series feature family measured at the noise floor.
m_neuronsCount is already in the fingerprint, so the width change re-keys existing caches
by itself, which is correct - the input vector genuinely changed shape.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-01 17:34:33 -04:00
|
|
|
// FOUR values, not one. This block used to feed only the bar-over-bar change ratio below.
|
|
|
|
|
// research/test_volume.py measured all four against the barrier label with a block-permutation
|
|
|
|
|
// null (blocks = the barrier horizon, because adjacent labels share almost their whole outcome
|
|
|
|
|
// window and a free shuffle produces a null far too tight): the LEVEL and the two
|
|
|
|
|
// volume-vs-range interactions each carry information the first difference does not, and the
|
|
|
|
|
// level beats the shipped feature outright on 4 of 6 instrument/geometry cells.
|
|
|
|
|
//
|
|
|
|
|
// Read the magnitudes before expecting much: the excess mutual information is ~2e-4 nats
|
|
|
|
|
// against a label entropy near 1.05, i.e. well under a tenth of one percent of the label's
|
|
|
|
|
// uncertainty. This is real and repeatable across instruments, and it is nowhere near an edge.
|
|
|
|
|
// It is worth having because it costs one 50-bar loop, not because it changes the answer.
|
|
|
|
|
double vNow = m_Volumes.Main(idx);
|
refactor: split CExpertSignalAIBase implementation by responsibility
ExpertSignalAIBase.mqh was 8216 lines: the class declaration followed by 87
method bodies covering training, labelling, feature extraction, persistence,
chart drawing, online learning, the GA auto-tuner and inference, all in one
file. Train() alone is 1492 lines; a change to arrow drawing meant scrolling
past the era loop.
Moved the bodies into Expert\AIBase\, included at the bottom of the original
after the class declaration:
Training.mqh 1607 era loop, plateau ladder, checkpoint select, deploy
Features.mqh 1093 indicator creation + per-bar input feature vector
ChartUI.mqh 634 arrows, arrow persistence, status panel, cleanup
Persistence.mqh 492 .stats/.cfg sidecars, CPU-inference validation, copy
OnlineLearning.mqh 461 live continual learning, EMA shadow, OOS simulator
Labels.mqh 309 ZigZag pivot labels, async label-cache prebuild
AutoTune.mqh 275 genetic tuner (population, crossover, halving)
Inference.mqh 235 softmax, prior calibration, class priors
ExpertSignalAIBase.mqh 8216 -> 3131 (declaration + topology build only)
This is a pure relocation - verified mechanically, not by eye: HEAD's file
reconstructed from the eight partials plus the surviving remainder is
byte-identical to HEAD, span for span (scratchpad verify_split.py). No
declaration moved, no signature changed, no code rewritten, so behaviour is
unchanged by construction.
Compiles 0 errors, 0 warnings.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-29 00:42:45 -04:00
|
|
|
double prevVolume = m_Volumes.Main(idx + 1);
|
feat(ai): widen the volume feature block from 1 value to 4
The block fed exactly one number: (v[i] - v[i-1]) / v[i-1]. That is the first difference,
and it cannot express three things that matter - the LEVEL relative to a baseline (two
dead bars and two frantic bars both read ~0 change), and the two volume-vs-range
interactions, where heavy participation that went NOWHERE (absorption) and heavy
participation that travelled (continuation) mean opposite things and currently collapse
onto the same value.
research/test_volume.py measures each candidate's mutual information with the triple-
barrier label across 3 instruments x 2 geometries, against a BLOCK-permutation null -
blocks sized to the barrier horizon, because adjacent labels share almost their entire
outcome window and a free shuffle yields a null so tight that everything looks
significant. Finite-sample MI bias (~7/n here) is reported alongside rather than
subtracted, since the permutation null already absorbs it.
Result: volLevel beats the shipped change ratio outright on 4 of 6 cells (EURUSD 2:3
+0.000118 excess at p=0.006, USDJPY 1:2 +0.000284 at p=0.002); absorption is the single
strongest reading anywhere in the sweep at EURUSD 1:2 (+0.000404, p=0.002) though it is
null on XAUUSD; vol x range clears on 4 of 6. The shipped change ratio is itself
significant on 5 of 6, so it stays.
Kept OUT: a session-relative z-score against the same hour-of-day's own recent history.
It was the weakest candidate - null on both EURUSD cells - and it is the only one needing
per-hour rolling bookkeeping in MQL5. Not worth the state for a reading that did not
survive its own null on the primary instrument.
Magnitudes, stated plainly because they are the point: the excess MI is ~2e-4 nats against
a label entropy near 1.05. That is under a tenth of one percent of the label's
uncertainty. It is real, it repeats across instruments, and it is nowhere near an edge -
this is worth having because it costs one 50-bar loop, not because it changes the answer.
Prior work stands: the whole single-series feature family measured at the noise floor.
m_neuronsCount is already in the fingerprint, so the width change re-keys existing caches
by itself, which is correct - the input vector genuinely changed shape.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-01 17:34:33 -04:00
|
|
|
double volumeDelta = vNow - prevVolume;
|
|
|
|
|
// Relative change - trading activity magnitude varies wildly across symbols/timeframes, so the
|
|
|
|
|
// previous bar's own volume is the scale reference, same logic as ATR-normalizing price above.
|
|
|
|
|
// Guard against a zero previous-bar volume (e.g. a holiday-thin session) instead of dividing by
|
|
|
|
|
// it. Clamped to +/-5: unlike the ATR-normalized price features this ratio has no natural
|
|
|
|
|
// ceiling (a 1-tick bar followed by a normal one produces a huge outlier).
|
refactor: split CExpertSignalAIBase implementation by responsibility
ExpertSignalAIBase.mqh was 8216 lines: the class declaration followed by 87
method bodies covering training, labelling, feature extraction, persistence,
chart drawing, online learning, the GA auto-tuner and inference, all in one
file. Train() alone is 1492 lines; a change to arrow drawing meant scrolling
past the era loop.
Moved the bodies into Expert\AIBase\, included at the bottom of the original
after the class declaration:
Training.mqh 1607 era loop, plateau ladder, checkpoint select, deploy
Features.mqh 1093 indicator creation + per-bar input feature vector
ChartUI.mqh 634 arrows, arrow persistence, status panel, cleanup
Persistence.mqh 492 .stats/.cfg sidecars, CPU-inference validation, copy
OnlineLearning.mqh 461 live continual learning, EMA shadow, OOS simulator
Labels.mqh 309 ZigZag pivot labels, async label-cache prebuild
AutoTune.mqh 275 genetic tuner (population, crossover, halving)
Inference.mqh 235 softmax, prior calibration, class priors
ExpertSignalAIBase.mqh 8216 -> 3131 (declaration + topology build only)
This is a pure relocation - verified mechanically, not by eye: HEAD's file
reconstructed from the eight partials plus the surviving remainder is
byte-identical to HEAD, span for span (scratchpad verify_split.py). No
declaration moved, no signature changed, no code rewritten, so behaviour is
unchanged by construction.
Compiles 0 errors, 0 warnings.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-29 00:42:45 -04:00
|
|
|
double volumeChangeRatio = prevVolume > 0.0 ? volumeDelta / prevVolume : 0.0;
|
feat(ai): widen the volume feature block from 1 value to 4
The block fed exactly one number: (v[i] - v[i-1]) / v[i-1]. That is the first difference,
and it cannot express three things that matter - the LEVEL relative to a baseline (two
dead bars and two frantic bars both read ~0 change), and the two volume-vs-range
interactions, where heavy participation that went NOWHERE (absorption) and heavy
participation that travelled (continuation) mean opposite things and currently collapse
onto the same value.
research/test_volume.py measures each candidate's mutual information with the triple-
barrier label across 3 instruments x 2 geometries, against a BLOCK-permutation null -
blocks sized to the barrier horizon, because adjacent labels share almost their entire
outcome window and a free shuffle yields a null so tight that everything looks
significant. Finite-sample MI bias (~7/n here) is reported alongside rather than
subtracted, since the permutation null already absorbs it.
Result: volLevel beats the shipped change ratio outright on 4 of 6 cells (EURUSD 2:3
+0.000118 excess at p=0.006, USDJPY 1:2 +0.000284 at p=0.002); absorption is the single
strongest reading anywhere in the sweep at EURUSD 1:2 (+0.000404, p=0.002) though it is
null on XAUUSD; vol x range clears on 4 of 6. The shipped change ratio is itself
significant on 5 of 6, so it stays.
Kept OUT: a session-relative z-score against the same hour-of-day's own recent history.
It was the weakest candidate - null on both EURUSD cells - and it is the only one needing
per-hour rolling bookkeeping in MQL5. Not worth the state for a reading that did not
survive its own null on the primary instrument.
Magnitudes, stated plainly because they are the point: the excess MI is ~2e-4 nats against
a label entropy near 1.05. That is under a tenth of one percent of the label's
uncertainty. It is real, it repeats across instruments, and it is nowhere near an edge -
this is worth having because it costs one 50-bar loop, not because it changes the answer.
Prior work stands: the whole single-series feature family measured at the noise floor.
m_neuronsCount is already in the fingerprint, so the width change re-keys existing caches
by itself, which is correct - the input vector genuinely changed shape.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-01 17:34:33 -04:00
|
|
|
// Baseline over the trailing 50 bars, walking toward OLDER bars only (increasing index), so
|
|
|
|
|
// nothing here can see the future. Degraded-but-usable at the oldest edge, same convention as
|
|
|
|
|
// the swing-context window above: a short early-history baseline beats rejecting the bar.
|
|
|
|
|
double volSum = vNow;
|
|
|
|
|
int volCnt = 1;
|
|
|
|
|
for(int w = 1; w < 50; w++)
|
|
|
|
|
{
|
|
|
|
|
double jv = m_Volumes.Main(idx + w);
|
|
|
|
|
if(jv <= 0.0)
|
|
|
|
|
break;
|
|
|
|
|
volSum += jv;
|
|
|
|
|
volCnt++;
|
|
|
|
|
}
|
|
|
|
|
double volBase = volSum / volCnt;
|
|
|
|
|
// LEVEL: is this an active bar or a dead one? The change ratio cannot express this at all -
|
|
|
|
|
// two consecutive dead bars and two consecutive frantic ones both read as ~0 change.
|
|
|
|
|
double volLevel = (volBase > 0.0) ? vNow / volBase : 1.0;
|
|
|
|
|
double rangeAtr = (high - low) / atr;
|
|
|
|
|
// ABSORPTION: range delivered per unit of activity. A low value means heavy participation that
|
|
|
|
|
// went nowhere - supply meeting demand - which is a categorically different bar from heavy
|
|
|
|
|
// participation that travelled. The single change ratio conflates the two.
|
|
|
|
|
double absorption = (volLevel > 0.05) ? rangeAtr / volLevel : 0.0;
|
|
|
|
|
// ...and its converse, effort AND result together, which is the continuation reading.
|
|
|
|
|
double volXrange = volLevel * rangeAtr;
|
|
|
|
|
if(!TempData.Add(MathMax(-5.0, MathMin(5.0, volumeChangeRatio))) ||
|
|
|
|
|
!TempData.Add(MathMax(0.0, MathMin(5.0, volLevel))) ||
|
|
|
|
|
!TempData.Add(MathMax(0.0, MathMin(5.0, absorption))) ||
|
|
|
|
|
!TempData.Add(MathMax(0.0, MathMin(5.0, volXrange))))
|
refactor: split CExpertSignalAIBase implementation by responsibility
ExpertSignalAIBase.mqh was 8216 lines: the class declaration followed by 87
method bodies covering training, labelling, feature extraction, persistence,
chart drawing, online learning, the GA auto-tuner and inference, all in one
file. Train() alone is 1492 lines; a change to arrow drawing meant scrolling
past the era loop.
Moved the bodies into Expert\AIBase\, included at the bottom of the original
after the class declaration:
Training.mqh 1607 era loop, plateau ladder, checkpoint select, deploy
Features.mqh 1093 indicator creation + per-bar input feature vector
ChartUI.mqh 634 arrows, arrow persistence, status panel, cleanup
Persistence.mqh 492 .stats/.cfg sidecars, CPU-inference validation, copy
OnlineLearning.mqh 461 live continual learning, EMA shadow, OOS simulator
Labels.mqh 309 ZigZag pivot labels, async label-cache prebuild
AutoTune.mqh 275 genetic tuner (population, crossover, halving)
Inference.mqh 235 softmax, prior calibration, class priors
ExpertSignalAIBase.mqh 8216 -> 3131 (declaration + topology build only)
This is a pure relocation - verified mechanically, not by eye: HEAD's file
reconstructed from the eight partials plus the surviving remainder is
byte-identical to HEAD, span for span (scratchpad verify_split.py). No
declaration moved, no signature changed, no code rewritten, so behaviour is
unchanged by construction.
Compiles 0 errors, 0 warnings.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-29 00:42:45 -04:00
|
|
|
return false;
|
|
|
|
|
}
|
|
|
|
|
if(m_useTime)
|
|
|
|
|
{
|
|
|
|
|
// Normalize time (cyclical encoding)
|
|
|
|
|
if(!TempData.Add(sin(2 * M_PI * sTime.hour / 24.0)))
|
|
|
|
|
return false;
|
|
|
|
|
if(!TempData.Add(cos(2 * M_PI * sTime.hour / 24.0)))
|
|
|
|
|
return false;
|
|
|
|
|
if(!TempData.Add(sin(2 * M_PI * sTime.day_of_week / 7.0)))
|
|
|
|
|
return false;
|
|
|
|
|
if(!TempData.Add(cos(2 * M_PI * sTime.day_of_week / 7.0)))
|
|
|
|
|
return false;
|
|
|
|
|
if(!TempData.Add(sin(2 * M_PI * sTime.mon / 12.0)))
|
|
|
|
|
return false;
|
|
|
|
|
if(!TempData.Add(cos(2 * M_PI * sTime.mon / 12.0)))
|
|
|
|
|
return false;
|
|
|
|
|
}
|
|
|
|
|
if(m_useATR)
|
|
|
|
|
{
|
|
|
|
|
// ATR/close (volatility as a fraction of price), not raw ATR - the raw absolute value is
|
|
|
|
|
// itself unnormalized (e.g. ~0.0012 on EURUSD vs. ~1.5 on gold, and drifts over time even on
|
|
|
|
|
// one symbol as its price level changes), which is exactly the kind of scale-dependent
|
|
|
|
|
// feature this whole normalization pass is fixing everywhere else.
|
|
|
|
|
if(!TempData.Add(close != 0.0 ? atr / close : 0.0))
|
|
|
|
|
return false;
|
|
|
|
|
}
|
|
|
|
|
if(m_useMA)
|
|
|
|
|
{
|
|
|
|
|
// Same ATR-normalized distance-from-level convention as the base OHLC-from-open features above,
|
|
|
|
|
// just measured against the MA instead of the bar's own open - lets the network read where
|
|
|
|
|
// price sits relative to the same MA Signals\SignalMA.mqh votes on. Plus the MA's own
|
|
|
|
|
// bar-over-bar change (also ATR-normalized, since the MA lives in price units and ATR is
|
|
|
|
|
// already this codebase's scale reference for that - see m_useMA's declaration comment for why
|
|
|
|
|
// this isn't volume's previous-bar-ratio scheme instead).
|
|
|
|
|
double maNow = m_MA.GetData(0, idx);
|
|
|
|
|
double maPrev = m_MA.GetData(0, idx + 1);
|
|
|
|
|
if(maNow == EMPTY_VALUE || maPrev == EMPTY_VALUE)
|
fix(features): a cold indicator is TRANSIENT, not a permanent miss - and back off instead of re-sweeping
Six fresh instances on USDJPY and XAUUSD swept 33,965-50,162 bars and produced
ZERO usable feature windows, repeatedly, for 40 minutes and 239 stall reports,
without ever completing era 0. The four instances already warmed up before those
charts were attached trained normally throughout.
THE STALL REPORT NAMED THE SPOT EXACTLY: 'lookback slot 0 REJECTED the bar
(window had 24 of 832 values)', and 24 is the core block to the value - 4 price +
5 swing + 4 range + 4 volume + 6 time + 1 ATR. So feature 25 was the wall, and
feature 25 is the first value of the MA block. The same 24 appeared on XAUUSD
against a 816-value window (51 features/bar vs 52), which is what ruled out any
symbol-specific data gap: the wall sits at a fixed feature index, not a date.
ADMovingAverage is a CUSTOM indicator, so MT5 fills its buffer asynchronously and
returns EMPTY_VALUE for EVERY index until it has calculated - not just the
warm-up tail. That guard did not set m_featureFailTransient, so every bar of the
sweep was cached as a PERMANENT miss. This is precisely the failure the ATR guard
twenty lines above it was fixed for on 2026-08-10; the fix was never propagated
to the indicator blocks that follow. RSI, MACD and Ichimoku had the same defect
and are fixed too. (The Donchian high/low guard is a break into a
degraded-but-usable path, not a rejection, and is deliberately left alone.)
IT ALSO SELF-SUSTAINED, which is why it never recovered. The ok=0 self-heal drops
the feature cache and re-sweeps immediately, so each stuck instance spent every
millisecond re-reading 30-50k bars - six of them at once, on a six-core box,
competing for CPU with the very indicator calculation they were all waiting on.
The recovery was preventing the recovery. A transient total failure now re-arms
m_warmupPassesRemaining, yielding the CPU for a few separately-scheduled Train()
calls - the same mechanism a fresh model already uses to let history sync finish,
pointed at indicator warm-up instead.
Verified in the terminal journal first: indicators load and unload in matched
counts and there is no OOM, so this is NOT the 33f106d handle leak.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-17 12:47:25 -04:00
|
|
|
{
|
|
|
|
|
//--- TRANSIENT, for exactly the reason spelled out at the ATR guard above, and this is the
|
|
|
|
|
//--- guard that proved it: 2026-08-17, six fresh instances on USDJPY and XAUUSD swept
|
|
|
|
|
//--- 33,965-50,162 bars and produced ZERO usable windows, over and over, for 40 minutes.
|
|
|
|
|
//--- The stall report named the spot precisely - "lookback slot 0 REJECTED (window had 24 of
|
|
|
|
|
//--- 832 values)" - and 24 is exactly the core block (4 price + 5 swing + 4 range + 4 volume
|
|
|
|
|
//--- + 6 time + 1 ATR), so feature 25, the first value of THIS block, was the wall.
|
|
|
|
|
//--- ADMovingAverage is a CUSTOM indicator: MT5 fills its buffer asynchronously, so before it
|
|
|
|
|
//--- has calculated it returns EMPTY_VALUE for EVERY index, not just the warm-up tail. Without
|
|
|
|
|
//--- this flag every bar of the sweep was cached as a PERMANENT miss, which is the identical
|
|
|
|
|
//--- failure the ATR guard above was fixed for on 2026-08-10 - the fix was simply never
|
|
|
|
|
//--- propagated to the indicator blocks that follow it.
|
|
|
|
|
//--- Worse, it self-sustained: the discarded era immediately re-swept all 50k bars, and six
|
|
|
|
|
//--- instances doing that in a loop on a six-core box starved the very indicator they were
|
|
|
|
|
//--- waiting on. Marking it transient is what lets the cache retry instead of poisoning.
|
|
|
|
|
m_featureFailTransient = true;
|
fix(ensemble+depth): the barrier had no liveness escape, and the depth gate could not report the one state the evidence pointed at
Two charts (USDJPY 50,179 bars / XAUUSD 33,982) sat at era 0 for 38 minutes with
four of their eight members completely silent. Nothing in this commit guesses at
why the sweep fails - the last five guesses were all wrong. It makes the failure
say what it is, and stops one broken member taking its whole chart down with it.
WHAT THE LOG ACTUALLY SAYS, before any of this.
- The running build IS d9f834d (pulled 14:18, compiled 14:19:01, 0 errors), so
every depth instrument from 1dda479/7e63a8b/45c9e21 was live.
- It printed NOTHING. Zero "PRIMING", zero "CAPPED", zero "Per-indicator depth"
in 27 MB of journal. The instrument built to find the depth shortfall returned
"not this".
- On USDJPY at 14:24, CONV-cad8 completed eras 0 AND 1 across all 50,179 bars -
same chart, same 832-value window, same indicators, byte-identical fingerprint -
while LSTM-cad8 and HYB-cad8 reported ok=0 failed=50163. So it is not the
symbol, the history, the bar count or the indicator depth. It is per-member.
- ok=0 means the NEWEST anchors failed too, and a short indicator cannot do that.
The depth reading in project_silent_block_failures is therefore retired by its own
instrumentation. THE ROOT CAUSE IS STILL UNKNOWN and this commit does not claim one.
1. THE DEPTH GATE'S SILENT PATH WAS THE STATE IT WAS HUNTING.
ServableBars() read `if(servable <= 0 || servable >= want) return want;` - one
branch over three unrelated states, silent in all of them:
enabled == 0 -> nothing tunable is on. No cap. Healthy.
enabled > 0, servable == -1 -> a handle answered INVALID.
enabled > 0, servable == 0 -> created, never calculated.
BarsCalculated() returns -1 for a dead handle, so a dead MA is indistinguishable
from "no tunable indicators enabled" - and both returned `want` without printing a
character. That is exactly the state a per-member, every-index, depth-independent
failure produces, and it is the single reason a build carrying full depth
instrumentation logged nothing through the whole outage.
TunableBarsCalculated() now also reports HOW MANY indicators it consulted, and the
dead-handle case is reported (latched, with per-handle depths). The RETURN is
deliberately unchanged - what to do about a dead handle is not yet known, and
changing control flow on an unproven cause is how the last four fixes here went
wrong. SettledBars() routes its three pass-through states via ServableBars() so the
report is reachable from the training sweep, which is the only caller that hits it.
2. THE STALL REPORT NAMED A SLOT, NEVER A BLOCK.
"lookback slot 0 REJECTED (window had 24 of 832 values)" plus a guess ("an
indicator warm-up or a history-edge read"). Which guard fired was INFERRED by
counting 4+5+4+4+6+1 = 24 and concluding feature 25 must be the MA. The arithmetic
was right; every conclusion drawn from it was wrong, because a value count names a
POSITION and a position cannot tell cold from capped from invalid from off-the-end.
Every guard that can reject a bar now records itself - m_featureFailBlock - and the
report carries it, the series index, IndicatorDepthReport()'s per-handle depths,
and for each indicator whether the NEWEST bar reads. That last field is the whole
diagnosis in one word: newest-also-EMPTY means the buffer is unreadable everywhere
(cold or dead handle), newest-reads means a genuine history edge. Instrumented:
open, ATR, MA, RSI, MACD, Ichimoku, and all five AD blocks via ADIndicatorCold().
3. THE TOTAL-FAILURE BACKOFF WAS GATED ON THE WRONG QUESTION.
It armed only when m_featureFailTransient was set. Keeping that flag correct across
every guard is a list that has to stay right forever - the same shape of fix the
feature cache abandoned for the same reason - and the gate is pointless anyway: a
sweep where ZERO of 50,163 bars produced a window will produce zero again if it
restarts a millisecond later, transient or not. Doing that at full speed is what
starved six indicator threads on a six-core box. The backoff is now unconditional
on a total failure. The flag keeps its real job, deciding whether a MISS may be
cached, which is a per-bar question and not a scheduling one.
4. THE ERA BARRIER DEADLOCKED, AND SILENCED THE MEMBERS IT FROZE.
EnsembleMinTrainingEra() exempted deployed, stopped and paused members and its
comment concluded "so nothing deadlocks". Those three are all VOLUNTARY. A member
that simply CANNOT finish an era is none of them, so it pinned the minimum at its
own era with no time limit - and the hold branch's only action was
`m_lastEraCompleteTick = GetTickCount()`, which silences the stall watchdog. So on
USDJPY the two members that could not train reported, and the two healthy members
frozen behind them wrote nothing anywhere. The outage was visible only through the
members that were not suffering it.
- BarrierEraHeartbeat() stamps a clock on real era CHANGE, kept separate from
m_lastEraCompleteTick precisely because the barrier resets that one. Only a
member AT the minimum can be a blocker; a member ahead is idle by design and is
never counted as stuck.
- After ENSEMBLE_BARRIER_STUCK_MS (12 min) a non-advancing member is dropped from
the barrier minimum. It keeps training and rejoins the instant it completes an
era - at which point, being behind, it legitimately becomes the minimum again,
which is the documented resumed-laggard behaviour.
- Both transitions say so loudly, and the release states plainly that the
combined-vote score cannot be computed while the ensemble is desynchronised.
- A held member now writes a rate-limited journal line naming WHICH members it is
waiting on, so the blocker is read off one line.
5. THE PANEL FLICKER.
OnTickHandler gates its terse writer on !m_trainRunActive, and a barrier-held member
returns from Train() before ever setting it - so both writers thought they were the
only one updating the label and fought every tick. That is the reported "Getting
ready..." <-> "Waiting at era N for slower ensemble members" oscillation, and it hit
Perceptron but not Convolutional purely because Convolutional had a run active from
a completed era and Perceptron, resumed from disk, never did. Train()'s message is
the specific one, so it wins.
NEXT STEP once this is running: the stall line now ends in "REJECTED BY: ..." and
the per-handle depths. Read it. Do not reason around it.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-17 15:21:15 -04:00
|
|
|
//--- WHICH of the two reads failed, and whether the indicator is empty EVERYWHERE or only here.
|
|
|
|
|
//--- Those are different faults with different fixes and the old report could not separate them:
|
|
|
|
|
//--- newest bar also EMPTY -> the whole buffer is unreadable (cold, or a dead/invalid handle).
|
|
|
|
|
//--- Every anchor in the sweep fails, ok=0, and no amount of re-sweeping helps.
|
|
|
|
|
//--- newest bar READS -> the buffer is fine and only this depth is missing, i.e. a genuine
|
|
|
|
|
//--- history edge. Recent anchors succeed and ok > 0, so a total failure RULES THIS OUT.
|
|
|
|
|
//--- ok=0 across 50,163 anchors on 2026-08-17 therefore meant the first case, which is exactly
|
|
|
|
|
//--- what "the handle is short on deep history" would NOT produce - and that is the reading the
|
|
|
|
|
//--- report has to hand over instead of leaving to be inferred.
|
|
|
|
|
string maNewest = "reads (buffer live; this is a history-edge miss)";
|
|
|
|
|
if(m_MA.GetData(0, 0) == EMPTY_VALUE)
|
|
|
|
|
maNewest = "ALSO EMPTY (whole buffer unreadable - cold or dead handle, NOT a depth shortfall)";
|
|
|
|
|
m_featureFailBlock = StringFormat("MA (ADMovingAverage) - GetData(%d)=%s GetData(%d)=%s,"
|
|
|
|
|
" newest bar %s, BarsCalculated=%d",
|
|
|
|
|
idx, maNow == EMPTY_VALUE ? "EMPTY" : "ok",
|
|
|
|
|
idx + 1, maPrev == EMPTY_VALUE ? "EMPTY" : "ok",
|
|
|
|
|
maNewest, m_MA.BarsCalculated());
|
refactor: split CExpertSignalAIBase implementation by responsibility
ExpertSignalAIBase.mqh was 8216 lines: the class declaration followed by 87
method bodies covering training, labelling, feature extraction, persistence,
chart drawing, online learning, the GA auto-tuner and inference, all in one
file. Train() alone is 1492 lines; a change to arrow drawing meant scrolling
past the era loop.
Moved the bodies into Expert\AIBase\, included at the bottom of the original
after the class declaration:
Training.mqh 1607 era loop, plateau ladder, checkpoint select, deploy
Features.mqh 1093 indicator creation + per-bar input feature vector
ChartUI.mqh 634 arrows, arrow persistence, status panel, cleanup
Persistence.mqh 492 .stats/.cfg sidecars, CPU-inference validation, copy
OnlineLearning.mqh 461 live continual learning, EMA shadow, OOS simulator
Labels.mqh 309 ZigZag pivot labels, async label-cache prebuild
AutoTune.mqh 275 genetic tuner (population, crossover, halving)
Inference.mqh 235 softmax, prior calibration, class priors
ExpertSignalAIBase.mqh 8216 -> 3131 (declaration + topology build only)
This is a pure relocation - verified mechanically, not by eye: HEAD's file
reconstructed from the eight partials plus the surviving remainder is
byte-identical to HEAD, span for span (scratchpad verify_split.py). No
declaration moved, no signature changed, no code rewritten, so behaviour is
unchanged by construction.
Compiles 0 errors, 0 warnings.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-29 00:42:45 -04:00
|
|
|
return false;
|
fix(features): a cold indicator is TRANSIENT, not a permanent miss - and back off instead of re-sweeping
Six fresh instances on USDJPY and XAUUSD swept 33,965-50,162 bars and produced
ZERO usable feature windows, repeatedly, for 40 minutes and 239 stall reports,
without ever completing era 0. The four instances already warmed up before those
charts were attached trained normally throughout.
THE STALL REPORT NAMED THE SPOT EXACTLY: 'lookback slot 0 REJECTED the bar
(window had 24 of 832 values)', and 24 is the core block to the value - 4 price +
5 swing + 4 range + 4 volume + 6 time + 1 ATR. So feature 25 was the wall, and
feature 25 is the first value of the MA block. The same 24 appeared on XAUUSD
against a 816-value window (51 features/bar vs 52), which is what ruled out any
symbol-specific data gap: the wall sits at a fixed feature index, not a date.
ADMovingAverage is a CUSTOM indicator, so MT5 fills its buffer asynchronously and
returns EMPTY_VALUE for EVERY index until it has calculated - not just the
warm-up tail. That guard did not set m_featureFailTransient, so every bar of the
sweep was cached as a PERMANENT miss. This is precisely the failure the ATR guard
twenty lines above it was fixed for on 2026-08-10; the fix was never propagated
to the indicator blocks that follow. RSI, MACD and Ichimoku had the same defect
and are fixed too. (The Donchian high/low guard is a break into a
degraded-but-usable path, not a rejection, and is deliberately left alone.)
IT ALSO SELF-SUSTAINED, which is why it never recovered. The ok=0 self-heal drops
the feature cache and re-sweeps immediately, so each stuck instance spent every
millisecond re-reading 30-50k bars - six of them at once, on a six-core box,
competing for CPU with the very indicator calculation they were all waiting on.
The recovery was preventing the recovery. A transient total failure now re-arms
m_warmupPassesRemaining, yielding the CPU for a few separately-scheduled Train()
calls - the same mechanism a fresh model already uses to let history sync finish,
pointed at indicator warm-up instead.
Verified in the terminal journal first: indicators load and unload in matched
counts and there is no OOM, so this is NOT the 33f106d handle leak.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-17 12:47:25 -04:00
|
|
|
}
|
refactor: split CExpertSignalAIBase implementation by responsibility
ExpertSignalAIBase.mqh was 8216 lines: the class declaration followed by 87
method bodies covering training, labelling, feature extraction, persistence,
chart drawing, online learning, the GA auto-tuner and inference, all in one
file. Train() alone is 1492 lines; a change to arrow drawing meant scrolling
past the era loop.
Moved the bodies into Expert\AIBase\, included at the bottom of the original
after the class declaration:
Training.mqh 1607 era loop, plateau ladder, checkpoint select, deploy
Features.mqh 1093 indicator creation + per-bar input feature vector
ChartUI.mqh 634 arrows, arrow persistence, status panel, cleanup
Persistence.mqh 492 .stats/.cfg sidecars, CPU-inference validation, copy
OnlineLearning.mqh 461 live continual learning, EMA shadow, OOS simulator
Labels.mqh 309 ZigZag pivot labels, async label-cache prebuild
AutoTune.mqh 275 genetic tuner (population, crossover, halving)
Inference.mqh 235 softmax, prior calibration, class priors
ExpertSignalAIBase.mqh 8216 -> 3131 (declaration + topology build only)
This is a pure relocation - verified mechanically, not by eye: HEAD's file
reconstructed from the eight partials plus the surviving remainder is
byte-identical to HEAD, span for span (scratchpad verify_split.py). No
declaration moved, no signature changed, no code rewritten, so behaviour is
unchanged by construction.
Compiles 0 errors, 0 warnings.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-29 00:42:45 -04:00
|
|
|
if(!TempData.Add((open - maNow) / atr) ||
|
|
|
|
|
!TempData.Add((high - maNow) / atr) ||
|
|
|
|
|
!TempData.Add((low - maNow) / atr) ||
|
|
|
|
|
!TempData.Add((close - maNow) / atr) ||
|
|
|
|
|
!TempData.Add((maNow - maPrev) / atr))
|
|
|
|
|
return false;
|
|
|
|
|
}
|
|
|
|
|
if(m_useRSI)
|
|
|
|
|
{
|
|
|
|
|
// Already a 0-100 oscillator - /100 is the only transform needed to match the rest of the
|
|
|
|
|
// feature vector's scale (see m_useRSI's declaration comment).
|
|
|
|
|
double rsiNow = m_RSI.Main(idx);
|
|
|
|
|
if(rsiNow == EMPTY_VALUE)
|
fix(features): a cold indicator is TRANSIENT, not a permanent miss - and back off instead of re-sweeping
Six fresh instances on USDJPY and XAUUSD swept 33,965-50,162 bars and produced
ZERO usable feature windows, repeatedly, for 40 minutes and 239 stall reports,
without ever completing era 0. The four instances already warmed up before those
charts were attached trained normally throughout.
THE STALL REPORT NAMED THE SPOT EXACTLY: 'lookback slot 0 REJECTED the bar
(window had 24 of 832 values)', and 24 is the core block to the value - 4 price +
5 swing + 4 range + 4 volume + 6 time + 1 ATR. So feature 25 was the wall, and
feature 25 is the first value of the MA block. The same 24 appeared on XAUUSD
against a 816-value window (51 features/bar vs 52), which is what ruled out any
symbol-specific data gap: the wall sits at a fixed feature index, not a date.
ADMovingAverage is a CUSTOM indicator, so MT5 fills its buffer asynchronously and
returns EMPTY_VALUE for EVERY index until it has calculated - not just the
warm-up tail. That guard did not set m_featureFailTransient, so every bar of the
sweep was cached as a PERMANENT miss. This is precisely the failure the ATR guard
twenty lines above it was fixed for on 2026-08-10; the fix was never propagated
to the indicator blocks that follow. RSI, MACD and Ichimoku had the same defect
and are fixed too. (The Donchian high/low guard is a break into a
degraded-but-usable path, not a rejection, and is deliberately left alone.)
IT ALSO SELF-SUSTAINED, which is why it never recovered. The ok=0 self-heal drops
the feature cache and re-sweeps immediately, so each stuck instance spent every
millisecond re-reading 30-50k bars - six of them at once, on a six-core box,
competing for CPU with the very indicator calculation they were all waiting on.
The recovery was preventing the recovery. A transient total failure now re-arms
m_warmupPassesRemaining, yielding the CPU for a few separately-scheduled Train()
calls - the same mechanism a fresh model already uses to let history sync finish,
pointed at indicator warm-up instead.
Verified in the terminal journal first: indicators load and unload in matched
counts and there is no OOM, so this is NOT the 33f106d handle leak.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-17 12:47:25 -04:00
|
|
|
{
|
|
|
|
|
m_featureFailTransient = true; // not-ready, not no-data - see the MA guard above
|
fix(ensemble+depth): the barrier had no liveness escape, and the depth gate could not report the one state the evidence pointed at
Two charts (USDJPY 50,179 bars / XAUUSD 33,982) sat at era 0 for 38 minutes with
four of their eight members completely silent. Nothing in this commit guesses at
why the sweep fails - the last five guesses were all wrong. It makes the failure
say what it is, and stops one broken member taking its whole chart down with it.
WHAT THE LOG ACTUALLY SAYS, before any of this.
- The running build IS d9f834d (pulled 14:18, compiled 14:19:01, 0 errors), so
every depth instrument from 1dda479/7e63a8b/45c9e21 was live.
- It printed NOTHING. Zero "PRIMING", zero "CAPPED", zero "Per-indicator depth"
in 27 MB of journal. The instrument built to find the depth shortfall returned
"not this".
- On USDJPY at 14:24, CONV-cad8 completed eras 0 AND 1 across all 50,179 bars -
same chart, same 832-value window, same indicators, byte-identical fingerprint -
while LSTM-cad8 and HYB-cad8 reported ok=0 failed=50163. So it is not the
symbol, the history, the bar count or the indicator depth. It is per-member.
- ok=0 means the NEWEST anchors failed too, and a short indicator cannot do that.
The depth reading in project_silent_block_failures is therefore retired by its own
instrumentation. THE ROOT CAUSE IS STILL UNKNOWN and this commit does not claim one.
1. THE DEPTH GATE'S SILENT PATH WAS THE STATE IT WAS HUNTING.
ServableBars() read `if(servable <= 0 || servable >= want) return want;` - one
branch over three unrelated states, silent in all of them:
enabled == 0 -> nothing tunable is on. No cap. Healthy.
enabled > 0, servable == -1 -> a handle answered INVALID.
enabled > 0, servable == 0 -> created, never calculated.
BarsCalculated() returns -1 for a dead handle, so a dead MA is indistinguishable
from "no tunable indicators enabled" - and both returned `want` without printing a
character. That is exactly the state a per-member, every-index, depth-independent
failure produces, and it is the single reason a build carrying full depth
instrumentation logged nothing through the whole outage.
TunableBarsCalculated() now also reports HOW MANY indicators it consulted, and the
dead-handle case is reported (latched, with per-handle depths). The RETURN is
deliberately unchanged - what to do about a dead handle is not yet known, and
changing control flow on an unproven cause is how the last four fixes here went
wrong. SettledBars() routes its three pass-through states via ServableBars() so the
report is reachable from the training sweep, which is the only caller that hits it.
2. THE STALL REPORT NAMED A SLOT, NEVER A BLOCK.
"lookback slot 0 REJECTED (window had 24 of 832 values)" plus a guess ("an
indicator warm-up or a history-edge read"). Which guard fired was INFERRED by
counting 4+5+4+4+6+1 = 24 and concluding feature 25 must be the MA. The arithmetic
was right; every conclusion drawn from it was wrong, because a value count names a
POSITION and a position cannot tell cold from capped from invalid from off-the-end.
Every guard that can reject a bar now records itself - m_featureFailBlock - and the
report carries it, the series index, IndicatorDepthReport()'s per-handle depths,
and for each indicator whether the NEWEST bar reads. That last field is the whole
diagnosis in one word: newest-also-EMPTY means the buffer is unreadable everywhere
(cold or dead handle), newest-reads means a genuine history edge. Instrumented:
open, ATR, MA, RSI, MACD, Ichimoku, and all five AD blocks via ADIndicatorCold().
3. THE TOTAL-FAILURE BACKOFF WAS GATED ON THE WRONG QUESTION.
It armed only when m_featureFailTransient was set. Keeping that flag correct across
every guard is a list that has to stay right forever - the same shape of fix the
feature cache abandoned for the same reason - and the gate is pointless anyway: a
sweep where ZERO of 50,163 bars produced a window will produce zero again if it
restarts a millisecond later, transient or not. Doing that at full speed is what
starved six indicator threads on a six-core box. The backoff is now unconditional
on a total failure. The flag keeps its real job, deciding whether a MISS may be
cached, which is a per-bar question and not a scheduling one.
4. THE ERA BARRIER DEADLOCKED, AND SILENCED THE MEMBERS IT FROZE.
EnsembleMinTrainingEra() exempted deployed, stopped and paused members and its
comment concluded "so nothing deadlocks". Those three are all VOLUNTARY. A member
that simply CANNOT finish an era is none of them, so it pinned the minimum at its
own era with no time limit - and the hold branch's only action was
`m_lastEraCompleteTick = GetTickCount()`, which silences the stall watchdog. So on
USDJPY the two members that could not train reported, and the two healthy members
frozen behind them wrote nothing anywhere. The outage was visible only through the
members that were not suffering it.
- BarrierEraHeartbeat() stamps a clock on real era CHANGE, kept separate from
m_lastEraCompleteTick precisely because the barrier resets that one. Only a
member AT the minimum can be a blocker; a member ahead is idle by design and is
never counted as stuck.
- After ENSEMBLE_BARRIER_STUCK_MS (12 min) a non-advancing member is dropped from
the barrier minimum. It keeps training and rejoins the instant it completes an
era - at which point, being behind, it legitimately becomes the minimum again,
which is the documented resumed-laggard behaviour.
- Both transitions say so loudly, and the release states plainly that the
combined-vote score cannot be computed while the ensemble is desynchronised.
- A held member now writes a rate-limited journal line naming WHICH members it is
waiting on, so the blocker is read off one line.
5. THE PANEL FLICKER.
OnTickHandler gates its terse writer on !m_trainRunActive, and a barrier-held member
returns from Train() before ever setting it - so both writers thought they were the
only one updating the label and fought every tick. That is the reported "Getting
ready..." <-> "Waiting at era N for slower ensemble members" oscillation, and it hit
Perceptron but not Convolutional purely because Convolutional had a run active from
a completed era and Perceptron, resumed from disk, never did. Train()'s message is
the specific one, so it wins.
NEXT STEP once this is running: the stall line now ends in "REJECTED BY: ..." and
the per-handle depths. Read it. Do not reason around it.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-17 15:21:15 -04:00
|
|
|
m_featureFailBlock = StringFormat("RSI - Main(%d)=EMPTY, newest bar %s, BarsCalculated=%d", idx,
|
|
|
|
|
m_RSI.Main(0) == EMPTY_VALUE ? "ALSO EMPTY" : "reads",
|
|
|
|
|
m_RSI.BarsCalculated());
|
refactor: split CExpertSignalAIBase implementation by responsibility
ExpertSignalAIBase.mqh was 8216 lines: the class declaration followed by 87
method bodies covering training, labelling, feature extraction, persistence,
chart drawing, online learning, the GA auto-tuner and inference, all in one
file. Train() alone is 1492 lines; a change to arrow drawing meant scrolling
past the era loop.
Moved the bodies into Expert\AIBase\, included at the bottom of the original
after the class declaration:
Training.mqh 1607 era loop, plateau ladder, checkpoint select, deploy
Features.mqh 1093 indicator creation + per-bar input feature vector
ChartUI.mqh 634 arrows, arrow persistence, status panel, cleanup
Persistence.mqh 492 .stats/.cfg sidecars, CPU-inference validation, copy
OnlineLearning.mqh 461 live continual learning, EMA shadow, OOS simulator
Labels.mqh 309 ZigZag pivot labels, async label-cache prebuild
AutoTune.mqh 275 genetic tuner (population, crossover, halving)
Inference.mqh 235 softmax, prior calibration, class priors
ExpertSignalAIBase.mqh 8216 -> 3131 (declaration + topology build only)
This is a pure relocation - verified mechanically, not by eye: HEAD's file
reconstructed from the eight partials plus the surviving remainder is
byte-identical to HEAD, span for span (scratchpad verify_split.py). No
declaration moved, no signature changed, no code rewritten, so behaviour is
unchanged by construction.
Compiles 0 errors, 0 warnings.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-29 00:42:45 -04:00
|
|
|
return false;
|
fix(features): a cold indicator is TRANSIENT, not a permanent miss - and back off instead of re-sweeping
Six fresh instances on USDJPY and XAUUSD swept 33,965-50,162 bars and produced
ZERO usable feature windows, repeatedly, for 40 minutes and 239 stall reports,
without ever completing era 0. The four instances already warmed up before those
charts were attached trained normally throughout.
THE STALL REPORT NAMED THE SPOT EXACTLY: 'lookback slot 0 REJECTED the bar
(window had 24 of 832 values)', and 24 is the core block to the value - 4 price +
5 swing + 4 range + 4 volume + 6 time + 1 ATR. So feature 25 was the wall, and
feature 25 is the first value of the MA block. The same 24 appeared on XAUUSD
against a 816-value window (51 features/bar vs 52), which is what ruled out any
symbol-specific data gap: the wall sits at a fixed feature index, not a date.
ADMovingAverage is a CUSTOM indicator, so MT5 fills its buffer asynchronously and
returns EMPTY_VALUE for EVERY index until it has calculated - not just the
warm-up tail. That guard did not set m_featureFailTransient, so every bar of the
sweep was cached as a PERMANENT miss. This is precisely the failure the ATR guard
twenty lines above it was fixed for on 2026-08-10; the fix was never propagated
to the indicator blocks that follow. RSI, MACD and Ichimoku had the same defect
and are fixed too. (The Donchian high/low guard is a break into a
degraded-but-usable path, not a rejection, and is deliberately left alone.)
IT ALSO SELF-SUSTAINED, which is why it never recovered. The ok=0 self-heal drops
the feature cache and re-sweeps immediately, so each stuck instance spent every
millisecond re-reading 30-50k bars - six of them at once, on a six-core box,
competing for CPU with the very indicator calculation they were all waiting on.
The recovery was preventing the recovery. A transient total failure now re-arms
m_warmupPassesRemaining, yielding the CPU for a few separately-scheduled Train()
calls - the same mechanism a fresh model already uses to let history sync finish,
pointed at indicator warm-up instead.
Verified in the terminal journal first: indicators load and unload in matched
counts and there is no OOM, so this is NOT the 33f106d handle leak.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-17 12:47:25 -04:00
|
|
|
}
|
refactor: split CExpertSignalAIBase implementation by responsibility
ExpertSignalAIBase.mqh was 8216 lines: the class declaration followed by 87
method bodies covering training, labelling, feature extraction, persistence,
chart drawing, online learning, the GA auto-tuner and inference, all in one
file. Train() alone is 1492 lines; a change to arrow drawing meant scrolling
past the era loop.
Moved the bodies into Expert\AIBase\, included at the bottom of the original
after the class declaration:
Training.mqh 1607 era loop, plateau ladder, checkpoint select, deploy
Features.mqh 1093 indicator creation + per-bar input feature vector
ChartUI.mqh 634 arrows, arrow persistence, status panel, cleanup
Persistence.mqh 492 .stats/.cfg sidecars, CPU-inference validation, copy
OnlineLearning.mqh 461 live continual learning, EMA shadow, OOS simulator
Labels.mqh 309 ZigZag pivot labels, async label-cache prebuild
AutoTune.mqh 275 genetic tuner (population, crossover, halving)
Inference.mqh 235 softmax, prior calibration, class priors
ExpertSignalAIBase.mqh 8216 -> 3131 (declaration + topology build only)
This is a pure relocation - verified mechanically, not by eye: HEAD's file
reconstructed from the eight partials plus the surviving remainder is
byte-identical to HEAD, span for span (scratchpad verify_split.py). No
declaration moved, no signature changed, no code rewritten, so behaviour is
unchanged by construction.
Compiles 0 errors, 0 warnings.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-29 00:42:45 -04:00
|
|
|
if(!TempData.Add(rsiNow / 100.0))
|
|
|
|
|
return false;
|
|
|
|
|
}
|
|
|
|
|
if(m_useMACD)
|
|
|
|
|
{
|
|
|
|
|
// Main and signal lines are price-domain differences of two EMAs, so the same ATR normalization
|
|
|
|
|
// every other price-unit feature here uses applies unchanged. The third value is the histogram
|
|
|
|
|
// (main - signal): algebraically derivable from the first two, but handed over explicitly for the
|
|
|
|
|
// same reason the bullish/bearish flag is handed to the network alongside (close-open)/atr - a
|
|
|
|
|
// value the network would otherwise have to learn to subtract is better given directly, and the
|
|
|
|
|
// histogram (momentum ACCELERATION) is the one term nothing else in this vector carries.
|
|
|
|
|
double macdMain = m_MACDFeature.Main(idx);
|
|
|
|
|
double macdSignal = m_MACDFeature.Signal(idx);
|
|
|
|
|
if(macdMain == EMPTY_VALUE || macdSignal == EMPTY_VALUE)
|
fix(features): a cold indicator is TRANSIENT, not a permanent miss - and back off instead of re-sweeping
Six fresh instances on USDJPY and XAUUSD swept 33,965-50,162 bars and produced
ZERO usable feature windows, repeatedly, for 40 minutes and 239 stall reports,
without ever completing era 0. The four instances already warmed up before those
charts were attached trained normally throughout.
THE STALL REPORT NAMED THE SPOT EXACTLY: 'lookback slot 0 REJECTED the bar
(window had 24 of 832 values)', and 24 is the core block to the value - 4 price +
5 swing + 4 range + 4 volume + 6 time + 1 ATR. So feature 25 was the wall, and
feature 25 is the first value of the MA block. The same 24 appeared on XAUUSD
against a 816-value window (51 features/bar vs 52), which is what ruled out any
symbol-specific data gap: the wall sits at a fixed feature index, not a date.
ADMovingAverage is a CUSTOM indicator, so MT5 fills its buffer asynchronously and
returns EMPTY_VALUE for EVERY index until it has calculated - not just the
warm-up tail. That guard did not set m_featureFailTransient, so every bar of the
sweep was cached as a PERMANENT miss. This is precisely the failure the ATR guard
twenty lines above it was fixed for on 2026-08-10; the fix was never propagated
to the indicator blocks that follow. RSI, MACD and Ichimoku had the same defect
and are fixed too. (The Donchian high/low guard is a break into a
degraded-but-usable path, not a rejection, and is deliberately left alone.)
IT ALSO SELF-SUSTAINED, which is why it never recovered. The ok=0 self-heal drops
the feature cache and re-sweeps immediately, so each stuck instance spent every
millisecond re-reading 30-50k bars - six of them at once, on a six-core box,
competing for CPU with the very indicator calculation they were all waiting on.
The recovery was preventing the recovery. A transient total failure now re-arms
m_warmupPassesRemaining, yielding the CPU for a few separately-scheduled Train()
calls - the same mechanism a fresh model already uses to let history sync finish,
pointed at indicator warm-up instead.
Verified in the terminal journal first: indicators load and unload in matched
counts and there is no OOM, so this is NOT the 33f106d handle leak.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-17 12:47:25 -04:00
|
|
|
{
|
|
|
|
|
m_featureFailTransient = true; // not-ready, not no-data - see the MA guard above
|
fix(ensemble+depth): the barrier had no liveness escape, and the depth gate could not report the one state the evidence pointed at
Two charts (USDJPY 50,179 bars / XAUUSD 33,982) sat at era 0 for 38 minutes with
four of their eight members completely silent. Nothing in this commit guesses at
why the sweep fails - the last five guesses were all wrong. It makes the failure
say what it is, and stops one broken member taking its whole chart down with it.
WHAT THE LOG ACTUALLY SAYS, before any of this.
- The running build IS d9f834d (pulled 14:18, compiled 14:19:01, 0 errors), so
every depth instrument from 1dda479/7e63a8b/45c9e21 was live.
- It printed NOTHING. Zero "PRIMING", zero "CAPPED", zero "Per-indicator depth"
in 27 MB of journal. The instrument built to find the depth shortfall returned
"not this".
- On USDJPY at 14:24, CONV-cad8 completed eras 0 AND 1 across all 50,179 bars -
same chart, same 832-value window, same indicators, byte-identical fingerprint -
while LSTM-cad8 and HYB-cad8 reported ok=0 failed=50163. So it is not the
symbol, the history, the bar count or the indicator depth. It is per-member.
- ok=0 means the NEWEST anchors failed too, and a short indicator cannot do that.
The depth reading in project_silent_block_failures is therefore retired by its own
instrumentation. THE ROOT CAUSE IS STILL UNKNOWN and this commit does not claim one.
1. THE DEPTH GATE'S SILENT PATH WAS THE STATE IT WAS HUNTING.
ServableBars() read `if(servable <= 0 || servable >= want) return want;` - one
branch over three unrelated states, silent in all of them:
enabled == 0 -> nothing tunable is on. No cap. Healthy.
enabled > 0, servable == -1 -> a handle answered INVALID.
enabled > 0, servable == 0 -> created, never calculated.
BarsCalculated() returns -1 for a dead handle, so a dead MA is indistinguishable
from "no tunable indicators enabled" - and both returned `want` without printing a
character. That is exactly the state a per-member, every-index, depth-independent
failure produces, and it is the single reason a build carrying full depth
instrumentation logged nothing through the whole outage.
TunableBarsCalculated() now also reports HOW MANY indicators it consulted, and the
dead-handle case is reported (latched, with per-handle depths). The RETURN is
deliberately unchanged - what to do about a dead handle is not yet known, and
changing control flow on an unproven cause is how the last four fixes here went
wrong. SettledBars() routes its three pass-through states via ServableBars() so the
report is reachable from the training sweep, which is the only caller that hits it.
2. THE STALL REPORT NAMED A SLOT, NEVER A BLOCK.
"lookback slot 0 REJECTED (window had 24 of 832 values)" plus a guess ("an
indicator warm-up or a history-edge read"). Which guard fired was INFERRED by
counting 4+5+4+4+6+1 = 24 and concluding feature 25 must be the MA. The arithmetic
was right; every conclusion drawn from it was wrong, because a value count names a
POSITION and a position cannot tell cold from capped from invalid from off-the-end.
Every guard that can reject a bar now records itself - m_featureFailBlock - and the
report carries it, the series index, IndicatorDepthReport()'s per-handle depths,
and for each indicator whether the NEWEST bar reads. That last field is the whole
diagnosis in one word: newest-also-EMPTY means the buffer is unreadable everywhere
(cold or dead handle), newest-reads means a genuine history edge. Instrumented:
open, ATR, MA, RSI, MACD, Ichimoku, and all five AD blocks via ADIndicatorCold().
3. THE TOTAL-FAILURE BACKOFF WAS GATED ON THE WRONG QUESTION.
It armed only when m_featureFailTransient was set. Keeping that flag correct across
every guard is a list that has to stay right forever - the same shape of fix the
feature cache abandoned for the same reason - and the gate is pointless anyway: a
sweep where ZERO of 50,163 bars produced a window will produce zero again if it
restarts a millisecond later, transient or not. Doing that at full speed is what
starved six indicator threads on a six-core box. The backoff is now unconditional
on a total failure. The flag keeps its real job, deciding whether a MISS may be
cached, which is a per-bar question and not a scheduling one.
4. THE ERA BARRIER DEADLOCKED, AND SILENCED THE MEMBERS IT FROZE.
EnsembleMinTrainingEra() exempted deployed, stopped and paused members and its
comment concluded "so nothing deadlocks". Those three are all VOLUNTARY. A member
that simply CANNOT finish an era is none of them, so it pinned the minimum at its
own era with no time limit - and the hold branch's only action was
`m_lastEraCompleteTick = GetTickCount()`, which silences the stall watchdog. So on
USDJPY the two members that could not train reported, and the two healthy members
frozen behind them wrote nothing anywhere. The outage was visible only through the
members that were not suffering it.
- BarrierEraHeartbeat() stamps a clock on real era CHANGE, kept separate from
m_lastEraCompleteTick precisely because the barrier resets that one. Only a
member AT the minimum can be a blocker; a member ahead is idle by design and is
never counted as stuck.
- After ENSEMBLE_BARRIER_STUCK_MS (12 min) a non-advancing member is dropped from
the barrier minimum. It keeps training and rejoins the instant it completes an
era - at which point, being behind, it legitimately becomes the minimum again,
which is the documented resumed-laggard behaviour.
- Both transitions say so loudly, and the release states plainly that the
combined-vote score cannot be computed while the ensemble is desynchronised.
- A held member now writes a rate-limited journal line naming WHICH members it is
waiting on, so the blocker is read off one line.
5. THE PANEL FLICKER.
OnTickHandler gates its terse writer on !m_trainRunActive, and a barrier-held member
returns from Train() before ever setting it - so both writers thought they were the
only one updating the label and fought every tick. That is the reported "Getting
ready..." <-> "Waiting at era N for slower ensemble members" oscillation, and it hit
Perceptron but not Convolutional purely because Convolutional had a run active from
a completed era and Perceptron, resumed from disk, never did. Train()'s message is
the specific one, so it wins.
NEXT STEP once this is running: the stall line now ends in "REJECTED BY: ..." and
the per-handle depths. Read it. Do not reason around it.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-17 15:21:15 -04:00
|
|
|
m_featureFailBlock = StringFormat("MACD - Main(%d)=%s Signal(%d)=%s, newest bar %s,"
|
|
|
|
|
" BarsCalculated=%d", idx,
|
|
|
|
|
macdMain == EMPTY_VALUE ? "EMPTY" : "ok", idx,
|
|
|
|
|
macdSignal == EMPTY_VALUE ? "EMPTY" : "ok",
|
|
|
|
|
m_MACDFeature.Main(0) == EMPTY_VALUE ? "ALSO EMPTY" : "reads",
|
|
|
|
|
m_MACDFeature.BarsCalculated());
|
refactor: split CExpertSignalAIBase implementation by responsibility
ExpertSignalAIBase.mqh was 8216 lines: the class declaration followed by 87
method bodies covering training, labelling, feature extraction, persistence,
chart drawing, online learning, the GA auto-tuner and inference, all in one
file. Train() alone is 1492 lines; a change to arrow drawing meant scrolling
past the era loop.
Moved the bodies into Expert\AIBase\, included at the bottom of the original
after the class declaration:
Training.mqh 1607 era loop, plateau ladder, checkpoint select, deploy
Features.mqh 1093 indicator creation + per-bar input feature vector
ChartUI.mqh 634 arrows, arrow persistence, status panel, cleanup
Persistence.mqh 492 .stats/.cfg sidecars, CPU-inference validation, copy
OnlineLearning.mqh 461 live continual learning, EMA shadow, OOS simulator
Labels.mqh 309 ZigZag pivot labels, async label-cache prebuild
AutoTune.mqh 275 genetic tuner (population, crossover, halving)
Inference.mqh 235 softmax, prior calibration, class priors
ExpertSignalAIBase.mqh 8216 -> 3131 (declaration + topology build only)
This is a pure relocation - verified mechanically, not by eye: HEAD's file
reconstructed from the eight partials plus the surviving remainder is
byte-identical to HEAD, span for span (scratchpad verify_split.py). No
declaration moved, no signature changed, no code rewritten, so behaviour is
unchanged by construction.
Compiles 0 errors, 0 warnings.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-29 00:42:45 -04:00
|
|
|
return false;
|
fix(features): a cold indicator is TRANSIENT, not a permanent miss - and back off instead of re-sweeping
Six fresh instances on USDJPY and XAUUSD swept 33,965-50,162 bars and produced
ZERO usable feature windows, repeatedly, for 40 minutes and 239 stall reports,
without ever completing era 0. The four instances already warmed up before those
charts were attached trained normally throughout.
THE STALL REPORT NAMED THE SPOT EXACTLY: 'lookback slot 0 REJECTED the bar
(window had 24 of 832 values)', and 24 is the core block to the value - 4 price +
5 swing + 4 range + 4 volume + 6 time + 1 ATR. So feature 25 was the wall, and
feature 25 is the first value of the MA block. The same 24 appeared on XAUUSD
against a 816-value window (51 features/bar vs 52), which is what ruled out any
symbol-specific data gap: the wall sits at a fixed feature index, not a date.
ADMovingAverage is a CUSTOM indicator, so MT5 fills its buffer asynchronously and
returns EMPTY_VALUE for EVERY index until it has calculated - not just the
warm-up tail. That guard did not set m_featureFailTransient, so every bar of the
sweep was cached as a PERMANENT miss. This is precisely the failure the ATR guard
twenty lines above it was fixed for on 2026-08-10; the fix was never propagated
to the indicator blocks that follow. RSI, MACD and Ichimoku had the same defect
and are fixed too. (The Donchian high/low guard is a break into a
degraded-but-usable path, not a rejection, and is deliberately left alone.)
IT ALSO SELF-SUSTAINED, which is why it never recovered. The ok=0 self-heal drops
the feature cache and re-sweeps immediately, so each stuck instance spent every
millisecond re-reading 30-50k bars - six of them at once, on a six-core box,
competing for CPU with the very indicator calculation they were all waiting on.
The recovery was preventing the recovery. A transient total failure now re-arms
m_warmupPassesRemaining, yielding the CPU for a few separately-scheduled Train()
calls - the same mechanism a fresh model already uses to let history sync finish,
pointed at indicator warm-up instead.
Verified in the terminal journal first: indicators load and unload in matched
counts and there is no OOM, so this is NOT the 33f106d handle leak.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-17 12:47:25 -04:00
|
|
|
}
|
refactor: split CExpertSignalAIBase implementation by responsibility
ExpertSignalAIBase.mqh was 8216 lines: the class declaration followed by 87
method bodies covering training, labelling, feature extraction, persistence,
chart drawing, online learning, the GA auto-tuner and inference, all in one
file. Train() alone is 1492 lines; a change to arrow drawing meant scrolling
past the era loop.
Moved the bodies into Expert\AIBase\, included at the bottom of the original
after the class declaration:
Training.mqh 1607 era loop, plateau ladder, checkpoint select, deploy
Features.mqh 1093 indicator creation + per-bar input feature vector
ChartUI.mqh 634 arrows, arrow persistence, status panel, cleanup
Persistence.mqh 492 .stats/.cfg sidecars, CPU-inference validation, copy
OnlineLearning.mqh 461 live continual learning, EMA shadow, OOS simulator
Labels.mqh 309 ZigZag pivot labels, async label-cache prebuild
AutoTune.mqh 275 genetic tuner (population, crossover, halving)
Inference.mqh 235 softmax, prior calibration, class priors
ExpertSignalAIBase.mqh 8216 -> 3131 (declaration + topology build only)
This is a pure relocation - verified mechanically, not by eye: HEAD's file
reconstructed from the eight partials plus the surviving remainder is
byte-identical to HEAD, span for span (scratchpad verify_split.py). No
declaration moved, no signature changed, no code rewritten, so behaviour is
unchanged by construction.
Compiles 0 errors, 0 warnings.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-29 00:42:45 -04:00
|
|
|
if(!TempData.Add(macdMain / atr) ||
|
|
|
|
|
!TempData.Add(macdSignal / atr) ||
|
|
|
|
|
!TempData.Add((macdMain - macdSignal) / atr))
|
|
|
|
|
return false;
|
|
|
|
|
}
|
|
|
|
|
if(m_useIchimoku)
|
|
|
|
|
{
|
|
|
|
|
// LOOKAHEAD, the one thing that matters in this block. MT5's iIchimoku does NOT pre-shift its
|
|
|
|
|
// buffers - it stores raw per-bar values and shifts only the DRAWING (Ichimoku.mq5 sets
|
|
|
|
|
// PLOT_SHIFT=+Kijun on the Senkou A/B cloud plot and -Kijun on the Chikou plot). In series
|
|
|
|
|
// indexing that means:
|
|
|
|
|
// - SenkouSpan*(i) is computed FROM bar i and drawn Kijun bars into the FUTURE, so the cloud
|
|
|
|
|
// actually sitting under bar idx is SenkouSpan*(idx + kijun) - built from bar idx+kijun and
|
|
|
|
|
// older, hence strictly past data. Reading SenkouSpan*(idx) as "the cloud here" is the classic
|
|
|
|
|
// Ichimoku backtest bug and would leak Kijun bars of future information into every example.
|
|
|
|
|
// - SenkouSpan*(idx) with NO offset IS legitimate as the PROJECTED cloud - the part of the chart
|
|
|
|
|
// already drawn ahead of the current bar. A live bar at idx genuinely knows it (it is computed
|
|
|
|
|
// from bar idx), which is why it appears below as its own feature rather than being avoided.
|
|
|
|
|
// - ChinkouSpan(i) is just Close(i) drawn at i+Kijun, so the Chikou plotted AT bar idx would be
|
|
|
|
|
// Close(idx - kijun) - a FUTURE bar. It is never read. The lookahead-free statement of the same
|
|
|
|
|
// reading is "how far is this close from the close Kijun bars ago", the last feature below.
|
|
|
|
|
// Signals\SignalIchimoku.mqh's class comment documents the identical convention for the vote side.
|
|
|
|
|
int kijunShift = m_indicatorTuner.ichiKijun;
|
|
|
|
|
double tenkan = m_Ichimoku.TenkanSen(idx);
|
|
|
|
|
double kijun = m_Ichimoku.KijunSen(idx);
|
|
|
|
|
double spanA = m_Ichimoku.SenkouSpanA(idx + kijunShift); // cloud AS PLOTTED AT bar idx
|
|
|
|
|
double spanB = m_Ichimoku.SenkouSpanB(idx + kijunShift);
|
|
|
|
|
double futureSpanA = m_Ichimoku.SenkouSpanA(idx); // cloud projected AHEAD of bar idx
|
|
|
|
|
double futureSpanB = m_Ichimoku.SenkouSpanB(idx);
|
|
|
|
|
double closeLagRef = m_Close.GetData(idx + kijunShift); // Chikou reference, never idx - kijunShift
|
|
|
|
|
if(tenkan == EMPTY_VALUE || kijun == EMPTY_VALUE ||
|
|
|
|
|
spanA == EMPTY_VALUE || spanB == EMPTY_VALUE ||
|
|
|
|
|
futureSpanA == EMPTY_VALUE || futureSpanB == EMPTY_VALUE ||
|
|
|
|
|
closeLagRef == EMPTY_VALUE || closeLagRef <= 0.0)
|
fix(features): a cold indicator is TRANSIENT, not a permanent miss - and back off instead of re-sweeping
Six fresh instances on USDJPY and XAUUSD swept 33,965-50,162 bars and produced
ZERO usable feature windows, repeatedly, for 40 minutes and 239 stall reports,
without ever completing era 0. The four instances already warmed up before those
charts were attached trained normally throughout.
THE STALL REPORT NAMED THE SPOT EXACTLY: 'lookback slot 0 REJECTED the bar
(window had 24 of 832 values)', and 24 is the core block to the value - 4 price +
5 swing + 4 range + 4 volume + 6 time + 1 ATR. So feature 25 was the wall, and
feature 25 is the first value of the MA block. The same 24 appeared on XAUUSD
against a 816-value window (51 features/bar vs 52), which is what ruled out any
symbol-specific data gap: the wall sits at a fixed feature index, not a date.
ADMovingAverage is a CUSTOM indicator, so MT5 fills its buffer asynchronously and
returns EMPTY_VALUE for EVERY index until it has calculated - not just the
warm-up tail. That guard did not set m_featureFailTransient, so every bar of the
sweep was cached as a PERMANENT miss. This is precisely the failure the ATR guard
twenty lines above it was fixed for on 2026-08-10; the fix was never propagated
to the indicator blocks that follow. RSI, MACD and Ichimoku had the same defect
and are fixed too. (The Donchian high/low guard is a break into a
degraded-but-usable path, not a rejection, and is deliberately left alone.)
IT ALSO SELF-SUSTAINED, which is why it never recovered. The ok=0 self-heal drops
the feature cache and re-sweeps immediately, so each stuck instance spent every
millisecond re-reading 30-50k bars - six of them at once, on a six-core box,
competing for CPU with the very indicator calculation they were all waiting on.
The recovery was preventing the recovery. A transient total failure now re-arms
m_warmupPassesRemaining, yielding the CPU for a few separately-scheduled Train()
calls - the same mechanism a fresh model already uses to let history sync finish,
pointed at indicator warm-up instead.
Verified in the terminal journal first: indicators load and unload in matched
counts and there is no OOM, so this is NOT the 33f106d handle leak.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-17 12:47:25 -04:00
|
|
|
{
|
|
|
|
|
m_featureFailTransient = true; // not-ready, not no-data - see the MA guard above
|
fix(ensemble+depth): the barrier had no liveness escape, and the depth gate could not report the one state the evidence pointed at
Two charts (USDJPY 50,179 bars / XAUUSD 33,982) sat at era 0 for 38 minutes with
four of their eight members completely silent. Nothing in this commit guesses at
why the sweep fails - the last five guesses were all wrong. It makes the failure
say what it is, and stops one broken member taking its whole chart down with it.
WHAT THE LOG ACTUALLY SAYS, before any of this.
- The running build IS d9f834d (pulled 14:18, compiled 14:19:01, 0 errors), so
every depth instrument from 1dda479/7e63a8b/45c9e21 was live.
- It printed NOTHING. Zero "PRIMING", zero "CAPPED", zero "Per-indicator depth"
in 27 MB of journal. The instrument built to find the depth shortfall returned
"not this".
- On USDJPY at 14:24, CONV-cad8 completed eras 0 AND 1 across all 50,179 bars -
same chart, same 832-value window, same indicators, byte-identical fingerprint -
while LSTM-cad8 and HYB-cad8 reported ok=0 failed=50163. So it is not the
symbol, the history, the bar count or the indicator depth. It is per-member.
- ok=0 means the NEWEST anchors failed too, and a short indicator cannot do that.
The depth reading in project_silent_block_failures is therefore retired by its own
instrumentation. THE ROOT CAUSE IS STILL UNKNOWN and this commit does not claim one.
1. THE DEPTH GATE'S SILENT PATH WAS THE STATE IT WAS HUNTING.
ServableBars() read `if(servable <= 0 || servable >= want) return want;` - one
branch over three unrelated states, silent in all of them:
enabled == 0 -> nothing tunable is on. No cap. Healthy.
enabled > 0, servable == -1 -> a handle answered INVALID.
enabled > 0, servable == 0 -> created, never calculated.
BarsCalculated() returns -1 for a dead handle, so a dead MA is indistinguishable
from "no tunable indicators enabled" - and both returned `want` without printing a
character. That is exactly the state a per-member, every-index, depth-independent
failure produces, and it is the single reason a build carrying full depth
instrumentation logged nothing through the whole outage.
TunableBarsCalculated() now also reports HOW MANY indicators it consulted, and the
dead-handle case is reported (latched, with per-handle depths). The RETURN is
deliberately unchanged - what to do about a dead handle is not yet known, and
changing control flow on an unproven cause is how the last four fixes here went
wrong. SettledBars() routes its three pass-through states via ServableBars() so the
report is reachable from the training sweep, which is the only caller that hits it.
2. THE STALL REPORT NAMED A SLOT, NEVER A BLOCK.
"lookback slot 0 REJECTED (window had 24 of 832 values)" plus a guess ("an
indicator warm-up or a history-edge read"). Which guard fired was INFERRED by
counting 4+5+4+4+6+1 = 24 and concluding feature 25 must be the MA. The arithmetic
was right; every conclusion drawn from it was wrong, because a value count names a
POSITION and a position cannot tell cold from capped from invalid from off-the-end.
Every guard that can reject a bar now records itself - m_featureFailBlock - and the
report carries it, the series index, IndicatorDepthReport()'s per-handle depths,
and for each indicator whether the NEWEST bar reads. That last field is the whole
diagnosis in one word: newest-also-EMPTY means the buffer is unreadable everywhere
(cold or dead handle), newest-reads means a genuine history edge. Instrumented:
open, ATR, MA, RSI, MACD, Ichimoku, and all five AD blocks via ADIndicatorCold().
3. THE TOTAL-FAILURE BACKOFF WAS GATED ON THE WRONG QUESTION.
It armed only when m_featureFailTransient was set. Keeping that flag correct across
every guard is a list that has to stay right forever - the same shape of fix the
feature cache abandoned for the same reason - and the gate is pointless anyway: a
sweep where ZERO of 50,163 bars produced a window will produce zero again if it
restarts a millisecond later, transient or not. Doing that at full speed is what
starved six indicator threads on a six-core box. The backoff is now unconditional
on a total failure. The flag keeps its real job, deciding whether a MISS may be
cached, which is a per-bar question and not a scheduling one.
4. THE ERA BARRIER DEADLOCKED, AND SILENCED THE MEMBERS IT FROZE.
EnsembleMinTrainingEra() exempted deployed, stopped and paused members and its
comment concluded "so nothing deadlocks". Those three are all VOLUNTARY. A member
that simply CANNOT finish an era is none of them, so it pinned the minimum at its
own era with no time limit - and the hold branch's only action was
`m_lastEraCompleteTick = GetTickCount()`, which silences the stall watchdog. So on
USDJPY the two members that could not train reported, and the two healthy members
frozen behind them wrote nothing anywhere. The outage was visible only through the
members that were not suffering it.
- BarrierEraHeartbeat() stamps a clock on real era CHANGE, kept separate from
m_lastEraCompleteTick precisely because the barrier resets that one. Only a
member AT the minimum can be a blocker; a member ahead is idle by design and is
never counted as stuck.
- After ENSEMBLE_BARRIER_STUCK_MS (12 min) a non-advancing member is dropped from
the barrier minimum. It keeps training and rejoins the instant it completes an
era - at which point, being behind, it legitimately becomes the minimum again,
which is the documented resumed-laggard behaviour.
- Both transitions say so loudly, and the release states plainly that the
combined-vote score cannot be computed while the ensemble is desynchronised.
- A held member now writes a rate-limited journal line naming WHICH members it is
waiting on, so the blocker is read off one line.
5. THE PANEL FLICKER.
OnTickHandler gates its terse writer on !m_trainRunActive, and a barrier-held member
returns from Train() before ever setting it - so both writers thought they were the
only one updating the label and fought every tick. That is the reported "Getting
ready..." <-> "Waiting at era N for slower ensemble members" oscillation, and it hit
Perceptron but not Convolutional purely because Convolutional had a run active from
a completed era and Perceptron, resumed from disk, never did. Train()'s message is
the specific one, so it wins.
NEXT STEP once this is running: the stall line now ends in "REJECTED BY: ..." and
the per-handle depths. Read it. Do not reason around it.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-17 15:21:15 -04:00
|
|
|
//--- closeLagRef is called out separately because it is the one term here that reads the CLOSE
|
|
|
|
|
//--- series at idx + kijunShift, so it fails on the oldest kijunShift bars by construction (see
|
|
|
|
|
//--- ResizeBuffers' clamp note) rather than because Ichimoku is unready.
|
|
|
|
|
m_featureFailBlock = StringFormat("Ichimoku - tenkan=%s kijun=%s spanA=%s spanB=%s fA=%s fB=%s"
|
|
|
|
|
" closeLag(idx+%d)=%s, newest bar %s, BarsCalculated=%d",
|
|
|
|
|
tenkan == EMPTY_VALUE ? "EMPTY" : "ok",
|
|
|
|
|
kijun == EMPTY_VALUE ? "EMPTY" : "ok",
|
|
|
|
|
spanA == EMPTY_VALUE ? "EMPTY" : "ok",
|
|
|
|
|
spanB == EMPTY_VALUE ? "EMPTY" : "ok",
|
|
|
|
|
futureSpanA == EMPTY_VALUE ? "EMPTY" : "ok",
|
|
|
|
|
futureSpanB == EMPTY_VALUE ? "EMPTY" : "ok",
|
|
|
|
|
kijunShift,
|
|
|
|
|
(closeLagRef == EMPTY_VALUE || closeLagRef <= 0.0) ? "EMPTY" : "ok",
|
|
|
|
|
m_Ichimoku.TenkanSen(0) == EMPTY_VALUE ? "ALSO EMPTY" : "reads",
|
|
|
|
|
m_Ichimoku.BarsCalculated());
|
refactor: split CExpertSignalAIBase implementation by responsibility
ExpertSignalAIBase.mqh was 8216 lines: the class declaration followed by 87
method bodies covering training, labelling, feature extraction, persistence,
chart drawing, online learning, the GA auto-tuner and inference, all in one
file. Train() alone is 1492 lines; a change to arrow drawing meant scrolling
past the era loop.
Moved the bodies into Expert\AIBase\, included at the bottom of the original
after the class declaration:
Training.mqh 1607 era loop, plateau ladder, checkpoint select, deploy
Features.mqh 1093 indicator creation + per-bar input feature vector
ChartUI.mqh 634 arrows, arrow persistence, status panel, cleanup
Persistence.mqh 492 .stats/.cfg sidecars, CPU-inference validation, copy
OnlineLearning.mqh 461 live continual learning, EMA shadow, OOS simulator
Labels.mqh 309 ZigZag pivot labels, async label-cache prebuild
AutoTune.mqh 275 genetic tuner (population, crossover, halving)
Inference.mqh 235 softmax, prior calibration, class priors
ExpertSignalAIBase.mqh 8216 -> 3131 (declaration + topology build only)
This is a pure relocation - verified mechanically, not by eye: HEAD's file
reconstructed from the eight partials plus the surviving remainder is
byte-identical to HEAD, span for span (scratchpad verify_split.py). No
declaration moved, no signature changed, no code rewritten, so behaviour is
unchanged by construction.
Compiles 0 errors, 0 warnings.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-29 00:42:45 -04:00
|
|
|
return false;
|
fix(features): a cold indicator is TRANSIENT, not a permanent miss - and back off instead of re-sweeping
Six fresh instances on USDJPY and XAUUSD swept 33,965-50,162 bars and produced
ZERO usable feature windows, repeatedly, for 40 minutes and 239 stall reports,
without ever completing era 0. The four instances already warmed up before those
charts were attached trained normally throughout.
THE STALL REPORT NAMED THE SPOT EXACTLY: 'lookback slot 0 REJECTED the bar
(window had 24 of 832 values)', and 24 is the core block to the value - 4 price +
5 swing + 4 range + 4 volume + 6 time + 1 ATR. So feature 25 was the wall, and
feature 25 is the first value of the MA block. The same 24 appeared on XAUUSD
against a 816-value window (51 features/bar vs 52), which is what ruled out any
symbol-specific data gap: the wall sits at a fixed feature index, not a date.
ADMovingAverage is a CUSTOM indicator, so MT5 fills its buffer asynchronously and
returns EMPTY_VALUE for EVERY index until it has calculated - not just the
warm-up tail. That guard did not set m_featureFailTransient, so every bar of the
sweep was cached as a PERMANENT miss. This is precisely the failure the ATR guard
twenty lines above it was fixed for on 2026-08-10; the fix was never propagated
to the indicator blocks that follow. RSI, MACD and Ichimoku had the same defect
and are fixed too. (The Donchian high/low guard is a break into a
degraded-but-usable path, not a rejection, and is deliberately left alone.)
IT ALSO SELF-SUSTAINED, which is why it never recovered. The ok=0 self-heal drops
the feature cache and re-sweeps immediately, so each stuck instance spent every
millisecond re-reading 30-50k bars - six of them at once, on a six-core box,
competing for CPU with the very indicator calculation they were all waiting on.
The recovery was preventing the recovery. A transient total failure now re-arms
m_warmupPassesRemaining, yielding the CPU for a few separately-scheduled Train()
calls - the same mechanism a fresh model already uses to let history sync finish,
pointed at indicator warm-up instead.
Verified in the terminal journal first: indicators load and unload in matched
counts and there is no OOM, so this is NOT the 33f106d handle leak.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-17 12:47:25 -04:00
|
|
|
}
|
refactor: split CExpertSignalAIBase implementation by responsibility
ExpertSignalAIBase.mqh was 8216 lines: the class declaration followed by 87
method bodies covering training, labelling, feature extraction, persistence,
chart drawing, online learning, the GA auto-tuner and inference, all in one
file. Train() alone is 1492 lines; a change to arrow drawing meant scrolling
past the era loop.
Moved the bodies into Expert\AIBase\, included at the bottom of the original
after the class declaration:
Training.mqh 1607 era loop, plateau ladder, checkpoint select, deploy
Features.mqh 1093 indicator creation + per-bar input feature vector
ChartUI.mqh 634 arrows, arrow persistence, status panel, cleanup
Persistence.mqh 492 .stats/.cfg sidecars, CPU-inference validation, copy
OnlineLearning.mqh 461 live continual learning, EMA shadow, OOS simulator
Labels.mqh 309 ZigZag pivot labels, async label-cache prebuild
AutoTune.mqh 275 genetic tuner (population, crossover, halving)
Inference.mqh 235 softmax, prior calibration, class priors
ExpertSignalAIBase.mqh 8216 -> 3131 (declaration + topology build only)
This is a pure relocation - verified mechanically, not by eye: HEAD's file
reconstructed from the eight partials plus the surviving remainder is
byte-identical to HEAD, span for span (scratchpad verify_split.py). No
declaration moved, no signature changed, no code rewritten, so behaviour is
unchanged by construction.
Compiles 0 errors, 0 warnings.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-29 00:42:45 -04:00
|
|
|
if(!TempData.Add((close - tenkan) / atr) || // distance to the fast line
|
|
|
|
|
!TempData.Add((close - kijun) / atr) || // distance to the equilibrium line
|
|
|
|
|
!TempData.Add((tenkan - kijun) / atr) || // TK spread: sign = cross state, size = conviction
|
|
|
|
|
!TempData.Add((close - spanA) / atr) || // distance to each cloud edge, so the network can
|
|
|
|
|
!TempData.Add((close - spanB) / atr) || // place price above / inside / below the cloud
|
|
|
|
|
!TempData.Add((spanA - spanB) / atr) || // signed cloud thickness here: sign = regime, size = strength
|
|
|
|
|
!TempData.Add((futureSpanA - futureSpanB) / atr) || // same for the projected cloud - the "twist" ahead
|
|
|
|
|
!TempData.Add((close - closeLagRef) / atr)) // Chikou displacement, in its lookahead-free form
|
|
|
|
|
return false;
|
|
|
|
|
}
|
|
|
|
|
if(m_useNews)
|
|
|
|
|
{
|
|
|
|
|
// Event proximity + impact only - see this member's declaration comment and
|
|
|
|
|
// System\NewsRelevance.mqh's ImpactWeightedProximity() for why the forward-looking half
|
|
|
|
|
// (searchForward=true) isn't lookahead bias despite being computed for a historical bar.
|
fix(signals): revive a dead MA model, and demote Sanyaku from state to event
Two defects surfaced by research/test_classic.py, both verified fixed by re-running the
transcription against 178k bars of EURUSD H1.
CSignalMA model 1 could never fire. For any recursive average - and MA_TYPE_EMA is the
shipped default - MA(i) = a*Close(i) + (1-a)*MA(i+1), so
DiffMA(i) = a * (Close(i) - MA(i+1))
DiffCloseMA(i) = (1-a) * (Close(i) - MA(i+1))
are positive multiples of one quantity and always share a sign. Model 1 asks for a close
BELOW a RISING average, which is precisely the combination that identity forbids: 0.000%
of bars, either direction, any symbol. The MQL5 standard library this was ported from
defaults to MODE_SMA, where the two are merely correlated - the bug arrived with the EMA
default, not with the port. Reading the slope one bar back (DiffMAPrev) breaks the tie for
every MA type while keeping the model's stated meaning. Now fires on 7.92% of bars.
CSignalIchimoku model 11 fired on 27% of bars at weight 100. Sanyaku is three standing
STATES conjoined with no transition term, so it held across long stretches - and being
last in the if-chain at the top weight, the module's highest-conviction reading was also
its most common one, overwriting all eight event models below it on a quarter of all bars.
The old comment rejected an event form because "demanding all three flip on the same bar
would fire almost never" - true, but that is not the alternative. Kouten is the TURN: the
ALIGNMENT transitions, and only one role need change for it to. Testing !Sanyaku(idx+1)
fires once per aligned stretch. Now 2.17%, in line with Kumo breakout (2.4%) and the
strong TK cross (1.1%). DataReady() extended one bar deeper to cover the lookback.
Neither pattern showed edge before or after; this is about the models meaning what they
say and the vote not being dominated by a constant.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-01 17:14:34 -04:00
|
|
|
// What is DELIBERATELY not here is actual-vs-forecast surprise. Release TIMES are published
|
|
|
|
|
// in advance and never revised, so reading them for a historical bar is legitimate; released
|
|
|
|
|
// VALUES are neither. MqlCalendarValue.actual_value returns the FINAL figure, and the calendar
|
|
|
|
|
// keeps no as-of-release snapshot (revised_prev_value exists precisely because revisions
|
|
|
|
|
// happen), so a surprise feature computed for a 2019 bar would be built from a number nobody
|
|
|
|
|
// had in 2019. That is the same class of leak that made the RSI/MACD divergence models read
|
|
|
|
|
// +4 sigma in research/test_classic.py until two bars of lookahead were closed - except this
|
|
|
|
|
// one would survive into production and be paid for in real money.
|
refactor: split CExpertSignalAIBase implementation by responsibility
ExpertSignalAIBase.mqh was 8216 lines: the class declaration followed by 87
method bodies covering training, labelling, feature extraction, persistence,
chart drawing, online learning, the GA auto-tuner and inference, all in one
file. Train() alone is 1492 lines; a change to arrow drawing meant scrolling
past the era loop.
Moved the bodies into Expert\AIBase\, included at the bottom of the original
after the class declaration:
Training.mqh 1607 era loop, plateau ladder, checkpoint select, deploy
Features.mqh 1093 indicator creation + per-bar input feature vector
ChartUI.mqh 634 arrows, arrow persistence, status panel, cleanup
Persistence.mqh 492 .stats/.cfg sidecars, CPU-inference validation, copy
OnlineLearning.mqh 461 live continual learning, EMA shadow, OOS simulator
Labels.mqh 309 ZigZag pivot labels, async label-cache prebuild
AutoTune.mqh 275 genetic tuner (population, crossover, halving)
Inference.mqh 235 softmax, prior calibration, class priors
ExpertSignalAIBase.mqh 8216 -> 3131 (declaration + topology build only)
This is a pure relocation - verified mechanically, not by eye: HEAD's file
reconstructed from the eight partials plus the surviving remainder is
byte-identical to HEAD, span for span (scratchpad verify_split.py). No
declaration moved, no signature changed, no code rewritten, so behaviour is
unchanged by construction.
Compiles 0 errors, 0 warnings.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-29 00:42:45 -04:00
|
|
|
datetime barTime = m_Time.GetData(idx);
|
|
|
|
|
double newsRecency = ImpactWeightedProximity(m_symbol.Name(), barTime, m_newsFeatureWindowMinutes, false);
|
|
|
|
|
double newsProximity = ImpactWeightedProximity(m_symbol.Name(), barTime, m_newsFeatureWindowMinutes, true);
|
|
|
|
|
if(!TempData.Add(newsRecency) || !TempData.Add(newsProximity))
|
|
|
|
|
return false;
|
|
|
|
|
}
|
feat(ai): spread as a volatility-regime feature, and fix a stale-index cache in both new blocks
Adds spread/ATR and the spread change ratio as network inputs (EnableSpreadFeature,
default on). Spread is the one microstructure channel that is both FX-available and
genuinely historical in the Strategy Tester - "during testing, the spread is not modeled
but is taken from historical data" - so unlike swap, signed tick flow or depth of market it
is something a backtest can honestly validate.
What it encodes, stated precisely because the raw measurement overstates it.
research/test_spread.py found spr/atr the strongest single feature in this codebase, on 5
of 8 instrument/geometry cells at 2-4x any volume feature. But the barrier LABEL charges
the spread inside its own barriers, so a wide-spread bar is mechanically likelier to
resolve as a loss and the feature would partly be predicting its own cost model. Relabelling
at zero cost and re-measuring the identical feature showed 20-40% of it WAS that tautology
and the majority was not (XAUUSD retained 97%). What survives is a volatility-regime
reading: spread is near-fixed while ATR is not, so the ratio runs high exactly when
realised volatility is below its own ATR estimate, which genuinely predicts whether
ATR-scaled barriers get reached. It is UNSIGNED - Neutral-vs-directional only, never a side.
Also fixes a stale-index bug I introduced with the cross-asset panel and had just repeated
in the spread series. Both cached on length alone:
if(m_crossAsset.Bars() >= bars) return true;
MQL5 series indices are relative to NOW, so one new closed candle shifts every index by
one. Keyed only on length, the panel keeps serving its index 0 as a bar that is no longer
the newest, and every cross-asset value is read one bar out of step with the price features
sitting beside it in the same vector - silently, with no error and no shape change. This is
the same class of defect as the dtStudied watermark behind the zero-direction backtests.
Both now carry a datetime anchor on m_Time.GetData(0), the same invalidation key the
label/feature bar caches already use.
And a performance fix that fell out of it: with correct invalidation the panel rebuilds on
every new bar, and RefreshConvergedSignal runs per bar - which in the tester would mean one
full multi-symbol resample per simulated bar at training depth. Inference only reads bars
0..m_historyBars-1 plus the panel's own slow window, so it now requests exactly that. The
cache check is >=, so a deeper panel left from training still satisfies it.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-01 17:42:40 -04:00
|
|
|
if(m_useSpreadFeature)
|
|
|
|
|
{
|
|
|
|
|
// TWO values. What this block actually encodes is worth stating precisely, because the raw
|
|
|
|
|
// measurement overstates it.
|
|
|
|
|
//
|
|
|
|
|
// spr/atr measured as the single strongest feature in this codebase (research/test_spread.py):
|
|
|
|
|
// significant on 5 of 8 instrument/geometry cells at 2-4x any volume feature. But the barrier
|
|
|
|
|
// LABEL is computed with the spread charged inside it, so a high-spread bar has its barriers
|
|
|
|
|
// shifted adversely and is mechanically likelier to resolve as a loss - the feature would
|
|
|
|
|
// partly be predicting its own cost model, which is not tradeable. Re-labelling at zero cost
|
|
|
|
|
// and re-measuring the identical feature showed 20-40% of the signal WAS that tautology and
|
|
|
|
|
// the majority was not (XAUUSD kept 97%).
|
|
|
|
|
//
|
|
|
|
|
// What survives is a VOLATILITY-REGIME reading: the spread is near-fixed while ATR is not, so
|
|
|
|
|
// this ratio runs high exactly when realised volatility is below its own ATR estimate - which
|
|
|
|
|
// genuinely predicts whether ATR-scaled barriers get reached at all. Note it is UNSIGNED, like
|
|
|
|
|
// volume: it informs Neutral-vs-directional and can never pick a side.
|
|
|
|
|
double sprRatio = 0.0, sprChange = 0.0;
|
|
|
|
|
if(idx + 1 < m_spreadSeriesBars)
|
|
|
|
|
{
|
|
|
|
|
double sNow = (double)m_spreadSeries[idx] * m_symbol.Point();
|
|
|
|
|
double sPrev = (double)m_spreadSeries[idx + 1] * m_symbol.Point();
|
|
|
|
|
sprRatio = sNow / atr;
|
|
|
|
|
if(sPrev > 0.0)
|
|
|
|
|
sprChange = (sNow - sPrev) / sPrev;
|
|
|
|
|
}
|
|
|
|
|
if(!TempData.Add(MathMax(0.0, MathMin(5.0, sprRatio))) ||
|
|
|
|
|
!TempData.Add(MathMax(-5.0, MathMin(5.0, sprChange))))
|
|
|
|
|
return false;
|
|
|
|
|
}
|
fix(signals): revive a dead MA model, and demote Sanyaku from state to event
Two defects surfaced by research/test_classic.py, both verified fixed by re-running the
transcription against 178k bars of EURUSD H1.
CSignalMA model 1 could never fire. For any recursive average - and MA_TYPE_EMA is the
shipped default - MA(i) = a*Close(i) + (1-a)*MA(i+1), so
DiffMA(i) = a * (Close(i) - MA(i+1))
DiffCloseMA(i) = (1-a) * (Close(i) - MA(i+1))
are positive multiples of one quantity and always share a sign. Model 1 asks for a close
BELOW a RISING average, which is precisely the combination that identity forbids: 0.000%
of bars, either direction, any symbol. The MQL5 standard library this was ported from
defaults to MODE_SMA, where the two are merely correlated - the bug arrived with the EMA
default, not with the port. Reading the slope one bar back (DiffMAPrev) breaks the tie for
every MA type while keeping the model's stated meaning. Now fires on 7.92% of bars.
CSignalIchimoku model 11 fired on 27% of bars at weight 100. Sanyaku is three standing
STATES conjoined with no transition term, so it held across long stretches - and being
last in the if-chain at the top weight, the module's highest-conviction reading was also
its most common one, overwriting all eight event models below it on a quarter of all bars.
The old comment rejected an event form because "demanding all three flip on the same bar
would fire almost never" - true, but that is not the alternative. Kouten is the TURN: the
ALIGNMENT transitions, and only one role need change for it to. Testing !Sanyaku(idx+1)
fires once per aligned stretch. Now 2.17%, in line with Kumo breakout (2.4%) and the
strong TK cross (1.1%). DataReady() extended one bar deeper to cover the lookback.
Neither pattern showed edge before or after; this is about the models meaning what they
say and the vote not being dominated by a constant.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-01 17:14:34 -04:00
|
|
|
if(m_useCrossAsset)
|
|
|
|
|
{
|
|
|
|
|
// What every OTHER instrument was doing at this bar's timestamp - the one feature block here
|
|
|
|
|
// that is not a function of this symbol's own series. See System\CrossAsset.mqh.
|
|
|
|
|
// A panel that failed to build (no Market Watch pairs, unsynchronised history) yields a
|
|
|
|
|
// neutral 0-fill rather than rejecting the bar: the block is additive context, and losing
|
|
|
|
|
// every bar of training because a reference symbol was missing would be a far worse failure
|
|
|
|
|
// than training without the context. Features() reports that by returning false, which is
|
|
|
|
|
// logged once at build time rather than per bar.
|
|
|
|
|
double xa[];
|
|
|
|
|
m_crossAsset.Features(idx, xa);
|
|
|
|
|
for(int k = 0; k < CROSSASSET_FEATURES; k++)
|
|
|
|
|
if(!TempData.Add(xa[k]))
|
|
|
|
|
return false;
|
|
|
|
|
}
|
refactor: split CExpertSignalAIBase implementation by responsibility
ExpertSignalAIBase.mqh was 8216 lines: the class declaration followed by 87
method bodies covering training, labelling, feature extraction, persistence,
chart drawing, online learning, the GA auto-tuner and inference, all in one
file. Train() alone is 1492 lines; a change to arrow drawing meant scrolling
past the era loop.
Moved the bodies into Expert\AIBase\, included at the bottom of the original
after the class declaration:
Training.mqh 1607 era loop, plateau ladder, checkpoint select, deploy
Features.mqh 1093 indicator creation + per-bar input feature vector
ChartUI.mqh 634 arrows, arrow persistence, status panel, cleanup
Persistence.mqh 492 .stats/.cfg sidecars, CPU-inference validation, copy
OnlineLearning.mqh 461 live continual learning, EMA shadow, OOS simulator
Labels.mqh 309 ZigZag pivot labels, async label-cache prebuild
AutoTune.mqh 275 genetic tuner (population, crossover, halving)
Inference.mqh 235 softmax, prior calibration, class priors
ExpertSignalAIBase.mqh 8216 -> 3131 (declaration + topology build only)
This is a pure relocation - verified mechanically, not by eye: HEAD's file
reconstructed from the eight partials plus the surviving remainder is
byte-identical to HEAD, span for span (scratchpad verify_split.py). No
declaration moved, no signature changed, no code rewritten, so behaviour is
unchanged by construction.
Compiles 0 errors, 0 warnings.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-29 00:42:45 -04:00
|
|
|
if(m_useADCumulativeDelta)
|
|
|
|
|
{
|
fix: the trailing incumbent read the future across eras; cold AD blocks cached zeros as truth
Three findings from the 2026-08-11 audit:
1. The excursion head's trailing-quantile ring was deliberately never cleared
between eras ("a rolling estimate of the market, not of the era") - but
pass 3 re-walks the SAME OOS window every era, so at each walk's restart
the ring still held the outcome masks of the newest OOS bars from the
previous walk: the chronological FUTURE of the bars about to be scored.
For the first ~window+horizon pushes of every era the "trailing" incumbent
was partly a leading one - conservative for the gate (an informed incumbent
is a harder hurdle) but exactly the self-made-artifact class 06d4785 hunts.
The ring now clears at era-score reset; the warm-up bars simply don't score
the trail race, which the m_excTrailN gating already accounts for.
2. skillTrail compared the head's FULL-block Brier (pro-rated by coverage)
against the incumbent's subset sum - valid only if head skill is uniform
across the OOS walk, while the trail-scored subset systematically excludes
each era's warm-up bars. The audit also found m_excBrierHeadD/BaseD/
m_excOosHitsD declared, zeroed and never accumulated (dead since e2c9593
made every scored bar disjoint). The dead trio is replaced by
m_excBrierHeadT: the head's Brier accumulated only on the bars the warm
incumbent also scored, so the race now compares both predictors on an
identical bar set.
3. The AD/Wyckoff feature blocks read GetData with no EMPTY_VALUE guard; a
cold (still-calculating) indicator returns EMPTY_VALUE everywhere, the
sanitize loop rewrote that to 0.0, and the bar SUCCEEDED - so
BufferTempData cached an all-zero Wyckoff block as a success for the whole
bar frame: the one path the f6150ee only-cache-successes rule cannot see,
because it never fails (the ba13eef class, arriving through values that
never fail; a resumed model's era-0 prebuild starts milliseconds after
OnInit). ADIndicatorCold() probes the NEWEST bar - EMPTY_VALUE there means
async warm-up (transient reject, retried), while deep bars beyond the
buffered depth keep the sanitize loop's neutral-fill so degraded history
still trains. Also fixed m_featureCacheValid's declaration comment, which
still described the pre-f6150ee cached-miss semantics.
Compile: 0 errors, 0 warnings.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-11 18:20:24 -04:00
|
|
|
//--- COLD IS TRANSIENT, NOT ZERO (2026-08-11). These raw GetData reads have no EMPTY_VALUE
|
|
|
|
|
//--- guard of their own; a not-yet-calculated indicator returns EMPTY_VALUE for EVERY index,
|
|
|
|
|
//--- the sanitize loop at the bottom rewrites that to 0.0, the bar then SUCCEEDS - and
|
|
|
|
|
//--- BufferTempData caches it as a success for the whole bar frame. That is the one path the
|
|
|
|
|
//--- f6150ee only-cache-successes rule cannot see, because it never fails: on a resumed model
|
|
|
|
|
//--- the era-0 prebuild/MI report start milliseconds after OnInit and could train on all-zero
|
|
|
|
|
//--- Wyckoff/AD blocks for up to a full bar (the ba13eef failure class, arriving through
|
|
|
|
|
//--- values that never fail). ADIndicatorCold probes the NEWEST bar: EMPTY_VALUE there means
|
|
|
|
|
//--- the async calculation hasn't filled yet -> transient reject (never cached, retried like
|
|
|
|
|
//--- the cold-ATR guard above). A warm indicator whose DEEP bars read EMPTY_VALUE (beyond its
|
|
|
|
|
//--- buffer depth) is different - that stays the sanitize loop's neutral-fill, since rejecting
|
|
|
|
|
//--- those bars would starve training of legitimately degraded history.
|
fix(ensemble+depth): the barrier had no liveness escape, and the depth gate could not report the one state the evidence pointed at
Two charts (USDJPY 50,179 bars / XAUUSD 33,982) sat at era 0 for 38 minutes with
four of their eight members completely silent. Nothing in this commit guesses at
why the sweep fails - the last five guesses were all wrong. It makes the failure
say what it is, and stops one broken member taking its whole chart down with it.
WHAT THE LOG ACTUALLY SAYS, before any of this.
- The running build IS d9f834d (pulled 14:18, compiled 14:19:01, 0 errors), so
every depth instrument from 1dda479/7e63a8b/45c9e21 was live.
- It printed NOTHING. Zero "PRIMING", zero "CAPPED", zero "Per-indicator depth"
in 27 MB of journal. The instrument built to find the depth shortfall returned
"not this".
- On USDJPY at 14:24, CONV-cad8 completed eras 0 AND 1 across all 50,179 bars -
same chart, same 832-value window, same indicators, byte-identical fingerprint -
while LSTM-cad8 and HYB-cad8 reported ok=0 failed=50163. So it is not the
symbol, the history, the bar count or the indicator depth. It is per-member.
- ok=0 means the NEWEST anchors failed too, and a short indicator cannot do that.
The depth reading in project_silent_block_failures is therefore retired by its own
instrumentation. THE ROOT CAUSE IS STILL UNKNOWN and this commit does not claim one.
1. THE DEPTH GATE'S SILENT PATH WAS THE STATE IT WAS HUNTING.
ServableBars() read `if(servable <= 0 || servable >= want) return want;` - one
branch over three unrelated states, silent in all of them:
enabled == 0 -> nothing tunable is on. No cap. Healthy.
enabled > 0, servable == -1 -> a handle answered INVALID.
enabled > 0, servable == 0 -> created, never calculated.
BarsCalculated() returns -1 for a dead handle, so a dead MA is indistinguishable
from "no tunable indicators enabled" - and both returned `want` without printing a
character. That is exactly the state a per-member, every-index, depth-independent
failure produces, and it is the single reason a build carrying full depth
instrumentation logged nothing through the whole outage.
TunableBarsCalculated() now also reports HOW MANY indicators it consulted, and the
dead-handle case is reported (latched, with per-handle depths). The RETURN is
deliberately unchanged - what to do about a dead handle is not yet known, and
changing control flow on an unproven cause is how the last four fixes here went
wrong. SettledBars() routes its three pass-through states via ServableBars() so the
report is reachable from the training sweep, which is the only caller that hits it.
2. THE STALL REPORT NAMED A SLOT, NEVER A BLOCK.
"lookback slot 0 REJECTED (window had 24 of 832 values)" plus a guess ("an
indicator warm-up or a history-edge read"). Which guard fired was INFERRED by
counting 4+5+4+4+6+1 = 24 and concluding feature 25 must be the MA. The arithmetic
was right; every conclusion drawn from it was wrong, because a value count names a
POSITION and a position cannot tell cold from capped from invalid from off-the-end.
Every guard that can reject a bar now records itself - m_featureFailBlock - and the
report carries it, the series index, IndicatorDepthReport()'s per-handle depths,
and for each indicator whether the NEWEST bar reads. That last field is the whole
diagnosis in one word: newest-also-EMPTY means the buffer is unreadable everywhere
(cold or dead handle), newest-reads means a genuine history edge. Instrumented:
open, ATR, MA, RSI, MACD, Ichimoku, and all five AD blocks via ADIndicatorCold().
3. THE TOTAL-FAILURE BACKOFF WAS GATED ON THE WRONG QUESTION.
It armed only when m_featureFailTransient was set. Keeping that flag correct across
every guard is a list that has to stay right forever - the same shape of fix the
feature cache abandoned for the same reason - and the gate is pointless anyway: a
sweep where ZERO of 50,163 bars produced a window will produce zero again if it
restarts a millisecond later, transient or not. Doing that at full speed is what
starved six indicator threads on a six-core box. The backoff is now unconditional
on a total failure. The flag keeps its real job, deciding whether a MISS may be
cached, which is a per-bar question and not a scheduling one.
4. THE ERA BARRIER DEADLOCKED, AND SILENCED THE MEMBERS IT FROZE.
EnsembleMinTrainingEra() exempted deployed, stopped and paused members and its
comment concluded "so nothing deadlocks". Those three are all VOLUNTARY. A member
that simply CANNOT finish an era is none of them, so it pinned the minimum at its
own era with no time limit - and the hold branch's only action was
`m_lastEraCompleteTick = GetTickCount()`, which silences the stall watchdog. So on
USDJPY the two members that could not train reported, and the two healthy members
frozen behind them wrote nothing anywhere. The outage was visible only through the
members that were not suffering it.
- BarrierEraHeartbeat() stamps a clock on real era CHANGE, kept separate from
m_lastEraCompleteTick precisely because the barrier resets that one. Only a
member AT the minimum can be a blocker; a member ahead is idle by design and is
never counted as stuck.
- After ENSEMBLE_BARRIER_STUCK_MS (12 min) a non-advancing member is dropped from
the barrier minimum. It keeps training and rejoins the instant it completes an
era - at which point, being behind, it legitimately becomes the minimum again,
which is the documented resumed-laggard behaviour.
- Both transitions say so loudly, and the release states plainly that the
combined-vote score cannot be computed while the ensemble is desynchronised.
- A held member now writes a rate-limited journal line naming WHICH members it is
waiting on, so the blocker is read off one line.
5. THE PANEL FLICKER.
OnTickHandler gates its terse writer on !m_trainRunActive, and a barrier-held member
returns from Train() before ever setting it - so both writers thought they were the
only one updating the label and fought every tick. That is the reported "Getting
ready..." <-> "Waiting at era N for slower ensemble members" oscillation, and it hit
Perceptron but not Convolutional purely because Convolutional had a run active from
a completed era and Perceptron, resumed from disk, never did. Train()'s message is
the specific one, so it wins.
NEXT STEP once this is running: the stall line now ends in "REJECTED BY: ..." and
the per-handle depths. Read it. Do not reason around it.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-17 15:21:15 -04:00
|
|
|
if(ADIndicatorCold(m_ADCumulativeDelta, "ADCumulativeDelta"))
|
fix: the trailing incumbent read the future across eras; cold AD blocks cached zeros as truth
Three findings from the 2026-08-11 audit:
1. The excursion head's trailing-quantile ring was deliberately never cleared
between eras ("a rolling estimate of the market, not of the era") - but
pass 3 re-walks the SAME OOS window every era, so at each walk's restart
the ring still held the outcome masks of the newest OOS bars from the
previous walk: the chronological FUTURE of the bars about to be scored.
For the first ~window+horizon pushes of every era the "trailing" incumbent
was partly a leading one - conservative for the gate (an informed incumbent
is a harder hurdle) but exactly the self-made-artifact class 06d4785 hunts.
The ring now clears at era-score reset; the warm-up bars simply don't score
the trail race, which the m_excTrailN gating already accounts for.
2. skillTrail compared the head's FULL-block Brier (pro-rated by coverage)
against the incumbent's subset sum - valid only if head skill is uniform
across the OOS walk, while the trail-scored subset systematically excludes
each era's warm-up bars. The audit also found m_excBrierHeadD/BaseD/
m_excOosHitsD declared, zeroed and never accumulated (dead since e2c9593
made every scored bar disjoint). The dead trio is replaced by
m_excBrierHeadT: the head's Brier accumulated only on the bars the warm
incumbent also scored, so the race now compares both predictors on an
identical bar set.
3. The AD/Wyckoff feature blocks read GetData with no EMPTY_VALUE guard; a
cold (still-calculating) indicator returns EMPTY_VALUE everywhere, the
sanitize loop rewrote that to 0.0, and the bar SUCCEEDED - so
BufferTempData cached an all-zero Wyckoff block as a success for the whole
bar frame: the one path the f6150ee only-cache-successes rule cannot see,
because it never fails (the ba13eef class, arriving through values that
never fail; a resumed model's era-0 prebuild starts milliseconds after
OnInit). ADIndicatorCold() probes the NEWEST bar - EMPTY_VALUE there means
async warm-up (transient reject, retried), while deep bars beyond the
buffered depth keep the sanitize loop's neutral-fill so degraded history
still trains. Also fixed m_featureCacheValid's declaration comment, which
still described the pre-f6150ee cached-miss semantics.
Compile: 0 errors, 0 warnings.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-11 18:20:24 -04:00
|
|
|
return false;
|
refactor: split CExpertSignalAIBase implementation by responsibility
ExpertSignalAIBase.mqh was 8216 lines: the class declaration followed by 87
method bodies covering training, labelling, feature extraction, persistence,
chart drawing, online learning, the GA auto-tuner and inference, all in one
file. Train() alone is 1492 lines; a change to arrow drawing meant scrolling
past the era loop.
Moved the bodies into Expert\AIBase\, included at the bottom of the original
after the class declaration:
Training.mqh 1607 era loop, plateau ladder, checkpoint select, deploy
Features.mqh 1093 indicator creation + per-bar input feature vector
ChartUI.mqh 634 arrows, arrow persistence, status panel, cleanup
Persistence.mqh 492 .stats/.cfg sidecars, CPU-inference validation, copy
OnlineLearning.mqh 461 live continual learning, EMA shadow, OOS simulator
Labels.mqh 309 ZigZag pivot labels, async label-cache prebuild
AutoTune.mqh 275 genetic tuner (population, crossover, halving)
Inference.mqh 235 softmax, prior calibration, class priors
ExpertSignalAIBase.mqh 8216 -> 3131 (declaration + topology build only)
This is a pure relocation - verified mechanically, not by eye: HEAD's file
reconstructed from the eight partials plus the surviving remainder is
byte-identical to HEAD, span for span (scratchpad verify_split.py). No
declaration moved, no signature changed, no code rewritten, so behaviour is
unchanged by construction.
Compiles 0 errors, 0 warnings.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-29 00:42:45 -04:00
|
|
|
// buffers: 0=Pressure, 1=CumulativeDelta, 2=BullishPressure, 3=BearishPressure, 4=Absorption, 5=Initiative.
|
|
|
|
|
// CumulativeDelta (buffer 1) is now cumulativeDelta/sumVolume clamped +/-2 (same scale as every
|
|
|
|
|
// other buffer here, see ADCumulativeDelta.mq5) - a pure order-flow-imbalance ratio, distinct from
|
|
|
|
|
// Pressure (buffer 0), which is this same term further adjusted by Initiative/Absorption.
|
|
|
|
|
if(!TempData.Add(m_ADCumulativeDelta.GetData(0, idx)) || // Pressure
|
|
|
|
|
!TempData.Add(m_ADCumulativeDelta.GetData(1, idx)) || // CumulativeDelta
|
|
|
|
|
!TempData.Add(m_ADCumulativeDelta.GetData(2, idx)) || // BullishPressure
|
|
|
|
|
!TempData.Add(m_ADCumulativeDelta.GetData(3, idx)) || // BearishPressure
|
|
|
|
|
!TempData.Add(m_ADCumulativeDelta.GetData(4, idx)) || // Absorption
|
|
|
|
|
!TempData.Add(m_ADCumulativeDelta.GetData(5, idx))) // Initiative
|
|
|
|
|
return false;
|
|
|
|
|
}
|
|
|
|
|
if(m_useADShorteningOfThrust)
|
|
|
|
|
{
|
fix(ensemble+depth): the barrier had no liveness escape, and the depth gate could not report the one state the evidence pointed at
Two charts (USDJPY 50,179 bars / XAUUSD 33,982) sat at era 0 for 38 minutes with
four of their eight members completely silent. Nothing in this commit guesses at
why the sweep fails - the last five guesses were all wrong. It makes the failure
say what it is, and stops one broken member taking its whole chart down with it.
WHAT THE LOG ACTUALLY SAYS, before any of this.
- The running build IS d9f834d (pulled 14:18, compiled 14:19:01, 0 errors), so
every depth instrument from 1dda479/7e63a8b/45c9e21 was live.
- It printed NOTHING. Zero "PRIMING", zero "CAPPED", zero "Per-indicator depth"
in 27 MB of journal. The instrument built to find the depth shortfall returned
"not this".
- On USDJPY at 14:24, CONV-cad8 completed eras 0 AND 1 across all 50,179 bars -
same chart, same 832-value window, same indicators, byte-identical fingerprint -
while LSTM-cad8 and HYB-cad8 reported ok=0 failed=50163. So it is not the
symbol, the history, the bar count or the indicator depth. It is per-member.
- ok=0 means the NEWEST anchors failed too, and a short indicator cannot do that.
The depth reading in project_silent_block_failures is therefore retired by its own
instrumentation. THE ROOT CAUSE IS STILL UNKNOWN and this commit does not claim one.
1. THE DEPTH GATE'S SILENT PATH WAS THE STATE IT WAS HUNTING.
ServableBars() read `if(servable <= 0 || servable >= want) return want;` - one
branch over three unrelated states, silent in all of them:
enabled == 0 -> nothing tunable is on. No cap. Healthy.
enabled > 0, servable == -1 -> a handle answered INVALID.
enabled > 0, servable == 0 -> created, never calculated.
BarsCalculated() returns -1 for a dead handle, so a dead MA is indistinguishable
from "no tunable indicators enabled" - and both returned `want` without printing a
character. That is exactly the state a per-member, every-index, depth-independent
failure produces, and it is the single reason a build carrying full depth
instrumentation logged nothing through the whole outage.
TunableBarsCalculated() now also reports HOW MANY indicators it consulted, and the
dead-handle case is reported (latched, with per-handle depths). The RETURN is
deliberately unchanged - what to do about a dead handle is not yet known, and
changing control flow on an unproven cause is how the last four fixes here went
wrong. SettledBars() routes its three pass-through states via ServableBars() so the
report is reachable from the training sweep, which is the only caller that hits it.
2. THE STALL REPORT NAMED A SLOT, NEVER A BLOCK.
"lookback slot 0 REJECTED (window had 24 of 832 values)" plus a guess ("an
indicator warm-up or a history-edge read"). Which guard fired was INFERRED by
counting 4+5+4+4+6+1 = 24 and concluding feature 25 must be the MA. The arithmetic
was right; every conclusion drawn from it was wrong, because a value count names a
POSITION and a position cannot tell cold from capped from invalid from off-the-end.
Every guard that can reject a bar now records itself - m_featureFailBlock - and the
report carries it, the series index, IndicatorDepthReport()'s per-handle depths,
and for each indicator whether the NEWEST bar reads. That last field is the whole
diagnosis in one word: newest-also-EMPTY means the buffer is unreadable everywhere
(cold or dead handle), newest-reads means a genuine history edge. Instrumented:
open, ATR, MA, RSI, MACD, Ichimoku, and all five AD blocks via ADIndicatorCold().
3. THE TOTAL-FAILURE BACKOFF WAS GATED ON THE WRONG QUESTION.
It armed only when m_featureFailTransient was set. Keeping that flag correct across
every guard is a list that has to stay right forever - the same shape of fix the
feature cache abandoned for the same reason - and the gate is pointless anyway: a
sweep where ZERO of 50,163 bars produced a window will produce zero again if it
restarts a millisecond later, transient or not. Doing that at full speed is what
starved six indicator threads on a six-core box. The backoff is now unconditional
on a total failure. The flag keeps its real job, deciding whether a MISS may be
cached, which is a per-bar question and not a scheduling one.
4. THE ERA BARRIER DEADLOCKED, AND SILENCED THE MEMBERS IT FROZE.
EnsembleMinTrainingEra() exempted deployed, stopped and paused members and its
comment concluded "so nothing deadlocks". Those three are all VOLUNTARY. A member
that simply CANNOT finish an era is none of them, so it pinned the minimum at its
own era with no time limit - and the hold branch's only action was
`m_lastEraCompleteTick = GetTickCount()`, which silences the stall watchdog. So on
USDJPY the two members that could not train reported, and the two healthy members
frozen behind them wrote nothing anywhere. The outage was visible only through the
members that were not suffering it.
- BarrierEraHeartbeat() stamps a clock on real era CHANGE, kept separate from
m_lastEraCompleteTick precisely because the barrier resets that one. Only a
member AT the minimum can be a blocker; a member ahead is idle by design and is
never counted as stuck.
- After ENSEMBLE_BARRIER_STUCK_MS (12 min) a non-advancing member is dropped from
the barrier minimum. It keeps training and rejoins the instant it completes an
era - at which point, being behind, it legitimately becomes the minimum again,
which is the documented resumed-laggard behaviour.
- Both transitions say so loudly, and the release states plainly that the
combined-vote score cannot be computed while the ensemble is desynchronised.
- A held member now writes a rate-limited journal line naming WHICH members it is
waiting on, so the blocker is read off one line.
5. THE PANEL FLICKER.
OnTickHandler gates its terse writer on !m_trainRunActive, and a barrier-held member
returns from Train() before ever setting it - so both writers thought they were the
only one updating the label and fought every tick. That is the reported "Getting
ready..." <-> "Waiting at era N for slower ensemble members" oscillation, and it hit
Perceptron but not Convolutional purely because Convolutional had a run active from
a completed era and Perceptron, resumed from disk, never did. Train()'s message is
the specific one, so it wins.
NEXT STEP once this is running: the stall line now ends in "REJECTED BY: ..." and
the per-handle depths. Read it. Do not reason around it.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-17 15:21:15 -04:00
|
|
|
if(ADIndicatorCold(m_ADShorteningOfThrust, "ADShorteningOfThrust")) // see the CumulativeDelta block's comment
|
fix: the trailing incumbent read the future across eras; cold AD blocks cached zeros as truth
Three findings from the 2026-08-11 audit:
1. The excursion head's trailing-quantile ring was deliberately never cleared
between eras ("a rolling estimate of the market, not of the era") - but
pass 3 re-walks the SAME OOS window every era, so at each walk's restart
the ring still held the outcome masks of the newest OOS bars from the
previous walk: the chronological FUTURE of the bars about to be scored.
For the first ~window+horizon pushes of every era the "trailing" incumbent
was partly a leading one - conservative for the gate (an informed incumbent
is a harder hurdle) but exactly the self-made-artifact class 06d4785 hunts.
The ring now clears at era-score reset; the warm-up bars simply don't score
the trail race, which the m_excTrailN gating already accounts for.
2. skillTrail compared the head's FULL-block Brier (pro-rated by coverage)
against the incumbent's subset sum - valid only if head skill is uniform
across the OOS walk, while the trail-scored subset systematically excludes
each era's warm-up bars. The audit also found m_excBrierHeadD/BaseD/
m_excOosHitsD declared, zeroed and never accumulated (dead since e2c9593
made every scored bar disjoint). The dead trio is replaced by
m_excBrierHeadT: the head's Brier accumulated only on the bars the warm
incumbent also scored, so the race now compares both predictors on an
identical bar set.
3. The AD/Wyckoff feature blocks read GetData with no EMPTY_VALUE guard; a
cold (still-calculating) indicator returns EMPTY_VALUE everywhere, the
sanitize loop rewrote that to 0.0, and the bar SUCCEEDED - so
BufferTempData cached an all-zero Wyckoff block as a success for the whole
bar frame: the one path the f6150ee only-cache-successes rule cannot see,
because it never fails (the ba13eef class, arriving through values that
never fail; a resumed model's era-0 prebuild starts milliseconds after
OnInit). ADIndicatorCold() probes the NEWEST bar - EMPTY_VALUE there means
async warm-up (transient reject, retried), while deep bars beyond the
buffered depth keep the sanitize loop's neutral-fill so degraded history
still trains. Also fixed m_featureCacheValid's declaration comment, which
still described the pre-f6150ee cached-miss semantics.
Compile: 0 errors, 0 warnings.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-11 18:20:24 -04:00
|
|
|
return false;
|
refactor: split CExpertSignalAIBase implementation by responsibility
ExpertSignalAIBase.mqh was 8216 lines: the class declaration followed by 87
method bodies covering training, labelling, feature extraction, persistence,
chart drawing, online learning, the GA auto-tuner and inference, all in one
file. Train() alone is 1492 lines; a change to arrow drawing meant scrolling
past the era loop.
Moved the bodies into Expert\AIBase\, included at the bottom of the original
after the class declaration:
Training.mqh 1607 era loop, plateau ladder, checkpoint select, deploy
Features.mqh 1093 indicator creation + per-bar input feature vector
ChartUI.mqh 634 arrows, arrow persistence, status panel, cleanup
Persistence.mqh 492 .stats/.cfg sidecars, CPU-inference validation, copy
OnlineLearning.mqh 461 live continual learning, EMA shadow, OOS simulator
Labels.mqh 309 ZigZag pivot labels, async label-cache prebuild
AutoTune.mqh 275 genetic tuner (population, crossover, halving)
Inference.mqh 235 softmax, prior calibration, class priors
ExpertSignalAIBase.mqh 8216 -> 3131 (declaration + topology build only)
This is a pure relocation - verified mechanically, not by eye: HEAD's file
reconstructed from the eight partials plus the surviving remainder is
byte-identical to HEAD, span for span (scratchpad verify_split.py). No
declaration moved, no signature changed, no code rewritten, so behaviour is
unchanged by construction.
Compiles 0 errors, 0 warnings.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-29 00:42:45 -04:00
|
|
|
// buffers: 0=SOT, 1=SOTEffortRegime, 2=SOTConfirmation, 3=SOTPushRegime
|
|
|
|
|
if(!TempData.Add(m_ADShorteningOfThrust.GetData(0, idx)) || // SOT
|
|
|
|
|
!TempData.Add(m_ADShorteningOfThrust.GetData(1, idx)) || // SOTEffortRegime
|
|
|
|
|
!TempData.Add(m_ADShorteningOfThrust.GetData(2, idx)) || // SOTConfirmation
|
|
|
|
|
!TempData.Add(m_ADShorteningOfThrust.GetData(3, idx))) // SOTPushRegime
|
|
|
|
|
return false;
|
|
|
|
|
}
|
|
|
|
|
if(m_useADWyckoffEventStream)
|
|
|
|
|
{
|
|
|
|
|
// buffers: 0=EventCode, 1=EventPhase, 2=ZoneTop, 3=ZoneBottom, 4=EventPrice, 5=StructuralPhase,
|
|
|
|
|
// 6=CHoCHTrendToRange, 7=CHoCHRangeToTrend, 8=SlopeAccumulationBullish, 9=SlopeAccumulationBearish,
|
|
|
|
|
// 10=SlopeDistributionBullish, 11=SlopeDistributionBearish, 12=Reaccumulation, 13=Redistribution.
|
|
|
|
|
// Buffer 4 (EventPrice) is deliberately skipped below - per the indicator's own source
|
|
|
|
|
// (ADWyckoffEventStream.mq5: "BufColor[wi]=(ev!=0)?C[i]:0;"), it's just this bar's close price
|
|
|
|
|
// echoed back when an event fires (0 otherwise), kept only so a charting/backtesting tool like
|
|
|
|
|
// StrategyQuant can anchor an arrow to a price. It carries no information the network doesn't
|
|
|
|
|
// already have (EventCode already flags whether an event fired; the close is already in the
|
|
|
|
|
// base OHLC features), and normalizing it as a "distance from close" like ZoneTop/ZoneBottom
|
|
|
|
|
// below would be actively wrong: it's close-close=0 on event bars but 0-close=-close (a raw,
|
|
|
|
|
// ATR-blown-up price) on every other bar - a huge, meaningless outlier feature.
|
2026-08-02 17:08:48 -04:00
|
|
|
// Buffer 1 (EventPhase) USED to be skipped for the same kind of reason - it was written as
|
|
|
|
|
// "BufPhase[wi]=(double)ev;", a byte-for-byte copy of EventCode. The 2026-08-02 rewrite made it a
|
|
|
|
|
// real reading: "BufPhase[wi]=(double)(phaseNow*((dirNow>=0)?1:-1))", i.e. the live range's own
|
|
|
|
|
// Wyckoff phase 1..5 signed by whether that range is accumulation (+) or distribution (-). That is
|
|
|
|
|
// NOT what StructuralPhase (buffer 5) carries: StructuralPhase is derived from the EVENT on this
|
|
|
|
|
// bar (MapStructuralPhase(ev)) and so is 0 on every bar where nothing fires, while EventPhase
|
|
|
|
|
// persists for the whole life of the range. The pair gives the network both "an event just put us
|
|
|
|
|
// in phase C" and "we are still in phase C" - so it is included below.
|
feat: mini-batch gradient accumulation (F4), front-end-aware capacity budget (F6), split Wyckoff categoricals (N1)
Completes the 2026-08-09 training audit. FORCES A RETRAIN of every
Wyckoff-enabled config (N1 re-keys the fingerprint), and BOTH DLLs must be
redeployed alongside the .ex5 - they carry new exports.
F4 - mini-batch accumulation, TRAIN_BATCH_SIZE=32. Training was pure online
SGD (one weight update per bar), which is the mechanical source of the
era-to-era whipsaw every downstream guard was built to cope with. The O(n^2)
outer product is native - AccumulateWeightGrad / AccumulateWeightGradConv /
AccumulateBufferInto in Network.cl, WarriorCPU and WarriorDML - while the
optimizer step is host-side MQL5 shared by all tiers (ApplyAccumToBlock), so
there is one Adam/SGD implementation instead of four that can drift.
- the LSTM needs no outer-product kernel (WeightsGradient already holds the
sample's full dW) but could NOT simply be left un-zeroed between samples:
CPU_LSTMSeqBackward/DML_LSTMSeqBackward memset it on entry. Hence a
separate accumulator plus an elementwise add.
- batch-norm gamma/beta accumulate in host arrays, not new BatchOptions
slots - BN_OPT_STRIDE is baked into every persisted .nnw.
- scoped to pass 2; online learning keeps immediate updates. Every save /
checkpoint / scoring boundary flushes, scaling by the real sample count.
- degrades to per-sample updates (one log line) on a tier that cannot
accumulate, so old devices and DLL-free builds are unaffected.
- verified offline: DirectML/batch_accum_check.cpp drives the real exports
against an independent reference; at B=1 the accumulator matches the
shipped unbatched kernel's own gradient to 1.1e-16. Math only - the
in-situ check remains the per-layer dW/W report on a real era.
F6 - ComputeFirstLayerWidth budgeted against the RAW input width even where a
conv/LSTM front end had already reduced it, so an LSTM's dense stack was
charged for 1,280 inputs when it receives 64. Confirmed from the deployed
.cfg files: CONV, LSTM and HYBRID were all pinned at the 16-unit floor. Now
budgeted against the front-end output and capped at it (never fan out), with
the derivation reordered so both stages settle first.
N1 - EventCode/EventPhase/StructuralPhase are signed categoricals packing
direction and Wyckoff stage into one scalar across a sign discontinuity. Split
into direction + [0,1] magnitude, the same convention the base OHLC block uses.
Information-preserving; 13 readings now occupy 16 inputs.
Compiled clean (0 errors, 0 warnings); both DLLs rebuilt.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-09 11:48:03 -04:00
|
|
|
// SIGN AND MAGNITUDE SPLIT (2026-08-09 audit, N1). All three of these buffers are signed
|
|
|
|
|
// categoricals of the form (stage * direction), packed into one scalar:
|
|
|
|
|
// EventCode +-1..7 sign = accumulation/distribution, |v| = the Wyckoff schematic stage
|
|
|
|
|
// (1 PS, 2 SC, 3 AR, 4 ST, 5 Spring/UTAD, 6 LPS/LPSY, 7 SOS/SOW)
|
|
|
|
|
// EventPhase +-1..5 the LIVE range's phase, same sign convention, persists between events
|
|
|
|
|
// StructuralPhase +-1..5 this bar's event mapped to a phase, 0 when nothing fired
|
|
|
|
|
// Fed raw, each one asks the network to disentangle "which way" from "how far through the
|
|
|
|
|
// schematic" out of a single continuous value - and to do it across a sign change, where the
|
|
|
|
|
// ordinal jumps from -1 to +1 with nothing in between. That is the exact ambiguity the base OHLC
|
|
|
|
|
// block calls out and fixes by handing direction its own +1/-1/0 flag beside (close-open)/atr;
|
|
|
|
|
// these are the same shape of value and get the same treatment.
|
|
|
|
|
// Information-preserving: (dir, mag) reconstructs the original exactly, so this is a re-encoding
|
|
|
|
|
// and not a feature change. Magnitudes are scaled onto [0,1] by their own maxima so they sit in
|
|
|
|
|
// the same range as the rest of the vector instead of reaching 7.
|
|
|
|
|
// The magnitudes ARE meaningfully ordinal - the schematic is a sequence, not a set of unrelated
|
|
|
|
|
// labels - which is why they stay scalars rather than being one-hot expanded across 7 inputs.
|
fix(ensemble+depth): the barrier had no liveness escape, and the depth gate could not report the one state the evidence pointed at
Two charts (USDJPY 50,179 bars / XAUUSD 33,982) sat at era 0 for 38 minutes with
four of their eight members completely silent. Nothing in this commit guesses at
why the sweep fails - the last five guesses were all wrong. It makes the failure
say what it is, and stops one broken member taking its whole chart down with it.
WHAT THE LOG ACTUALLY SAYS, before any of this.
- The running build IS d9f834d (pulled 14:18, compiled 14:19:01, 0 errors), so
every depth instrument from 1dda479/7e63a8b/45c9e21 was live.
- It printed NOTHING. Zero "PRIMING", zero "CAPPED", zero "Per-indicator depth"
in 27 MB of journal. The instrument built to find the depth shortfall returned
"not this".
- On USDJPY at 14:24, CONV-cad8 completed eras 0 AND 1 across all 50,179 bars -
same chart, same 832-value window, same indicators, byte-identical fingerprint -
while LSTM-cad8 and HYB-cad8 reported ok=0 failed=50163. So it is not the
symbol, the history, the bar count or the indicator depth. It is per-member.
- ok=0 means the NEWEST anchors failed too, and a short indicator cannot do that.
The depth reading in project_silent_block_failures is therefore retired by its own
instrumentation. THE ROOT CAUSE IS STILL UNKNOWN and this commit does not claim one.
1. THE DEPTH GATE'S SILENT PATH WAS THE STATE IT WAS HUNTING.
ServableBars() read `if(servable <= 0 || servable >= want) return want;` - one
branch over three unrelated states, silent in all of them:
enabled == 0 -> nothing tunable is on. No cap. Healthy.
enabled > 0, servable == -1 -> a handle answered INVALID.
enabled > 0, servable == 0 -> created, never calculated.
BarsCalculated() returns -1 for a dead handle, so a dead MA is indistinguishable
from "no tunable indicators enabled" - and both returned `want` without printing a
character. That is exactly the state a per-member, every-index, depth-independent
failure produces, and it is the single reason a build carrying full depth
instrumentation logged nothing through the whole outage.
TunableBarsCalculated() now also reports HOW MANY indicators it consulted, and the
dead-handle case is reported (latched, with per-handle depths). The RETURN is
deliberately unchanged - what to do about a dead handle is not yet known, and
changing control flow on an unproven cause is how the last four fixes here went
wrong. SettledBars() routes its three pass-through states via ServableBars() so the
report is reachable from the training sweep, which is the only caller that hits it.
2. THE STALL REPORT NAMED A SLOT, NEVER A BLOCK.
"lookback slot 0 REJECTED (window had 24 of 832 values)" plus a guess ("an
indicator warm-up or a history-edge read"). Which guard fired was INFERRED by
counting 4+5+4+4+6+1 = 24 and concluding feature 25 must be the MA. The arithmetic
was right; every conclusion drawn from it was wrong, because a value count names a
POSITION and a position cannot tell cold from capped from invalid from off-the-end.
Every guard that can reject a bar now records itself - m_featureFailBlock - and the
report carries it, the series index, IndicatorDepthReport()'s per-handle depths,
and for each indicator whether the NEWEST bar reads. That last field is the whole
diagnosis in one word: newest-also-EMPTY means the buffer is unreadable everywhere
(cold or dead handle), newest-reads means a genuine history edge. Instrumented:
open, ATR, MA, RSI, MACD, Ichimoku, and all five AD blocks via ADIndicatorCold().
3. THE TOTAL-FAILURE BACKOFF WAS GATED ON THE WRONG QUESTION.
It armed only when m_featureFailTransient was set. Keeping that flag correct across
every guard is a list that has to stay right forever - the same shape of fix the
feature cache abandoned for the same reason - and the gate is pointless anyway: a
sweep where ZERO of 50,163 bars produced a window will produce zero again if it
restarts a millisecond later, transient or not. Doing that at full speed is what
starved six indicator threads on a six-core box. The backoff is now unconditional
on a total failure. The flag keeps its real job, deciding whether a MISS may be
cached, which is a per-bar question and not a scheduling one.
4. THE ERA BARRIER DEADLOCKED, AND SILENCED THE MEMBERS IT FROZE.
EnsembleMinTrainingEra() exempted deployed, stopped and paused members and its
comment concluded "so nothing deadlocks". Those three are all VOLUNTARY. A member
that simply CANNOT finish an era is none of them, so it pinned the minimum at its
own era with no time limit - and the hold branch's only action was
`m_lastEraCompleteTick = GetTickCount()`, which silences the stall watchdog. So on
USDJPY the two members that could not train reported, and the two healthy members
frozen behind them wrote nothing anywhere. The outage was visible only through the
members that were not suffering it.
- BarrierEraHeartbeat() stamps a clock on real era CHANGE, kept separate from
m_lastEraCompleteTick precisely because the barrier resets that one. Only a
member AT the minimum can be a blocker; a member ahead is idle by design and is
never counted as stuck.
- After ENSEMBLE_BARRIER_STUCK_MS (12 min) a non-advancing member is dropped from
the barrier minimum. It keeps training and rejoins the instant it completes an
era - at which point, being behind, it legitimately becomes the minimum again,
which is the documented resumed-laggard behaviour.
- Both transitions say so loudly, and the release states plainly that the
combined-vote score cannot be computed while the ensemble is desynchronised.
- A held member now writes a rate-limited journal line naming WHICH members it is
waiting on, so the blocker is read off one line.
5. THE PANEL FLICKER.
OnTickHandler gates its terse writer on !m_trainRunActive, and a barrier-held member
returns from Train() before ever setting it - so both writers thought they were the
only one updating the label and fought every tick. That is the reported "Getting
ready..." <-> "Waiting at era N for slower ensemble members" oscillation, and it hit
Perceptron but not Convolutional purely because Convolutional had a run active from
a completed era and Perceptron, resumed from disk, never did. Train()'s message is
the specific one, so it wins.
NEXT STEP once this is running: the stall line now ends in "REJECTED BY: ..." and
the per-handle depths. Read it. Do not reason around it.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-17 15:21:15 -04:00
|
|
|
if(ADIndicatorCold(m_ADWyckoffEventStream, "ADWyckoffEventStream")) // see the CumulativeDelta block's comment
|
fix: the trailing incumbent read the future across eras; cold AD blocks cached zeros as truth
Three findings from the 2026-08-11 audit:
1. The excursion head's trailing-quantile ring was deliberately never cleared
between eras ("a rolling estimate of the market, not of the era") - but
pass 3 re-walks the SAME OOS window every era, so at each walk's restart
the ring still held the outcome masks of the newest OOS bars from the
previous walk: the chronological FUTURE of the bars about to be scored.
For the first ~window+horizon pushes of every era the "trailing" incumbent
was partly a leading one - conservative for the gate (an informed incumbent
is a harder hurdle) but exactly the self-made-artifact class 06d4785 hunts.
The ring now clears at era-score reset; the warm-up bars simply don't score
the trail race, which the m_excTrailN gating already accounts for.
2. skillTrail compared the head's FULL-block Brier (pro-rated by coverage)
against the incumbent's subset sum - valid only if head skill is uniform
across the OOS walk, while the trail-scored subset systematically excludes
each era's warm-up bars. The audit also found m_excBrierHeadD/BaseD/
m_excOosHitsD declared, zeroed and never accumulated (dead since e2c9593
made every scored bar disjoint). The dead trio is replaced by
m_excBrierHeadT: the head's Brier accumulated only on the bars the warm
incumbent also scored, so the race now compares both predictors on an
identical bar set.
3. The AD/Wyckoff feature blocks read GetData with no EMPTY_VALUE guard; a
cold (still-calculating) indicator returns EMPTY_VALUE everywhere, the
sanitize loop rewrote that to 0.0, and the bar SUCCEEDED - so
BufferTempData cached an all-zero Wyckoff block as a success for the whole
bar frame: the one path the f6150ee only-cache-successes rule cannot see,
because it never fails (the ba13eef class, arriving through values that
never fail; a resumed model's era-0 prebuild starts milliseconds after
OnInit). ADIndicatorCold() probes the NEWEST bar - EMPTY_VALUE there means
async warm-up (transient reject, retried), while deep bars beyond the
buffered depth keep the sanitize loop's neutral-fill so degraded history
still trains. Also fixed m_featureCacheValid's declaration comment, which
still described the pre-f6150ee cached-miss semantics.
Compile: 0 errors, 0 warnings.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-11 18:20:24 -04:00
|
|
|
return false;
|
feat: mini-batch gradient accumulation (F4), front-end-aware capacity budget (F6), split Wyckoff categoricals (N1)
Completes the 2026-08-09 training audit. FORCES A RETRAIN of every
Wyckoff-enabled config (N1 re-keys the fingerprint), and BOTH DLLs must be
redeployed alongside the .ex5 - they carry new exports.
F4 - mini-batch accumulation, TRAIN_BATCH_SIZE=32. Training was pure online
SGD (one weight update per bar), which is the mechanical source of the
era-to-era whipsaw every downstream guard was built to cope with. The O(n^2)
outer product is native - AccumulateWeightGrad / AccumulateWeightGradConv /
AccumulateBufferInto in Network.cl, WarriorCPU and WarriorDML - while the
optimizer step is host-side MQL5 shared by all tiers (ApplyAccumToBlock), so
there is one Adam/SGD implementation instead of four that can drift.
- the LSTM needs no outer-product kernel (WeightsGradient already holds the
sample's full dW) but could NOT simply be left un-zeroed between samples:
CPU_LSTMSeqBackward/DML_LSTMSeqBackward memset it on entry. Hence a
separate accumulator plus an elementwise add.
- batch-norm gamma/beta accumulate in host arrays, not new BatchOptions
slots - BN_OPT_STRIDE is baked into every persisted .nnw.
- scoped to pass 2; online learning keeps immediate updates. Every save /
checkpoint / scoring boundary flushes, scaling by the real sample count.
- degrades to per-sample updates (one log line) on a tier that cannot
accumulate, so old devices and DLL-free builds are unaffected.
- verified offline: DirectML/batch_accum_check.cpp drives the real exports
against an independent reference; at B=1 the accumulator matches the
shipped unbatched kernel's own gradient to 1.1e-16. Math only - the
in-situ check remains the per-layer dW/W report on a real era.
F6 - ComputeFirstLayerWidth budgeted against the RAW input width even where a
conv/LSTM front end had already reduced it, so an LSTM's dense stack was
charged for 1,280 inputs when it receives 64. Confirmed from the deployed
.cfg files: CONV, LSTM and HYBRID were all pinned at the 16-unit floor. Now
budgeted against the front-end output and capped at it (never fan out), with
the derivation reordered so both stages settle first.
N1 - EventCode/EventPhase/StructuralPhase are signed categoricals packing
direction and Wyckoff stage into one scalar across a sign discontinuity. Split
into direction + [0,1] magnitude, the same convention the base OHLC block uses.
Information-preserving; 13 readings now occupy 16 inputs.
Compiled clean (0 errors, 0 warnings); both DLLs rebuilt.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-09 11:48:03 -04:00
|
|
|
double wesEvent = m_ADWyckoffEventStream.GetData(0, idx);
|
|
|
|
|
double wesLivePhase = m_ADWyckoffEventStream.GetData(1, idx);
|
|
|
|
|
double wesStructPhase = m_ADWyckoffEventStream.GetData(5, idx);
|
|
|
|
|
if(!TempData.Add(wesEvent > 0 ? 1.0 : (wesEvent < 0 ? -1.0 : 0.0)) || // event direction
|
|
|
|
|
!TempData.Add(MathMin(1.0, MathAbs(wesEvent) / 7.0)) || // event stage
|
|
|
|
|
!TempData.Add(wesLivePhase > 0 ? 1.0 : (wesLivePhase < 0 ? -1.0 : 0.0)) || // live-range direction
|
|
|
|
|
!TempData.Add(MathMin(1.0, MathAbs(wesLivePhase) / 5.0)) || // live-range phase
|
refactor: split CExpertSignalAIBase implementation by responsibility
ExpertSignalAIBase.mqh was 8216 lines: the class declaration followed by 87
method bodies covering training, labelling, feature extraction, persistence,
chart drawing, online learning, the GA auto-tuner and inference, all in one
file. Train() alone is 1492 lines; a change to arrow drawing meant scrolling
past the era loop.
Moved the bodies into Expert\AIBase\, included at the bottom of the original
after the class declaration:
Training.mqh 1607 era loop, plateau ladder, checkpoint select, deploy
Features.mqh 1093 indicator creation + per-bar input feature vector
ChartUI.mqh 634 arrows, arrow persistence, status panel, cleanup
Persistence.mqh 492 .stats/.cfg sidecars, CPU-inference validation, copy
OnlineLearning.mqh 461 live continual learning, EMA shadow, OOS simulator
Labels.mqh 309 ZigZag pivot labels, async label-cache prebuild
AutoTune.mqh 275 genetic tuner (population, crossover, halving)
Inference.mqh 235 softmax, prior calibration, class priors
ExpertSignalAIBase.mqh 8216 -> 3131 (declaration + topology build only)
This is a pure relocation - verified mechanically, not by eye: HEAD's file
reconstructed from the eight partials plus the surviving remainder is
byte-identical to HEAD, span for span (scratchpad verify_split.py). No
declaration moved, no signature changed, no code rewritten, so behaviour is
unchanged by construction.
Compiles 0 errors, 0 warnings.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-29 00:42:45 -04:00
|
|
|
!TempData.Add((m_ADWyckoffEventStream.GetData(2, idx) - close) / atr) || // ZoneTop
|
|
|
|
|
!TempData.Add((m_ADWyckoffEventStream.GetData(3, idx) - close) / atr) || // ZoneBottom
|
feat: mini-batch gradient accumulation (F4), front-end-aware capacity budget (F6), split Wyckoff categoricals (N1)
Completes the 2026-08-09 training audit. FORCES A RETRAIN of every
Wyckoff-enabled config (N1 re-keys the fingerprint), and BOTH DLLs must be
redeployed alongside the .ex5 - they carry new exports.
F4 - mini-batch accumulation, TRAIN_BATCH_SIZE=32. Training was pure online
SGD (one weight update per bar), which is the mechanical source of the
era-to-era whipsaw every downstream guard was built to cope with. The O(n^2)
outer product is native - AccumulateWeightGrad / AccumulateWeightGradConv /
AccumulateBufferInto in Network.cl, WarriorCPU and WarriorDML - while the
optimizer step is host-side MQL5 shared by all tiers (ApplyAccumToBlock), so
there is one Adam/SGD implementation instead of four that can drift.
- the LSTM needs no outer-product kernel (WeightsGradient already holds the
sample's full dW) but could NOT simply be left un-zeroed between samples:
CPU_LSTMSeqBackward/DML_LSTMSeqBackward memset it on entry. Hence a
separate accumulator plus an elementwise add.
- batch-norm gamma/beta accumulate in host arrays, not new BatchOptions
slots - BN_OPT_STRIDE is baked into every persisted .nnw.
- scoped to pass 2; online learning keeps immediate updates. Every save /
checkpoint / scoring boundary flushes, scaling by the real sample count.
- degrades to per-sample updates (one log line) on a tier that cannot
accumulate, so old devices and DLL-free builds are unaffected.
- verified offline: DirectML/batch_accum_check.cpp drives the real exports
against an independent reference; at B=1 the accumulator matches the
shipped unbatched kernel's own gradient to 1.1e-16. Math only - the
in-situ check remains the per-layer dW/W report on a real era.
F6 - ComputeFirstLayerWidth budgeted against the RAW input width even where a
conv/LSTM front end had already reduced it, so an LSTM's dense stack was
charged for 1,280 inputs when it receives 64. Confirmed from the deployed
.cfg files: CONV, LSTM and HYBRID were all pinned at the 16-unit floor. Now
budgeted against the front-end output and capped at it (never fan out), with
the derivation reordered so both stages settle first.
N1 - EventCode/EventPhase/StructuralPhase are signed categoricals packing
direction and Wyckoff stage into one scalar across a sign discontinuity. Split
into direction + [0,1] magnitude, the same convention the base OHLC block uses.
Information-preserving; 13 readings now occupy 16 inputs.
Compiled clean (0 errors, 0 warnings); both DLLs rebuilt.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-09 11:48:03 -04:00
|
|
|
!TempData.Add(wesStructPhase > 0 ? 1.0 : (wesStructPhase < 0 ? -1.0 : 0.0)) || // struct direction
|
|
|
|
|
!TempData.Add(MathMin(1.0, MathAbs(wesStructPhase) / 5.0)) || // struct phase
|
refactor: split CExpertSignalAIBase implementation by responsibility
ExpertSignalAIBase.mqh was 8216 lines: the class declaration followed by 87
method bodies covering training, labelling, feature extraction, persistence,
chart drawing, online learning, the GA auto-tuner and inference, all in one
file. Train() alone is 1492 lines; a change to arrow drawing meant scrolling
past the era loop.
Moved the bodies into Expert\AIBase\, included at the bottom of the original
after the class declaration:
Training.mqh 1607 era loop, plateau ladder, checkpoint select, deploy
Features.mqh 1093 indicator creation + per-bar input feature vector
ChartUI.mqh 634 arrows, arrow persistence, status panel, cleanup
Persistence.mqh 492 .stats/.cfg sidecars, CPU-inference validation, copy
OnlineLearning.mqh 461 live continual learning, EMA shadow, OOS simulator
Labels.mqh 309 ZigZag pivot labels, async label-cache prebuild
AutoTune.mqh 275 genetic tuner (population, crossover, halving)
Inference.mqh 235 softmax, prior calibration, class priors
ExpertSignalAIBase.mqh 8216 -> 3131 (declaration + topology build only)
This is a pure relocation - verified mechanically, not by eye: HEAD's file
reconstructed from the eight partials plus the surviving remainder is
byte-identical to HEAD, span for span (scratchpad verify_split.py). No
declaration moved, no signature changed, no code rewritten, so behaviour is
unchanged by construction.
Compiles 0 errors, 0 warnings.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-29 00:42:45 -04:00
|
|
|
!TempData.Add(m_ADWyckoffEventStream.GetData(6, idx)) || // CHoCHTrendToRange
|
|
|
|
|
!TempData.Add(m_ADWyckoffEventStream.GetData(7, idx)) || // CHoCHRangeToTrend
|
|
|
|
|
!TempData.Add(m_ADWyckoffEventStream.GetData(8, idx)) || // SlopeAccumulationBullish
|
|
|
|
|
!TempData.Add(m_ADWyckoffEventStream.GetData(9, idx)) || // SlopeAccumulationBearish
|
|
|
|
|
!TempData.Add(m_ADWyckoffEventStream.GetData(10, idx)) || // SlopeDistributionBullish
|
|
|
|
|
!TempData.Add(m_ADWyckoffEventStream.GetData(11, idx)) || // SlopeDistributionBearish
|
|
|
|
|
!TempData.Add(m_ADWyckoffEventStream.GetData(12, idx)) || // Reaccumulation
|
|
|
|
|
!TempData.Add(m_ADWyckoffEventStream.GetData(13, idx))) // Redistribution
|
|
|
|
|
return false;
|
|
|
|
|
}
|
|
|
|
|
if(m_useADWyckoffFailedStructure)
|
|
|
|
|
{
|
fix(ensemble+depth): the barrier had no liveness escape, and the depth gate could not report the one state the evidence pointed at
Two charts (USDJPY 50,179 bars / XAUUSD 33,982) sat at era 0 for 38 minutes with
four of their eight members completely silent. Nothing in this commit guesses at
why the sweep fails - the last five guesses were all wrong. It makes the failure
say what it is, and stops one broken member taking its whole chart down with it.
WHAT THE LOG ACTUALLY SAYS, before any of this.
- The running build IS d9f834d (pulled 14:18, compiled 14:19:01, 0 errors), so
every depth instrument from 1dda479/7e63a8b/45c9e21 was live.
- It printed NOTHING. Zero "PRIMING", zero "CAPPED", zero "Per-indicator depth"
in 27 MB of journal. The instrument built to find the depth shortfall returned
"not this".
- On USDJPY at 14:24, CONV-cad8 completed eras 0 AND 1 across all 50,179 bars -
same chart, same 832-value window, same indicators, byte-identical fingerprint -
while LSTM-cad8 and HYB-cad8 reported ok=0 failed=50163. So it is not the
symbol, the history, the bar count or the indicator depth. It is per-member.
- ok=0 means the NEWEST anchors failed too, and a short indicator cannot do that.
The depth reading in project_silent_block_failures is therefore retired by its own
instrumentation. THE ROOT CAUSE IS STILL UNKNOWN and this commit does not claim one.
1. THE DEPTH GATE'S SILENT PATH WAS THE STATE IT WAS HUNTING.
ServableBars() read `if(servable <= 0 || servable >= want) return want;` - one
branch over three unrelated states, silent in all of them:
enabled == 0 -> nothing tunable is on. No cap. Healthy.
enabled > 0, servable == -1 -> a handle answered INVALID.
enabled > 0, servable == 0 -> created, never calculated.
BarsCalculated() returns -1 for a dead handle, so a dead MA is indistinguishable
from "no tunable indicators enabled" - and both returned `want` without printing a
character. That is exactly the state a per-member, every-index, depth-independent
failure produces, and it is the single reason a build carrying full depth
instrumentation logged nothing through the whole outage.
TunableBarsCalculated() now also reports HOW MANY indicators it consulted, and the
dead-handle case is reported (latched, with per-handle depths). The RETURN is
deliberately unchanged - what to do about a dead handle is not yet known, and
changing control flow on an unproven cause is how the last four fixes here went
wrong. SettledBars() routes its three pass-through states via ServableBars() so the
report is reachable from the training sweep, which is the only caller that hits it.
2. THE STALL REPORT NAMED A SLOT, NEVER A BLOCK.
"lookback slot 0 REJECTED (window had 24 of 832 values)" plus a guess ("an
indicator warm-up or a history-edge read"). Which guard fired was INFERRED by
counting 4+5+4+4+6+1 = 24 and concluding feature 25 must be the MA. The arithmetic
was right; every conclusion drawn from it was wrong, because a value count names a
POSITION and a position cannot tell cold from capped from invalid from off-the-end.
Every guard that can reject a bar now records itself - m_featureFailBlock - and the
report carries it, the series index, IndicatorDepthReport()'s per-handle depths,
and for each indicator whether the NEWEST bar reads. That last field is the whole
diagnosis in one word: newest-also-EMPTY means the buffer is unreadable everywhere
(cold or dead handle), newest-reads means a genuine history edge. Instrumented:
open, ATR, MA, RSI, MACD, Ichimoku, and all five AD blocks via ADIndicatorCold().
3. THE TOTAL-FAILURE BACKOFF WAS GATED ON THE WRONG QUESTION.
It armed only when m_featureFailTransient was set. Keeping that flag correct across
every guard is a list that has to stay right forever - the same shape of fix the
feature cache abandoned for the same reason - and the gate is pointless anyway: a
sweep where ZERO of 50,163 bars produced a window will produce zero again if it
restarts a millisecond later, transient or not. Doing that at full speed is what
starved six indicator threads on a six-core box. The backoff is now unconditional
on a total failure. The flag keeps its real job, deciding whether a MISS may be
cached, which is a per-bar question and not a scheduling one.
4. THE ERA BARRIER DEADLOCKED, AND SILENCED THE MEMBERS IT FROZE.
EnsembleMinTrainingEra() exempted deployed, stopped and paused members and its
comment concluded "so nothing deadlocks". Those three are all VOLUNTARY. A member
that simply CANNOT finish an era is none of them, so it pinned the minimum at its
own era with no time limit - and the hold branch's only action was
`m_lastEraCompleteTick = GetTickCount()`, which silences the stall watchdog. So on
USDJPY the two members that could not train reported, and the two healthy members
frozen behind them wrote nothing anywhere. The outage was visible only through the
members that were not suffering it.
- BarrierEraHeartbeat() stamps a clock on real era CHANGE, kept separate from
m_lastEraCompleteTick precisely because the barrier resets that one. Only a
member AT the minimum can be a blocker; a member ahead is idle by design and is
never counted as stuck.
- After ENSEMBLE_BARRIER_STUCK_MS (12 min) a non-advancing member is dropped from
the barrier minimum. It keeps training and rejoins the instant it completes an
era - at which point, being behind, it legitimately becomes the minimum again,
which is the documented resumed-laggard behaviour.
- Both transitions say so loudly, and the release states plainly that the
combined-vote score cannot be computed while the ensemble is desynchronised.
- A held member now writes a rate-limited journal line naming WHICH members it is
waiting on, so the blocker is read off one line.
5. THE PANEL FLICKER.
OnTickHandler gates its terse writer on !m_trainRunActive, and a barrier-held member
returns from Train() before ever setting it - so both writers thought they were the
only one updating the label and fought every tick. That is the reported "Getting
ready..." <-> "Waiting at era N for slower ensemble members" oscillation, and it hit
Perceptron but not Convolutional purely because Convolutional had a run active from
a completed era and Perceptron, resumed from disk, never did. Train()'s message is
the specific one, so it wins.
NEXT STEP once this is running: the stall line now ends in "REJECTED BY: ..." and
the per-handle depths. Read it. Do not reason around it.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-17 15:21:15 -04:00
|
|
|
if(ADIndicatorCold(m_ADWyckoffFailedStructure, "ADWyckoffFailedStructure")) // see the CumulativeDelta block's comment
|
fix: the trailing incumbent read the future across eras; cold AD blocks cached zeros as truth
Three findings from the 2026-08-11 audit:
1. The excursion head's trailing-quantile ring was deliberately never cleared
between eras ("a rolling estimate of the market, not of the era") - but
pass 3 re-walks the SAME OOS window every era, so at each walk's restart
the ring still held the outcome masks of the newest OOS bars from the
previous walk: the chronological FUTURE of the bars about to be scored.
For the first ~window+horizon pushes of every era the "trailing" incumbent
was partly a leading one - conservative for the gate (an informed incumbent
is a harder hurdle) but exactly the self-made-artifact class 06d4785 hunts.
The ring now clears at era-score reset; the warm-up bars simply don't score
the trail race, which the m_excTrailN gating already accounts for.
2. skillTrail compared the head's FULL-block Brier (pro-rated by coverage)
against the incumbent's subset sum - valid only if head skill is uniform
across the OOS walk, while the trail-scored subset systematically excludes
each era's warm-up bars. The audit also found m_excBrierHeadD/BaseD/
m_excOosHitsD declared, zeroed and never accumulated (dead since e2c9593
made every scored bar disjoint). The dead trio is replaced by
m_excBrierHeadT: the head's Brier accumulated only on the bars the warm
incumbent also scored, so the race now compares both predictors on an
identical bar set.
3. The AD/Wyckoff feature blocks read GetData with no EMPTY_VALUE guard; a
cold (still-calculating) indicator returns EMPTY_VALUE everywhere, the
sanitize loop rewrote that to 0.0, and the bar SUCCEEDED - so
BufferTempData cached an all-zero Wyckoff block as a success for the whole
bar frame: the one path the f6150ee only-cache-successes rule cannot see,
because it never fails (the ba13eef class, arriving through values that
never fail; a resumed model's era-0 prebuild starts milliseconds after
OnInit). ADIndicatorCold() probes the NEWEST bar - EMPTY_VALUE there means
async warm-up (transient reject, retried), while deep bars beyond the
buffered depth keep the sanitize loop's neutral-fill so degraded history
still trains. Also fixed m_featureCacheValid's declaration comment, which
still described the pre-f6150ee cached-miss semantics.
Compile: 0 errors, 0 warnings.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-11 18:20:24 -04:00
|
|
|
return false;
|
refactor: split CExpertSignalAIBase implementation by responsibility
ExpertSignalAIBase.mqh was 8216 lines: the class declaration followed by 87
method bodies covering training, labelling, feature extraction, persistence,
chart drawing, online learning, the GA auto-tuner and inference, all in one
file. Train() alone is 1492 lines; a change to arrow drawing meant scrolling
past the era loop.
Moved the bodies into Expert\AIBase\, included at the bottom of the original
after the class declaration:
Training.mqh 1607 era loop, plateau ladder, checkpoint select, deploy
Features.mqh 1093 indicator creation + per-bar input feature vector
ChartUI.mqh 634 arrows, arrow persistence, status panel, cleanup
Persistence.mqh 492 .stats/.cfg sidecars, CPU-inference validation, copy
OnlineLearning.mqh 461 live continual learning, EMA shadow, OOS simulator
Labels.mqh 309 ZigZag pivot labels, async label-cache prebuild
AutoTune.mqh 275 genetic tuner (population, crossover, halving)
Inference.mqh 235 softmax, prior calibration, class priors
ExpertSignalAIBase.mqh 8216 -> 3131 (declaration + topology build only)
This is a pure relocation - verified mechanically, not by eye: HEAD's file
reconstructed from the eight partials plus the surviving remainder is
byte-identical to HEAD, span for span (scratchpad verify_split.py). No
declaration moved, no signature changed, no code rewritten, so behaviour is
unchanged by construction.
Compiles 0 errors, 0 warnings.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-29 00:42:45 -04:00
|
|
|
// buffers: 0=Value, 1=BullishStructuralFailure, 2=BearishStructuralFailure, 3=FailedAccumulation, 4=FailedDistribution
|
|
|
|
|
if(!TempData.Add(m_ADWyckoffFailedStructure.GetData(0, idx)) || // Value
|
|
|
|
|
!TempData.Add(m_ADWyckoffFailedStructure.GetData(1, idx)) || // BullishStructuralFailure
|
|
|
|
|
!TempData.Add(m_ADWyckoffFailedStructure.GetData(2, idx)) || // BearishStructuralFailure
|
|
|
|
|
!TempData.Add(m_ADWyckoffFailedStructure.GetData(3, idx)) || // FailedAccumulation
|
|
|
|
|
!TempData.Add(m_ADWyckoffFailedStructure.GetData(4, idx))) // FailedDistribution
|
|
|
|
|
return false;
|
|
|
|
|
}
|
|
|
|
|
if(m_useADWyckoffSignificantBarInversion)
|
|
|
|
|
{
|
fix(ensemble+depth): the barrier had no liveness escape, and the depth gate could not report the one state the evidence pointed at
Two charts (USDJPY 50,179 bars / XAUUSD 33,982) sat at era 0 for 38 minutes with
four of their eight members completely silent. Nothing in this commit guesses at
why the sweep fails - the last five guesses were all wrong. It makes the failure
say what it is, and stops one broken member taking its whole chart down with it.
WHAT THE LOG ACTUALLY SAYS, before any of this.
- The running build IS d9f834d (pulled 14:18, compiled 14:19:01, 0 errors), so
every depth instrument from 1dda479/7e63a8b/45c9e21 was live.
- It printed NOTHING. Zero "PRIMING", zero "CAPPED", zero "Per-indicator depth"
in 27 MB of journal. The instrument built to find the depth shortfall returned
"not this".
- On USDJPY at 14:24, CONV-cad8 completed eras 0 AND 1 across all 50,179 bars -
same chart, same 832-value window, same indicators, byte-identical fingerprint -
while LSTM-cad8 and HYB-cad8 reported ok=0 failed=50163. So it is not the
symbol, the history, the bar count or the indicator depth. It is per-member.
- ok=0 means the NEWEST anchors failed too, and a short indicator cannot do that.
The depth reading in project_silent_block_failures is therefore retired by its own
instrumentation. THE ROOT CAUSE IS STILL UNKNOWN and this commit does not claim one.
1. THE DEPTH GATE'S SILENT PATH WAS THE STATE IT WAS HUNTING.
ServableBars() read `if(servable <= 0 || servable >= want) return want;` - one
branch over three unrelated states, silent in all of them:
enabled == 0 -> nothing tunable is on. No cap. Healthy.
enabled > 0, servable == -1 -> a handle answered INVALID.
enabled > 0, servable == 0 -> created, never calculated.
BarsCalculated() returns -1 for a dead handle, so a dead MA is indistinguishable
from "no tunable indicators enabled" - and both returned `want` without printing a
character. That is exactly the state a per-member, every-index, depth-independent
failure produces, and it is the single reason a build carrying full depth
instrumentation logged nothing through the whole outage.
TunableBarsCalculated() now also reports HOW MANY indicators it consulted, and the
dead-handle case is reported (latched, with per-handle depths). The RETURN is
deliberately unchanged - what to do about a dead handle is not yet known, and
changing control flow on an unproven cause is how the last four fixes here went
wrong. SettledBars() routes its three pass-through states via ServableBars() so the
report is reachable from the training sweep, which is the only caller that hits it.
2. THE STALL REPORT NAMED A SLOT, NEVER A BLOCK.
"lookback slot 0 REJECTED (window had 24 of 832 values)" plus a guess ("an
indicator warm-up or a history-edge read"). Which guard fired was INFERRED by
counting 4+5+4+4+6+1 = 24 and concluding feature 25 must be the MA. The arithmetic
was right; every conclusion drawn from it was wrong, because a value count names a
POSITION and a position cannot tell cold from capped from invalid from off-the-end.
Every guard that can reject a bar now records itself - m_featureFailBlock - and the
report carries it, the series index, IndicatorDepthReport()'s per-handle depths,
and for each indicator whether the NEWEST bar reads. That last field is the whole
diagnosis in one word: newest-also-EMPTY means the buffer is unreadable everywhere
(cold or dead handle), newest-reads means a genuine history edge. Instrumented:
open, ATR, MA, RSI, MACD, Ichimoku, and all five AD blocks via ADIndicatorCold().
3. THE TOTAL-FAILURE BACKOFF WAS GATED ON THE WRONG QUESTION.
It armed only when m_featureFailTransient was set. Keeping that flag correct across
every guard is a list that has to stay right forever - the same shape of fix the
feature cache abandoned for the same reason - and the gate is pointless anyway: a
sweep where ZERO of 50,163 bars produced a window will produce zero again if it
restarts a millisecond later, transient or not. Doing that at full speed is what
starved six indicator threads on a six-core box. The backoff is now unconditional
on a total failure. The flag keeps its real job, deciding whether a MISS may be
cached, which is a per-bar question and not a scheduling one.
4. THE ERA BARRIER DEADLOCKED, AND SILENCED THE MEMBERS IT FROZE.
EnsembleMinTrainingEra() exempted deployed, stopped and paused members and its
comment concluded "so nothing deadlocks". Those three are all VOLUNTARY. A member
that simply CANNOT finish an era is none of them, so it pinned the minimum at its
own era with no time limit - and the hold branch's only action was
`m_lastEraCompleteTick = GetTickCount()`, which silences the stall watchdog. So on
USDJPY the two members that could not train reported, and the two healthy members
frozen behind them wrote nothing anywhere. The outage was visible only through the
members that were not suffering it.
- BarrierEraHeartbeat() stamps a clock on real era CHANGE, kept separate from
m_lastEraCompleteTick precisely because the barrier resets that one. Only a
member AT the minimum can be a blocker; a member ahead is idle by design and is
never counted as stuck.
- After ENSEMBLE_BARRIER_STUCK_MS (12 min) a non-advancing member is dropped from
the barrier minimum. It keeps training and rejoins the instant it completes an
era - at which point, being behind, it legitimately becomes the minimum again,
which is the documented resumed-laggard behaviour.
- Both transitions say so loudly, and the release states plainly that the
combined-vote score cannot be computed while the ensemble is desynchronised.
- A held member now writes a rate-limited journal line naming WHICH members it is
waiting on, so the blocker is read off one line.
5. THE PANEL FLICKER.
OnTickHandler gates its terse writer on !m_trainRunActive, and a barrier-held member
returns from Train() before ever setting it - so both writers thought they were the
only one updating the label and fought every tick. That is the reported "Getting
ready..." <-> "Waiting at era N for slower ensemble members" oscillation, and it hit
Perceptron but not Convolutional purely because Convolutional had a run active from
a completed era and Perceptron, resumed from disk, never did. Train()'s message is
the specific one, so it wins.
NEXT STEP once this is running: the stall line now ends in "REJECTED BY: ..." and
the per-handle depths. Read it. Do not reason around it.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-17 15:21:15 -04:00
|
|
|
if(ADIndicatorCold(m_ADWyckoffSignificantBarInversion, "ADWyckoffSignificantBarInversion")) // see the CumulativeDelta block's comment
|
fix: the trailing incumbent read the future across eras; cold AD blocks cached zeros as truth
Three findings from the 2026-08-11 audit:
1. The excursion head's trailing-quantile ring was deliberately never cleared
between eras ("a rolling estimate of the market, not of the era") - but
pass 3 re-walks the SAME OOS window every era, so at each walk's restart
the ring still held the outcome masks of the newest OOS bars from the
previous walk: the chronological FUTURE of the bars about to be scored.
For the first ~window+horizon pushes of every era the "trailing" incumbent
was partly a leading one - conservative for the gate (an informed incumbent
is a harder hurdle) but exactly the self-made-artifact class 06d4785 hunts.
The ring now clears at era-score reset; the warm-up bars simply don't score
the trail race, which the m_excTrailN gating already accounts for.
2. skillTrail compared the head's FULL-block Brier (pro-rated by coverage)
against the incumbent's subset sum - valid only if head skill is uniform
across the OOS walk, while the trail-scored subset systematically excludes
each era's warm-up bars. The audit also found m_excBrierHeadD/BaseD/
m_excOosHitsD declared, zeroed and never accumulated (dead since e2c9593
made every scored bar disjoint). The dead trio is replaced by
m_excBrierHeadT: the head's Brier accumulated only on the bars the warm
incumbent also scored, so the race now compares both predictors on an
identical bar set.
3. The AD/Wyckoff feature blocks read GetData with no EMPTY_VALUE guard; a
cold (still-calculating) indicator returns EMPTY_VALUE everywhere, the
sanitize loop rewrote that to 0.0, and the bar SUCCEEDED - so
BufferTempData cached an all-zero Wyckoff block as a success for the whole
bar frame: the one path the f6150ee only-cache-successes rule cannot see,
because it never fails (the ba13eef class, arriving through values that
never fail; a resumed model's era-0 prebuild starts milliseconds after
OnInit). ADIndicatorCold() probes the NEWEST bar - EMPTY_VALUE there means
async warm-up (transient reject, retried), while deep bars beyond the
buffered depth keep the sanitize loop's neutral-fill so degraded history
still trains. Also fixed m_featureCacheValid's declaration comment, which
still described the pre-f6150ee cached-miss semantics.
Compile: 0 errors, 0 warnings.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-11 18:20:24 -04:00
|
|
|
return false;
|
refactor: split CExpertSignalAIBase implementation by responsibility
ExpertSignalAIBase.mqh was 8216 lines: the class declaration followed by 87
method bodies covering training, labelling, feature extraction, persistence,
chart drawing, online learning, the GA auto-tuner and inference, all in one
file. Train() alone is 1492 lines; a change to arrow drawing meant scrolling
past the era loop.
Moved the bodies into Expert\AIBase\, included at the bottom of the original
after the class declaration:
Training.mqh 1607 era loop, plateau ladder, checkpoint select, deploy
Features.mqh 1093 indicator creation + per-bar input feature vector
ChartUI.mqh 634 arrows, arrow persistence, status panel, cleanup
Persistence.mqh 492 .stats/.cfg sidecars, CPU-inference validation, copy
OnlineLearning.mqh 461 live continual learning, EMA shadow, OOS simulator
Labels.mqh 309 ZigZag pivot labels, async label-cache prebuild
AutoTune.mqh 275 genetic tuner (population, crossover, halving)
Inference.mqh 235 softmax, prior calibration, class priors
ExpertSignalAIBase.mqh 8216 -> 3131 (declaration + topology build only)
This is a pure relocation - verified mechanically, not by eye: HEAD's file
reconstructed from the eight partials plus the surviving remainder is
byte-identical to HEAD, span for span (scratchpad verify_split.py). No
declaration moved, no signature changed, no code rewritten, so behaviour is
unchanged by construction.
Compiles 0 errors, 0 warnings.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-29 00:42:45 -04:00
|
|
|
// buffers: 0=SignificantBarQuality, 1=BullishSignificantBar, 2=BearishSignificantBar, 3=BullishControlFlip, 4=BearishControlFlip
|
|
|
|
|
if(!TempData.Add(m_ADWyckoffSignificantBarInversion.GetData(0, idx)) || // SignificantBarQuality
|
|
|
|
|
!TempData.Add(m_ADWyckoffSignificantBarInversion.GetData(1, idx)) || // BullishSignificantBar
|
|
|
|
|
!TempData.Add(m_ADWyckoffSignificantBarInversion.GetData(2, idx)) || // BearishSignificantBar
|
|
|
|
|
!TempData.Add(m_ADWyckoffSignificantBarInversion.GetData(3, idx)) || // BullishControlFlip
|
|
|
|
|
!TempData.Add(m_ADWyckoffSignificantBarInversion.GetData(4, idx))) // BearishControlFlip
|
|
|
|
|
return false;
|
|
|
|
|
}
|
2026-08-16 13:39:00 -04:00
|
|
|
if(m_useAltData)
|
|
|
|
|
{
|
|
|
|
|
//--- External publication-stamped block (COT/VIX/macro) - see System\AltData.mqh and the
|
|
|
|
|
//--- matching m_neuronsCount block in Topology.mqh. As-of lookup by THIS bar's open time, so
|
|
|
|
|
//--- a bar can only read values the live run would have had. Gaps 0-fill like cross-asset:
|
|
|
|
|
//--- the block is additive context and "no reading yet" (pre-history warm-up, collector
|
|
|
|
|
//--- outage, pinned column gone from the file) must degrade, never reject the bar. m_Time is
|
|
|
|
|
//--- unconditional (see InitTime), so GetData here is safe on every path that reaches this.
|
|
|
|
|
double av[];
|
|
|
|
|
m_altData.Features((datetime)m_Time.GetData(idx), av);
|
|
|
|
|
int an = m_altData.FeatureCount();
|
|
|
|
|
for(int k = 0; k < an; k++)
|
|
|
|
|
if(!TempData.Add(av[k]))
|
|
|
|
|
return false;
|
|
|
|
|
}
|
2026-08-02 01:09:18 -04:00
|
|
|
//--- ONE finiteness/plausibility gate for the whole bar, rather than 60-odd individually guarded
|
|
|
|
|
//--- Add() calls. Most blocks above already clamp their own output; the AD/Wyckoff blocks
|
|
|
|
|
//--- deliberately do not, because those indicators emit plain readings with no natural range. But
|
|
|
|
|
//--- MQL5's CDoubleBuffer::At() returns EMPTY_VALUE (DBL_MAX) for any index it holds no data for,
|
|
|
|
|
//--- and (EMPTY_VALUE - close) / atr is ~1e307: still FINITE, so it sails through every downstream
|
|
|
|
|
//--- isfinite() check, and still large enough to overflow the first batch-norm layer's running
|
|
|
|
|
//--- variance and latch that layer to NaN permanently (see NormalizeHost in AI\NeuronBatchNorm.mqh -
|
|
|
|
|
//--- that is the 2026-08-02 "BufferWrite failed for buffer 3" run). Neutral-fill rather than reject
|
|
|
|
|
//--- the bar: "this indicator has no reading here" is the degraded-but-usable case that the swing,
|
|
|
|
|
//--- cross-asset and spread blocks above all already handle the same way.
|
|
|
|
|
int featureEnd = TempData.Total();
|
|
|
|
|
for(int f = featureStart; f < featureEnd; f++)
|
|
|
|
|
{
|
|
|
|
|
double v = TempData.At(f);
|
|
|
|
|
if((!MathIsValidNumber(v) || MathAbs(v) > FEATURE_ABS_MAX) && !TempData.Update(f, 0.0))
|
|
|
|
|
return false;
|
|
|
|
|
}
|
refactor: split CExpertSignalAIBase implementation by responsibility
ExpertSignalAIBase.mqh was 8216 lines: the class declaration followed by 87
method bodies covering training, labelling, feature extraction, persistence,
chart drawing, online learning, the GA auto-tuner and inference, all in one
file. Train() alone is 1492 lines; a change to arrow drawing meant scrolling
past the era loop.
Moved the bodies into Expert\AIBase\, included at the bottom of the original
after the class declaration:
Training.mqh 1607 era loop, plateau ladder, checkpoint select, deploy
Features.mqh 1093 indicator creation + per-bar input feature vector
ChartUI.mqh 634 arrows, arrow persistence, status panel, cleanup
Persistence.mqh 492 .stats/.cfg sidecars, CPU-inference validation, copy
OnlineLearning.mqh 461 live continual learning, EMA shadow, OOS simulator
Labels.mqh 309 ZigZag pivot labels, async label-cache prebuild
AutoTune.mqh 275 genetic tuner (population, crossover, halving)
Inference.mqh 235 softmax, prior calibration, class priors
ExpertSignalAIBase.mqh 8216 -> 3131 (declaration + topology build only)
This is a pure relocation - verified mechanically, not by eye: HEAD's file
reconstructed from the eight partials plus the surviving remainder is
byte-identical to HEAD, span for span (scratchpad verify_split.py). No
declaration moved, no signature changed, no code rewritten, so behaviour is
unchanged by construction.
Compiles 0 errors, 0 warnings.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-29 00:42:45 -04:00
|
|
|
return true;
|
|
|
|
|
}
|
|
|
|
|
//+------------------------------------------------------------------+
|
fix: the trailing incumbent read the future across eras; cold AD blocks cached zeros as truth
Three findings from the 2026-08-11 audit:
1. The excursion head's trailing-quantile ring was deliberately never cleared
between eras ("a rolling estimate of the market, not of the era") - but
pass 3 re-walks the SAME OOS window every era, so at each walk's restart
the ring still held the outcome masks of the newest OOS bars from the
previous walk: the chronological FUTURE of the bars about to be scored.
For the first ~window+horizon pushes of every era the "trailing" incumbent
was partly a leading one - conservative for the gate (an informed incumbent
is a harder hurdle) but exactly the self-made-artifact class 06d4785 hunts.
The ring now clears at era-score reset; the warm-up bars simply don't score
the trail race, which the m_excTrailN gating already accounts for.
2. skillTrail compared the head's FULL-block Brier (pro-rated by coverage)
against the incumbent's subset sum - valid only if head skill is uniform
across the OOS walk, while the trail-scored subset systematically excludes
each era's warm-up bars. The audit also found m_excBrierHeadD/BaseD/
m_excOosHitsD declared, zeroed and never accumulated (dead since e2c9593
made every scored bar disjoint). The dead trio is replaced by
m_excBrierHeadT: the head's Brier accumulated only on the bars the warm
incumbent also scored, so the race now compares both predictors on an
identical bar set.
3. The AD/Wyckoff feature blocks read GetData with no EMPTY_VALUE guard; a
cold (still-calculating) indicator returns EMPTY_VALUE everywhere, the
sanitize loop rewrote that to 0.0, and the bar SUCCEEDED - so
BufferTempData cached an all-zero Wyckoff block as a success for the whole
bar frame: the one path the f6150ee only-cache-successes rule cannot see,
because it never fails (the ba13eef class, arriving through values that
never fail; a resumed model's era-0 prebuild starts milliseconds after
OnInit). ADIndicatorCold() probes the NEWEST bar - EMPTY_VALUE there means
async warm-up (transient reject, retried), while deep bars beyond the
buffered depth keep the sanitize loop's neutral-fill so degraded history
still trains. Also fixed m_featureCacheValid's declaration comment, which
still described the pre-f6150ee cached-miss semantics.
Compile: 0 errors, 0 warnings.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-11 18:20:24 -04:00
|
|
|
//| "Is this AD indicator still calculating?" MT5 fills custom- |
|
|
|
|
|
//| indicator buffers asynchronously after the handle is created, and |
|
|
|
|
|
//| a cold one returns EMPTY_VALUE for EVERY index - including the |
|
|
|
|
|
//| newest bar, which a warm indicator always has. So probing buffer |
|
|
|
|
|
//| 0 at index 0 cleanly separates "async calc hasn't run yet" (cold: |
|
|
|
|
|
//| transient reject, retried next call like the cold-ATR guard) from |
|
|
|
|
|
//| "this deep bar is beyond the buffered depth" (warm: neutral-fill |
|
|
|
|
|
//| by the sanitize loop, since that history is degraded-but-usable). |
|
|
|
|
|
//| Cost is one array read per block per bar. (2026-08-11) |
|
|
|
|
|
//+------------------------------------------------------------------+
|
fix(ensemble+depth): the barrier had no liveness escape, and the depth gate could not report the one state the evidence pointed at
Two charts (USDJPY 50,179 bars / XAUUSD 33,982) sat at era 0 for 38 minutes with
four of their eight members completely silent. Nothing in this commit guesses at
why the sweep fails - the last five guesses were all wrong. It makes the failure
say what it is, and stops one broken member taking its whole chart down with it.
WHAT THE LOG ACTUALLY SAYS, before any of this.
- The running build IS d9f834d (pulled 14:18, compiled 14:19:01, 0 errors), so
every depth instrument from 1dda479/7e63a8b/45c9e21 was live.
- It printed NOTHING. Zero "PRIMING", zero "CAPPED", zero "Per-indicator depth"
in 27 MB of journal. The instrument built to find the depth shortfall returned
"not this".
- On USDJPY at 14:24, CONV-cad8 completed eras 0 AND 1 across all 50,179 bars -
same chart, same 832-value window, same indicators, byte-identical fingerprint -
while LSTM-cad8 and HYB-cad8 reported ok=0 failed=50163. So it is not the
symbol, the history, the bar count or the indicator depth. It is per-member.
- ok=0 means the NEWEST anchors failed too, and a short indicator cannot do that.
The depth reading in project_silent_block_failures is therefore retired by its own
instrumentation. THE ROOT CAUSE IS STILL UNKNOWN and this commit does not claim one.
1. THE DEPTH GATE'S SILENT PATH WAS THE STATE IT WAS HUNTING.
ServableBars() read `if(servable <= 0 || servable >= want) return want;` - one
branch over three unrelated states, silent in all of them:
enabled == 0 -> nothing tunable is on. No cap. Healthy.
enabled > 0, servable == -1 -> a handle answered INVALID.
enabled > 0, servable == 0 -> created, never calculated.
BarsCalculated() returns -1 for a dead handle, so a dead MA is indistinguishable
from "no tunable indicators enabled" - and both returned `want` without printing a
character. That is exactly the state a per-member, every-index, depth-independent
failure produces, and it is the single reason a build carrying full depth
instrumentation logged nothing through the whole outage.
TunableBarsCalculated() now also reports HOW MANY indicators it consulted, and the
dead-handle case is reported (latched, with per-handle depths). The RETURN is
deliberately unchanged - what to do about a dead handle is not yet known, and
changing control flow on an unproven cause is how the last four fixes here went
wrong. SettledBars() routes its three pass-through states via ServableBars() so the
report is reachable from the training sweep, which is the only caller that hits it.
2. THE STALL REPORT NAMED A SLOT, NEVER A BLOCK.
"lookback slot 0 REJECTED (window had 24 of 832 values)" plus a guess ("an
indicator warm-up or a history-edge read"). Which guard fired was INFERRED by
counting 4+5+4+4+6+1 = 24 and concluding feature 25 must be the MA. The arithmetic
was right; every conclusion drawn from it was wrong, because a value count names a
POSITION and a position cannot tell cold from capped from invalid from off-the-end.
Every guard that can reject a bar now records itself - m_featureFailBlock - and the
report carries it, the series index, IndicatorDepthReport()'s per-handle depths,
and for each indicator whether the NEWEST bar reads. That last field is the whole
diagnosis in one word: newest-also-EMPTY means the buffer is unreadable everywhere
(cold or dead handle), newest-reads means a genuine history edge. Instrumented:
open, ATR, MA, RSI, MACD, Ichimoku, and all five AD blocks via ADIndicatorCold().
3. THE TOTAL-FAILURE BACKOFF WAS GATED ON THE WRONG QUESTION.
It armed only when m_featureFailTransient was set. Keeping that flag correct across
every guard is a list that has to stay right forever - the same shape of fix the
feature cache abandoned for the same reason - and the gate is pointless anyway: a
sweep where ZERO of 50,163 bars produced a window will produce zero again if it
restarts a millisecond later, transient or not. Doing that at full speed is what
starved six indicator threads on a six-core box. The backoff is now unconditional
on a total failure. The flag keeps its real job, deciding whether a MISS may be
cached, which is a per-bar question and not a scheduling one.
4. THE ERA BARRIER DEADLOCKED, AND SILENCED THE MEMBERS IT FROZE.
EnsembleMinTrainingEra() exempted deployed, stopped and paused members and its
comment concluded "so nothing deadlocks". Those three are all VOLUNTARY. A member
that simply CANNOT finish an era is none of them, so it pinned the minimum at its
own era with no time limit - and the hold branch's only action was
`m_lastEraCompleteTick = GetTickCount()`, which silences the stall watchdog. So on
USDJPY the two members that could not train reported, and the two healthy members
frozen behind them wrote nothing anywhere. The outage was visible only through the
members that were not suffering it.
- BarrierEraHeartbeat() stamps a clock on real era CHANGE, kept separate from
m_lastEraCompleteTick precisely because the barrier resets that one. Only a
member AT the minimum can be a blocker; a member ahead is idle by design and is
never counted as stuck.
- After ENSEMBLE_BARRIER_STUCK_MS (12 min) a non-advancing member is dropped from
the barrier minimum. It keeps training and rejoins the instant it completes an
era - at which point, being behind, it legitimately becomes the minimum again,
which is the documented resumed-laggard behaviour.
- Both transitions say so loudly, and the release states plainly that the
combined-vote score cannot be computed while the ensemble is desynchronised.
- A held member now writes a rate-limited journal line naming WHICH members it is
waiting on, so the blocker is read off one line.
5. THE PANEL FLICKER.
OnTickHandler gates its terse writer on !m_trainRunActive, and a barrier-held member
returns from Train() before ever setting it - so both writers thought they were the
only one updating the label and fought every tick. That is the reported "Getting
ready..." <-> "Waiting at era N for slower ensemble members" oscillation, and it hit
Perceptron but not Convolutional purely because Convolutional had a run active from
a completed era and Perceptron, resumed from disk, never did. Train()'s message is
the specific one, so it wins.
NEXT STEP once this is running: the stall line now ends in "REJECTED BY: ..." and
the per-handle depths. Read it. Do not reason around it.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-17 15:21:15 -04:00
|
|
|
bool CExpertSignalAIBase::ADIndicatorCold(CiCustom &ind, string block)
|
fix: the trailing incumbent read the future across eras; cold AD blocks cached zeros as truth
Three findings from the 2026-08-11 audit:
1. The excursion head's trailing-quantile ring was deliberately never cleared
between eras ("a rolling estimate of the market, not of the era") - but
pass 3 re-walks the SAME OOS window every era, so at each walk's restart
the ring still held the outcome masks of the newest OOS bars from the
previous walk: the chronological FUTURE of the bars about to be scored.
For the first ~window+horizon pushes of every era the "trailing" incumbent
was partly a leading one - conservative for the gate (an informed incumbent
is a harder hurdle) but exactly the self-made-artifact class 06d4785 hunts.
The ring now clears at era-score reset; the warm-up bars simply don't score
the trail race, which the m_excTrailN gating already accounts for.
2. skillTrail compared the head's FULL-block Brier (pro-rated by coverage)
against the incumbent's subset sum - valid only if head skill is uniform
across the OOS walk, while the trail-scored subset systematically excludes
each era's warm-up bars. The audit also found m_excBrierHeadD/BaseD/
m_excOosHitsD declared, zeroed and never accumulated (dead since e2c9593
made every scored bar disjoint). The dead trio is replaced by
m_excBrierHeadT: the head's Brier accumulated only on the bars the warm
incumbent also scored, so the race now compares both predictors on an
identical bar set.
3. The AD/Wyckoff feature blocks read GetData with no EMPTY_VALUE guard; a
cold (still-calculating) indicator returns EMPTY_VALUE everywhere, the
sanitize loop rewrote that to 0.0, and the bar SUCCEEDED - so
BufferTempData cached an all-zero Wyckoff block as a success for the whole
bar frame: the one path the f6150ee only-cache-successes rule cannot see,
because it never fails (the ba13eef class, arriving through values that
never fail; a resumed model's era-0 prebuild starts milliseconds after
OnInit). ADIndicatorCold() probes the NEWEST bar - EMPTY_VALUE there means
async warm-up (transient reject, retried), while deep bars beyond the
buffered depth keep the sanitize loop's neutral-fill so degraded history
still trains. Also fixed m_featureCacheValid's declaration comment, which
still described the pre-f6150ee cached-miss semantics.
Compile: 0 errors, 0 warnings.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-11 18:20:24 -04:00
|
|
|
{
|
|
|
|
|
if(ind.GetData(0, 0) != EMPTY_VALUE)
|
|
|
|
|
return false;
|
|
|
|
|
m_featureFailTransient = true;
|
fix(ensemble+depth): the barrier had no liveness escape, and the depth gate could not report the one state the evidence pointed at
Two charts (USDJPY 50,179 bars / XAUUSD 33,982) sat at era 0 for 38 minutes with
four of their eight members completely silent. Nothing in this commit guesses at
why the sweep fails - the last five guesses were all wrong. It makes the failure
say what it is, and stops one broken member taking its whole chart down with it.
WHAT THE LOG ACTUALLY SAYS, before any of this.
- The running build IS d9f834d (pulled 14:18, compiled 14:19:01, 0 errors), so
every depth instrument from 1dda479/7e63a8b/45c9e21 was live.
- It printed NOTHING. Zero "PRIMING", zero "CAPPED", zero "Per-indicator depth"
in 27 MB of journal. The instrument built to find the depth shortfall returned
"not this".
- On USDJPY at 14:24, CONV-cad8 completed eras 0 AND 1 across all 50,179 bars -
same chart, same 832-value window, same indicators, byte-identical fingerprint -
while LSTM-cad8 and HYB-cad8 reported ok=0 failed=50163. So it is not the
symbol, the history, the bar count or the indicator depth. It is per-member.
- ok=0 means the NEWEST anchors failed too, and a short indicator cannot do that.
The depth reading in project_silent_block_failures is therefore retired by its own
instrumentation. THE ROOT CAUSE IS STILL UNKNOWN and this commit does not claim one.
1. THE DEPTH GATE'S SILENT PATH WAS THE STATE IT WAS HUNTING.
ServableBars() read `if(servable <= 0 || servable >= want) return want;` - one
branch over three unrelated states, silent in all of them:
enabled == 0 -> nothing tunable is on. No cap. Healthy.
enabled > 0, servable == -1 -> a handle answered INVALID.
enabled > 0, servable == 0 -> created, never calculated.
BarsCalculated() returns -1 for a dead handle, so a dead MA is indistinguishable
from "no tunable indicators enabled" - and both returned `want` without printing a
character. That is exactly the state a per-member, every-index, depth-independent
failure produces, and it is the single reason a build carrying full depth
instrumentation logged nothing through the whole outage.
TunableBarsCalculated() now also reports HOW MANY indicators it consulted, and the
dead-handle case is reported (latched, with per-handle depths). The RETURN is
deliberately unchanged - what to do about a dead handle is not yet known, and
changing control flow on an unproven cause is how the last four fixes here went
wrong. SettledBars() routes its three pass-through states via ServableBars() so the
report is reachable from the training sweep, which is the only caller that hits it.
2. THE STALL REPORT NAMED A SLOT, NEVER A BLOCK.
"lookback slot 0 REJECTED (window had 24 of 832 values)" plus a guess ("an
indicator warm-up or a history-edge read"). Which guard fired was INFERRED by
counting 4+5+4+4+6+1 = 24 and concluding feature 25 must be the MA. The arithmetic
was right; every conclusion drawn from it was wrong, because a value count names a
POSITION and a position cannot tell cold from capped from invalid from off-the-end.
Every guard that can reject a bar now records itself - m_featureFailBlock - and the
report carries it, the series index, IndicatorDepthReport()'s per-handle depths,
and for each indicator whether the NEWEST bar reads. That last field is the whole
diagnosis in one word: newest-also-EMPTY means the buffer is unreadable everywhere
(cold or dead handle), newest-reads means a genuine history edge. Instrumented:
open, ATR, MA, RSI, MACD, Ichimoku, and all five AD blocks via ADIndicatorCold().
3. THE TOTAL-FAILURE BACKOFF WAS GATED ON THE WRONG QUESTION.
It armed only when m_featureFailTransient was set. Keeping that flag correct across
every guard is a list that has to stay right forever - the same shape of fix the
feature cache abandoned for the same reason - and the gate is pointless anyway: a
sweep where ZERO of 50,163 bars produced a window will produce zero again if it
restarts a millisecond later, transient or not. Doing that at full speed is what
starved six indicator threads on a six-core box. The backoff is now unconditional
on a total failure. The flag keeps its real job, deciding whether a MISS may be
cached, which is a per-bar question and not a scheduling one.
4. THE ERA BARRIER DEADLOCKED, AND SILENCED THE MEMBERS IT FROZE.
EnsembleMinTrainingEra() exempted deployed, stopped and paused members and its
comment concluded "so nothing deadlocks". Those three are all VOLUNTARY. A member
that simply CANNOT finish an era is none of them, so it pinned the minimum at its
own era with no time limit - and the hold branch's only action was
`m_lastEraCompleteTick = GetTickCount()`, which silences the stall watchdog. So on
USDJPY the two members that could not train reported, and the two healthy members
frozen behind them wrote nothing anywhere. The outage was visible only through the
members that were not suffering it.
- BarrierEraHeartbeat() stamps a clock on real era CHANGE, kept separate from
m_lastEraCompleteTick precisely because the barrier resets that one. Only a
member AT the minimum can be a blocker; a member ahead is idle by design and is
never counted as stuck.
- After ENSEMBLE_BARRIER_STUCK_MS (12 min) a non-advancing member is dropped from
the barrier minimum. It keeps training and rejoins the instant it completes an
era - at which point, being behind, it legitimately becomes the minimum again,
which is the documented resumed-laggard behaviour.
- Both transitions say so loudly, and the release states plainly that the
combined-vote score cannot be computed while the ensemble is desynchronised.
- A held member now writes a rate-limited journal line naming WHICH members it is
waiting on, so the blocker is read off one line.
5. THE PANEL FLICKER.
OnTickHandler gates its terse writer on !m_trainRunActive, and a barrier-held member
returns from Train() before ever setting it - so both writers thought they were the
only one updating the label and fought every tick. That is the reported "Getting
ready..." <-> "Waiting at era N for slower ensemble members" oscillation, and it hit
Perceptron but not Convolutional purely because Convolutional had a run active from
a completed era and Perceptron, resumed from disk, never did. Train()'s message is
the specific one, so it wins.
NEXT STEP once this is running: the stall line now ends in "REJECTED BY: ..." and
the per-handle depths. Read it. Do not reason around it.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-17 15:21:15 -04:00
|
|
|
m_featureFailBlock = StringFormat("%s - COLD (newest bar EMPTY, whole buffer unreadable),"
|
|
|
|
|
" BarsCalculated=%d", block, ind.BarsCalculated());
|
fix: the trailing incumbent read the future across eras; cold AD blocks cached zeros as truth
Three findings from the 2026-08-11 audit:
1. The excursion head's trailing-quantile ring was deliberately never cleared
between eras ("a rolling estimate of the market, not of the era") - but
pass 3 re-walks the SAME OOS window every era, so at each walk's restart
the ring still held the outcome masks of the newest OOS bars from the
previous walk: the chronological FUTURE of the bars about to be scored.
For the first ~window+horizon pushes of every era the "trailing" incumbent
was partly a leading one - conservative for the gate (an informed incumbent
is a harder hurdle) but exactly the self-made-artifact class 06d4785 hunts.
The ring now clears at era-score reset; the warm-up bars simply don't score
the trail race, which the m_excTrailN gating already accounts for.
2. skillTrail compared the head's FULL-block Brier (pro-rated by coverage)
against the incumbent's subset sum - valid only if head skill is uniform
across the OOS walk, while the trail-scored subset systematically excludes
each era's warm-up bars. The audit also found m_excBrierHeadD/BaseD/
m_excOosHitsD declared, zeroed and never accumulated (dead since e2c9593
made every scored bar disjoint). The dead trio is replaced by
m_excBrierHeadT: the head's Brier accumulated only on the bars the warm
incumbent also scored, so the race now compares both predictors on an
identical bar set.
3. The AD/Wyckoff feature blocks read GetData with no EMPTY_VALUE guard; a
cold (still-calculating) indicator returns EMPTY_VALUE everywhere, the
sanitize loop rewrote that to 0.0, and the bar SUCCEEDED - so
BufferTempData cached an all-zero Wyckoff block as a success for the whole
bar frame: the one path the f6150ee only-cache-successes rule cannot see,
because it never fails (the ba13eef class, arriving through values that
never fail; a resumed model's era-0 prebuild starts milliseconds after
OnInit). ADIndicatorCold() probes the NEWEST bar - EMPTY_VALUE there means
async warm-up (transient reject, retried), while deep bars beyond the
buffered depth keep the sanitize loop's neutral-fill so degraded history
still trains. Also fixed m_featureCacheValid's declaration comment, which
still described the pre-f6150ee cached-miss semantics.
Compile: 0 errors, 0 warnings.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-11 18:20:24 -04:00
|
|
|
return true;
|
|
|
|
|
}
|
|
|
|
|
//+------------------------------------------------------------------+
|
refactor: split CExpertSignalAIBase implementation by responsibility
ExpertSignalAIBase.mqh was 8216 lines: the class declaration followed by 87
method bodies covering training, labelling, feature extraction, persistence,
chart drawing, online learning, the GA auto-tuner and inference, all in one
file. Train() alone is 1492 lines; a change to arrow drawing meant scrolling
past the era loop.
Moved the bodies into Expert\AIBase\, included at the bottom of the original
after the class declaration:
Training.mqh 1607 era loop, plateau ladder, checkpoint select, deploy
Features.mqh 1093 indicator creation + per-bar input feature vector
ChartUI.mqh 634 arrows, arrow persistence, status panel, cleanup
Persistence.mqh 492 .stats/.cfg sidecars, CPU-inference validation, copy
OnlineLearning.mqh 461 live continual learning, EMA shadow, OOS simulator
Labels.mqh 309 ZigZag pivot labels, async label-cache prebuild
AutoTune.mqh 275 genetic tuner (population, crossover, halving)
Inference.mqh 235 softmax, prior calibration, class priors
ExpertSignalAIBase.mqh 8216 -> 3131 (declaration + topology build only)
This is a pure relocation - verified mechanically, not by eye: HEAD's file
reconstructed from the eight partials plus the surviving remainder is
byte-identical to HEAD, span for span (scratchpad verify_split.py). No
declaration moved, no signature changed, no code rewritten, so behaviour is
unchanged by construction.
Compiles 0 errors, 0 warnings.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-29 00:42:45 -04:00
|
|
|
//| Initialize Open indicators. |
|
|
|
|
|
//+------------------------------------------------------------------+
|
|
|
|
|
bool CExpertSignalAIBase::InitOpen(CIndicators * indicators)
|
|
|
|
|
{
|
|
|
|
|
//--- check pointer
|
|
|
|
|
if(indicators == NULL)
|
|
|
|
|
return (false);
|
|
|
|
|
//--- add object to collection
|
|
|
|
|
if(!indicators.Add(GetPointer(m_Open)))
|
|
|
|
|
{
|
|
|
|
|
printf(__FUNCTION__ + ": error adding object");
|
|
|
|
|
return (false);
|
|
|
|
|
}
|
|
|
|
|
//--- initialize object
|
|
|
|
|
if(!m_Open.Create(m_symbol.Name(), m_period))
|
|
|
|
|
{
|
|
|
|
|
printf(__FUNCTION__ + ": error initializing object");
|
|
|
|
|
return (false);
|
|
|
|
|
}
|
|
|
|
|
//--- ok
|
|
|
|
|
return (true);
|
|
|
|
|
}
|
|
|
|
|
//+------------------------------------------------------------------+
|
|
|
|
|
//| Initialize Close indicators. |
|
|
|
|
|
//+------------------------------------------------------------------+
|
|
|
|
|
bool CExpertSignalAIBase::InitClose(CIndicators * indicators)
|
|
|
|
|
{
|
|
|
|
|
//--- check pointer
|
|
|
|
|
if(indicators == NULL)
|
|
|
|
|
return (false);
|
|
|
|
|
//--- add object to collection
|
|
|
|
|
if(!indicators.Add(GetPointer(m_Close)))
|
|
|
|
|
{
|
|
|
|
|
printf(__FUNCTION__ + ": error adding object");
|
|
|
|
|
return (false);
|
|
|
|
|
}
|
|
|
|
|
//--- initialize object
|
|
|
|
|
if(!m_Close.Create(m_symbol.Name(), m_period))
|
|
|
|
|
{
|
|
|
|
|
printf(__FUNCTION__ + ": error initializing object");
|
|
|
|
|
return (false);
|
|
|
|
|
}
|
|
|
|
|
//--- ok
|
|
|
|
|
return (true);
|
|
|
|
|
}
|
|
|
|
|
//+------------------------------------------------------------------+
|
|
|
|
|
//| Initialize High indicators. |
|
|
|
|
|
//+------------------------------------------------------------------+
|
|
|
|
|
bool CExpertSignalAIBase::InitHigh(CIndicators * indicators)
|
|
|
|
|
{
|
|
|
|
|
//--- check pointer
|
|
|
|
|
if(indicators == NULL)
|
|
|
|
|
return (false);
|
|
|
|
|
//--- add object to collection
|
|
|
|
|
if(!indicators.Add(GetPointer(m_High)))
|
|
|
|
|
{
|
|
|
|
|
printf(__FUNCTION__ + ": error adding object");
|
|
|
|
|
return (false);
|
|
|
|
|
}
|
|
|
|
|
//--- initialize object
|
|
|
|
|
if(!m_High.Create(m_symbol.Name(), m_period))
|
|
|
|
|
{
|
|
|
|
|
printf(__FUNCTION__ + ": error initializing object");
|
|
|
|
|
return (false);
|
|
|
|
|
}
|
|
|
|
|
//--- ok
|
|
|
|
|
return (true);
|
|
|
|
|
}
|
|
|
|
|
//+------------------------------------------------------------------+
|
|
|
|
|
//| Initialize Low indicators. |
|
|
|
|
|
//+------------------------------------------------------------------+
|
|
|
|
|
bool CExpertSignalAIBase::InitLow(CIndicators * indicators)
|
|
|
|
|
{
|
|
|
|
|
//--- check pointer
|
|
|
|
|
if(indicators == NULL)
|
|
|
|
|
return (false);
|
|
|
|
|
//--- add object to collection
|
|
|
|
|
if(!indicators.Add(GetPointer(m_Low)))
|
|
|
|
|
{
|
|
|
|
|
printf(__FUNCTION__ + ": error adding object");
|
|
|
|
|
return (false);
|
|
|
|
|
}
|
|
|
|
|
//--- initialize object
|
|
|
|
|
if(!m_Low.Create(m_symbol.Name(), m_period))
|
|
|
|
|
{
|
|
|
|
|
printf(__FUNCTION__ + ": error initializing object");
|
|
|
|
|
return (false);
|
|
|
|
|
}
|
|
|
|
|
//--- ok
|
|
|
|
|
return (true);
|
|
|
|
|
}
|
|
|
|
|
//+------------------------------------------------------------------+
|
|
|
|
|
//| Initialize Time indicators. |
|
|
|
|
|
//+------------------------------------------------------------------+
|
|
|
|
|
bool CExpertSignalAIBase::InitTime(CIndicators * indicators)
|
|
|
|
|
{
|
|
|
|
|
//--- check pointer
|
|
|
|
|
if(indicators == NULL)
|
|
|
|
|
return (false);
|
|
|
|
|
//--- add object to collection
|
|
|
|
|
if(!indicators.Add(GetPointer(m_Time)))
|
|
|
|
|
{
|
|
|
|
|
printf(__FUNCTION__ + ": error adding object");
|
|
|
|
|
return (false);
|
|
|
|
|
}
|
|
|
|
|
//--- initialize object
|
|
|
|
|
if(!m_Time.Create(m_symbol.Name(), m_period))
|
|
|
|
|
{
|
|
|
|
|
printf(__FUNCTION__ + ": error initializing object");
|
|
|
|
|
return (false);
|
|
|
|
|
}
|
|
|
|
|
//--- ok
|
|
|
|
|
return (true);
|
|
|
|
|
}
|
|
|
|
|
//+------------------------------------------------------------------+
|
|
|
|
|
//| Initialize Volumes indicators. |
|
|
|
|
|
//+------------------------------------------------------------------+
|
|
|
|
|
bool CExpertSignalAIBase::InitVolumes(CIndicators * indicators)
|
|
|
|
|
{
|
|
|
|
|
//--- check pointer
|
|
|
|
|
if(indicators == NULL)
|
|
|
|
|
return (false);
|
|
|
|
|
//--- add object to collection
|
|
|
|
|
if(!indicators.Add(GetPointer(m_Volumes)))
|
|
|
|
|
{
|
|
|
|
|
printf(__FUNCTION__ + ": error adding object");
|
|
|
|
|
return (false);
|
|
|
|
|
}
|
|
|
|
|
//--- initialize object
|
|
|
|
|
if(!m_Volumes.Create(m_symbol.Name(), m_period, VolumeData))
|
|
|
|
|
{
|
|
|
|
|
printf(__FUNCTION__ + ": error initializing object");
|
|
|
|
|
return (false);
|
|
|
|
|
}
|
|
|
|
|
//--- ok
|
|
|
|
|
return (true);
|
|
|
|
|
}
|
|
|
|
|
//+------------------------------------------------------------------+
|
|
|
|
|
//| Initialize MA indicator (feature use - see m_useMA). Period comes |
|
|
|
|
|
//| from m_indicatorTuner.maPeriod, not the raw PeriodMA input - it |
|
|
|
|
|
//| starts equal to it (see CADIndicatorTuner's constructor) but may |
|
|
|
|
|
//| diverge once AutoTuneIndicators actually searches a trial. The |
|
|
|
|
|
//| Classic Signals MA vote is unaffected - see m_useMA's declaration |
|
|
|
|
|
//| comment. |
|
|
|
|
|
//+------------------------------------------------------------------+
|
|
|
|
|
bool CExpertSignalAIBase::InitMA(CIndicators * indicators, bool addToCollection)
|
|
|
|
|
{
|
|
|
|
|
if(indicators == NULL)
|
|
|
|
|
return (false);
|
|
|
|
|
if(addToCollection && !indicators.Add(GetPointer(m_MA)))
|
|
|
|
|
{
|
|
|
|
|
printf(__FUNCTION__ + ": error adding object");
|
|
|
|
|
return (false);
|
|
|
|
|
}
|
|
|
|
|
//--- unified MA custom indicator (CustomIndicators\ADMovingAverage.mq5); type AND period are both tuner-
|
|
|
|
|
//--- driven (m_indicatorTuner.maType/maPeriod). params[1..] mirror the indicator's own input order.
|
|
|
|
|
MqlParam params[9];
|
|
|
|
|
params[0].type = TYPE_STRING; params[0].string_value = WARRIOR_CI("ADMovingAverage");
|
|
|
|
|
params[1].type = TYPE_INT; params[1].integer_value = m_indicatorTuner.maType; // InpType
|
|
|
|
|
params[2].type = TYPE_INT; params[2].integer_value = m_indicatorTuner.maPeriod; // InpPeriod
|
|
|
|
|
params[3].type = TYPE_INT; params[3].integer_value = PRICE_CLOSE; // InpAppliedPrice
|
|
|
|
|
params[4].type = TYPE_DOUBLE; params[4].double_value = 0.85; // InpOffset (ALMA)
|
|
|
|
|
params[5].type = TYPE_DOUBLE; params[5].double_value = 6.0; // InpSigma (ALMA)
|
|
|
|
|
params[6].type = TYPE_DOUBLE; params[6].double_value = 0.7; // InpVolumeFactor (T3)
|
|
|
|
|
params[7].type = TYPE_DOUBLE; params[7].double_value = 0.001; // InpProcessNoise (Kalman)
|
|
|
|
|
params[8].type = TYPE_DOUBLE; params[8].double_value = 0.1; // InpMeasurementNoise (Kalman)
|
|
|
|
|
if(!m_MA.Create(m_symbol.Name(), m_period, IND_CUSTOM, 9, params))
|
|
|
|
|
{
|
|
|
|
|
printf(__FUNCTION__ + ": error initializing object");
|
|
|
|
|
return (false);
|
|
|
|
|
}
|
|
|
|
|
m_MA.NumBuffers(1);
|
|
|
|
|
return (true);
|
|
|
|
|
}
|
|
|
|
|
//+------------------------------------------------------------------+
|
|
|
|
|
//| Initialize RSI indicator (feature use - see m_useRSI). Period |
|
|
|
|
|
//| comes from m_indicatorTuner.rsiPeriod - see InitMA()'s comment. |
|
|
|
|
|
//+------------------------------------------------------------------+
|
|
|
|
|
bool CExpertSignalAIBase::InitRSI(CIndicators * indicators, bool addToCollection)
|
|
|
|
|
{
|
|
|
|
|
if(indicators == NULL)
|
|
|
|
|
return (false);
|
|
|
|
|
if(addToCollection && !indicators.Add(GetPointer(m_RSI)))
|
|
|
|
|
{
|
|
|
|
|
printf(__FUNCTION__ + ": error adding object");
|
|
|
|
|
return (false);
|
|
|
|
|
}
|
|
|
|
|
if(!m_RSI.Create(m_symbol.Name(), m_period, m_indicatorTuner.rsiPeriod, PRICE_CLOSE))
|
|
|
|
|
{
|
|
|
|
|
printf(__FUNCTION__ + ": error initializing object");
|
|
|
|
|
return (false);
|
|
|
|
|
}
|
|
|
|
|
return (true);
|
|
|
|
|
}
|
|
|
|
|
//+------------------------------------------------------------------+
|
|
|
|
|
//| Initialize MACD indicator (feature use - see m_useMACD). Periods |
|
|
|
|
|
//| come from m_indicatorTuner.macdFast/macdSlow/macdSignal - see |
|
|
|
|
|
//| InitMA()'s comment for the "starts at the input, may diverge once |
|
|
|
|
|
//| the tuner searches" split, and note the Classic Signals MACD vote |
|
|
|
|
|
//| (Signals\SignalMACD.mqh) keeps its own separate instance. |
|
|
|
|
|
//+------------------------------------------------------------------+
|
|
|
|
|
bool CExpertSignalAIBase::InitMACDFeature(CIndicators * indicators, bool addToCollection)
|
|
|
|
|
{
|
|
|
|
|
if(indicators == NULL)
|
|
|
|
|
return (false);
|
|
|
|
|
if(addToCollection && !indicators.Add(GetPointer(m_MACDFeature)))
|
|
|
|
|
{
|
|
|
|
|
printf(__FUNCTION__ + ": error adding object");
|
|
|
|
|
return (false);
|
|
|
|
|
}
|
|
|
|
|
if(!m_MACDFeature.Create(m_symbol.Name(), m_period, m_indicatorTuner.macdFast, m_indicatorTuner.macdSlow,
|
|
|
|
|
m_indicatorTuner.macdSignal, PRICE_CLOSE))
|
|
|
|
|
{
|
|
|
|
|
printf(__FUNCTION__ + ": error initializing object");
|
|
|
|
|
return (false);
|
|
|
|
|
}
|
|
|
|
|
return (true);
|
|
|
|
|
}
|
|
|
|
|
//+------------------------------------------------------------------+
|
|
|
|
|
//| Initialize Ichimoku indicator (feature use - see m_useIchimoku). |
|
|
|
|
|
//| Periods come from m_indicatorTuner.ichiTenkan/ichiKijun/ |
|
|
|
|
|
//| ichiSenkou - see InitMA()'s comment. The Classic Signals Ichimoku |
|
|
|
|
|
//| vote (Signals\SignalIchimoku.mqh) keeps its own instance. |
|
|
|
|
|
//+------------------------------------------------------------------+
|
|
|
|
|
bool CExpertSignalAIBase::InitIchimoku(CIndicators * indicators, bool addToCollection)
|
|
|
|
|
{
|
|
|
|
|
if(indicators == NULL)
|
|
|
|
|
return (false);
|
|
|
|
|
if(addToCollection && !indicators.Add(GetPointer(m_Ichimoku)))
|
|
|
|
|
{
|
|
|
|
|
printf(__FUNCTION__ + ": error adding object");
|
|
|
|
|
return (false);
|
|
|
|
|
}
|
|
|
|
|
if(!m_Ichimoku.Create(m_symbol.Name(), m_period, m_indicatorTuner.ichiTenkan, m_indicatorTuner.ichiKijun,
|
|
|
|
|
m_indicatorTuner.ichiSenkou))
|
|
|
|
|
{
|
|
|
|
|
printf(__FUNCTION__ + ": error initializing object");
|
|
|
|
|
return (false);
|
|
|
|
|
}
|
|
|
|
|
return (true);
|
|
|
|
|
}
|
|
|
|
|
//+------------------------------------------------------------------+
|
|
|
|
|
//| Initialize AD Cumulative Delta (CustomIndicators\ADCumulativeDelta.mq5) |
|
|
|
|
|
//| Loaded via iCustom/CiCustom, not a built-in Ci* class - the compiled |
|
|
|
|
|
//| indicator must be present under MQL5\Indicators\ (see |
|
|
|
|
|
//| ExtractCustomIndicators() in Warrior_EA.mq5). Uses the indicator's |
|
|
|
|
|
//| own input defaults; 6 output buffers, one TempData feature each. |
|
|
|
|
|
//+------------------------------------------------------------------+
|
|
|
|
|
bool CExpertSignalAIBase::InitADCumulativeDelta(CIndicators * indicators, bool addToCollection)
|
|
|
|
|
{
|
|
|
|
|
//--- check pointer
|
|
|
|
|
if(indicators == NULL)
|
|
|
|
|
return (false);
|
|
|
|
|
//--- add object to collection
|
|
|
|
|
if(addToCollection && !indicators.Add(GetPointer(m_ADCumulativeDelta)))
|
|
|
|
|
{
|
|
|
|
|
printf(__FUNCTION__ + ": error adding object");
|
|
|
|
|
return (false);
|
|
|
|
|
}
|
|
|
|
|
//--- initialize object; params[1..] mirror ADCumulativeDelta.mq5's own input order exactly
|
|
|
|
|
MqlParam params[11];
|
|
|
|
|
params[0].type = TYPE_STRING;
|
|
|
|
|
params[0].string_value = WARRIOR_CI("ADCumulativeDelta");
|
|
|
|
|
params[1].type = TYPE_INT;
|
|
|
|
|
params[1].integer_value = m_indicatorTuner.adCumDelta.lookback; // InpLookbackPeriod
|
|
|
|
|
params[2].type = TYPE_DOUBLE;
|
|
|
|
|
params[2].double_value = m_indicatorTuner.adCumDelta.volClimax; // InpVolumeClimaxMultiplier
|
|
|
|
|
params[3].type = TYPE_DOUBLE;
|
|
|
|
|
params[3].double_value = m_indicatorTuner.adCumDelta.volHigh; // InpVolumeHighMultiplier
|
|
|
|
|
params[4].type = TYPE_DOUBLE;
|
|
|
|
|
params[4].double_value = m_indicatorTuner.adCumDelta.rangeClimax; // InpRangeClimaxMultiplier
|
|
|
|
|
params[5].type = TYPE_DOUBLE;
|
|
|
|
|
params[5].double_value = m_indicatorTuner.adCumDelta.rangeSignificant; // InpRangeSignificantMult
|
|
|
|
|
params[6].type = TYPE_DOUBLE;
|
|
|
|
|
params[6].double_value = m_indicatorTuner.adCumDelta.stVolRatio; // InpSTVolumeRatio
|
|
|
|
|
params[7].type = TYPE_DOUBLE;
|
|
|
|
|
params[7].double_value = m_indicatorTuner.adCumDelta.atrMult; // InpATRMultiplier
|
|
|
|
|
params[8].type = TYPE_INT;
|
|
|
|
|
params[8].integer_value = 0; // InpContextMode - DO NOT tune
|
|
|
|
|
params[9].type = TYPE_INT;
|
|
|
|
|
params[9].integer_value = 5; // InpSessionType - DO NOT tune
|
|
|
|
|
params[10].type = TYPE_INT;
|
|
|
|
|
params[10].integer_value = 1; // InpSessionCount - DO NOT tune
|
|
|
|
|
if(!m_ADCumulativeDelta.Create(m_symbol.Name(), m_period, IND_CUSTOM, 11, params))
|
|
|
|
|
{
|
|
|
|
|
printf(__FUNCTION__ + ": error initializing object");
|
|
|
|
|
return (false);
|
|
|
|
|
}
|
|
|
|
|
m_ADCumulativeDelta.NumBuffers(6);
|
|
|
|
|
//--- ok
|
|
|
|
|
return (true);
|
|
|
|
|
}
|
|
|
|
|
//+------------------------------------------------------------------+
|
|
|
|
|
//| Initialize AD Shortening of Thrust (CustomIndicators\ADShorteningOfThrust.mq5) |
|
|
|
|
|
//+------------------------------------------------------------------+
|
|
|
|
|
bool CExpertSignalAIBase::InitADShorteningOfThrust(CIndicators * indicators, bool addToCollection)
|
|
|
|
|
{
|
|
|
|
|
//--- check pointer
|
|
|
|
|
if(indicators == NULL)
|
|
|
|
|
return (false);
|
|
|
|
|
//--- add object to collection
|
|
|
|
|
if(addToCollection && !indicators.Add(GetPointer(m_ADShorteningOfThrust)))
|
|
|
|
|
{
|
|
|
|
|
printf(__FUNCTION__ + ": error adding object");
|
|
|
|
|
return (false);
|
|
|
|
|
}
|
|
|
|
|
//--- initialize object; params[1..] mirror ADShorteningOfThrust.mq5's own input order exactly
|
|
|
|
|
MqlParam params[7];
|
|
|
|
|
params[0].type = TYPE_STRING;
|
|
|
|
|
params[0].string_value = WARRIOR_CI("ADShorteningOfThrust");
|
|
|
|
|
params[1].type = TYPE_INT;
|
|
|
|
|
params[1].integer_value = m_indicatorTuner.adSOT.thrustLookback; // InpThrustLookback
|
|
|
|
|
params[2].type = TYPE_INT;
|
|
|
|
|
params[2].integer_value = m_indicatorTuner.adSOT.minImpulses; // InpMinImpulses
|
|
|
|
|
params[3].type = TYPE_DOUBLE;
|
|
|
|
|
params[3].double_value = m_indicatorTuner.adSOT.sotThreshold; // InpSOTThreshold
|
|
|
|
|
params[4].type = TYPE_INT;
|
|
|
|
|
params[4].integer_value = 0; // InpContextMode - DO NOT tune
|
|
|
|
|
params[5].type = TYPE_INT;
|
|
|
|
|
params[5].integer_value = 5; // InpSessionType - DO NOT tune
|
|
|
|
|
params[6].type = TYPE_INT;
|
|
|
|
|
params[6].integer_value = 1; // InpSessionCount - DO NOT tune
|
|
|
|
|
if(!m_ADShorteningOfThrust.Create(m_symbol.Name(), m_period, IND_CUSTOM, 7, params))
|
|
|
|
|
{
|
|
|
|
|
printf(__FUNCTION__ + ": error initializing object");
|
|
|
|
|
return (false);
|
|
|
|
|
}
|
|
|
|
|
m_ADShorteningOfThrust.NumBuffers(4);
|
|
|
|
|
//--- ok
|
|
|
|
|
return (true);
|
|
|
|
|
}
|
|
|
|
|
//+------------------------------------------------------------------+
|
|
|
|
|
//| Initialize AD Wyckoff Event Stream (CustomIndicators\ADWyckoffEventStream.mq5) |
|
|
|
|
|
//+------------------------------------------------------------------+
|
|
|
|
|
bool CExpertSignalAIBase::InitADWyckoffEventStream(CIndicators * indicators, bool addToCollection)
|
|
|
|
|
{
|
|
|
|
|
//--- check pointer
|
|
|
|
|
if(indicators == NULL)
|
|
|
|
|
return (false);
|
|
|
|
|
//--- add object to collection
|
|
|
|
|
if(addToCollection && !indicators.Add(GetPointer(m_ADWyckoffEventStream)))
|
|
|
|
|
{
|
|
|
|
|
printf(__FUNCTION__ + ": error adding object");
|
|
|
|
|
return (false);
|
|
|
|
|
}
|
2026-08-02 17:08:48 -04:00
|
|
|
//--- initialize object; params[1..] mirror ADWyckoffEventStream.mq5's own input order exactly.
|
|
|
|
|
//--- NOTE the order is NOT grouped by meaning: the three range-lifecycle knobs the indicator gained on
|
|
|
|
|
//--- 2026-08-02 were appended AFTER the session inputs, not next to the thresholds they belong with.
|
|
|
|
|
//--- MqlParam is positional, so this list follows the indicator's declaration order, not a tidier one.
|
|
|
|
|
MqlParam params[17];
|
refactor: split CExpertSignalAIBase implementation by responsibility
ExpertSignalAIBase.mqh was 8216 lines: the class declaration followed by 87
method bodies covering training, labelling, feature extraction, persistence,
chart drawing, online learning, the GA auto-tuner and inference, all in one
file. Train() alone is 1492 lines; a change to arrow drawing meant scrolling
past the era loop.
Moved the bodies into Expert\AIBase\, included at the bottom of the original
after the class declaration:
Training.mqh 1607 era loop, plateau ladder, checkpoint select, deploy
Features.mqh 1093 indicator creation + per-bar input feature vector
ChartUI.mqh 634 arrows, arrow persistence, status panel, cleanup
Persistence.mqh 492 .stats/.cfg sidecars, CPU-inference validation, copy
OnlineLearning.mqh 461 live continual learning, EMA shadow, OOS simulator
Labels.mqh 309 ZigZag pivot labels, async label-cache prebuild
AutoTune.mqh 275 genetic tuner (population, crossover, halving)
Inference.mqh 235 softmax, prior calibration, class priors
ExpertSignalAIBase.mqh 8216 -> 3131 (declaration + topology build only)
This is a pure relocation - verified mechanically, not by eye: HEAD's file
reconstructed from the eight partials plus the surviving remainder is
byte-identical to HEAD, span for span (scratchpad verify_split.py). No
declaration moved, no signature changed, no code rewritten, so behaviour is
unchanged by construction.
Compiles 0 errors, 0 warnings.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-29 00:42:45 -04:00
|
|
|
params[0].type = TYPE_STRING;
|
|
|
|
|
params[0].string_value = WARRIOR_CI("ADWyckoffEventStream");
|
|
|
|
|
params[1].type = TYPE_INT;
|
|
|
|
|
params[1].integer_value = m_indicatorTuner.adWES.lookback; // InpLookback
|
|
|
|
|
params[2].type = TYPE_INT;
|
|
|
|
|
params[2].integer_value = m_indicatorTuner.adWES.zigzag; // InpZigZag
|
|
|
|
|
params[3].type = TYPE_DOUBLE;
|
|
|
|
|
params[3].double_value = m_indicatorTuner.adWES.volClimax; // InpVolClimax
|
|
|
|
|
params[4].type = TYPE_DOUBLE;
|
|
|
|
|
params[4].double_value = m_indicatorTuner.adWES.volHigh; // InpVolHigh
|
|
|
|
|
params[5].type = TYPE_DOUBLE;
|
|
|
|
|
params[5].double_value = m_indicatorTuner.adWES.rangeClimax; // InpRangeClimax
|
|
|
|
|
params[6].type = TYPE_DOUBLE;
|
|
|
|
|
params[6].double_value = m_indicatorTuner.adWES.rangeSignificant; // InpRangeSignificant
|
|
|
|
|
params[7].type = TYPE_DOUBLE;
|
|
|
|
|
params[7].double_value = m_indicatorTuner.adWES.stVolRatio; // InpSTVolRatio
|
|
|
|
|
params[8].type = TYPE_DOUBLE;
|
|
|
|
|
params[8].double_value = m_indicatorTuner.adWES.atr; // InpATR
|
|
|
|
|
params[9].type = TYPE_INT;
|
|
|
|
|
params[9].integer_value = 0; // InpContextMode - DO NOT tune
|
|
|
|
|
params[10].type = TYPE_INT;
|
|
|
|
|
params[10].integer_value = 5; // InpSessionType - DO NOT tune
|
|
|
|
|
params[11].type = TYPE_INT;
|
|
|
|
|
params[11].integer_value = 1; // InpSessionCount - DO NOT tune
|
2026-08-02 17:08:48 -04:00
|
|
|
params[12].type = TYPE_DOUBLE;
|
|
|
|
|
params[12].double_value = m_indicatorTuner.adWES.touchATR; // InpTouchATR
|
|
|
|
|
params[13].type = TYPE_DOUBLE;
|
|
|
|
|
params[13].double_value = m_indicatorTuner.adWES.arMinATR; // InpARMinATR
|
|
|
|
|
params[14].type = TYPE_INT;
|
|
|
|
|
params[14].integer_value = m_indicatorTuner.adWES.maxRangeBars; // InpMaxRangeBars
|
|
|
|
|
//--- InpShowLabels/InpShowZones - forced OFF, and deliberately NOT tunable. This handle exists purely
|
|
|
|
|
//--- to read buffers as network features; it is never the user's chart indicator. Left at their own
|
|
|
|
|
//--- `true` defaults the indicator would litter the traded chart with AWY_-prefixed labels and range
|
|
|
|
|
//--- rectangles that the EA does not own and its OnDeinit chart sweep does not know to remove.
|
|
|
|
|
params[15].type = TYPE_BOOL;
|
|
|
|
|
params[15].integer_value = 0; // InpShowLabels
|
|
|
|
|
params[16].type = TYPE_BOOL;
|
|
|
|
|
params[16].integer_value = 0; // InpShowZones
|
|
|
|
|
if(!m_ADWyckoffEventStream.Create(m_symbol.Name(), m_period, IND_CUSTOM, 17, params))
|
refactor: split CExpertSignalAIBase implementation by responsibility
ExpertSignalAIBase.mqh was 8216 lines: the class declaration followed by 87
method bodies covering training, labelling, feature extraction, persistence,
chart drawing, online learning, the GA auto-tuner and inference, all in one
file. Train() alone is 1492 lines; a change to arrow drawing meant scrolling
past the era loop.
Moved the bodies into Expert\AIBase\, included at the bottom of the original
after the class declaration:
Training.mqh 1607 era loop, plateau ladder, checkpoint select, deploy
Features.mqh 1093 indicator creation + per-bar input feature vector
ChartUI.mqh 634 arrows, arrow persistence, status panel, cleanup
Persistence.mqh 492 .stats/.cfg sidecars, CPU-inference validation, copy
OnlineLearning.mqh 461 live continual learning, EMA shadow, OOS simulator
Labels.mqh 309 ZigZag pivot labels, async label-cache prebuild
AutoTune.mqh 275 genetic tuner (population, crossover, halving)
Inference.mqh 235 softmax, prior calibration, class priors
ExpertSignalAIBase.mqh 8216 -> 3131 (declaration + topology build only)
This is a pure relocation - verified mechanically, not by eye: HEAD's file
reconstructed from the eight partials plus the surviving remainder is
byte-identical to HEAD, span for span (scratchpad verify_split.py). No
declaration moved, no signature changed, no code rewritten, so behaviour is
unchanged by construction.
Compiles 0 errors, 0 warnings.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-29 00:42:45 -04:00
|
|
|
{
|
|
|
|
|
printf(__FUNCTION__ + ": error initializing object");
|
|
|
|
|
return (false);
|
|
|
|
|
}
|
|
|
|
|
m_ADWyckoffEventStream.NumBuffers(14);
|
|
|
|
|
//--- ok
|
|
|
|
|
return (true);
|
|
|
|
|
}
|
|
|
|
|
//+------------------------------------------------------------------+
|
|
|
|
|
//| Initialize AD Wyckoff Failed Structure (CustomIndicators\ADWyckoffFailedStructure.mq5) |
|
|
|
|
|
//+------------------------------------------------------------------+
|
|
|
|
|
bool CExpertSignalAIBase::InitADWyckoffFailedStructure(CIndicators * indicators, bool addToCollection)
|
|
|
|
|
{
|
|
|
|
|
//--- check pointer
|
|
|
|
|
if(indicators == NULL)
|
|
|
|
|
return (false);
|
|
|
|
|
//--- add object to collection
|
|
|
|
|
if(addToCollection && !indicators.Add(GetPointer(m_ADWyckoffFailedStructure)))
|
|
|
|
|
{
|
|
|
|
|
printf(__FUNCTION__ + ": error adding object");
|
|
|
|
|
return (false);
|
|
|
|
|
}
|
|
|
|
|
//--- initialize object; params[1..] mirror ADWyckoffFailedStructure.mq5's own input order exactly
|
|
|
|
|
MqlParam params[12];
|
|
|
|
|
params[0].type = TYPE_STRING;
|
|
|
|
|
params[0].string_value = WARRIOR_CI("ADWyckoffFailedStructure");
|
|
|
|
|
params[1].type = TYPE_INT;
|
|
|
|
|
params[1].integer_value = m_indicatorTuner.adWFS.lookback; // InpLookbackPeriod
|
|
|
|
|
params[2].type = TYPE_INT;
|
|
|
|
|
params[2].integer_value = m_indicatorTuner.adWFS.zigzagStrength; // InpZigZagStrength
|
|
|
|
|
params[3].type = TYPE_DOUBLE;
|
|
|
|
|
params[3].double_value = m_indicatorTuner.adWFS.volClimax; // InpVolumeClimaxMultiplier
|
|
|
|
|
params[4].type = TYPE_DOUBLE;
|
|
|
|
|
params[4].double_value = m_indicatorTuner.adWFS.volHigh; // InpVolumeHighMultiplier
|
|
|
|
|
params[5].type = TYPE_DOUBLE;
|
|
|
|
|
params[5].double_value = m_indicatorTuner.adWFS.rangeClimax; // InpRangeClimaxMultiplier
|
|
|
|
|
params[6].type = TYPE_DOUBLE;
|
|
|
|
|
params[6].double_value = m_indicatorTuner.adWFS.rangeSignificant; // InpRangeSignificantMult
|
|
|
|
|
params[7].type = TYPE_DOUBLE;
|
|
|
|
|
params[7].double_value = m_indicatorTuner.adWFS.stVolRatio; // InpSTVolumeRatio
|
|
|
|
|
params[8].type = TYPE_DOUBLE;
|
|
|
|
|
params[8].double_value = m_indicatorTuner.adWFS.atrMult; // InpATRMultiplier
|
|
|
|
|
params[9].type = TYPE_INT;
|
|
|
|
|
params[9].integer_value = 0; // InpContextMode - DO NOT tune
|
|
|
|
|
params[10].type = TYPE_INT;
|
|
|
|
|
params[10].integer_value = 5; // InpSessionType - DO NOT tune
|
|
|
|
|
params[11].type = TYPE_INT;
|
|
|
|
|
params[11].integer_value = 1; // InpSessionCount - DO NOT tune
|
|
|
|
|
if(!m_ADWyckoffFailedStructure.Create(m_symbol.Name(), m_period, IND_CUSTOM, 12, params))
|
|
|
|
|
{
|
|
|
|
|
printf(__FUNCTION__ + ": error initializing object");
|
|
|
|
|
return (false);
|
|
|
|
|
}
|
|
|
|
|
m_ADWyckoffFailedStructure.NumBuffers(5);
|
|
|
|
|
//--- ok
|
|
|
|
|
return (true);
|
|
|
|
|
}
|
|
|
|
|
//+------------------------------------------------------------------+
|
|
|
|
|
//| Initialize AD Wyckoff Significant Bar Inversion (CustomIndicators\ADWyckoffSignificantBarInversion.mq5) |
|
|
|
|
|
//+------------------------------------------------------------------+
|
|
|
|
|
bool CExpertSignalAIBase::InitADWyckoffSignificantBarInversion(CIndicators * indicators, bool addToCollection)
|
|
|
|
|
{
|
|
|
|
|
//--- check pointer
|
|
|
|
|
if(indicators == NULL)
|
|
|
|
|
return (false);
|
|
|
|
|
//--- add object to collection
|
|
|
|
|
if(addToCollection && !indicators.Add(GetPointer(m_ADWyckoffSignificantBarInversion)))
|
|
|
|
|
{
|
|
|
|
|
printf(__FUNCTION__ + ": error adding object");
|
|
|
|
|
return (false);
|
|
|
|
|
}
|
|
|
|
|
//--- initialize object; params[1..] mirror ADWyckoffSignificantBarInversion.mq5's own input order exactly
|
|
|
|
|
MqlParam params[8];
|
|
|
|
|
params[0].type = TYPE_STRING;
|
|
|
|
|
params[0].string_value = WARRIOR_CI("ADWyckoffSignificantBarInversion");
|
|
|
|
|
params[1].type = TYPE_INT;
|
|
|
|
|
params[1].integer_value = m_indicatorTuner.adWSBI.lookback; // InpLookback
|
|
|
|
|
params[2].type = TYPE_DOUBLE;
|
|
|
|
|
params[2].double_value = m_indicatorTuner.adWSBI.rangeSignificant; // InpRangeSignificant
|
|
|
|
|
params[3].type = TYPE_DOUBLE;
|
|
|
|
|
params[3].double_value = m_indicatorTuner.adWSBI.volumeHigh; // InpVolumeHigh
|
|
|
|
|
params[4].type = TYPE_DOUBLE;
|
|
|
|
|
params[4].double_value = m_indicatorTuner.adWSBI.atr; // InpATR
|
|
|
|
|
params[5].type = TYPE_INT;
|
|
|
|
|
params[5].integer_value = 0; // InpContextMode - DO NOT tune
|
|
|
|
|
params[6].type = TYPE_INT;
|
|
|
|
|
params[6].integer_value = 5; // InpSessionType - DO NOT tune
|
|
|
|
|
params[7].type = TYPE_INT;
|
|
|
|
|
params[7].integer_value = 1; // InpSessionCount - DO NOT tune
|
|
|
|
|
if(!m_ADWyckoffSignificantBarInversion.Create(m_symbol.Name(), m_period, IND_CUSTOM, 8, params))
|
|
|
|
|
{
|
|
|
|
|
printf(__FUNCTION__ + ": error initializing object");
|
|
|
|
|
return (false);
|
|
|
|
|
}
|
|
|
|
|
m_ADWyckoffSignificantBarInversion.NumBuffers(5);
|
|
|
|
|
//--- ok
|
|
|
|
|
return (true);
|
|
|
|
|
}
|
|
|
|
|
//+------------------------------------------------------------------+
|
|
|
|
|
//| Initialize AD ZigZag (CustomIndicators\ADZigZag.mq5) - the |
|
|
|
|
|
//| training-label source (see m_ADZigZag's declaration comment). |
|
|
|
|
|
//| Always run at its own stock defaults (Depth=12, Deviation=5, |
|
|
|
|
|
//| Backstep=3) - unlike the AD* feature indicators above, this has |
|
|
|
|
|
//| no tunable-param struct and is never touched by AutoTuneIndicators.|
|
|
|
|
|
//+------------------------------------------------------------------+
|
|
|
|
|
bool CExpertSignalAIBase::InitADZigZag(CIndicators * indicators, bool addToCollection)
|
|
|
|
|
{
|
|
|
|
|
//--- check pointer
|
|
|
|
|
if(indicators == NULL)
|
|
|
|
|
return (false);
|
|
|
|
|
//--- add object to collection
|
|
|
|
|
if(addToCollection && !indicators.Add(GetPointer(m_ADZigZag)))
|
|
|
|
|
{
|
|
|
|
|
printf(__FUNCTION__ + ": error adding object");
|
|
|
|
|
return (false);
|
|
|
|
|
}
|
|
|
|
|
//--- initialize object; params[1..] mirror ADZigZag.mq5's own input order exactly - stock defaults,
|
|
|
|
|
//--- intentionally not sourced from a tunable params struct (see this function's declaration comment)
|
|
|
|
|
MqlParam params[4];
|
|
|
|
|
params[0].type = TYPE_STRING;
|
|
|
|
|
params[0].string_value = WARRIOR_CI("ADZigZag");
|
|
|
|
|
params[1].type = TYPE_INT;
|
|
|
|
|
params[1].integer_value = 12; // InpDepth
|
|
|
|
|
params[2].type = TYPE_INT;
|
|
|
|
|
params[2].integer_value = 5; // InpDeviation
|
|
|
|
|
params[3].type = TYPE_INT;
|
|
|
|
|
params[3].integer_value = 3; // InpBackstep
|
|
|
|
|
if(!m_ADZigZag.Create(m_symbol.Name(), m_period, IND_CUSTOM, 4, params))
|
|
|
|
|
{
|
|
|
|
|
printf(__FUNCTION__ + ": error initializing object");
|
|
|
|
|
return (false);
|
|
|
|
|
}
|
|
|
|
|
// Must match ADZigZag.mq5's #property indicator_buffers exactly (3: main ZigZag buffer + 2
|
|
|
|
|
// internal INDICATOR_CALCULATIONS buffers), even though only buffer 0 is ever read via
|
|
|
|
|
// GetData() - see the working AD Wyckoff indicators' InitAD*() for the same pattern.
|
|
|
|
|
m_ADZigZag.NumBuffers(3);
|
|
|
|
|
//--- ok
|
|
|
|
|
return (true);
|
|
|
|
|
}
|
|
|
|
|
#endif // WARRIOR_AIBASE_FEATURES_MQH
|