Warrior_EA/Expert/AIBase/Topology.mqh

1569 lines
108 KiB
MQL5
Raw Permalink Normal View History

//+------------------------------------------------------------------+
//| Topology.mqh |
//| |
//| Network bootstrap and topology construction: the derived shape |
//| (width/taper/depth/conv filters/LSTM hidden), the conv, LSTM and |
//| batch-norm stages, BuildFreshTopology and InitIndicators. |
//| |
//| PARTIAL IMPLEMENTATION FILE - not standalone. |
//| CExpertSignalAIBase method BODIES only. The class declaration |
//| lives in Expert\ExpertSignalAIBase.mqh, which includes this file |
//| at the bottom, after the declaration. Do not include it |
//| anywhere else and do not compile it on its own. |
//+------------------------------------------------------------------+
#ifndef WARRIOR_AIBASE_TOPOLOGY_MQH
#define WARRIOR_AIBASE_TOPOLOGY_MQH
//+------------------------------------------------------------------+
//| Common network bootstrap shared by every AI signal: sets up |
//| indicators, then loads a saved network or builds a fresh one |
//| whose only per-signal-type difference is AddCustomLayers(). |
//+------------------------------------------------------------------+
bool CExpertSignalAIBase::InitNeuralNetwork(CIndicators *indicators)
{
if(m_isInitialized)
return true;
if(indicators == NULL)
return false;
m_indicatorsPtr = indicators;
if(!CExpertSignalCustom::InitIndicators(indicators))
return false;
if(!CExpertSignalAIBase::InitIndicators(indicators))
return false;
//--- Kick the terminal's async history sync for every cross-asset reference symbol NOW, at init,
//--- so the ~minute of cross-symbol download runs while the model loads and the label cache
//--- prebuilds - instead of starting only when the first Build() call finds the symbols unselected
//--- and the first era (and the one-shot MI report) runs with the panel absent. Non-blocking.
if(m_useCrossAsset)
m_crossAsset.Warm((ENUM_TIMEFRAMES)m_period);
Net = new CNet(NULL);
if(CheckPointer(Net) == POINTER_INVALID)
return false;
//--- Size the first dense layer to the data. HERE and only here: it must be settled before the
//--- fingerprint below (which hashes it) and must never move afterwards - see ComputeFirstLayerWidth()
//--- and the note on fingerprint-feeding members at the top of this file. InitIndicators() above is
//--- what finalises m_neuronsCount, so this is the earliest point the input width is actually known.
feat: mini-batch gradient accumulation (F4), front-end-aware capacity budget (F6), split Wyckoff categoricals (N1) Completes the 2026-08-09 training audit. FORCES A RETRAIN of every Wyckoff-enabled config (N1 re-keys the fingerprint), and BOTH DLLs must be redeployed alongside the .ex5 - they carry new exports. F4 - mini-batch accumulation, TRAIN_BATCH_SIZE=32. Training was pure online SGD (one weight update per bar), which is the mechanical source of the era-to-era whipsaw every downstream guard was built to cope with. The O(n^2) outer product is native - AccumulateWeightGrad / AccumulateWeightGradConv / AccumulateBufferInto in Network.cl, WarriorCPU and WarriorDML - while the optimizer step is host-side MQL5 shared by all tiers (ApplyAccumToBlock), so there is one Adam/SGD implementation instead of four that can drift. - the LSTM needs no outer-product kernel (WeightsGradient already holds the sample's full dW) but could NOT simply be left un-zeroed between samples: CPU_LSTMSeqBackward/DML_LSTMSeqBackward memset it on entry. Hence a separate accumulator plus an elementwise add. - batch-norm gamma/beta accumulate in host arrays, not new BatchOptions slots - BN_OPT_STRIDE is baked into every persisted .nnw. - scoped to pass 2; online learning keeps immediate updates. Every save / checkpoint / scoring boundary flushes, scaling by the real sample count. - degrades to per-sample updates (one log line) on a tier that cannot accumulate, so old devices and DLL-free builds are unaffected. - verified offline: DirectML/batch_accum_check.cpp drives the real exports against an independent reference; at B=1 the accumulator matches the shipped unbatched kernel's own gradient to 1.1e-16. Math only - the in-situ check remains the per-layer dW/W report on a real era. F6 - ComputeFirstLayerWidth budgeted against the RAW input width even where a conv/LSTM front end had already reduced it, so an LSTM's dense stack was charged for 1,280 inputs when it receives 64. Confirmed from the deployed .cfg files: CONV, LSTM and HYBRID were all pinned at the 16-unit floor. Now budgeted against the front-end output and capped at it (never fan out), with the derivation reordered so both stages settle first. N1 - EventCode/EventPhase/StructuralPhase are signed categoricals packing direction and Wyckoff stage into one scalar across a sign discontinuity. Split into direction + [0,1] magnitude, the same convention the base OHLC block uses. Information-preserving; 13 readings now occupy 16 inputs. Compiled clean (0 errors, 0 warnings); both DLLs rebuilt. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-09 11:48:03 -04:00
//--- ORDER MATTERS, and it changed on 2026-08-09. The conv and LSTM stages settle FIRST, because
//--- ComputeFirstLayerWidth() now budgets the dense stack against the width that actually reaches it
//--- - which, on any topology with a front end, is that stage's output and not the raw input vector.
//--- Neither of these two depends on m_initialNeuronsCount, so moving them ahead of it is safe;
//--- depth still comes last because it is derived FROM the first-layer width.
//--- Both are also sized from the data rather than configured. Unconditional - a plain MLP simply
//--- never builds the stages these describe, and branching on the topology type would make the
//--- .cfg contents depend on which subclass is asking.
//--- The input WINDOW settles first of all: every shape below multiplies by it. Derived for a
//--- genuinely new model; an existing model ADOPTS the window it was trained with from its .cfg
//--- further down, exactly like the four shape fields (the .nnw remains the ultimate authority).
//--- Deliberately after InitIndicators (which needs none of it) and before the fingerprint, whose
//--- window slot is a LEGACY literal precisely so this measurement cannot re-key a filename.
m_historyBars = DeriveHistoryBars();
m_convFilterCount = ComputeConvFilterCount();
m_lstmHiddenSize = ComputeLstmHiddenSize();
feat: mini-batch gradient accumulation (F4), front-end-aware capacity budget (F6), split Wyckoff categoricals (N1) Completes the 2026-08-09 training audit. FORCES A RETRAIN of every Wyckoff-enabled config (N1 re-keys the fingerprint), and BOTH DLLs must be redeployed alongside the .ex5 - they carry new exports. F4 - mini-batch accumulation, TRAIN_BATCH_SIZE=32. Training was pure online SGD (one weight update per bar), which is the mechanical source of the era-to-era whipsaw every downstream guard was built to cope with. The O(n^2) outer product is native - AccumulateWeightGrad / AccumulateWeightGradConv / AccumulateBufferInto in Network.cl, WarriorCPU and WarriorDML - while the optimizer step is host-side MQL5 shared by all tiers (ApplyAccumToBlock), so there is one Adam/SGD implementation instead of four that can drift. - the LSTM needs no outer-product kernel (WeightsGradient already holds the sample's full dW) but could NOT simply be left un-zeroed between samples: CPU_LSTMSeqBackward/DML_LSTMSeqBackward memset it on entry. Hence a separate accumulator plus an elementwise add. - batch-norm gamma/beta accumulate in host arrays, not new BatchOptions slots - BN_OPT_STRIDE is baked into every persisted .nnw. - scoped to pass 2; online learning keeps immediate updates. Every save / checkpoint / scoring boundary flushes, scaling by the real sample count. - degrades to per-sample updates (one log line) on a tier that cannot accumulate, so old devices and DLL-free builds are unaffected. - verified offline: DirectML/batch_accum_check.cpp drives the real exports against an independent reference; at B=1 the accumulator matches the shipped unbatched kernel's own gradient to 1.1e-16. Math only - the in-situ check remains the per-layer dW/W report on a real era. F6 - ComputeFirstLayerWidth budgeted against the RAW input width even where a conv/LSTM front end had already reduced it, so an LSTM's dense stack was charged for 1,280 inputs when it receives 64. Confirmed from the deployed .cfg files: CONV, LSTM and HYBRID were all pinned at the 16-unit floor. Now budgeted against the front-end output and capped at it (never fan out), with the derivation reordered so both stages settle first. N1 - EventCode/EventPhase/StructuralPhase are signed categoricals packing direction and Wyckoff stage into one scalar across a sign discontinuity. Split into direction + [0,1] magnitude, the same convention the base OHLC block uses. Information-preserving; 13 readings now occupy 16 inputs. Compiled clean (0 errors, 0 warnings); both DLLs rebuilt. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-09 11:48:03 -04:00
m_initialNeuronsCount = ComputeFirstLayerWidth();
//--- Depth LAST of the four: it is derived from the first-layer width above, so it cannot be settled
//--- before that one is. All four are overwritten from the .cfg further below if this configuration
//--- already has a trained model - see the adopt-don't-compare block there.
m_hiddenLayersCount = ComputeHiddenLayerCount();
fix(labels): the 128-bar horizon ceiling was truncating the shipped label The corrected geometry scan exposed something bigger than the geometry question it was asked. Every pairing from 2:6 upward came back CLAMPED - including 2:6, the SHIPPED configuration. First-passage time for a driftless walk leaving [-m,+k] goes as m*k, and the measured swing median here is ~12 bars at m*k=1, so 2:6 wants ~144 bars and 3:10 wants ~360. The ladder stopped at 128. A clamped label stops meaning "does the target come before the stop" and quietly becomes "...within 128 bars", while the deployed EA holds until SL or TP with no bar limit. So the target the models have been trained on all along was not the strategy the EA executes, and the trades it silently reclassified as Neutral were the SLOW WINNERS - precisely the ones a 1:3 barrier exists to capture. Timeout share stayed ~0% throughout, which is why this never showed up: the truncation lands in Neutral, not in the timeout counter that was watching for it. Ladder extended to 384 (12..128, 192, 256, 384) so every selectable geometry gets an honest horizon. Cost is one embargo of at most 384 bars out of ~38k. Second fix, same class of error as the H(Y) one: the scan's "best eligible" was 2:2, a 1:1 barrier, against a shipped Min_Risk_Reward_Ratio of 1:2. Training four topologies on that target would have produced a model whose every setup is rejected at the door - the exact failure behind four consecutive Market rejections for "no trading operations". Sub-minRR geometries are now ineligible and marked [<minRR], printed rather than hidden. Also drops the dense-depth tag from the display name ("Perceptron 3L" -> "Perceptron"). Depth is derived, so it names nothing a user chose; the config tag [PAI-0be2] already disambiguates concurrent charts and does it for every input rather than one. Full topology still logged by "config -". Compiles 0 errors / 0 warnings, standard and Market. Build tag horizon-384-v1. Changes the LABEL for every geometry, so the next scan supersedes the previous numbers - and a retrain is required before any model trained under the truncated target means anything. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-01 14:54:45 -04:00
//--- The name used to carry a dense-depth tag ("Perceptron 3L"), from when AIType let a user pick
//--- MLP_3L vs MLP_4L and the depth was the only thing separating two charts of the same family.
//--- Depth is DERIVED now (see ComputeHiddenLayerCount), so it names nothing anyone chose - it is an
//--- internal shape detail leaking into a product surface a customer reads. Dropped: the config tag
//--- appended below ([PAI-0be2]) already disambiguates concurrent charts, and does it correctly for
//--- every input rather than just this one. The full topology is still logged once at startup by the
//--- "config -" line, which is where that detail belongs.
//--- Per-configuration fingerprint appended to the weights filename so that every distinct
//--- combination of RETRAIN-AFFECTING inputs gets its OWN persistent .nnw/.cfg, instead of all
//--- combinations sharing one file keyed only on symbol/period/output/optimizer. This is what lets a
//--- genetic/complete optimization that sweeps network params (neuron counts, layers, reduction,
//--- history bars, study period, feature set, focal gamma, OOS split, recall/WR targets, ...) build
//--- each combo's model exactly ONCE and then reuse it on every later pass that revisits that combo -
//--- previously each differing pass overwrote the single shared cache and retrained from scratch, so
//--- there was no cross-combination reuse at all. The topology .cfg check further below still runs as
//--- a secondary guard (and catches a rare hash collision by mismatching and retraining).
//--- Deliberately covers ONLY params that change the trained weights. Inference-only gates
//--- (Min_Vote_Open's confidence floor, SignalClusterWindow) and post-training/live settings (money
//--- management, trailing, entry, filters) are excluded, so changing those still reuses the exact
//--- same model - matching the pre-existing behavior the optimizer already relied on.
//--- 2026-08-01: SL/TP LEFT that exempt list. They used to be pure execution settings; the
//--- triple-barrier relabel makes them the barriers the TARGET is defined by (see TripleBarrierLabel),
//--- so changing either now changes every label and therefore every weight. A model trained at
//--- 1:3 must never be silently reused at 1:1. This is the rule from the fingerprint audit applied
//--- to the newest weight-affecting inputs: what shapes the labels shapes the hash.
//--- 2026-07-30: every DERIVED value left this hash - first-layer width, dense depth, conv filters,
//--- LSTM hidden size, and the retired reduction/minNeurons pair. They were legitimately here while
//--- they were functions of hashed INPUTS, which made them redundant-but-harmless. They stopped being
//--- that when the capacity budget started measuring the symbol's real bar count: a filename keyed on
//--- a measured quantity changes the moment more history downloads, so the EA would look for a file
//--- that does not exist, start from era 0, and orphan a fully-trained model - silently, since a
//--- missing cache is the normal first-run state and logs as such. The derived shape is pinned in the
//--- .cfg instead (see the adopt-don't-compare block in LoadAndCompareTopologyConfiguration), which is
//--- the correct home for it: it describes the model that EXISTS, not the config that asked for it.
//--- m_studyPeriod is gone for a simpler reason - the input it mirrored no longer exists.
string fp = StringFormat("%d|%d|%d|%d|%d|%d|%.2f|%d|%d|%d|%d|%d|%d|%d|%d|%d|%d",
//--- LEGACY_HISTORY_BARS_SLOT: the window left this hash 2026-08-11 when
//--- it became DERIVED - same rule and reason as every derived field above;
//--- keyed on a measured quantity, the filename would change the moment more
//--- history downloads. The .cfg is the record (adopt-don't-compare).
m_optimizationAlgo, LEGACY_HISTORY_BARS_SLOT, m_outputNeuronsCount,
m_neuronsCount, m_minTrainYear, LEGACY_CONVERGE_WR_SLOT, m_fractalPeriods,
//--- LEGACY SLOT (was m_focalGamma, removed 2026-07-31). It was a double fed
//--- to a %d conversion, so it always contributed the literal below rather
//--- than the configured gamma - the shipped fingerprints read ...|40|0|30|...
//--- Writing the same literal keeps every existing model's filename intact.
m_minDirectionalRecallPct, 0, m_oosSplitPct, m_swingConfirmationBars,
(int)m_useVolumes, (int)m_useTime, (int)m_useATR, (int)m_useSwingContext,
(int)m_useNews, m_newsFeatureWindowMinutes);
//--- The one derived value that DOES belong here, and only when it is not derived at all: a forced
//--- depth is a developer override (see ForceHiddenLayers), so a build that pins one must not adopt
//--- the .cfg of a build that derived it. Conditional, so the shipping value of 0 leaves the hash
//--- exactly as it reads above.
if(ForceHiddenLayers > 0)
fp += StringFormat("|FHL:%d", ForceHiddenLayers);
feat: entry/SL/TP stop being inputs - the barrier geometry is measured Three enums left the Inputs tab. They were three things a user had to pick and, in the tester, three more axes for a genetic optimization to overfit. Entry_Multiplier is pinned to MARKET. Its pending modes place the entry at a LEVEL while the rest of the pipeline measures from the bar open - the exact mismatch that manufactured the +0.097 R "retail fade" result later retracted as a fill artifact. This codebase's fill model cannot honestly simulate a pending entry, so it is no longer offered. SL_Mode/TP_Mode become a STARTING pair. ReportBarrierGeometryScan now ADOPTS its winner instead of printing "set SL_Mode/TP_Mode to X and retrain": - only when it clears the family-wise gate from 04ee2e1 (beat the null of the MAXIMUM, not merely the incumbent). This is why that gate had to land first: without it, removing the inputs would hand a noise-picked geometry direct control over the training target with no human in the loop - strictly worse than the input it replaced. On SP500 H1 today it does NOT clear (p=0.1463), so 2:6 is what you get - now chosen by measurement rather than assumed. - only at m_eraCount == 0. Relabelling a partly-trained net moves the target out from under weights already fitted to the old one. THE GEOMETRY LEFT THE WEIGHTS-FILENAME HASH, because it is now measured. Same rule that moved the horizon and the derived topology values out: a filename keyed on a measured quantity changes the moment the measurement does - a few more bars shift which pairing wins - and the EA then looks for a file that does not exist, starts from era 0 and orphans a trained model silently. It is PINNED IN THE .cfg instead: appended at the end (the only backward-safe change), length-guarded like the 2026-07-30 derived pair, and ADOPTED on load rather than compared, so a trained model keeps the barriers it actually learned and never re-measures. Two traps closed while wiring it, neither of which announces itself: - m_barrierHorizonResolved latches the horizon ONCE PER PROCESS. Adopting 2:8 (wants ~192 bars) after it settled for 2:6 (128) would label the new target against the old ceiling - the truncation fixed in 168422f, where every model learned "target within 128 bars" while the EA holds to SL/TP. It lands in Neutral, not in the timeout counter watching for it. Unlatched on adoption, along with the label cache the old barriers filled. - the .cfg adopt runs at init, before the horizon latches and before any label is computed, so a resumed model has its pinned pair in place first. Verified, not assumed. FORCES A FULL RETRAIN: the fingerprint change orphans every existing .nnw. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-07 09:39:30 -04:00
//--- THE TRIPLE-BARRIER SHAPE LEFT THIS HASH ON 2026-08-07, when SL_Mode/TP_Mode stopped being inputs
//--- and became MEASURED by ReportBarrierGeometryScan. It is the same rule that moved the horizon out
//--- (and the derived topology values before it, see above): a filename keyed on a measured quantity
//--- changes the moment the measurement does - more history downloads, a few more bars shift which
//--- pairing wins - and the EA then looks for a file that does not exist, starts from era 0, and
//--- orphans a fully-trained model silently. Measured values are PINNED IN THE .cfg instead, which is
//--- read back and adopted on load, so a trained model keeps the geometry it was actually trained on.
//--- Nothing replaces it here on purpose: the .cfg is the record, and re-measuring never happens for a
//--- model that already exists.
//--- MA/RSI + AD feature flags appended separately to keep each StringFormat call's arg list modest.
//--- m_useMA/m_useRSI belong here for the same reason every other feature flag does: they change the
//--- input-vector width (see InitIndicators()'s m_neuronsCount += 5/+1), so a model trained with them
//--- on must never silently reuse a cache trained with them off. m_neuronsCount alone (listed above)
//--- captured the WIDTH but not the composition, so two different feature sets summing to the same
//--- width could have collided onto one cache file - these two flags close that gap.
fp += StringFormat("|%d|%d|%d|%d|%d|%d|%d|%d",
(int)m_useMA, (int)m_useRSI,
(int)m_useADCumulativeDelta, (int)m_useADShorteningOfThrust,
(int)m_useADWyckoffEventStream, (int)m_useADWyckoffFailedStructure,
(int)m_useADWyckoffSignificantBarInversion,
//--- starting MA TYPE (MA_Type input): changes the MA feature's values, so a change
//--- must invalidate the cache. The auto-tuned type/period themselves live in the
//--- .nnw indicator-param block (Flatten/Unflatten), not here - this is the seed only.
(int)MA_Type);
//--- MACD/Ichimoku feature flags, appended ONLY WHEN ENABLED rather than unconditionally like every
//--- flag above. Both spellings are equally correct as a fingerprint (deterministic either way, and a
//--- model with these on can never collide with one that has them off), but appending them
//--- unconditionally would have changed the hash of EVERY existing config the moment this feature
//--- shipped - re-keying and forcing a full retrain of already-converged models that don't use MACD or
//--- Ichimoku at all. Conditional append leaves those fingerprints byte-identical. The seed periods go
//--- in for the same reason MA_Type does above: they change the feature's values. Anything added here
//--- in future should follow the same rule.
if(m_useMACD)
fp += StringFormat("|MACD:%d:%d:%d", (int)MACD_PeriodFast, (int)MACD_PeriodSlow, (int)MACD_PeriodSignal);
if(m_useIchimoku)
fp += StringFormat("|ICHI:%d:%d:%d", (int)Ichimoku_PeriodTenkan, (int)Ichimoku_PeriodKijun, (int)Ichimoku_PeriodSenkou);
fix(signals): revive a dead MA model, and demote Sanyaku from state to event Two defects surfaced by research/test_classic.py, both verified fixed by re-running the transcription against 178k bars of EURUSD H1. CSignalMA model 1 could never fire. For any recursive average - and MA_TYPE_EMA is the shipped default - MA(i) = a*Close(i) + (1-a)*MA(i+1), so DiffMA(i) = a * (Close(i) - MA(i+1)) DiffCloseMA(i) = (1-a) * (Close(i) - MA(i+1)) are positive multiples of one quantity and always share a sign. Model 1 asks for a close BELOW a RISING average, which is precisely the combination that identity forbids: 0.000% of bars, either direction, any symbol. The MQL5 standard library this was ported from defaults to MODE_SMA, where the two are merely correlated - the bug arrived with the EMA default, not with the port. Reading the slope one bar back (DiffMAPrev) breaks the tie for every MA type while keeping the model's stated meaning. Now fires on 7.92% of bars. CSignalIchimoku model 11 fired on 27% of bars at weight 100. Sanyaku is three standing STATES conjoined with no transition term, so it held across long stretches - and being last in the if-chain at the top weight, the module's highest-conviction reading was also its most common one, overwriting all eight event models below it on a quarter of all bars. The old comment rejected an event form because "demanding all three flip on the same bar would fire almost never" - true, but that is not the alternative. Kouten is the TURN: the ALIGNMENT transitions, and only one role need change for it to. Testing !Sanyaku(idx+1) fires once per aligned stretch. Now 2.17%, in line with Kumo breakout (2.4%) and the strong TK cross (1.1%). DataReady() extended one bar deeper to cover the lookback. Neither pattern showed edge before or after; this is about the models meaning what they say and the vote not being dominated by a constant. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-01 17:14:34 -04:00
//--- Cross-asset panel: conditional append, per the rule above, so existing fingerprints are untouched.
//--- ONLY the flag and the feature count go in. The panel's actual composition - which reference pairs
//--- were found in Market Watch, and therefore which currencies it can index - is a MEASURED property
//--- of the terminal, exactly like the bar count the header warns about. Keying the filename on it
//--- would orphan a fully-trained model the moment the user adds or removes a Market Watch symbol,
//--- silently, since a missing cache reads as a normal first run. The composition is logged at build
//--- time and pinned in the .cfg instead.
if(m_useCrossAsset)
{
fix(signals): revive a dead MA model, and demote Sanyaku from state to event Two defects surfaced by research/test_classic.py, both verified fixed by re-running the transcription against 178k bars of EURUSD H1. CSignalMA model 1 could never fire. For any recursive average - and MA_TYPE_EMA is the shipped default - MA(i) = a*Close(i) + (1-a)*MA(i+1), so DiffMA(i) = a * (Close(i) - MA(i+1)) DiffCloseMA(i) = (1-a) * (Close(i) - MA(i+1)) are positive multiples of one quantity and always share a sign. Model 1 asks for a close BELOW a RISING average, which is precisely the combination that identity forbids: 0.000% of bars, either direction, any symbol. The MQL5 standard library this was ported from defaults to MODE_SMA, where the two are merely correlated - the bug arrived with the EMA default, not with the port. Reading the slope one bar back (DiffMAPrev) breaks the tie for every MA type while keeping the model's stated meaning. Now fires on 7.92% of bars. CSignalIchimoku model 11 fired on 27% of bars at weight 100. Sanyaku is three standing STATES conjoined with no transition term, so it held across long stretches - and being last in the if-chain at the top weight, the module's highest-conviction reading was also its most common one, overwriting all eight event models below it on a quarter of all bars. The old comment rejected an event form because "demanding all three flip on the same bar would fire almost never" - true, but that is not the alternative. Kouten is the TURN: the ALIGNMENT transitions, and only one role need change for it to. Testing !Sanyaku(idx+1) fires once per aligned stretch. Now 2.17%, in line with Kumo breakout (2.4%) and the strong TK cross (1.1%). DataReady() extended one bar deeper to cover the lookback. Neither pattern showed edge before or after; this is about the models meaning what they say and the vote not being dominated by a constant. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-01 17:14:34 -04:00
fp += StringFormat("|XA:%d", CROSSASSET_FEATURES);
//--- INDEX-MODE RE-ENCODE (2026-08-11). When the traded symbol reports the same currency on
//--- both sides (SP500 -> USD/USD) the panel's slots changed meaning: denomination + risk-proxy
//--- strength and a denominator-adjusted divergence, instead of duplicated base/quote series
//--- (see System\CrossAsset.mqh header). Same width, different SEMANTICS - so models trained
//--- under the old degenerate encoding must re-key. Appended only for base==quote symbols:
//--- FX-pair semantics are untouched and their models keep their filenames, per the
//--- conditional-append rule above. Base/quote is a SYMBOL property, not a measured one, so it
//--- is fingerprint-safe - it cannot change under a trained model the way Market Watch can.
if(SymbolInfoString(m_symbol.Name(), SYMBOL_CURRENCY_BASE) ==
SymbolInfoString(m_symbol.Name(), SYMBOL_CURRENCY_PROFIT))
fp += ":IDX2";
}
feat(ai): spread as a volatility-regime feature, and fix a stale-index cache in both new blocks Adds spread/ATR and the spread change ratio as network inputs (EnableSpreadFeature, default on). Spread is the one microstructure channel that is both FX-available and genuinely historical in the Strategy Tester - "during testing, the spread is not modeled but is taken from historical data" - so unlike swap, signed tick flow or depth of market it is something a backtest can honestly validate. What it encodes, stated precisely because the raw measurement overstates it. research/test_spread.py found spr/atr the strongest single feature in this codebase, on 5 of 8 instrument/geometry cells at 2-4x any volume feature. But the barrier LABEL charges the spread inside its own barriers, so a wide-spread bar is mechanically likelier to resolve as a loss and the feature would partly be predicting its own cost model. Relabelling at zero cost and re-measuring the identical feature showed 20-40% of it WAS that tautology and the majority was not (XAUUSD retained 97%). What survives is a volatility-regime reading: spread is near-fixed while ATR is not, so the ratio runs high exactly when realised volatility is below its own ATR estimate, which genuinely predicts whether ATR-scaled barriers get reached. It is UNSIGNED - Neutral-vs-directional only, never a side. Also fixes a stale-index bug I introduced with the cross-asset panel and had just repeated in the spread series. Both cached on length alone: if(m_crossAsset.Bars() >= bars) return true; MQL5 series indices are relative to NOW, so one new closed candle shifts every index by one. Keyed only on length, the panel keeps serving its index 0 as a bar that is no longer the newest, and every cross-asset value is read one bar out of step with the price features sitting beside it in the same vector - silently, with no error and no shape change. This is the same class of defect as the dtStudied watermark behind the zero-direction backtests. Both now carry a datetime anchor on m_Time.GetData(0), the same invalidation key the label/feature bar caches already use. And a performance fix that fell out of it: with correct invalidation the panel rebuilds on every new bar, and RefreshConvergedSignal runs per bar - which in the tester would mean one full multi-symbol resample per simulated bar at training depth. Inference only reads bars 0..m_historyBars-1 plus the panel's own slow window, so it now requests exactly that. The cache check is >=, so a deeper panel left from training still satisfies it. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-01 17:42:40 -04:00
//--- Spread feature: conditional append, same rule. Nothing measured goes in - the spread series
//--- itself is market data, not configuration.
if(m_useSpreadFeature)
fp += "|SPR:2";
perf(features): the external block enters the window once, not once per bar Measured on the live SP500 D1 export (6073 rows, 13 features, 5888 simulated 16-bar windows): distinct values per feature per window : 1.7 - 2.7 of 16 slots variance in the first 13 PCs : 96.5 - 97.0% components for 95% / 99% : 12 / 17-19 effective rank (entropy) : ~11.5 208 inputs carrying about 12 dimensions. Only 6 of the 13 features move daily (VIX complex, USD, the rates trio); 5 are weekly (COT, EIA, output gap) and 2 monthly (CPI, unemployment). The lookup is as-of by bar open time into a DAILY file, so bars sharing a calendar day are byte-identical by construction. The cost is NOT overfitting capacity - collinear copies span ~12 directions, not 208, so an earlier claim that this wasted 26% of the model overstated it. It is GRADIENT WEIGHTING. Batch norm standardizes each of the 208 coordinates independently; that rescales the copies without decorrelating them, so one factor arrives on 16 unit-variance coordinates, each weight takes a full-size step, and the factor's aggregate coefficient moves ~16x faster than a per-bar price feature's. The network was biased toward the external block by a factor of the window length - and pointing the wrong way, since these features cleared only a marginal incremental screen while price is the base signal. Zeroed at WINDOW ASSEMBLY, not in BufferTempData: that output is cached PER BAR and a bar sits at slot 15 of one window and slot 0 of the next, so a slot-dependent value there would poison the cache or force a recompute per slot. The cache keeps true values; only this window's copies are cleared. Width contract untouched - same count, same positions - so conv/LSTM/HYBRID keep their bar-major rectangle unchanged and the block arrives at the newest bar, which for the LSTM is the final timestep. Zero-variance coordinates are safe through batch norm (divisor is MathMax(MathSqrt(var + BN_EPSILON), BN_MIN_STD)). Fingerprint gains |ALTW:1 when alt data is on. Same width and same .cfg, so nothing else would have caught a model trained under the replicated layout resuming under this one. Conditional append per the existing rule: configs without alt data keep their fingerprints and their trained models. NOT the concat branch. CNet is a strictly linear stack (CLayerDescription has no input-source field; NetBuild wires i to i+1 and stores layer L's weights on L-1), so a real two-tower model needs a new multi-input layer type across WarriorCPU, WarriorDML and the OpenCL kernels plus an .nnw format change - the highest-risk change in this repo, in the code that produced the transposed dense gradient, the Adam second-moment bug and the reversed LSTM window. This captures the part of that idea the measurement actually supports, at no engine risk. Compile-verified: 0 errors, 0 warnings. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-16 22:05:10 -04:00
//--- ALT-DATA WINDOW LAYOUT (2026-08-16). The external block now enters the input window ONCE, on
//--- the newest bar, instead of being replicated on all m_historyBars bars (see BuildFeatureWindow
//--- for the measurement and the reason). Same width, same slots, DIFFERENT input distribution - so
//--- a model trained under the replicated layout must never silently resume under this one. It
//--- cannot be caught by the .cfg guard either: m_neuronsCount is unchanged, so nothing else about
//--- this configuration moved. Conditional append, per the rule above - a config without alt data
//--- keeps the fingerprint it already had and its trained models stay loadable.
fix(features): collapse only the anchor's own run - leave lagged readings put User's call before deploy: "I would rather avoid lagging so the NN finds accurate patterns." Correct instinct, and it picks the conservative variant. 110b384 deduplicated the WHOLE window, so every distinct reading survived at one slot. The flaw is which slot: it depends on where the calendar-day boundary falls inside that particular window, and on H4 that boundary cycles through ~6 phases. A dense layer holds a separate weight per (slot, feature), so a given lag would have landed on a different coordinate from one window to the next - turning a stable lagged input into a moving one. Now it blanks only bars carrying a BYTE-IDENTICAL copy of the anchor's reading and stops at the first bar that differs. An as-of lookup into a daily file is a step function in time, so those copies are exactly the contiguous run of bars sharing the anchor's calendar day. Everything older keeps its natural replicated run, in the same slots it always occupied - whatever the net learned to read there, it still reads there. Why the anchor's reading is the right one to isolate: the window's newest slot IS the bar being predicted (BuildFeatureWindow's final iteration lands on r, and pass 3 grades that same index), so it is the reading contemporaneous with the decision - and the only one the alt screens ever validated. They measured the CURRENT reading's MI against forward range and never tested lags, so the lagged content is unproven, which is a reason to leave it undisturbed rather than a licence to rearrange it. What is still fixed: the anchor's reading reaches the first layer on one coordinate instead of once per bar of its day, removing the ~16x gradient upweight for the validated signal. And this is IDENTICAL to full dedup exactly where replication was worst - on M15/H1 the whole window sits inside one calendar day, so the anchor's run is the whole window - and a no-op on D1, where the bar before the anchor is already a different day and the loop breaks immediately. The two differ only on middle timeframes, and there this is the safe side. Fingerprint |ALTW:1 -> |ALTW:2 so nothing trained under the hour-old full-dedup semantics can silently resume under these. Compile-verified: 0 errors, 0 warnings. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-16 22:21:22 -04:00
//--- :2 = only the ANCHOR's run is collapsed (older readings keep their replicated runs). :1 was a
//--- brief full-window dedup - re-keyed rather than reused, so any model that trained under it in
//--- the hour it existed cannot silently resume on different input semantics.
perf(features): the external block enters the window once, not once per bar Measured on the live SP500 D1 export (6073 rows, 13 features, 5888 simulated 16-bar windows): distinct values per feature per window : 1.7 - 2.7 of 16 slots variance in the first 13 PCs : 96.5 - 97.0% components for 95% / 99% : 12 / 17-19 effective rank (entropy) : ~11.5 208 inputs carrying about 12 dimensions. Only 6 of the 13 features move daily (VIX complex, USD, the rates trio); 5 are weekly (COT, EIA, output gap) and 2 monthly (CPI, unemployment). The lookup is as-of by bar open time into a DAILY file, so bars sharing a calendar day are byte-identical by construction. The cost is NOT overfitting capacity - collinear copies span ~12 directions, not 208, so an earlier claim that this wasted 26% of the model overstated it. It is GRADIENT WEIGHTING. Batch norm standardizes each of the 208 coordinates independently; that rescales the copies without decorrelating them, so one factor arrives on 16 unit-variance coordinates, each weight takes a full-size step, and the factor's aggregate coefficient moves ~16x faster than a per-bar price feature's. The network was biased toward the external block by a factor of the window length - and pointing the wrong way, since these features cleared only a marginal incremental screen while price is the base signal. Zeroed at WINDOW ASSEMBLY, not in BufferTempData: that output is cached PER BAR and a bar sits at slot 15 of one window and slot 0 of the next, so a slot-dependent value there would poison the cache or force a recompute per slot. The cache keeps true values; only this window's copies are cleared. Width contract untouched - same count, same positions - so conv/LSTM/HYBRID keep their bar-major rectangle unchanged and the block arrives at the newest bar, which for the LSTM is the final timestep. Zero-variance coordinates are safe through batch norm (divisor is MathMax(MathSqrt(var + BN_EPSILON), BN_MIN_STD)). Fingerprint gains |ALTW:1 when alt data is on. Same width and same .cfg, so nothing else would have caught a model trained under the replicated layout resuming under this one. Conditional append per the existing rule: configs without alt data keep their fingerprints and their trained models. NOT the concat branch. CNet is a strictly linear stack (CLayerDescription has no input-source field; NetBuild wires i to i+1 and stores layer L's weights on L-1), so a real two-tower model needs a new multi-input layer type across WarriorCPU, WarriorDML and the OpenCL kernels plus an .nnw format change - the highest-risk change in this repo, in the code that produced the transposed dense gradient, the Adam second-moment bug and the reversed LSTM window. This captures the part of that idea the measurement actually supports, at no engine risk. Compile-verified: 0 errors, 0 warnings. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-16 22:05:10 -04:00
if(m_useAltData)
fix(features): collapse only the anchor's own run - leave lagged readings put User's call before deploy: "I would rather avoid lagging so the NN finds accurate patterns." Correct instinct, and it picks the conservative variant. 110b384 deduplicated the WHOLE window, so every distinct reading survived at one slot. The flaw is which slot: it depends on where the calendar-day boundary falls inside that particular window, and on H4 that boundary cycles through ~6 phases. A dense layer holds a separate weight per (slot, feature), so a given lag would have landed on a different coordinate from one window to the next - turning a stable lagged input into a moving one. Now it blanks only bars carrying a BYTE-IDENTICAL copy of the anchor's reading and stops at the first bar that differs. An as-of lookup into a daily file is a step function in time, so those copies are exactly the contiguous run of bars sharing the anchor's calendar day. Everything older keeps its natural replicated run, in the same slots it always occupied - whatever the net learned to read there, it still reads there. Why the anchor's reading is the right one to isolate: the window's newest slot IS the bar being predicted (BuildFeatureWindow's final iteration lands on r, and pass 3 grades that same index), so it is the reading contemporaneous with the decision - and the only one the alt screens ever validated. They measured the CURRENT reading's MI against forward range and never tested lags, so the lagged content is unproven, which is a reason to leave it undisturbed rather than a licence to rearrange it. What is still fixed: the anchor's reading reaches the first layer on one coordinate instead of once per bar of its day, removing the ~16x gradient upweight for the validated signal. And this is IDENTICAL to full dedup exactly where replication was worst - on M15/H1 the whole window sits inside one calendar day, so the anchor's run is the whole window - and a no-op on D1, where the bar before the anchor is already a different day and the loop breaks immediately. The two differ only on middle timeframes, and there this is the safe side. Fingerprint |ALTW:1 -> |ALTW:2 so nothing trained under the hour-old full-dedup semantics can silently resume under these. Compile-verified: 0 errors, 0 warnings. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-16 22:21:22 -04:00
fp += "|ALTW:2";
//--- Batch normalization changes the LAYER COUNT, not just the weights, so a model trained with it
//--- must never load into a topology built without it (and vice versa) - the .cfg guard would catch
//--- the mismatch and retrain, but only after a confusing failure. Appended conditionally, following
//--- the same rule as MACD/Ichimoku above: a config with batch norm off keeps the fingerprint it
//--- already had, so shipping this does not re-key and force a retrain of every existing model.
if(EnableBatchNorm && BatchNormWindow > 1)
fp += StringFormat("|BN:%d", BatchNormWindow);
//--- Changes the training gradient, so a model trained with it must never load into a run
//--- without it. Conditional append, same rule as MACD/Ichimoku/BN above: a config with
//--- this OFF keeps the fingerprint it already had, so the already-converged models on
//--- disk stay untouched and remain loadable as the fallback if this regresses.
fix(imbalance): the class-imbalance correction was subsidising the abstain class NOT COMPILED - user compiles. Root cause of the Neutral collapse. Logit adjustment (Menon et al. 2020) makes a classifier Bayes-optimal for BALANCED error by subsidising rare classes. It was wired here when Neutral was the DOMINANT class - the "big move up / big move down / nothing much" era, where the correction pulled the model off the majority. The triple-barrier relabel (b4a704d) inverted the distribution. The barriers are now the EA's own SL/TP, so ~89% of bars RESOLVE and only timeouts are Neutral. Measured on SP500 H4, from the EA's own log: measured priors Buy 48.26% Sell 41.13% Neutral 10.61% log-prior spread 1.52 | tau 1.00 CAPPED to 0.79 Neutral became the RAREST class, so the correction started subsidising it - by tau*(log pB - log pN) = 1.20 logits. With no directional edge to overcome that (direction is closed at best-of-999, p=1.0000), the model took the free lunch: OOS recall Buy:1% Sell:0% Neutral:100% OOS raw out spread avg 0.9993 (softmax saturated, near one-hot) dW/W bn1 0.000% bn3 2.0% bn5 6.5% (input weight block frozen; head twitching) The anti-collapse mechanism was the collapse. The recall gate needs >=40% on all three classes, so nothing could ever deploy and the plateau ladder burned eras. Present in both runs today (b6b5 froze bn1 by era ~719, 17ae by ~169), so it predates this week's work. FIX: the correction now spans the DECIDABLE classes only, Buy against Sell, centred on their midpoint, with Neutral pinned at offset 0. Neutral is the ABSTAIN outcome and abstention already has a better owner - m_dirConfThreshold, refitted every era on the held-out calibration band against a coverage floor and the measured break-even. Subsidising the abstain class does that job twice and spends the whole correction suppressing the only decisions that can pay. What still gets corrected is real: a trending symbol resolves more long barriers than short, and uncorrected the model inherits that as a standing directional bias. Here it is log(0.4826)-log(0.4113) = 0.16, so the offsets are tiny - the correct answer, not a broken one. The two traded classes were already balanced; the old spread of 1.52 only ever described how rare a timeout is. Everything is derived from the measured distribution, as requested - offsets from the priors, cap from the resulting spread. tau itself is deliberately NOT fitted: tuning it against the same data that selects the checkpoint would add another search dimension to a project that has been burned by exactly that. tau=1 is the theory value and the cap (now ~9.5x looser at spread 0.16) will rarely bind. Log line now reports both spreads and, when the abstain class is the rarest, says how much the old form would have boosted it. Fingerprint |LA:<tau> -> |LA:<tau>:BS so models trained under the all-three form re-key instead of resuming. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-16 22:43:30 -04:00
//--- ":BS" = the correction spans Buy/Sell only, with Neutral (the abstain outcome) never
//--- subsidised - see ApplyLogitAdjustment. Same tau, completely different gradient from the old
//--- all-three form, so models trained under that one must re-key rather than silently resume.
if(m_logitAdjustTau > 0.0)
fix(imbalance): the class-imbalance correction was subsidising the abstain class NOT COMPILED - user compiles. Root cause of the Neutral collapse. Logit adjustment (Menon et al. 2020) makes a classifier Bayes-optimal for BALANCED error by subsidising rare classes. It was wired here when Neutral was the DOMINANT class - the "big move up / big move down / nothing much" era, where the correction pulled the model off the majority. The triple-barrier relabel (b4a704d) inverted the distribution. The barriers are now the EA's own SL/TP, so ~89% of bars RESOLVE and only timeouts are Neutral. Measured on SP500 H4, from the EA's own log: measured priors Buy 48.26% Sell 41.13% Neutral 10.61% log-prior spread 1.52 | tau 1.00 CAPPED to 0.79 Neutral became the RAREST class, so the correction started subsidising it - by tau*(log pB - log pN) = 1.20 logits. With no directional edge to overcome that (direction is closed at best-of-999, p=1.0000), the model took the free lunch: OOS recall Buy:1% Sell:0% Neutral:100% OOS raw out spread avg 0.9993 (softmax saturated, near one-hot) dW/W bn1 0.000% bn3 2.0% bn5 6.5% (input weight block frozen; head twitching) The anti-collapse mechanism was the collapse. The recall gate needs >=40% on all three classes, so nothing could ever deploy and the plateau ladder burned eras. Present in both runs today (b6b5 froze bn1 by era ~719, 17ae by ~169), so it predates this week's work. FIX: the correction now spans the DECIDABLE classes only, Buy against Sell, centred on their midpoint, with Neutral pinned at offset 0. Neutral is the ABSTAIN outcome and abstention already has a better owner - m_dirConfThreshold, refitted every era on the held-out calibration band against a coverage floor and the measured break-even. Subsidising the abstain class does that job twice and spends the whole correction suppressing the only decisions that can pay. What still gets corrected is real: a trending symbol resolves more long barriers than short, and uncorrected the model inherits that as a standing directional bias. Here it is log(0.4826)-log(0.4113) = 0.16, so the offsets are tiny - the correct answer, not a broken one. The two traded classes were already balanced; the old spread of 1.52 only ever described how rare a timeout is. Everything is derived from the measured distribution, as requested - offsets from the priors, cap from the resulting spread. tau itself is deliberately NOT fitted: tuning it against the same data that selects the checkpoint would add another search dimension to a project that has been burned by exactly that. tau=1 is the theory value and the cap (now ~9.5x looser at spread 0.16) will rarely bind. Log line now reports both spreads and, when the abstain class is the rarest, says how much the old form would have boosted it. Fingerprint |LA:<tau> -> |LA:<tau>:BS so models trained under the all-three form re-key instead of resuming. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-16 22:43:30 -04:00
fp += StringFormat("|LA:%d:BS", (int)MathRound(m_logitAdjustTau * 100.0));
//--- 2026-07-29 audit of every input in Variables\Inputs.mqh against this hash. Five were changing the
//--- trained weights without changing the filename, so switching any of them re-adopted a model trained
//--- under the OLD value - the exact trap that the .nnw architecture incident already cost a day to
//--- (see EnforceTopologyContract): the .cfg guard would eventually mismatch and retrain, but only
//--- after a confusing failure, and a matching topology would not mismatch at all.
//--- LEGACY SLOT. The oversampling/replay inputs this encoded were removed 2026-07-31 (see the
//--- class-imbalance block in Variables\Inputs.mqh); the replay path itself is gone. The literal is
//--- the exact string the shipped defaults produced - EnableMinorityReplay=true, OversampleParity=90,
//--- ConstrainReplay=true - so every model already on disk keeps its filename and stays loadable.
//--- Dropping the segment instead would re-key EVERY model and force a from-scratch retrain of the
//--- one topology currently converged and trading, which is a steep price for cosmetics in a hash
//--- nobody reads. Same treatment as LEGACY_CONVERGE_WR_SLOT / LEGACY_STUDY_PERIOD_SLOT.
fp += "|MR:1:90:1";
//--- Feature-value inputs, each conditional on the feature that reads it actually being on - the same
//--- rule the MACD/Ichimoku blocks above follow. Tick vs real volume feeds different numbers into the
//--- same input slot (Features.mqh's m_Volumes.Create), and PeriodMA/PeriodRSI seed the tuner
//--- (ADIndicatorTuner.mqh) exactly as MA_Type does - MA_Type was already hashed, these two were not.
//--- All three also feed the CLASSIC MA/RSI votes, which are inference-only; gating on the AI feature
//--- flag is what keeps a classic-signal tweak from re-keying a model that never saw it.
if(m_useVolumes)
fp += StringFormat("|VOL:%d", (int)VolumeData);
if(m_useMA)
fp += StringFormat("|MAP:%d", (int)PeriodMA);
if(m_useRSI)
fp += StringFormat("|RSIP:%d", (int)PeriodRSI);
feat: expose the AD/Wyckoff parameters; default the indicator tuner off AutoTuneIndicators now defaults to FALSE, and the 33 AD/Wyckoff parameters it used to search are now inputs. WHY THE DEFAULT FLIPPED - not because the search is broken. It is correct, and its own Sidak gate is what proves it: 324 candidates per model on SP500 H1, "no improvement" on all four topologies (0.00236 -> 0.00236 on the AD configs, 0.00370 -> 0.00370 on PAI), winner rejected at p=1.0000. It cannot do better here by construction - it ranks candidates by MARGINAL MI, and the headline MI is 0.00370 nats against a shuffled null of 0.00379 +/- 0.00061 (p=0.4975), so every candidate is a noise draw and the maximum over N of them is noise too. The cost is 45-56 min per model in one synchronous call with no yield, and it was the amplifier for the handle leak fixed in 33f106d. The EA's own report says it plainest: "no per-feature indicator retuning will help." THE INPUT STAYS. TuneIndicatorsByFilter is one function of twelve in AIBase/AutoTune.mqh; the other eleven are the MI/lag/excursion/geometry diagnostics that produced every verdict this project relies on, and they run regardless of this flag. Removing the input invites removing the file. WHY THE INPUTS WERE NEEDED. All 33 were literals in CADIndicatorTuner's constructor with no input of any kind, while MA/RSI/MACD/Ichimoku have had their periods exposed from the start. On the AD configs those indicators contribute 28 of 64 features per bar. Survivable while the tuner searched them; indefensible with it off, where they would freeze at values nobody chose. CONSOLIDATED 33 -> 18. volClimax/volHigh/rangeClimax/rangeSignificant/ stVolRatio/atr were duplicated verbatim across CumulativeDelta, Wyckoff Events, Failed Structure and Bar Inversion - the same constants restated 3-4 times. One concept, one input. They are SEEDS: each fans out to the indicator's own struct field, so with the tuner on it retains full per-indicator freedom to move them apart. Same contract as PeriodMA. NO RETRAIN. Every default is byte-identical to the literal it replaces, and the fingerprint's new ADP token is appended ONLY on deviation (MACD/Ichimoku/BN/XA convention), gated on the AD features being enabled. At defaults the token is absent, so every model on disk keeps its filename and stays loadable. Without that guard, merely EXPOSING these parameters would have re-keyed every config and forced a from-scratch retrain of all four topologies for a change that alters no number anywhere. All-or-nothing rather than per-input, so the token can never encode a partial picture of what the features were built from. Compiles clean: 0 errors, 0 warnings. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-08 20:37:21 -04:00
//--- AD/WYCKOFF PARAMETERS. These SEED feature values (see CADIndicatorTuner's constructor), so by the
//--- rule this hash exists to enforce they belong in it - a model trained at one climactic-volume
//--- threshold must never silently re-adopt weights fitted at another. That rule is not theoretical
//--- here: the 2026-07-29 audit found five inputs changing trained weights without changing the
//--- filename, which is the exact trap the .nnw architecture incident already cost a day to.
//--- APPENDED ONLY ON DEVIATION, following the MACD/Ichimoku/BN/XA convention. All 18 inputs ship at
//--- byte-identical values to the literals they replaced, so at defaults this token is absent and every
//--- model already on disk keeps its filename and stays loadable. Without that guard, merely EXPOSING
//--- these parameters would have re-keyed every config and forced a from-scratch retrain of all four
//--- topologies - for a change that alters no number anywhere.
//--- All-or-nothing rather than per-input: one deviation writes the whole vector, so the token is
//--- either absent or complete and can never encode a partial picture of what the features were built
//--- from. Gated on the AD features actually being ON, since with them off none of this reaches a
//--- feature at all (m_useMA/m_useRSI already carry the two shared thresholds' other consumers).
//--- ADP TOKEN RETIRED 2026-08-16 with the menu pruning: the 18 AD/Wyckoff inputs are compile-time
//--- constants again (Variables\Inputs.mqh), so a deviation from defaults is impossible by
//--- construction and the conditional append above could never fire. Deleted rather than left as a
//--- tautology. TUNED values were never this token's job: the auto-tuner's winners are persisted in
//--- the .nnw beside the weights (CADIndicatorTuner::Flatten), which travels WITH the model - the
//--- fingerprint only ever guarded the operator-set STARTING point, which no longer varies.
fix: the sequence models were reading the window backwards BuildFeatureWindow() replaces eight hand-rolled copies of the same loop and feeds the window OLDEST BAR FIRST. Every copy fed it newest-first, because MQL5 timeseries indices run backwards and `r + b` with b ascending walks into the past. Harmless for PAI and CONV - a dense layer learns a weight per position either way, a conv learns time-mirrored kernels. Not harmless for the recurrent stacks: - LSTM_SeqStepForward reads `inputs + t*Iw`, so step t is block t. - It writes output[] only when t == steps-1: the visible output IS the last hidden state. - c_t = f*c_{t-1} + i*g decays toward the start of the sequence. lstm_seq_flowcheck.cpp measured block 0's influence on the output at 1.2e-2 of block T-1's, at the shipped forget bias of 1.0. So the bar being PREDICTED sat at the far end of the decay and the output was handed to the OLDEST bar in the window - the exact inverse of what the window is for. ~80x backwards on LSTM and HYBRID, on all three tiers (OpenCL kernel, CPU DLL, pure-MQL5 inference), which is why it never surfaced as a backend discrepancy. This does not create edge - the MI diagnostics read at the noise floor (p=0.4975) with a working positive control. It makes the one hypothesis those diagnostics explicitly do NOT cover testable: they are marginal and per-bar, and state they "cannot rule out one that only exists in combination or across time". The sequence model is the instrument for across-time structure and it has been crippled, so that hypothesis has never been honestly tested. Fingerprint gets an unconditional |WIN:2 - the vector keeps its shape and its features, so a stale .nnw would load cleanly and run a model fitted to one ordering against the other, silently. Re-keying every config is the point, not collateral damage. FORCES A FULL RETRAIN. Also: the now-relative bar caches are re-keyed on the two live paths. EnsureBarCachesCapacity() was only ever called from training paths, but once m_trainingComplete is set ScheduleTrainingIfNeeded() routes every bar to RefreshConvergedSignal() and Train() is never re-entered - so nothing cleared the feature cache again for the life of the process. A chart that trained to convergence kept replaying the rows computed for the last training era's bar grid: the live signal froze at its convergence-time value, and OnlineLearnStep() backpropped those stale features against freshly resolved labels. Backtests were never affected (an inference-only process never allocates the arrays, so every read recomputes). Compiles clean: 0 errors, 0 warnings. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-07 18:28:44 -04:00
//--- INPUT WINDOW ORDER. Appended UNCONDITIONALLY, which is a deliberate break from the "conditional
//--- append so existing fingerprints stay byte-identical" rule every block above follows - and the
//--- reason is exactly why that rule exists in the first place. Every model on disk was trained on a
//--- window fed NEWEST-BAR-FIRST; BuildFeatureWindow() now feeds it oldest-first (see its definition
//--- comment for the LSTM measurement that forced it). The vector has the same SHAPE and the same
//--- features, so nothing downstream would fail: a stale .nnw would load cleanly, pass the .cfg guard,
//--- and run a model fitted to one input ordering against the other - silently, forever. That is the
//--- precise failure this hash exists to make impossible, so here re-keying every config is the
//--- CORRECT outcome, not collateral damage. Version it rather than toggling a flag: if the ordering
//--- is ever revisited, bump the number instead of trying to reconstruct which models predate what.
fp += "|WIN:2";
feat: mini-batch gradient accumulation (F4), front-end-aware capacity budget (F6), split Wyckoff categoricals (N1) Completes the 2026-08-09 training audit. FORCES A RETRAIN of every Wyckoff-enabled config (N1 re-keys the fingerprint), and BOTH DLLs must be redeployed alongside the .ex5 - they carry new exports. F4 - mini-batch accumulation, TRAIN_BATCH_SIZE=32. Training was pure online SGD (one weight update per bar), which is the mechanical source of the era-to-era whipsaw every downstream guard was built to cope with. The O(n^2) outer product is native - AccumulateWeightGrad / AccumulateWeightGradConv / AccumulateBufferInto in Network.cl, WarriorCPU and WarriorDML - while the optimizer step is host-side MQL5 shared by all tiers (ApplyAccumToBlock), so there is one Adam/SGD implementation instead of four that can drift. - the LSTM needs no outer-product kernel (WeightsGradient already holds the sample's full dW) but could NOT simply be left un-zeroed between samples: CPU_LSTMSeqBackward/DML_LSTMSeqBackward memset it on entry. Hence a separate accumulator plus an elementwise add. - batch-norm gamma/beta accumulate in host arrays, not new BatchOptions slots - BN_OPT_STRIDE is baked into every persisted .nnw. - scoped to pass 2; online learning keeps immediate updates. Every save / checkpoint / scoring boundary flushes, scaling by the real sample count. - degrades to per-sample updates (one log line) on a tier that cannot accumulate, so old devices and DLL-free builds are unaffected. - verified offline: DirectML/batch_accum_check.cpp drives the real exports against an independent reference; at B=1 the accumulator matches the shipped unbatched kernel's own gradient to 1.1e-16. Math only - the in-situ check remains the per-layer dW/W report on a real era. F6 - ComputeFirstLayerWidth budgeted against the RAW input width even where a conv/LSTM front end had already reduced it, so an LSTM's dense stack was charged for 1,280 inputs when it receives 64. Confirmed from the deployed .cfg files: CONV, LSTM and HYBRID were all pinned at the 16-unit floor. Now budgeted against the front-end output and capped at it (never fan out), with the derivation reordered so both stages settle first. N1 - EventCode/EventPhase/StructuralPhase are signed categoricals packing direction and Wyckoff stage into one scalar across a sign discontinuity. Split into direction + [0,1] magnitude, the same convention the base OHLC block uses. Information-preserving; 13 readings now occupy 16 inputs. Compiled clean (0 errors, 0 warnings); both DLLs rebuilt. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-09 11:48:03 -04:00
//--- WYCKOFF CATEGORICAL ENCODING VERSION. Unconditional and versioned for exactly the reason WIN is
//--- (see above): the 2026-08-09 N1 split re-encodes EventCode/EventPhase/StructuralPhase as
//--- direction+magnitude pairs, so those input slots now MEAN something different. The vector width
//--- also changes (13 -> 16 readings), which m_neuronsCount above would catch on its own - but width
//--- alone is the weaker guard, and a future re-encoding that preserved width would slip past it and
//--- run a model fitted to one encoding against another, silently and forever. Bump the number
//--- rather than adding a flag if the encoding is ever revisited.
//--- Gated on the feature actually being on, so a config without the Wyckoff events keeps its
//--- existing fingerprint and its trained model.
if(m_useADWyckoffEventStream)
fp += "|WES:2";
feat: S2 meta-labeling head - binary trade-quality model over the classic-candidate corpus The NN now has a target that is not per-bar direction (closed, best-of-999 p=1.0000): P(win | this journaled candidate, at the EA's own SL/TP, net of cost). One net for all 52 pattern-sides, AIType=AI_META. - NetForward.mqh: the host-side softmax+CE gradient generalized total==3 -> 2||3 on both backprop paths; a 2-class softmax IS a logistic head, and no compute backend changes. - SignalMETA.mqh (new): corpus loaded read-only from the LARGEST signal DB on disk (decoupled from the config fingerprint that burned four S1 runs); the GMT->server offset is measured PER ROW against entryPrice vs bar open (DST-immune, histogram logged); a window-span regime filter drops the pre-2017 daily-backfill rows; 31-feature setup descriptor appended at the input (26 one-hot + side + tanh netVote + SL/TP ATR + spread/ATR). - Training.mqh: candidate-queued pass 1, binary-target pass 2, per-candidate calibration (2.5) and OOS (3) walks. Counter mapping win->Buy / loss->Sell lets checkpoint selection, the edge floor, the plateau ladder and the family-wise deploy gate run UNCHANGED: precision reads as win rate among traded candidates, chance as the base win rate, recalls as sensitivity/ specificity. Era-end META line: coverage x (p - break-even) vs the null. - Labels are the side-conditional triple-barrier win caches - never the DB's stop-and-reverse outcome. Logit adjustment deliberately skipped (~40% base rate). Live inference + online learning guarded off until S3. - Fingerprint: conditional |TGT:META1; State\META\ folder + 2-output filename slot keep meta models fully separate from direction models. Compiles clean (0 errors, 0 warnings). S2 run = attach a chart with AIType=AI_META; S3 wires the votes via the per-side hooks. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-13 06:52:31 -04:00
//--- TRAINING TARGET (meta-labeling). Conditional, so every existing direction model keeps its
//--- byte-identical fingerprint. Versioned like WIN/WES: the token covers the meta LABEL and the
//--- setup-descriptor layout (META_DESC_FEATURES) - bump the number if either ever changes, so a
//--- model trained on one meaning can never silently load under another. The 2-output head and the
//--- State\META\ folder already separate the FILES; this separates the SEMANTICS.
if(IsMetaTarget())
fp += "|TGT:META1";
feat(ai): TrainingTarget input - fractal-direction label for the direction models User direction (2026-08-15): back to predicting swing turns, D1 charts, fractals over ZigZag pivots (their call - balances classes, matches the reference library target, and a 5-bar fractal confirms 2 bars after its extreme so labels resolve nearly to the present with no repaint embargo). - TRAINING_TARGET enum + TrainingTarget input: TARGET_BARRIER (Market default - existing models keep their meaning and fingerprints) or TARGET_FRACTAL (private default). - FractalDirectionLabel (Labels.mqh): per-bar 3-class label = direction from the bar close to the next confirmed strict 5-bar fractal extreme, costs charged in the same bid-series convention as the barrier label, Neutral when the move cannot clear max(2 spreads, 0.10 ATR) or on an outside bar (both-extreme bars are unorderable within OHLC). - The barrier walk still runs in full: measured SL/TP geometry, the expectancy scan, excursion caches and the era gate all keep scoring what a trade at the EA's own stop/target actually collected - only the TRAINING label changes. NOT the pre-b4a704d "is this bar the pivot" form; that target's 31:1 imbalance stays retired. - Fingerprint token |TGT:FRA1 so switching targets trains a separate model; AI_META unaffected (guarded setter). - Private defaults: AIType back to AI_HYBRID (direction topology needed) + TrainingTarget=TARGET_FRACTAL = drop-on-D1-chart workflow. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-15 04:44:10 -04:00
//--- Same contract for the fractal-direction target (TrainingTarget input): the token covers the
//--- label's meaning (next confirmed 5-bar fractal extreme, min-move floor, tie handling) - bump the
//--- number if any of that changes. Absent for TARGET_BARRIER so every existing direction model
//--- keeps its byte-identical fingerprint.
if(IsFractalTarget())
fp += "|TGT:FRA1";
//--- Ensemble membership separates the FILES, not the semantics: the member's topology and label
//--- are identical to its solo twin, but the two must never share weights across charts (the
//--- duplicate-chart guard exists precisely to stop concurrent writers). Versioned like the target
//--- tokens in case ensemble semantics ever change what a member means.
if(m_ensembleMember)
fp += "|ENS1";
//--- The scheduled close-all became part of the LABEL'S MEANING on 2026-08-19: TripleBarrierLabel
//--- stops its walk at the next scheduled flat, so the same chart with a Friday-23:45 schedule and
//--- with an everyday-22:00 schedule trains two DIFFERENT targets. An active schedule therefore
//--- keys the fingerprint - changing the day/hour/minute re-keys the model and forces a fresh
//--- train instead of silently resuming weights fitted to a different target (the
//--- stale-enum-wrong-target family). A disabled schedule appends nothing, keeping every
//--- no-schedule model byte-identical. Charts on the DEFAULT Friday schedule re-key exactly once,
//--- at this change, and that is deliberate: their existing weights were trained on weekend-blind
//--- labels and are confounded - see the 3e467f9 commit message.
if((int)targetDayOfWeek != -1 && (int)targetHour != -1 && (int)targetMinutes != -1)
fp += "|CUT:" + IntegerToString((int)targetDayOfWeek) + "@" + IntegerToString((int)targetHour) +
":" + IntegerToString((int)targetMinutes);
//--- FNV-1a 32-bit -> 8 hex chars: compact, deterministic, order-stable, collision-safe enough for
//--- the small optimizer grids in play (a collision would merely fail the .cfg guard and retrain).
uint fpHash = 2166136261;
int fpLen = StringLen(fp);
for(int fpi = 0; fpi < fpLen; fpi++)
{
fpHash ^= (uint)StringGetCharacter(fp, fpi);
fpHash *= 16777619;
}
m_fileName += "_" + DoubleToString(MathRound(m_outputNeuronsCount)) + "_" + DoubleToString(MathRound(m_optimizationAlgo)) + "_" + StringFormat("%08x", fpHash);
//--- Finish the display name with the model's short id and the leading 4 hex digits of that same
//--- fingerprint, so every log line and panel names the model file it belongs to. The dense-depth tag
//--- added at the top of this function separates MLP_3L from MLP_4L, but NOT two charts that differ by
//--- anything else - the batch-norm control was 3L on both sides, which put two identical
//--- "Perceptron 3L" streams in the log the first time this was tried. Any config difference at all
//--- changes the hash, by construction, so it is the discriminator that cannot go stale as inputs are
//--- added - but it is NOT unique on its own. The fingerprint deliberately omits the topology TYPE,
//--- because the file path already separates it (State\CONV\ vs State\LSTM\ vs State\HYB\) and hashing
//--- a value that is constant within a folder would add no discriminating power while re-keying every
//--- trained model on disk into a forced retrain. The consequence is that CONV, LSTM and HYBRID at the
//--- same depth with the same inputs hash IDENTICALLY - a 2026-07-30 deploy came back with three charts
//--- all tagged [4109]. Their files were never at risk; the TAG was simply unable to do its one job.
//--- Prefixing m_id restores uniqueness on the display side without touching m_fileName, and the hex
//--- half still greps straight to the .nnw inside the folder the prefix names.
string cfgTag = " [" + m_id + "-" + StringSubstr(StringFormat("%08x", fpHash), 0, 4) + "]";
if(StringFind(ID, cfgTag) < 0)
ID += cfgTag;
//--- One self-verifying config line per chart, deliberately NOT gated on VerboseMode. A multi-chart
//--- comparison is only valid if every chart is identical except the axis under test, and until now
//--- a drifted setting was invisible: the filename carries a HASH, so two charts that should match
//--- and do not look merely "different" with no indication of WHICH field moved. Printing the raw
//--- fingerprint string makes the six lines directly diffable - any accidental divergence in study
//--- period, feature set, focal gamma or anything else that feeds training shows up as a textual
//--- difference at startup instead of an unexplained result three hours later.
Print(ID + ": config - " + IntegerToString(m_hiddenLayersCount) + " dense from " +
IntegerToString(m_initialNeuronsCount) + " units | batchnorm " +
((EnableBatchNorm && BatchNormWindow > 1) ? "ON(" + IntegerToString(BatchNormWindow) + ")" : "OFF") +
//--- "requested", not the bare number: the EFFECTIVE tau is capped against the head's usable
//--- logit range and cannot be known until the class priors are measured, so printing 1.00 here
//--- read as the value in force when every chart was actually running 0.35. The real figure is
//--- logged once per run by ApplyLogitAdjustment().
//--- The ONE class-imbalance mechanism. There is deliberately no "| replay ON/OFF" beside it any
//--- more: that field reported a path which had already been dead for the whole shipped
//--- configuration, which is exactly the kind of line that makes a log look informative while
//--- describing nothing (see the class-imbalance audit in Variables\Inputs.mqh).
" | class-imbalance " + (m_logitAdjustTau > 0.0
? "logit-adjust(tau " + DoubleToString(m_logitAdjustTau, 2) + " requested)" : "OFF") +
" | input " + IntegerToString((int)m_historyBars * m_neuronsCount) +
" (" + IntegerToString((int)m_historyBars) + " bars x " + IntegerToString(m_neuronsCount) + ")" +
//--- The front-end stages are DERIVED (see ComputeConvFilterCount/ComputeLstmHiddenSize), so
//--- without them this "self-verifying" line verified only half the topology - it printed the
//--- dense taper while the conv/recurrent stages that actually dominate CONV/LSTM/HYBRID were
//--- invisible. Shows the width flowing INTO each stage as well as out of it, because the
//--- interesting failure is a stage that expands rather than compresses.
FrontEndConfigSummary());
//--- Kept as its own line and deliberately free of any per-chart prefix INSIDE the string, so the six
//--- startup lines diff textually against each other. The model file goes on the line below rather than
//--- here for the same reason: it necessarily differs per topology (it carries the State\<id>\ folder),
//--- so folding it in would make every fingerprint line differ and destroy the diff.
Print(ID + ": fingerprint - " + fp);
//--- The resolved path is DebuggingMode-only: the tag above already names the folder (its m_id half)
//--- and the file's hash suffix (its hex half), so this line is derivable rather than new information,
//--- and a third startup line per chart is not worth spending on a user who will never open the file.
if(DebuggingMode)
Print(ID + ": model file - " + m_fileName + ".nnw");
//--- Strategy Tester / optimizer: target a LOCAL (agent-sandboxed, non-FILE_COMMON) cache file
//--- instead of the shared production weights, so genetic/complete optimization passes on this
//--- same agent can reuse an already-trained model whenever the topology-relevant inputs
//--- (neuron counts, layers, history bars, output count, opt algo, study period, ...) are
//--- unchanged from a previous pass, instead of re-running every training era from scratch each
//--- pass. The live/manual-chart production .nnw/.cfg under FILE_COMMON are never touched by
//--- this path, so a backtest can never corrupt or overwrite the deployed live model.
bool inTesterOrOpt = MQLInfoInteger(MQL_TESTER) || MQLInfoInteger(MQL_OPTIMIZATION) || MQLInfoInteger(MQL_FORWARD);
m_activeFileName = inTesterOrOpt ? (m_fileName + "_optcache") : m_fileName;
m_activeFileCommon = !inTesterOrOpt;
//--- Claim these files before anything reads or writes them, and refuse to start if another chart in
//--- this terminal already holds them (see AcquireConfigLock). Deliberately placed here: this is the
//--- first moment the resolved filename - i.e. the config's true identity - is known, and it is still
//--- ahead of every load, seed and save. Tester/optimizer agents are exempt: each is a separate
//--- process writing its own sandboxed _optcache, and running one config across many agents in
//--- parallel is the entire point of an optimization.
research: export the feature matrix and a raw OHLCV grid for offline work The bottleneck on this project has never been the modelling - it is that every hypothesis costs a compile, a deploy, an attach and a log read, and answers exactly one question. Days have gone into questions that are seconds of arithmetic once the data is in hand. Adds a RESEARCH-ONLY build, gated behind WARRIOR_EXPORT_FEATURES and never compiled into a shipped binary, which writes two things to Common\Files\Warrior_EA\Research\ and then does nothing at all: <symbol>_<tf>_features.csv - one row per bar: index, time, OHLC, ATR, and the m_neuronsCount feature values. Exactly what the network sees. The raw bars ride along on purpose: with OHLC and ATR offline, every barrier geometry, horizon and in-trade target is recomputable without MetaTrader in the loop. <symbol>_<tf>_rates.csv - raw OHLCV across a grid of 8 symbols x 5 timeframes. The 26 engineered features only exist for the attached chart (indicator handles bind to PERIOD_CURRENT); raw rates do not, so ONE attach yields the whole research grid. The bar time also makes session/hour/day-of-week derivable - the only inputs in play that are not a transform of the same OHLCV series. Safety, because this binary gets attached to a chart on a LIVE ACCOUNT to reach real history: - OnTick returns immediately, so Expert.OnTick() - the entire trading path - is unreachable regardless of the AlgoTrading toggle, the signal state or the inputs. Structurally incapable of sending an order, not merely unlikely to. - No config lock. It never trains and never saves a model, so it has nothing to protect against a concurrent chart - and taking the lock would make it refuse to start exactly when the config it wants to read is already open, which is when it is most useful. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-01 15:49:57 -04:00
#ifdef WARRIOR_EXPORT_FEATURES
//--- RESEARCH BUILD: no lock. This binary reads history and writes one CSV - it never trains, never saves
//--- a model (OnTick returns immediately, so no era ever completes) and therefore has nothing to protect
//--- against a concurrent chart. Claiming the lock would only make the exporter REFUSE to start whenever
//--- the config it wants to read is already open on a production chart, which is exactly when it is most
//--- useful to run.
#else
if(!inTesterOrOpt && !AcquireConfigLock())
return false;
research: export the feature matrix and a raw OHLCV grid for offline work The bottleneck on this project has never been the modelling - it is that every hypothesis costs a compile, a deploy, an attach and a log read, and answers exactly one question. Days have gone into questions that are seconds of arithmetic once the data is in hand. Adds a RESEARCH-ONLY build, gated behind WARRIOR_EXPORT_FEATURES and never compiled into a shipped binary, which writes two things to Common\Files\Warrior_EA\Research\ and then does nothing at all: <symbol>_<tf>_features.csv - one row per bar: index, time, OHLC, ATR, and the m_neuronsCount feature values. Exactly what the network sees. The raw bars ride along on purpose: with OHLC and ATR offline, every barrier geometry, horizon and in-trade target is recomputable without MetaTrader in the loop. <symbol>_<tf>_rates.csv - raw OHLCV across a grid of 8 symbols x 5 timeframes. The 26 engineered features only exist for the attached chart (indicator handles bind to PERIOD_CURRENT); raw rates do not, so ONE attach yields the whole research grid. The bar time also makes session/hour/day-of-week derivable - the only inputs in play that are not a transform of the same OHLCV series. Safety, because this binary gets attached to a chart on a LIVE ACCOUNT to reach real history: - OnTick returns immediately, so Expert.OnTick() - the entire trading path - is unreachable regardless of the AlgoTrading toggle, the signal state or the inputs. Structurally incapable of sending an order, not merely unlikely to. - No config lock. It never trains and never saves a model, so it has nothing to protect against a concurrent chart - and taking the lock would make it refuse to start exactly when the config it wants to read is already open, which is when it is most useful. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-01 15:49:57 -04:00
#endif
//--- Any Strategy-Tester run - a single backtest OR an optimization pass - runs pure inference on the
//--- deployed model, never trains. A user optimizing TRADING parameters (SL/TP, filters, MM, ...) wants
//--- the AI held fixed at the deployed weights so passes are fast and comparable; retraining the net
//--- per config would be slow and make every pass a different model. AI hyperparameters are tuned on a
//--- chart (the internal auto-tuner / a real training run), not via MT5 optimization. Training and the
//--- new online continual-learning step therefore run ONLY on a live chart (see OnlineLearnStep).
m_inferenceOnly = MQLInfoInteger(MQL_TESTER);
//--- Seed the agent-local optcache from the deployed production model on the first tester/opt pass.
//--- Without this, the tester's separate _optcache file starts empty and the run retrains from zero -
//--- so a buyer who loads a .set and hits "backtest" waits through a full training run instead of a
//--- backtest of the model they deployed. The optcache shares the production model's exact config
//--- fingerprint (same m_fileName base), so the copied weights are guaranteed topology-compatible.
//--- Copies FROM FILE_COMMON (the live/manual-chart model) INTO the agent-local sandbox only; the
//--- production files are read, never written, so a backtest still can't corrupt the deployed model.
//--- Re-seeds when the cache is MISSING *or* STALE. Staleness matters because a tester run no longer
//--- writes this file at all (see PersistWeightsOnShutdown's inference-only skip), so without a
//--- freshness check the very first seeded copy would be reused forever - meaning the obvious workflow
//--- "retrain/redeploy on the chart, then backtest" would silently keep testing the OLD model. The
//--- config fingerprint in the filename can't catch this: retraining changes the WEIGHTS, not the
//--- topology inputs the fingerprint hashes, so the name stays identical.
bool cacheMissing = !FileIsExist(m_activeFileName + ".nnw");
bool cacheStale = false;
if(inTesterOrOpt && !cacheMissing && FileIsExist(m_fileName + ".nnw", FILE_COMMON))
{
datetime prodModified = (datetime)FileGetInteger(m_fileName + ".nnw", FILE_MODIFY_DATE, true);
datetime cacheModified = (datetime)FileGetInteger(m_activeFileName + ".nnw", FILE_MODIFY_DATE, false);
//--- both timestamps must be readable before trusting the comparison; a 0 means "couldn't tell",
//--- and re-seeding on an unreadable timestamp every single pass would be worse than not checking.
cacheStale = (prodModified > 0 && cacheModified > 0 && prodModified > cacheModified);
if(cacheStale)
Print(__FUNCTION__ + ": the deployed model is newer than this agent's cached copy - re-seeding so the backtest runs the CURRENT model, not the previously cached one.");
}
if(inTesterOrOpt && (cacheMissing || cacheStale))
{
if(FileIsExist(m_fileName + ".nnw", FILE_COMMON))
{
//--- The .nnw is the only copy that MUST succeed - retried (see CopyFileWithRetry's declaration
//--- comment) because a live chart's own atomic Save() can be mid-rename on this exact file.
//--- Its return value used to be ignored entirely, so a failed copy still logged "seeded tester
//--- cache..." as if it had worked, and the run silently trained from scratch instead.
if(CopyFileWithRetry(m_fileName + ".nnw", m_activeFileName + ".nnw"))
{
//--- Best-effort sidecars: not retried - losing one just means a cold calibration/shadow-blend
//--- start rather than a wrong/untrained model, which the .nnw copy above already guards against.
//--- Still share-aware (CopySharedFile, not FileCopy): the live chart holds these open too, so
//--- plain FileCopy would fail on them for exactly the same reason it failed on the .nnw.
if(FileIsExist(m_fileName + ".cfg", FILE_COMMON))
CopySharedFile(m_fileName + ".cfg", m_activeFileName + ".cfg", false);
if(FileIsExist(m_fileName + "_shadow.nnw", FILE_COMMON))
CopySharedFile(m_fileName + "_shadow.nnw", m_activeFileName + "_shadow.nnw", false);
//--- carry the calibration sidecar into the agent sandbox too, so a seeded backtest calibrates its
//--- live decisions with the deployed model's priors instead of the un-adjusted cold defaults.
if(FileIsExist(m_fileName + ".stats", FILE_COMMON))
CopySharedFile(m_fileName + ".stats", m_activeFileName + ".stats", false);
Print(__FUNCTION__ + ": seeded tester cache from the deployed production model (" + m_fileName + ") - this run reuses the deployed weights instead of retraining");
}
//--- else: CopyFileWithRetry already logged why. Fall through - the Net.Load() below will
//--- correctly report "no file" and BuildFreshTopology() takes over, same as a genuine first pass.
}
else if(m_inferenceOnly)
//--- Name the exact file (symbol + timeframe + config fingerprint) it looked for: the model is
//--- keyed on the CHART TIMEFRAME, so the #1 cause of this is running the tester on a different
//--- timeframe than the model was trained on (e.g. an H4 model, tester set to H1) - which reads
//--- as "no model" when one exists under a different timeframe. Spelling out the filename makes
//--- that mismatch obvious instead of looking like the deploy silently failed.
Print(__FUNCTION__ + ": WARNING - no deployed production model found at '" + m_fileName +
".nnw' (shared folder) for " + _Symbol + " " + EnumToString((ENUM_TIMEFRAMES)_Period) +
". A single backtest runs inference only and will NOT train. Most common cause: the tester" +
" timeframe differs from the one the model was trained on (the filename is keyed on timeframe)." +
" Otherwise, train this configuration on a chart first, then re-run the backtest.");
}
if(!LoadAndCompareTopologyConfiguration(m_activeFileName, m_initialNeuronsCount, m_hiddenLayersCount, m_neuronsReduction, m_minNeuronsCount, m_optimizationAlgo, m_historyBars, m_outputNeuronsCount, m_neuronsCount, m_minTrainYear, m_isInitialized, LEGACY_CONVERGE_WR_SLOT, m_fractalPeriods, m_convFilterCount, m_lstmHiddenSize, m_activeFileCommon))
{
// Topology/input params diverged from what produced the saved .nnw (or no .cfg exists yet;
// for inTesterOrOpt this is also the normal "first pass on this agent" case). The stale .cfg
// was already deleted on a mismatch, but the .nnw weights themselves are shaped for the OLD
// topology - loading them into a network built to the NEW shape would corrupt state or crash.
// Drop the incompatible weights/checkpoint too so the Net.Load() below cleanly misses and
// BuildFreshTopology() takes over (i.e. this pass pays the training cost once, and the result
// gets cached below for the NEXT pass to reuse, same as a live topology change would).
if(FileIsExist(m_activeFileName + ".nnw", m_activeFileCommon ? FILE_COMMON : 0))
{
Print(__FUNCTION__ + ": " + m_activeFileName + " - topology/input params changed since last save; discarding incompatible saved weights and starting fresh");
FileDelete(m_activeFileName + ".nnw", m_activeFileCommon ? FILE_COMMON : 0);
//--- Reaching here means a TRAINED model was just thrown away, so its drawn signals are stale for
//--- exactly the same reason ResetWeights() clears them: they would otherwise be restored moments
//--- later (LoadChartSignals runs at the end of this function) and shown as if they belonged to the
//--- model about to be trained. The sibling "no .cfg yet" case does not reach here (there are no
//--- weights to delete), so it is handled separately at the fresh-topology branch below.
ClearPersistedChartSignals("saved weights discarded - topology/input params changed");
}
if(FileIsExist(m_activeFileName + "_ckpt.tmp", m_activeFileCommon ? FILE_COMMON : 0))
FileDelete(m_activeFileName + "_ckpt.tmp", m_activeFileCommon ? FILE_COMMON : 0);
// Same reasoning applies to the EMA shadow-weight file (see m_shadowNet's declaration comment) -
// it's shaped for the OLD topology too, and EnsureShadowNet() has no independent way to detect
// that mismatch on Load() (CNet::Load() doesn't cross-validate against an expected shape). Drop
// it so EnsureShadowNet() cleanly misses and re-bootstraps from the fresh Net instead.
if(FileIsExist(m_activeFileName + "_shadow.nnw", m_activeFileCommon ? FILE_COMMON : 0))
FileDelete(m_activeFileName + "_shadow.nnw", m_activeFileCommon ? FILE_COMMON : 0);
//--- the calibration sidecar is tied to the discarded weights - drop it too so a fresh run
//--- re-measures priors from scratch instead of adjusting with a stale model's base rates.
if(FileIsExist(m_activeFileName + ".stats", m_activeFileCommon ? FILE_COMMON : 0))
FileDelete(m_activeFileName + ".stats", m_activeFileCommon ? FILE_COMMON : 0);
fix: the DB backfill could never run, and HEAD did not compile Four defects in 64c5dd5/1a05e63, found by review + a baseline compile. Goals 1-8 of that session are unchanged; this makes 6 and 8 actually reachable. 1. HEAD DID NOT COMPILE - 6 errors. CControlPanel::Minimize/Maximize were declared `virtual bool ... override`, but CAppDialog declares both as `virtual void` (Controls\Dialog.mqh). errors 265 + 404 on each, plus 151 on `bool ok = CAppDialog::Minimize()`. Return type is void now; there was never a success flag to forward. Verified: 0 errors, 0 warnings. 2. THE BACKFILL COULD NEVER ADVANCE, and neither could the OOS continual simulation (that one has been dead since it was written). Both are armed at the instant convergence is declared, and both advance only from inside Train(), one chunk per call. But ScheduleTrainingIfNeeded's only per-tick ArmStudyEvent site sits in the `else` of a branch taken whenever m_trainingComplete is set and m_trainRunActive is clear - which is exactly the state FinalizeTrainRun() leaves behind one line before they are armed. Train() was never called again, so the walks sat at their start index forever: no "simulation complete" line, and not one row written to the DB this feature exists to fill. Only a manual Resume/Retrain unstuck them. Both flags now keep the model schedulable. 3. IN AI_HYBRID - the mode this ships in - the backfill was never even armed. Ensemble members deploy at Train() ENTRY and return immediately (so no era is wasted), which skips the era-end block the backfill was started from. All four members were a no-op for a second, independent reason. Armed on the ensemble deploy path too, from m_resumeBars/m_resumeOosCutoff. 4. RE-RUNS DUPLICATED ROWS. RegisterSignal inserts unconditionally - no key, no duplicate check - and m_dbBackfillDone is in-memory, so every later attach that retrained to convergence wrote a second full set of rows for the same bars. The ranking would count one bar once per model that ever deployed, weighting superseded opinions as heavily as the live one. A .dbfill marker stamps the deployed era; written only on completion (an interrupted walk redoes itself rather than ranking a partial window) and deleted with the other sidecars on reset-weights. Also: WarmBlocking's timeout was silent, which restored the exact silent pin failure it was added to prevent - it now says so in the journal, and returns true for "no reference pairs to wait for" so the warning stays rare enough to be read. Not addressed, needs a decision: the backfill scores the OOS window with the checkpoint that was SELECTED as best on that same window, then writes those win rates into the table filter weights rank on - the selection set consumed twice, undiscounted, while the deploy gate right next to it applies a family-wise correction for exactly that effect. The rows are also simulated triple-barrier outcomes at today's spread sharing a table with realised fills. The completion log line now states both plainly. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-16 21:25:51 -04:00
//--- and the pattern-database backfill marker (see StartPatternDatabaseBackfill): it records the
//--- era of the model whose OOS calls were written into the ranking tables. Those weights are
//--- being discarded here, so the marker must not survive to suppress the fresh model's own
//--- backfill - era numbers restart from 0 and could otherwise collide with the stale stamp.
if(FileIsExist(m_activeFileName + ".dbfill", m_activeFileCommon ? FILE_COMMON : 0))
FileDelete(m_activeFileName + ".dbfill", m_activeFileCommon ? FILE_COMMON : 0);
SaveTopologyConfiguration(m_activeFileName, m_initialNeuronsCount, m_hiddenLayersCount, m_neuronsReduction, m_minNeuronsCount, m_optimizationAlgo, m_historyBars, m_outputNeuronsCount, m_neuronsCount, LEGACY_STUDY_PERIOD_SLOT, m_minTrainYear, m_isInitialized, LEGACY_CONVERGE_WR_SLOT, m_fractalPeriods, m_convFilterCount, m_lstmHiddenSize, m_activeFileCommon);
}
double loadedIndicatorParams[];
//--- Inference-only backtest: if this deployed model was validated MQL5-inference-safe at deploy
//--- (marker in its .stats), load it host-only and run the pure-MQL5 forward path so the backtest
//--- never loads WarriorDML/WarriorCPU.dll - no DLL file-lock class of failure, and the exact math
//--- the Market build ships. Falls back to a compute backend just below if that load fails.
if(m_inferenceOnly && CheckPointer(Net) != POINTER_INVALID)
{
LoadModelStats(m_activeFileName, m_activeFileCommon); // reads m_mqlInferenceValidated (and priors)
if(m_mqlInferenceValidated)
{
Net.SetCpuInference(true);
PrintVerbose(__FUNCTION__ + ": " + ID + " - inference-only backtest running pure-MQL5 (DLL-free): the deployed model is validated MQL5-inference-safe");
}
}
bool netLoaded = LoadNetWithRetry(loadedIndicatorParams);
//--- Pure-MQL5 load failed unexpectedly (should not happen for a validated model) - drop back to a
//--- compute backend and retry once so the backtest still runs via the DLL rather than on a fresh net.
if(!netLoaded && CheckPointer(Net) != POINTER_INVALID && Net.CpuInference())
{
Print(__FUNCTION__ + ": " + ID + " - pure-MQL5 load failed; retrying with a compute backend (DLL)");
Net.SetCpuInference(false);
netLoaded = LoadNetWithRetry(loadedIndicatorParams);
}
//--- the file may carry a superseded architecture - correct it before anything reads the net
if(netLoaded)
EnforceTopologyContract();
//--- A superseded conv receptive field cannot be repaired in place (different weight-tensor shape), so
//--- the loaded net is discarded and the fresh-topology path below rebuilds and retrains. Deliberately
//--- routed through netLoaded rather than a separate branch: that path already cools the calibration,
//--- resets the era/trainingComplete state and re-arms the label-cache prebuild, all of which a genuine
//--- architecture change needs too.
if(netLoaded && m_topologySuperseded)
netLoaded = false;
//--- restore the calibration sidecar (priors + confidence scale) that pairs with these weights, so a
//--- restart - including a buyer's inference-only backtest - calibrates live decisions exactly as the
//--- saved model did instead of running with cold defaults (priors 0 => no adjustment). See LoadModelStats().
if(netLoaded)
LoadModelStats(m_activeFileName, m_activeFileCommon);
m_modelLoadedFromDisk = netLoaded;
//--- Make a successful resume visible (the counterpart to the fresh-start / mismatch messages below):
//--- on a live chart this confirms the saved model was found and loaded rather than silently retrained.
if(netLoaded && !inTesterOrOpt)
Print(ID + ": resumed saved model from era " + IntegerToString(m_eraCount) + " (trainingComplete=" + (string)m_trainingComplete + ") - continuing, not retraining from era 0.");
2026-08-13 10:23:11 -04:00
//--- RESUMED MODELS GET THE SAME WARM-UP AS FRESH ONES (2026-08-13; was `netLoaded ? 0 : 3`).
//--- The old rationale said a restart "already has a proven-synced history" - but the custom
//--- indicators recompute from scratch every PROCESS start regardless of what the .nnw proves,
//--- and skipping the warm-up on resume is the exact root cause that has now bitten three times:
//--- the cold ATR cached as permanent (ba13eef), the cold AD block training on zeros (the
//--- 2026-08-11 guard), and the 2026-08-13 resumed-META stall (features read milliseconds after
//--- OnInit while five AD indicators were still calculating 54k bars on a memory-starved box,
//--- pass 1 hot-looping 0->100% for 6+ minutes). Three no-op passes cost seconds.
//--- The label cache itself, however, is NEVER restored from the .nnw checkpoint - it lives only in
//--- the in-memory m_labelCacheBuy/Sell/HasValue arrays, which start empty every process start
//--- regardless of netLoaded. Previously this was set to netLoaded, which on a successful checkpoint
//--- load skipped the eager StartLabelCachePrebuild()/AdvanceLabelCachePrebuild() scan (Train()'s
//--- !m_labelCachePrebuilt gate) - every bar then fell through to the lazy per-bar fallback in the
//--- era loop, which calls ComputeLabelForBar(), a dead stub that unconditionally returns
//--- buy=false/sell=false (the real labeling logic lives ONLY in
//--- AdvanceBarrierLabelState(), reachable exclusively from the eager prebuild). The result: every
//--- restart that loaded a checkpoint silently force-labeled the entire era Neutral until something
//--- else (a topology mismatch, a fresh start) triggered a real prebuild. Always eager-prebuilding
//--- now, checkpoint or not, closes this at the root - the dead stub fallback then never matters.
2026-08-13 10:23:11 -04:00
m_warmupPassesRemaining = 3;
m_labelCachePrebuilt = false;
if(inTesterOrOpt && netLoaded)
Print(__FUNCTION__ + ": " + ID + " - reused cached weights from a previous optimization/tester pass on this agent (era " + IntegerToString(m_eraCount) + ", trainingComplete=" + (string)m_trainingComplete + ") - skipping redundant training for this unchanged config");
if(netLoaded && ArraySize(loadedIndicatorParams) == AD_TUNE_PARAM_COUNT)
{
2026-08-13 10:23:11 -04:00
// Restart deploying previously AutoTune-d indicator params even with AutoTuneIndicators=false now.
// AdoptIndicatorParams (NOT a bare ReInitADIndicators): when the saved params equal the values the
// indicators were JUST created with - the common case, since the MI tuner usually keeps the
// configured settings - re-creating is a pure destroy/rebuild churn paid at the worst possible
// moment (milliseconds after process start, history still syncing). Observed 2026-08-13 on a
// resumed META model: the churn threw away five freshly-calculating indicator instances and the
// replacements sat cold for 6+ minutes on a memory-starved box, stalling training entirely.
AdoptIndicatorParams(loadedIndicatorParams, indicators);
}
if(!netLoaded)
{
int error_code = GetLastError();
//--- Do NOT present error_code as the cause: on a no-GPU/CPU-DLL box it is the harmless 5100
//--- (OpenCL-not-found) left by the compute probe inside CNet::Load, NOT the reason the file was
//--- rejected. CNet::Load now prints the precise reason (bad marker / type mismatch / 0-layer stub /
//--- partial layer load) itself. Only clear the stale code here; the "rebuilding fresh" line below is
//--- the user-facing summary.
if(error_code != 5004) // not "file not found"
ResetLastError();
//--- CRITICAL: a failed load may have ALREADY overwritten the training-state out-params from the
//--- bad file's header before it was rejected - notably a corrupt/empty 0-layer stub whose header
//--- still says trainingComplete=1 (see CNet::Load's 0-layer guard). Left as-is, the freshly built,
//--- untrained topology below would be treated as an already-deployed converged model: it would
//--- never train, run inference on random weights (every bar scores Neutral, so the end-of-era NMS
//--- sweep deletes every chart arrow), and "save weights" would just re-persist that empty net.
//--- Force the state back to a genuine fresh start so BuildFreshTopology() actually gets trained.
m_trainingComplete = false;
m_eraCount = 0;
dtStudied = 0;
dForecast = 0;
//--- Cold the in-memory calibration so the freshly-rebuilt (untrained) topology below runs with no
//--- stale prior-correction until a retrain re-measures it (priors 0 => AdjustedSignalFromSoftmax is a
//--- no-op; scale 1.0 = the constructor default). In-memory ONLY - deliberately does NOT touch any
//--- file. (Earlier this session I also deleted the .stats/_shadow.nnw sidecars here; that was too
//--- destructive - a load failure can be transient/spurious (a not-yet-ready compute backend, a
//--- momentary file lock, or - on this CPU-DLL machine - GetLastError() being polluted with the
//--- harmless OpenCL-not-found 5100 from the probe inside CNet::Load), and wiping a user's calibration
//--- and deployed shadow on any such hiccup is the wrong default. The sidecars self-heal anyway: the
//--- shadow re-blends toward the retrained Net and .stats is overwritten on the next save.)
m_priorBuy = 0.0;
m_priorSell = 0.0;
m_priorNeutral = 0.0;
m_confidenceCalScale = 1.0;
//--- Accurate diagnostic (do NOT cite GetLastError() - inside CNet::Load the OpenCL probe leaves 5100
//--- there on a no-GPU/CPU-DLL box, which has nothing to do with the file). Distinguish an ordinary
//--- fresh start (no file yet) from a real read failure of an existing file by testing existence.
if(!inTesterOrOpt)
{
int loadFlags = m_activeFileCommon ? FILE_COMMON : 0;
if(FileIsExist(m_activeFileName + ".nnw", loadFlags))
Print(ID + ": could not read the existing model file " + m_activeFileName + ".nnw - rebuilding a fresh topology to retrain from era 0. Existing .stats/_shadow.nnw are KEPT (they refresh as training runs). If this recurs, that .nnw is likely corrupt - back it up, then use the panel's reset-weights to start clean.");
else
Print(ID + ": no saved model for this config yet - starting a fresh training run from era 0.");
}
//--- Re-seed before building a fresh topology so weight init is genuinely random. A prior
//--- genetic tuner sweep (TuneIndicatorsAndTrain's candidate eval loop, line ~5006) left the
//--- MQL5 RNG to a fixed seed; if this load-fail path then builds a production
//--- topology without re-seeding, the deployed model's weights would be deterministic/repeatable
//--- from whatever the last candidate's seed was — silently reproducible, not genuinely random.
//--- Matches ResetWeights() and Warrior_EA.mq5's OnInit.
MathSrand(GetTickCount());
//--- Era 0 with no weights behind it, so any arrow currently on this chart was drawn by a
//--- DIFFERENT model - the previous fingerprint's, or a corrupt .nnw's. Neither the panel reset
//--- nor the topology-mismatch discard above covers this path: both are gated on there being a
//--- saved .nnw to delete, and here there is none (a changed config produces a new m_fileName,
//--- so the old model's files are not "discarded", they are simply not this model's files).
//--- Left alone the stale arrows do NOT just look wrong - the chart objects survive deploys and
//--- restarts on their own, and SaveChartSignals() rebuilds the sidecar by scanning the chart,
//--- so the first save of this fresh run would adopt the dead model's calls as its own history.
//--- Deliberately at this call site rather than inside BuildFreshTopology(): the genetic tuner
//--- calls that for every throwaway candidate (AutoTune.mqh) and must not touch the chart.
ClearPersistedChartSignals("fresh topology at era 0 - arrows belong to a previous model");
if(!BuildFreshTopology())
return false;
}
TempData = new CArrayDouble();
if(CheckPointer(TempData) == POINTER_INVALID)
return false;
if(netLoaded)
// Populate dPrevSignal from the just-loaded weights immediately, rather than leaving it at
// its blank constructor default until the next (asynchronous, queued) training pass happens
// to run - matters most for the tester cache-reuse path above, where training may be skipped
// entirely for this run because dtStudied already covers the whole backtest window.
RefreshLatestSignal();
//--- Status line must match what the gate below (if(!m_trainingComplete && !m_inferenceOnly)) will
//--- actually do - otherwise an inference-only single backtest logs "resuming full training now" right
//--- under the "runs inference only and will NOT train" warning, which reads as a contradiction.
string trainState = m_trainingComplete
? "already complete - staying converged, no full retrain on this restart"
: (m_inferenceOnly
? "NOT complete, but this is an inference-only backtest - NOT training (see warning above); deploy a trained model for meaningful results"
: "NOT complete (interrupted or never converged) - resuming full training now");
Print(__FUNCTION__ + ": " + m_activeFileName + " - training " + trainState);
//--- Only kick off a full Train() run here if the loaded model genuinely isn't converged yet - an
//--- already-complete model used to get one full era-loop retrain (real Net.backProp() over the
//--- whole IS window) on every single EA restart/reattach for no reason, since this "Init" event
//--- bypassed ScheduleTrainingIfNeeded()'s m_trainingComplete gate entirely. dPrevSignal is already
//--- fresh from RefreshLatestSignal() above; ScheduleTrainingIfNeeded()'s normal per-tick check
//--- will call RefreshConvergedSignal() itself once a genuinely new bar closes.
if(!m_trainingComplete && !m_inferenceOnly)
fix(ensemble): responsive panel + synchronized eras + combined-vote accuracy Four user-reported/requested items, one root cause chain: 1) DEAD CONTROL PANEL in AI_HYBRID mode. All members posted custom event id 1 and handled id 1001, and CExpertCustom broadcasts every chart event to every filter - so each posted event ran a train chunk in ALL N members (N*N chunks per round) and the chart thread never idled long enough to deliver clicks/drags. profiling.csv: 99.45% of time in OnChartEventHandler. Fix: per-instance study-event ids (STUDY_EVENT_ID_BASE + construction order, offset above the Controls library's ON_* codes - id 1 was also ON_DBL_CLICK, so panel double-clicks fired training chunks). ArmStudyEvent() is the single post site; lost-event watchdog replaces the accidental sibling-clears-my-flag rescue. 2) WARM-UP DUPLICATION. The auto-tune sweep is deterministic over identical features/labels, and it ends in the full MI diagnostic suite, which the MI-share gate never intercepted on the sweep path - four members ran four identical ~36s sweep+report blocks. First member publishes outcome (g_ensembleChartTuneDone/Installed/Settings); the rest apply it and skip both. 3) DEINIT STRANDED PANEL+ARROWS (user repro 18:52). Root cause from the log: the 4,500ms budget runs from MetaTrader's stop REQUEST - a heavy autosave in flight ate it, OnDeinit got ~430ms and died in the first member's arrow persist ("Abnormal termination" 432ms in). Fix: early visible-UI sweep (native prefix deletes for status/panel/dialog) right after ClearStatusLabel, and a fast path for still-training models - their arrows are re-rendered every era, so they get one bulk purge instead of scan+atomic-write in the death window. 4) ENSEMBLE FEATURES (user requests): era BARRIER - members advance era by era together; a member ahead of the slowest still-training member declines Train() calls and its chunk budget is donated (TRAIN_TIME_BUDGET_MS = 120/activeTrainers, UI headroom constant). COMBINED-VOTE OOS SCORE - each member's pass-3 scan contributes its adjusted per-bar decision (0.0 on abstain) to a shared row buffer; the last member to finish the era scores the averaged vote vs the mirrored Min_Vote_Open against the same target-before-stop outcomes members grade themselves on, publishing an "Ensemble vote" line on the aggregated panel. Member headlines now carry their lifetime win rate with break-even. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-16 19:06:04 -04:00
ArmStudyEvent((long)MathMax(0, MathMin(iTime(_Symbol, PERIOD_CURRENT, (int)(100 * Net.recentAverageSmoothingFactor * (m_trainingComplete ? 1 : 10))), dtStudied)), "Init");
//--- Restore arrows persisted from a previous session (see SaveChartSignals). MUST run here, not in
//--- InitIndicators(): the arrows file is keyed on the FULL m_fileName including the per-config
//--- fingerprint, which is only appended above - see the note left at InitIndicators()'s old call site.
LoadChartSignals();
//--- bootstrap (or restore) the EMA shadow net now rather than waiting for the first
//--- RefreshLatestSignal()/era-blend call to lazily trigger it - see m_shadowNet's declaration
//--- comment.
EnsureShadowNet();
m_isInitialized = true;
research: export the feature matrix and a raw OHLCV grid for offline work The bottleneck on this project has never been the modelling - it is that every hypothesis costs a compile, a deploy, an attach and a log read, and answers exactly one question. Days have gone into questions that are seconds of arithmetic once the data is in hand. Adds a RESEARCH-ONLY build, gated behind WARRIOR_EXPORT_FEATURES and never compiled into a shipped binary, which writes two things to Common\Files\Warrior_EA\Research\ and then does nothing at all: <symbol>_<tf>_features.csv - one row per bar: index, time, OHLC, ATR, and the m_neuronsCount feature values. Exactly what the network sees. The raw bars ride along on purpose: with OHLC and ATR offline, every barrier geometry, horizon and in-trade target is recomputable without MetaTrader in the loop. <symbol>_<tf>_rates.csv - raw OHLCV across a grid of 8 symbols x 5 timeframes. The 26 engineered features only exist for the attached chart (indicator handles bind to PERIOD_CURRENT); raw rates do not, so ONE attach yields the whole research grid. The bar time also makes session/hour/day-of-week derivable - the only inputs in play that are not a transform of the same OHLCV series. Safety, because this binary gets attached to a chart on a LIVE ACCOUNT to reach real history: - OnTick returns immediately, so Expert.OnTick() - the entire trading path - is unreachable regardless of the AlgoTrading toggle, the signal state or the inputs. Structurally incapable of sending an order, not merely unlikely to. - No config lock. It never trains and never saves a model, so it has nothing to protect against a concurrent chart - and taking the lock would make it refuse to start exactly when the config it wants to read is already open, which is when it is most useful. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-01 15:49:57 -04:00
#ifdef WARRIOR_EXPORT_FEATURES
//--- Research build only. Runs here because this is the first point at which the indicators, the buffers
//--- and the derived barrier horizon are all settled, and it needs no model, no labels and no training.
ExportFeatureMatrix();
#endif
return true;
}
//+------------------------------------------------------------------+
//| Shared training-set size estimate - see the declaration comment. |
//+------------------------------------------------------------------+
double CExpertSignalAIBase::EstimatedInSampleBars(void) const
{
int secs = PeriodSeconds(m_period);
if(secs <= 0)
secs = PeriodSeconds(PERIOD_H1);
double barsPerYear = (SECONDS_PER_YEAR / (double)secs) * MARKET_OPEN_FRACTION;
double oosKept = (100.0 - (double)m_oosSplitPct) / 100.0;
//--- MEASURED from the symbol's real history, matching Train()'s window exactly (earliest available
//--- bar, floored by MinTrainYear) now that training covers everything available rather than a
//--- configured number of years. There used to be a hard rule against reading Bars() here, and it was
//--- right for what it guarded: what is downloaded grows over a terminal's lifetime, and a topology
//--- that silently widens as history fills in would re-key its own weights file and throw away a
//--- trained model. That hazard is now closed at the other end instead - the derived shape is written
//--- into the .cfg on first build and ADOPTED, not re-derived, on every subsequent load, and none of
//--- the derived values feed the weights-filename fingerprint any more. So this is measured once per
//--- model, at the moment the model is created, and never consulted again for an existing one.
datetime firstAvailableBar = (datetime)SeriesInfoInteger(_Symbol, m_period, SERIES_FIRSTDATE);
MqlDateTime floorTime;
TimeCurrent(floorTime);
floorTime.year = m_minTrainYear;
floorTime.mon = 1;
floorTime.day = 1;
floorTime.hour = 0;
floorTime.min = 0;
floorTime.sec = 0;
datetime windowStart = StructToTime(floorTime);
if(firstAvailableBar > windowStart)
windowStart = firstAvailableBar;
int available = Bars(_Symbol, m_period, windowStart, TimeCurrent());
//--- History may not have finished syncing when a chart first attaches, and a model whose capacity was
//--- pinned from a handful of bars would stay crippled for its whole life - the one failure mode that
//--- measuring instead of assuming introduces. Fall back to a conservative fixed span rather than
//--- pinning something absurd, and say so, because the fix (reattach once history has synced) is the
//--- user's to make and is invisible otherwise.
if(available < TOPOLOGY_BUDGET_MIN_TRUSTED_BARS)
{
Print(ID + ": WARNING - only " + IntegerToString(available) + " bars of " + _Symbol +
" history are available yet, too few to size the network from. Falling back to a " +
IntegerToString(TOPOLOGY_BUDGET_FALLBACK_YEARS) + "-year assumption. If this is a fresh" +
" install, let the terminal finish downloading history and then delete this model's weights" +
" from the panel so the topology is sized from the real data.");
return (double)TOPOLOGY_BUDGET_FALLBACK_YEARS * barsPerYear * oosKept;
}
return (double)available * oosKept;
}
//+------------------------------------------------------------------+
//| Derived input-window length - see the declaration comment. |
//+------------------------------------------------------------------+
int CExpertSignalAIBase::DeriveHistoryBars(void)
{
int availableBars = Bars(_Symbol, m_period);
int span = (int)MathMin(availableBars - 1, WINDOW_DERIVE_SPAN_BARS);
if(span < TOPOLOGY_BUDGET_MIN_TRUSTED_BARS)
{
Print(ID + ": WARNING - only " + IntegerToString(availableBars) + " bars of " + _Symbol +
" history are available yet, too few to measure the input window from. Falling back to " +
IntegerToString(HISTORY_BARS_FALLBACK) + " bars. If this is a fresh install, let history "
"finish downloading and delete this model's weights so the window is measured from real data.");
return HISTORY_BARS_FALLBACK;
}
//--- newest CLOSED bars only (start 1): the forming bar's extremes are still moving
double hi[], lo[];
ArraySetAsSeries(hi, true);
ArraySetAsSeries(lo, true);
if(CopyHigh(_Symbol, m_period, 1, span, hi) != span ||
CopyLow(_Symbol, m_period, 1, span, lo) != span)
{
Print(ID + ": WARNING - could not read " + IntegerToString(span) + " bars to measure the input "
"window; falling back to " + IntegerToString(HISTORY_BARS_FALLBACK) + " bars.");
return HISTORY_BARS_FALLBACK;
}
//--- Swing pivots: strict local extremum against the NEWER side, >= against the older side (the
//--- standard tie-break so a flat top counts once). Alternation is enforced by keeping the more
//--- extreme of two same-type candidates - a higher high before any low confirms extends the leg,
//--- it does not end one. Legs are the bar distances between consecutive ALTERNATING pivots: the
//--- same "confirmed leg" population the barrier horizon medians, measured here from raw price
//--- because at init the ZigZag indicator has no data yet.
int legs[];
ArrayResize(legs, 0, 256);
int lastType = 0; // +1 swing high, -1 swing low, 0 none yet
int lastPivotBar = -1;
double lastExtreme = 0.0;
for(int b = span - 1 - WINDOW_SWING_WING; b >= WINDOW_SWING_WING; b--) // oldest -> newest
{
bool isHigh = true, isLow = true;
for(int w = 1; w <= WINDOW_SWING_WING && (isHigh || isLow); w++)
{
if(hi[b] <= hi[b - w] || hi[b] < hi[b + w])
isHigh = false;
if(lo[b] >= lo[b - w] || lo[b] > lo[b + w])
isLow = false;
}
int type = 0;
if(isHigh != isLow)
type = isHigh ? 1 : -1; // a bar that is both is degenerate; skip it
if(type == 0)
continue;
if(type == lastType)
{
double x = (type == 1) ? hi[b] : lo[b];
if((type == 1 && x > lastExtreme) || (type == -1 && x < lastExtreme))
{
lastPivotBar = b;
lastExtreme = x;
}
continue;
}
if(lastType != 0)
{
int n = ArraySize(legs);
ArrayResize(legs, n + 1, 256);
legs[n] = lastPivotBar - b; // series indices: newer bar = smaller index
}
lastType = type;
lastPivotBar = b;
lastExtreme = (type == 1) ? hi[b] : lo[b];
}
if(ArraySize(legs) < WINDOW_DERIVE_MIN_LEGS)
{
Print(ID + ": WARNING - only " + IntegerToString(ArraySize(legs)) + " confirmed swing legs in " +
IntegerToString(span) + " bars, too few to trust a median. Falling back to " +
IntegerToString(HISTORY_BARS_FALLBACK) + " bars.");
return HISTORY_BARS_FALLBACK;
}
ArraySort(legs);
int median = legs[ArraySize(legs) / 2];
//--- snap DOWN to the ladder (see LEGACY_HISTORY_BARS_SLOT's comment for floor/cap rationale)
int ladder[] = {12, 16, 20, 24, 32};
int window = HISTORY_BARS_FLOOR;
for(int i = 0; i < ArraySize(ladder); i++)
if(ladder[i] <= median)
window = ladder[i];
PrintFormat("%s: derived input window - %d bars (median confirmed swing leg %d over %d legs in %d "
"bars, snapped down to the ladder%s). Measured once at model creation and pinned in the "
".cfg; an existing model adopts its own trained window instead.",
ID, window, median, ArraySize(legs), span,
median > 32 ? ", CAPPED at 32 - era time scales with the window" : "");
return window;
}
//+------------------------------------------------------------------+
//| Dense-taper depth - see the declaration comment. |
//+------------------------------------------------------------------+
int CExpertSignalAIBase::ComputeHiddenLayerCount(void) const
{
//--- Diagnostic escape hatch (compile-time, see ForceHiddenLayers). Deliberately not an input: this
//--- exists to run depth comparisons while working on the EA, and a user who picks a depth is
//--- contradicting the width and taper the code derived around it.
if(ForceHiddenLayers > 0)
return (int)MathMax(1, MathMin(MAX_HIDDEN_LAYERS, ForceHiddenLayers));
//--- Depth follows from the two ENDPOINTS the taper already has to connect - the derived first-layer
//--- width and the output-tied final hidden width (see BuildFreshTopology's taper block) - by asking
//--- how many steps it takes to get from one to the other at a sane per-layer compression ratio.
//--- Picking depth independently of those endpoints is what made it meaningless as an input: at 64
//--- units tapering to 12, four layers compress by 1.4x per step and five by barely 1.3x, so the extra
//--- depth buys no additional abstraction and costs a vanishing-gradient risk for nothing.
int lastHidden = (int)MathMax(HIDDEN_TAPER_OUTPUT_MULTIPLE * m_outputNeuronsCount, HIDDEN_TAPER_MIN_WIDTH);
lastHidden = (int)MathMin(lastHidden, m_initialNeuronsCount);
if(lastHidden <= 0 || m_initialNeuronsCount <= lastHidden)
feat: derived taper restored; DB ranking reads a reserved slice, shrunk TOPOLOGY - reverts the two constants and drops CausalHiddenLayerFloor. The MQL5 article's 30%-per-layer cut and floor of 20 are load-bearing on ITS first-layer width of 1000 (1000->300->90->27 needs a floor to stop). This codebase MEASURES that width, and on the live SP500 H4 config it is 16 units - already floored, with the budget printing "11360 estimated in-sample bars cannot support a 800-wide input ... roughly 1.1 weights per training bar - expect overfitting". At 16 units a floor of 20 makes lastHidden >= m_initialNeuronsCount, so ComputeHiddenLayerCount returns on its first branch and the width taper - the only part derived from this symbol's data - became dead code on all four ensemble members, with depth (2 -> 4) set entirely by counting feature domains. ComputeLayerWidths had already rejected this exact pair of constants in its own comment. The causal floor's premise does not hold either: layers are not inference steps. The "1 layer linear / 2 nonlinear / 3 multi-connected" result is Lippmann 1987 and is about hard-threshold units; with sigmoid/ReLU, Cybenko 1989 and Hornik 1991 give universal approximation from a single hidden layer. Depth buys parameter efficiency for compositional functions, not reasoning hops. ForceHiddenLayers remains for measuring depth directly. RANKING SLICE - the backfill no longer reads the window it is judged on. The deployed checkpoint is CHOSEN as the best-scoring era on the OOS window, so win rates measured back over it are selection-inflated, and the backfill was writing exactly those into the table filter weights rank on: the selection set consumed twice, beside a deploy gate that applies a Sidak correction for that effect. The newest RANK_SLICE_PCT_OF_OOS (20%) of the OOS window, plus a label-horizon purge, is now reserved and graded by nothing - not pass 3, not checkpoint selection, not the gate. The backfill reads only that. The gate keeps ~80% of its measurement (power goes as the square root, so ~10% of a sigma), and the slice is the newest data, which is the regime about to be traded. RankSliceBars returns 0 when no honest slice fits and the backfill then REFUSES and says so, rather than falling back to the scoring window and looking like a success. SHRINKAGE - per-tier win rates are shrunk toward the filter's own pooled rate by MIN_TRADES_FOR_WIN_RATE pseudo-trades before becoming weights. The raw ratio at the minimum sample count carries a ~15pp standard error, so a tier that went 8-2 was handed weight 80 and outranked a tier measured over hundreds of calls at 55 - the ranking was being driven by which small tier got lucky. Opt-in per call site (priorWeight 0 keeps the raw behaviour). Compile-verified: 0 errors, 0 warnings. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-16 21:49:52 -04:00
return MIN_HIDDEN_LAYERS;
double steps = MathLog((double)m_initialNeuronsCount / (double)lastHidden) / MathLog(HIDDEN_TAPER_TARGET_RATIO);
int layers = (int)MathRound(steps) + 1; // +1: the first layer IS the starting endpoint, not a step
return (int)MathMax(MIN_HIDDEN_LAYERS, MathMin(MAX_HIDDEN_LAYERS, layers));
}
//+------------------------------------------------------------------+
//| Conv output-filter count - see the declaration comment. |
//+------------------------------------------------------------------+
int CExpertSignalAIBase::ComputeConvFilterCount(void) const
{
//--- AddConvStage sets window = ConvReceptiveFieldBars() * m_neuronsCount and step = m_neuronsCount,
//--- so each sliding position covers that many BARS of features and the layer is a learned projection
//--- from the whole window down to this many filters. The meaningful reference point is therefore the
//--- WINDOW width, not one bar's feature count - budgeting against a single bar was correct only while
//--- the receptive field was 1, and at RF 3 it under-sized the stage 3x (8 filters for a 63-input
//--- window, an 8x squeeze, where the rule intends 2x). Same rule as before, applied to what the layer
//--- actually reads: halve the input. Keeping it tied to the window also means the receptive field and
//--- the filter count can never drift apart the way they did on 2026-07-31.
int chosen = (ConvReceptiveFieldBars() * m_neuronsCount) / CONV_COMPRESSION_DIVISOR;
//--- Snap DOWN to a power-of-two ladder for the same reason the first-layer width does: the target is
//--- approximate, and a value that moves with every feature toggle would re-key the weights file more
//--- often than the change in capacity justifies.
int ladder[] = {4, 8, 16, 32};
int snapped = CONV_FILTERS_MIN;
for(int i = 0; i < ArraySize(ladder); i++)
if(ladder[i] <= chosen)
snapped = ladder[i];
return (int)MathMax(CONV_FILTERS_MIN, MathMin(CONV_FILTERS_MAX, snapped));
}
//+------------------------------------------------------------------+
//| Derived front-end stages, for the startup config line. |
//+------------------------------------------------------------------+
string CExpertSignalAIBase::FrontEndConfigSummary(void) const
{
string s = "";
//--- conv slides a ConvReceptiveFieldBars()-bar window one bar at a time, emitting m_convFilterCount
//--- filters per position; the optional channel pool + second conv follow. Reported from the shape
//--- helpers rather than re-derived, so this line always describes what AddConvStage actually built.
if(UsesConvStage())
{
s += " | conv " + IntegerToString(ConvReceptiveFieldBars()) + " bars x" +
IntegerToString(m_neuronsCount) + "->" + IntegerToString(m_convFilterCount) +
" (" + IntegerToString(ConvFirstStagePositions()) + " pos)";
if(HasSecondConvStage())
s += " | pool /" + IntegerToString(m_convFilterCount) +
" | conv2 ->" + IntegerToString(ConvOutputPositions()) + " pos x" +
IntegerToString(m_convFilterCount) + " = " + IntegerToString(ConvOutputWidth());
else
s += " = " + IntegerToString(ConvOutputWidth());
}
if(UsesLstmStage())
s += " | lstm " + IntegerToString(LstmFanIn()) + "->" + IntegerToString(m_lstmHiddenSize);
//--- The dense stack is budgeted against the RAW input, so on any topology with a front-end it can be
//--- WIDER than the vector reaching it - a linear fan-out that cannot recover information the
//--- bottleneck already discarded, only add parameters. Flag it rather than silently reshaping a
//--- trained topology; see ComputeFirstLayerWidth.
int frontEndOut = UsesLstmStage() ? m_lstmHiddenSize
: (UsesConvStage() ? ConvOutputWidth() : 0);
if(frontEndOut > 0 && m_initialNeuronsCount > frontEndOut)
s += " | NOTE dense fans out " + IntegerToString(frontEndOut) + "->" +
IntegerToString(m_initialNeuronsCount);
return s;
}
//+------------------------------------------------------------------+
//| Input width the LSTM block actually receives. |
//+------------------------------------------------------------------+
int CExpertSignalAIBase::LstmFanIn(void) const
{
//--- LSTM-only: the layer sits directly on the input, so it sees the whole flattened vector.
//--- HYBRID: AddConvStage runs first, so the LSTM sees the CONV FEATURE MAP, not the input. That map
//--- is position-major - ConvOutputPositions() positions of m_convFilterCount filters each - so the
//--- LSTM's per-timestep width stays m_convFilterCount (AddLstmStage) and its step count is the
//--- POSITION count, which the conv chain shrinks below m_historyBars once a multi-bar window and a
//--- second conv are in play. Hardcoding historyBars here would over-state the fan-in and, worse,
//--- disagree with the width CNet actually hands the layer.
//--- Budgeting HYBRID's LSTM against the flattened 420 UNDER-sized it by a full ladder step: the
//--- quadratic in ComputeLstmHiddenSize is dominated by the inputs term, so overstating the fan-in
//--- buys a smaller H for no reason.
//--- Requires m_convFilterCount to be settled first - InitNeuralNetwork orders it that way.
if(HasConvBeforeLstm())
return ConvOutputWidth();
return (int)m_historyBars * m_neuronsCount;
}
//+------------------------------------------------------------------+
//| LSTM recurrent hidden width - see the declaration comment. |
//+------------------------------------------------------------------+
int CExpertSignalAIBase::ComputeLstmHiddenSize(void) const
{
//--- The LSTM block's parameter count is EXACTLY 4 * H * (H + inputs + 1) - see
//--- CNeuronLSTMOCL::SetInputs in AI\Network.mqh - and AddLstmStage feeds it the whole flattened
//--- input vector, so `inputs` is historyBars x neuronsCount. That makes this stage far and away the
//--- largest weight block in an LSTM or HYBRID model: at the shipped default of 32 units against a
//--- 540-wide input it is ~73k weights, more than DOUBLE the entire derived dense taper it feeds.
//--- It was the one part of the network the capacity budget never covered, which is why deriving the
//--- dense stack alone did not stop LSTM/HYBRID from being over-parameterized.
//--- Same budget as ComputeFirstLayerWidth: at most one weight per in-sample bar. Solving
//--- 4H(H+inputs+1) <= isBars for H is an ordinary quadratic, H = (-b + sqrt(b^2+4c))/2 with
//--- b = inputs+1 and c = isBars/4.
//--- PER-TIMESTEP width, not the flattened fan-in. The layer is now a recurrence: one shared gate
//--- block is applied at every step, so its parameter count is 4H(H + stepWidth + 1) - the whole
//--- point of weight sharing. Budgeting against the flattened width (420, or 160 behind conv) was
//--- correct for the old single-timestep layer and is now ~20x too pessimistic, which would starve
//--- the recurrence of hidden units for no reason.
//--- Must match what the layer is actually built as: a recurrence sizes its shared gate block on the
//--- PER-TIMESTEP width, while the single-timestep layer reads the whole flattened vector at once.
//--- Budgeting one against the other over-parameterizes by ~20x in one direction and starves the
//--- recurrence in the other. See LSTM_SEQUENCE_MODE.
int inputs = (LSTM_SEQUENCE_MODE ? (HasConvBeforeLstm() ? m_convFilterCount : m_neuronsCount)
: LstmFanIn());
double isBars = EstimatedInSampleBars();
if(inputs <= 0 || isBars <= 0.0)
return LSTM_HIDDEN_MIN;
double b = (double)(inputs + 1);
double budget = (-b + MathSqrt(b * b + 4.0 * (isBars / 4.0))) / 2.0;
int ladder[] = {8, 16, 32, 64, 128};
int snapped = LSTM_HIDDEN_MIN;
for(int i = 0; i < ArraySize(ladder); i++)
if((double)ladder[i] <= budget)
snapped = ladder[i];
return (int)MathMax(LSTM_HIDDEN_MIN, MathMin(LSTM_HIDDEN_MAX, snapped));
}
//+------------------------------------------------------------------+
//| Capacity budget for the first dense layer - see the declaration. |
//+------------------------------------------------------------------+
int CExpertSignalAIBase::ComputeFirstLayerWidth(void) const
{
feat: mini-batch gradient accumulation (F4), front-end-aware capacity budget (F6), split Wyckoff categoricals (N1) Completes the 2026-08-09 training audit. FORCES A RETRAIN of every Wyckoff-enabled config (N1 re-keys the fingerprint), and BOTH DLLs must be redeployed alongside the .ex5 - they carry new exports. F4 - mini-batch accumulation, TRAIN_BATCH_SIZE=32. Training was pure online SGD (one weight update per bar), which is the mechanical source of the era-to-era whipsaw every downstream guard was built to cope with. The O(n^2) outer product is native - AccumulateWeightGrad / AccumulateWeightGradConv / AccumulateBufferInto in Network.cl, WarriorCPU and WarriorDML - while the optimizer step is host-side MQL5 shared by all tiers (ApplyAccumToBlock), so there is one Adam/SGD implementation instead of four that can drift. - the LSTM needs no outer-product kernel (WeightsGradient already holds the sample's full dW) but could NOT simply be left un-zeroed between samples: CPU_LSTMSeqBackward/DML_LSTMSeqBackward memset it on entry. Hence a separate accumulator plus an elementwise add. - batch-norm gamma/beta accumulate in host arrays, not new BatchOptions slots - BN_OPT_STRIDE is baked into every persisted .nnw. - scoped to pass 2; online learning keeps immediate updates. Every save / checkpoint / scoring boundary flushes, scaling by the real sample count. - degrades to per-sample updates (one log line) on a tier that cannot accumulate, so old devices and DLL-free builds are unaffected. - verified offline: DirectML/batch_accum_check.cpp drives the real exports against an independent reference; at B=1 the accumulator matches the shipped unbatched kernel's own gradient to 1.1e-16. Math only - the in-situ check remains the per-layer dW/W report on a real era. F6 - ComputeFirstLayerWidth budgeted against the RAW input width even where a conv/LSTM front end had already reduced it, so an LSTM's dense stack was charged for 1,280 inputs when it receives 64. Confirmed from the deployed .cfg files: CONV, LSTM and HYBRID were all pinned at the 16-unit floor. Now budgeted against the front-end output and capped at it (never fan out), with the derivation reordered so both stages settle first. N1 - EventCode/EventPhase/StructuralPhase are signed categoricals packing direction and Wyckoff stage into one scalar across a sign discontinuity. Split into direction + [0,1] magnitude, the same convention the base OHLC block uses. Information-preserving; 13 readings now occupy 16 inputs. Compiled clean (0 errors, 0 warnings); both DLLs rebuilt. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-09 11:48:03 -04:00
//--- THE WIDTH THAT ACTUALLY REACHES THE DENSE STACK, not the raw input vector.
//--- Until 2026-08-09 this budgeted against m_historyBars * m_neuronsCount on every topology, which
//--- is only the truth for a plain MLP. On CONV/LSTM/HYBRID a front-end stage has already reduced
//--- the vector by the time the first dense layer sees it - an LSTM hands the taper m_lstmHiddenSize
//--- values (64 on the shipped SP500 H1 config), not 1,280 - so the budget was charging the dense
//--- layer for ~20x the fan-in it has. Measured on the deployed .cfg files that day: CONV, LSTM and
//--- HYBRID had all been pinned at FIRST_LAYER_MIN_WIDTH for their whole lives, with the
//--- "cannot support a N-wide input" warning below firing on a premise that was not true of them.
//--- The front-end widths come from the same helpers AddConvStage/AddLstmStage build from, so what is
//--- budgeted and what is constructed cannot disagree - and InitNeuralNetwork settles both stages
//--- before calling this (see the ordering note there).
int frontEndOut = UsesLstmStage() ? m_lstmHiddenSize
: (UsesConvStage() ? ConvOutputWidth() : 0);
int inputWidth = (frontEndOut > 0) ? frontEndOut : (int)m_historyBars * m_neuronsCount;
if(inputWidth <= 0)
return FIRST_LAYER_MIN_WIDTH;
double isBars = EstimatedInSampleBars();
//--- One first-layer weight per in-sample bar. That layer is (inputWidth+1) x width and dominates the
//--- model, so this is effectively a whole-model capacity budget. One parameter per sample is already
//--- generous for a signal this weak; it is a ceiling, not a target.
int budget = (int)(isBars / (double)(inputWidth + 1));
//--- Snap DOWN to the ladder: the estimate above is approximate, and a value that moves with every
//--- small change would re-key the weights file for no benefit. Rungs are far enough apart that the
//--- estimate would have to be wrong by ~2x to land on a different one.
int ladder[] = {16, 32, 64, 128, 256, 512, 1024};
int chosen = FIRST_LAYER_MIN_WIDTH;
for(int i = 0; i < ArraySize(ladder); i++)
if(ladder[i] <= budget)
chosen = ladder[i];
feat: mini-batch gradient accumulation (F4), front-end-aware capacity budget (F6), split Wyckoff categoricals (N1) Completes the 2026-08-09 training audit. FORCES A RETRAIN of every Wyckoff-enabled config (N1 re-keys the fingerprint), and BOTH DLLs must be redeployed alongside the .ex5 - they carry new exports. F4 - mini-batch accumulation, TRAIN_BATCH_SIZE=32. Training was pure online SGD (one weight update per bar), which is the mechanical source of the era-to-era whipsaw every downstream guard was built to cope with. The O(n^2) outer product is native - AccumulateWeightGrad / AccumulateWeightGradConv / AccumulateBufferInto in Network.cl, WarriorCPU and WarriorDML - while the optimizer step is host-side MQL5 shared by all tiers (ApplyAccumToBlock), so there is one Adam/SGD implementation instead of four that can drift. - the LSTM needs no outer-product kernel (WeightsGradient already holds the sample's full dW) but could NOT simply be left un-zeroed between samples: CPU_LSTMSeqBackward/DML_LSTMSeqBackward memset it on entry. Hence a separate accumulator plus an elementwise add. - batch-norm gamma/beta accumulate in host arrays, not new BatchOptions slots - BN_OPT_STRIDE is baked into every persisted .nnw. - scoped to pass 2; online learning keeps immediate updates. Every save / checkpoint / scoring boundary flushes, scaling by the real sample count. - degrades to per-sample updates (one log line) on a tier that cannot accumulate, so old devices and DLL-free builds are unaffected. - verified offline: DirectML/batch_accum_check.cpp drives the real exports against an independent reference; at B=1 the accumulator matches the shipped unbatched kernel's own gradient to 1.1e-16. Math only - the in-situ check remains the per-layer dW/W report on a real era. F6 - ComputeFirstLayerWidth budgeted against the RAW input width even where a conv/LSTM front end had already reduced it, so an LSTM's dense stack was charged for 1,280 inputs when it receives 64. Confirmed from the deployed .cfg files: CONV, LSTM and HYBRID were all pinned at the 16-unit floor. Now budgeted against the front-end output and capped at it (never fan out), with the derivation reordered so both stages settle first. N1 - EventCode/EventPhase/StructuralPhase are signed categoricals packing direction and Wyckoff stage into one scalar across a sign discontinuity. Split into direction + [0,1] magnitude, the same convention the base OHLC block uses. Information-preserving; 13 readings now occupy 16 inputs. Compiled clean (0 errors, 0 warnings); both DLLs rebuilt. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-09 11:48:03 -04:00
//--- NEVER WIDER THAN THE STAGE FEEDING IT. Charging the dense layer only for its real fan-in makes
//--- the budget generous on a narrow front end - an LSTM's 64 outputs would allow 256 units - and a
//--- 64 -> 256 first layer is a linear fan-out that cannot recover information the recurrence already
//--- discarded, it can only add parameters to overfit with. FrontEndConfigSummary() already calls
//--- that shape out as a defect when it happens; this stops it happening. The taper below this layer
//--- then funnels as intended. No effect on a plain MLP, which has no front end to be capped by.
if(frontEndOut > 0)
chosen = (int)MathMin(chosen, frontEndOut);
//--- Budget below the floor means this configuration cannot support even the narrowest usable layer -
//--- the model will be over-parameterized no matter what is chosen here, and no amount of
//--- regularization fixes having more weights than examples. Typical cause is a high timeframe
//--- (D1 over 10 years is under 2,000 bars) or too many features for the history available. Say so:
//--- the fix is fewer HistoryBars / fewer feature groups / a longer study period, none of which this
//--- function can choose on the user's behalf.
if(budget < FIRST_LAYER_MIN_WIDTH)
Print(ID + ": WARNING - " + IntegerToString((int)isBars) + " estimated in-sample bars cannot support a " +
feat: mini-batch gradient accumulation (F4), front-end-aware capacity budget (F6), split Wyckoff categoricals (N1) Completes the 2026-08-09 training audit. FORCES A RETRAIN of every Wyckoff-enabled config (N1 re-keys the fingerprint), and BOTH DLLs must be redeployed alongside the .ex5 - they carry new exports. F4 - mini-batch accumulation, TRAIN_BATCH_SIZE=32. Training was pure online SGD (one weight update per bar), which is the mechanical source of the era-to-era whipsaw every downstream guard was built to cope with. The O(n^2) outer product is native - AccumulateWeightGrad / AccumulateWeightGradConv / AccumulateBufferInto in Network.cl, WarriorCPU and WarriorDML - while the optimizer step is host-side MQL5 shared by all tiers (ApplyAccumToBlock), so there is one Adam/SGD implementation instead of four that can drift. - the LSTM needs no outer-product kernel (WeightsGradient already holds the sample's full dW) but could NOT simply be left un-zeroed between samples: CPU_LSTMSeqBackward/DML_LSTMSeqBackward memset it on entry. Hence a separate accumulator plus an elementwise add. - batch-norm gamma/beta accumulate in host arrays, not new BatchOptions slots - BN_OPT_STRIDE is baked into every persisted .nnw. - scoped to pass 2; online learning keeps immediate updates. Every save / checkpoint / scoring boundary flushes, scaling by the real sample count. - degrades to per-sample updates (one log line) on a tier that cannot accumulate, so old devices and DLL-free builds are unaffected. - verified offline: DirectML/batch_accum_check.cpp drives the real exports against an independent reference; at B=1 the accumulator matches the shipped unbatched kernel's own gradient to 1.1e-16. Math only - the in-situ check remains the per-layer dW/W report on a real era. F6 - ComputeFirstLayerWidth budgeted against the RAW input width even where a conv/LSTM front end had already reduced it, so an LSTM's dense stack was charged for 1,280 inputs when it receives 64. Confirmed from the deployed .cfg files: CONV, LSTM and HYBRID were all pinned at the 16-unit floor. Now budgeted against the front-end output and capped at it (never fan out), with the derivation reordered so both stages settle first. N1 - EventCode/EventPhase/StructuralPhase are signed categoricals packing direction and Wyckoff stage into one scalar across a sign discontinuity. Split into direction + [0,1] magnitude, the same convention the base OHLC block uses. Information-preserving; 13 readings now occupy 16 inputs. Compiled clean (0 errors, 0 warnings); both DLLs rebuilt. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-09 11:48:03 -04:00
IntegerToString(inputWidth) + "-wide " +
(frontEndOut > 0 ? "vector into the dense stack" : "input") + ". The first layer is being floored at " +
IntegerToString(FIRST_LAYER_MIN_WIDTH) + " units, which is still roughly " +
DoubleToString((double)(inputWidth + 1) * FIRST_LAYER_MIN_WIDTH / MathMax(1.0, isBars), 1) +
" weights per training bar - expect overfitting. Reduce HistoryBars or the feature set," +
" lengthen the study period, or train on a lower timeframe.");
return MathMax(FIRST_LAYER_MIN_WIDTH, chosen);
}
//+------------------------------------------------------------------+
//| Batch-normalization layer - see the declaration comment. |
//+------------------------------------------------------------------+
bool CExpertSignalAIBase::AddBatchNormStage(CArrayObj *topology, int units)
{
if(CheckPointer(topology) == POINTER_INVALID)
return false;
//--- Not an error: the input is off, so the topology simply has no normalization layers. Returning
//--- true keeps every call site a plain `if(!Add...) return false;` with no extra branching.
if(!EnableBatchNorm)
return true;
//--- A window of 1 makes the layer a no-op passthrough (mean==x, variance==0), which is a silently
//--- useless layer rather than an obviously absent one. Refuse to build it instead.
if(BatchNormWindow <= 1)
return true;
CLayerDescription *desc = new CLayerDescription();
if(CheckPointer(desc) == POINTER_INVALID)
return false;
desc.count = units;
desc.type = defNeuronBatchNorm;
desc.batch = BatchNormWindow;
//--- Identity forward transform. The non-linearity belongs to the dense layer stacked on top of this
//--- one; normalizing and then squashing in the same step would undo the normalization.
desc.activation = NONE;
desc.optimization = (ENUM_OPTIMIZATION)m_optimizationAlgo;
if(!topology.Add(desc))
{
delete desc;
return false;
}
return true;
}
//+------------------------------------------------------------------+
//| Convolution front-end: conv -> channel pool -> conv. Shared by |
//| CSignalCONV and CSignalHYBRID - see the declaration comment. |
//| |
//| MEMORY LAYOUT, which is what every decision here turns on: |
//| - The INPUT is bar-major: BufferTempData appends m_neuronsCount |
//| contiguous features per bar, bars in order. So a flat window of |
//| k*m_neuronsCount spans exactly k consecutive BARS, and a step |
//| of m_neuronsCount advances exactly one bar. A multi-bar |
//| receptive field therefore needs NO kernel change. |
//| - A CONV OUTPUT is position-major: FeedForwardConv (AI\Network.cl)|
//| emits matrix_o[out + window_out * i], so one position's |
//| window_out filter responses are CONTIGUOUS and consecutive |
//| positions sit window_out apart. |
//| - Both pool implementations (FeedForwardProof, CPU_FeedForwardProof)
//| slide FLAT: pos = i*step over `window` CONSECUTIVE elements. |
//| Over a position-major buffer those neighbours are the FILTERS of |
//| one position. So a pool here is a max-over-CHANNELS, never a |
//| pool across time. |
//| |
//| That is exactly the NeuroNet_DNG reference contract (see |
//| references\MQL5\Experts\EDL\Trajectory.mqh layers 2-5, kernels |
//| byte-identical to ours): conv(window=2, step=1, window_out=4) -> |
//| pool(window=4, step=4) -> conv -> pool. The pool is tied to the |
//| filter count, giving a clean non-overlapping channel reduction. |
//| |
//| WHY THE POOL WINDOW MUST STAY TIED TO window_out. Shipping |
//| window=3/step=2 against 16 filters overlapped windows across the |
//| filter axis and straddled position boundaries, collapsing |
//| unrelated detectors into whichever fired hardest below every |
//| learnable layer (CONV sat at ~40% balanced accuracy for 510 eras, |
//| Sell recall 0%). The opposite error is just as bad: dropping the |
//| pool but leaving conv at window=step=one bar is a 1x1 conv that |
//| never mixes across time at all. |
//| |
//| We stop one layer short of the reference and do NOT append the |
//| second pool. A channel pool emits one scalar per position, so a |
//| trailing pool would hand the dense stack ~18 values for a 420-wide |
//| input and force it to FAN OUT 18 -> 64 instead of funnelling. The |
//| reference affords that at window_out=4 against a far smaller |
//| input; here it is a bottleneck below every learnable layer. The |
//| full 8-filter map goes to the dense/LSTM stage. |
//+------------------------------------------------------------------+
bool CExpertSignalAIBase::AddConvStage(CArrayObj *topology)
{
if(CheckPointer(topology) == POINTER_INVALID)
return false;
//--- Stage 1: convolution across CONV_RECEPTIVE_FIELD_BARS bars, advancing one bar at a time.
CLayerDescription *desc = new CLayerDescription();
if(CheckPointer(desc) == POINTER_INVALID)
return false;
//--- desc.count here is the conv layer's own output-filter count (CNeuronConvOCL::Init's window_out
//--- param, AI\Network.mqh) - was m_hiddenLayersCount (an unrelated dense-taper-depth setting,
//--- defaulting to 4), bottlenecking every sliding position to just 4 filters regardless of how wide
//--- the rest of the network was. See ConvFilterCount's declaration comment (Variables\Inputs.mqh).
desc.count = m_convFilterCount;
desc.type = defNeuronConv;
// PRELU, not TANH: matches what CNeuronConv's CPU path (Network.mqh) has always hardcoded
// regardless of this setting (its activationFunction() override ignores `activation` entirely) -
// this used to silently diverge from the GPU/DirectML tier, which DOES honor this field and was
// therefore actually running tanh instead of the intended PReLU whenever hardware accel was active.
desc.activation = PRELU;
desc.optimization = (ENUM_OPTIMIZATION)m_optimizationAlgo;
//--- The whole point: a window spanning several bars. Guarded because m_historyBars can be small
//--- enough that a multi-bar window would not fit at all, in which case this degrades to the old
//--- per-bar projection rather than building a negative-width layer.
desc.window = ConvReceptiveFieldBars() * m_neuronsCount;
desc.step = m_neuronsCount;
if(!topology.Add(desc))
{
delete desc;
return false;
}
//--- NO POOL, and no second conv. See CONV_RECEPTIVE_FIELD_BARS' comment for the measurement and the
//--- reference-kernel reading behind that: the conv emits position-major output and the reference pool is
//--- a flat contiguous max, so a pool here can only ever reduce ACROSS FILTERS within a position, never
//--- over time. It threw away 87.5% of this layer's output and starved every non-argmax filter of
//--- gradient. The second conv was mis-shaped in the same change - its window was counted in raw elements
//--- while its comment claimed positions, so a "2-position" window actually spanned 2 FILTERS of position
//--- 0 - and it only existed to consume the pool's output.
//--- If a deeper hierarchy is wanted later, the correct shape on THIS layout is a strided conv over
//--- positions: window = k * m_convFilterCount, step = s * m_convFilterCount (both whole numbers of
//--- positions, which IS contiguous in position-major order), never a pool. Springenberg et al. ICLR 2015.
return true;
}
//+------------------------------------------------------------------+
//| Conv chain shape. SINGLE SOURCE OF TRUTH - AddConvStage builds |
//| from these and LstmFanIn/FrontEndConfigSummary report from them, |
//| so what is constructed and what is logged cannot drift apart. |
//+------------------------------------------------------------------+
int CExpertSignalAIBase::ConvReceptiveFieldBars(void) const
{
//--- Degrade to a per-bar projection rather than build an impossible layer when history is too short
//--- for a multi-bar window. MathMin against m_historyBars keeps window <= input width.
int bars = (int)MathMin((int)CONV_RECEPTIVE_FIELD_BARS, (int)m_historyBars);
return (bars > 0 ? bars : 1);
}
//+------------------------------------------------------------------+
int CExpertSignalAIBase::ConvFirstStagePositions(void) const
{
//--- Sliding positions of stage 1: window ConvReceptiveFieldBars() bars, step 1 bar.
int p = (int)m_historyBars - (ConvReceptiveFieldBars() - 1);
return (p > 0 ? p : 1);
}
//+------------------------------------------------------------------+
bool CExpertSignalAIBase::HasSecondConvStage(void) const
{
//--- Permanently false: the conv chain is ONE true convolution. Kept (rather than deleted along with the
//--- pool + second conv it used to gate) so ConvOutputPositions/ConvOutputWidth stay the single source of
//--- truth for the chain's shape and a future strided second stage has one place to switch itself on.
return false;
}
//+------------------------------------------------------------------+
int CExpertSignalAIBase::ConvOutputPositions(void) const
{
int p = ConvFirstStagePositions();
return (HasSecondConvStage() ? p - (ConvReceptiveFieldBars() - 1) : p);
}
//+------------------------------------------------------------------+
int CExpertSignalAIBase::ConvOutputWidth(void) const
{
//--- Total element count reaching whatever is stacked above the conv chain: the conv output is
//--- position-major, window_out filters per position.
return ConvOutputPositions() * m_convFilterCount;
}
//+------------------------------------------------------------------+
//| LSTM sequence stage. Shared by CSignalLSTM and CSignalHYBRID - |
//| see the declaration comment. |
//+------------------------------------------------------------------+
bool CExpertSignalAIBase::AddLstmStage(CArrayObj *topology)
{
if(CheckPointer(topology) == POINTER_INVALID)
return false;
CLayerDescription *desc = new CLayerDescription();
if(CheckPointer(desc) == POINTER_INVALID)
return false;
desc.count = m_lstmHiddenSize;
desc.type = defNeuronLSTM;
desc.activation = TANH;
//--- CNeuronLSTMOCL now has an accelerated SGD+momentum kernel (LSTM_UpdateWeightsMomentum,
//--- AI\Network.mqh/Network.cl/DirectML\WarriorCPU.cpp/WarriorDML.cpp) alongside the original
//--- Adam one, so this layer honors the same TrainingOptimizer input as PAI/CONV - see
//--- m_optimizationAlgo's declaration comment.
desc.optimization = (ENUM_OPTIMIZATION)m_optimizationAlgo;
//--- PER-TIMESTEP input width - the feature count for ONE bar as it reaches this layer. CNet passes
//--- this to CNeuronLSTMOCL::SetStepWidth(), which is what makes the layer an actual recurrence over
//--- m_historyBars steps instead of a single gated projection over the whole flattened vector. It
//--- must divide LstmFanIn() exactly, which it does by construction in both placements: on the raw
//--- input the vector is historyBars x m_neuronsCount, and behind the conv chain it is
//--- ConvOutputPositions() x m_convFilterCount (position-major, filters contiguous per position -
//--- see the layout note above AddConvStage). Note the step COUNT is the position count, which the
//--- conv chain shrinks below historyBars once a multi-bar window and a second conv are in play.
//--- 0 disables sequence mode in CNeuronLSTMOCL::SetStepWidth (which maps <=0 to "not a sequence"),
//--- restoring the single-timestep layer. See LSTM_SEQUENCE_MODE.
desc.window = (LSTM_SEQUENCE_MODE ? (HasConvBeforeLstm() ? m_convFilterCount : m_neuronsCount) : 0);
//--- MathMax(1,...) guard taken from the HYBRID copy: the CSignalLSTM copy divided unguarded, so a
//--- historyBars of 1 produced step 0 there and step 1 here for what is meant to be the same layer.
desc.step = MathMax(1, (int)m_historyBars / 2);
if(!topology.Add(desc))
{
delete desc;
return false;
}
return true;
}
//+------------------------------------------------------------------+
//| Builds a fresh, untrained topology into Net - the exact layer |
//| construction InitNeuralNetwork() used to inline for the |
//| "no saved .nnw" case; factored out so TuneIndicatorsAndTrain() can|
//| get a clean-slate Net per trial without touching indicator init. |
//+------------------------------------------------------------------+
bool CExpertSignalAIBase::BuildFreshTopology()
{
CArrayObj *Topology = new CArrayObj();
if(CheckPointer(Topology) == POINTER_INVALID)
return false;
//--- Input Layer
CLayerDescription *desc = new CLayerDescription();
if(CheckPointer(desc) == POINTER_INVALID)
{
delete Topology;
return false;
}
feat: S2 meta-labeling head - binary trade-quality model over the classic-candidate corpus The NN now has a target that is not per-bar direction (closed, best-of-999 p=1.0000): P(win | this journaled candidate, at the EA's own SL/TP, net of cost). One net for all 52 pattern-sides, AIType=AI_META. - NetForward.mqh: the host-side softmax+CE gradient generalized total==3 -> 2||3 on both backprop paths; a 2-class softmax IS a logistic head, and no compute backend changes. - SignalMETA.mqh (new): corpus loaded read-only from the LARGEST signal DB on disk (decoupled from the config fingerprint that burned four S1 runs); the GMT->server offset is measured PER ROW against entryPrice vs bar open (DST-immune, histogram logged); a window-span regime filter drops the pre-2017 daily-backfill rows; 31-feature setup descriptor appended at the input (26 one-hot + side + tanh netVote + SL/TP ATR + spread/ATR). - Training.mqh: candidate-queued pass 1, binary-target pass 2, per-candidate calibration (2.5) and OOS (3) walks. Counter mapping win->Buy / loss->Sell lets checkpoint selection, the edge floor, the plateau ladder and the family-wise deploy gate run UNCHANGED: precision reads as win rate among traded candidates, chance as the base win rate, recalls as sensitivity/ specificity. Era-end META line: coverage x (p - break-even) vs the null. - Labels are the side-conditional triple-barrier win caches - never the DB's stop-and-reverse outcome. Logit adjustment deliberately skipped (~40% base rate). Live inference + online learning guarded off until S3. - Fingerprint: conditional |TGT:META1; State\META\ folder + 2-output filename slot keep meta models fully separate from direction models. Compiles clean (0 errors, 0 warnings). S2 run = attach a chart with AIType=AI_META; S3 wires the votes via the per-side hooks. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-13 06:52:31 -04:00
//--- NetInputWidth = the bar window plus (meta target only) the per-candidate setup descriptor
//--- appended after it - see AppendCandidateFeatures. Zero-delta for every direction model.
desc.count = NetInputWidth();
desc.type = defNeuron;
desc.activation = NONE;
desc.optimization = (ENUM_OPTIMIZATION)m_optimizationAlgo;
if(!Topology.Add(desc))
{
delete Topology;
return false;
}
//--- neuron-type-specific layers (Conv+Pool, LSTM, or none for a plain perceptron)
if(!AddCustomLayers(Topology))
{
delete Topology;
return false;
}
//--- Hidden Layers, tapering from m_initialNeuronsCount down to m_minNeuronsCount, each preceded by
//--- a batch-normalization layer (no-op when EnableBatchNorm is off). Placed BETWEEN layers rather
//--- than inside them because every layer here computes activation(W.x+b) in a single kernel - there
//--- is no seam between the matmul and the non-linearity to insert anything into. Normalizing the
//--- previous layer's OUTPUT is the equivalent formulation and is exactly what the NeuroNet_DNG
//--- reference's own worked example does (input -> BatchNorm -> hidden -> output).
//--- The first one also normalizes whatever the conv/pool/LSTM stage produced, which is the widest
//--- unbounded stage in the whole network and the one whose scale drift hurts most.
//--- GEOMETRIC taper from the derived first-layer width down to a final hidden width tied to the
//--- output count, spread evenly over however many layers the architecture asks for. This replaces a
//--- pair of inputs (NeuronsReduction, MinNeuronsCount) that were calibrated when the first layer was
//--- a hand-picked 500: they produced a genuine 500 -> 150 -> 45 funnel there, but against the derived
//--- width they degenerate. At 64 units, "keep 30% with a floor of 20" gives 64 -> 20 -> 20 - the
//--- reduction stops mattering after one step and the "minimum" silently becomes the width of every
//--- layer but the first. Deriving the ratio from the endpoints keeps the funnel shape correct at any
//--- width, which is the whole point of having derived the width in the first place.
int lastHidden = MathMax(HIDDEN_TAPER_OUTPUT_MULTIPLE * m_outputNeuronsCount, HIDDEN_TAPER_MIN_WIDTH);
//--- Never wider than where the taper starts: a narrow first layer (see the D1 case in
//--- ComputeFirstLayerWidth) must still funnel DOWN, not fan back out.
lastHidden = MathMin(lastHidden, m_initialNeuronsCount);
double taperRatio = (m_hiddenLayersCount > 1)
? MathPow((double)lastHidden / (double)m_initialNeuronsCount, 1.0 / (double)(m_hiddenLayersCount - 1))
: 1.0;
//--- Width of the layer immediately below the next batch-norm layer. Only advisory (CNet sizes each
//--- batch-norm layer from whatever it actually sits on), but kept honest so the descriptor list
//--- reads correctly. Seeded with the input width - which is also a lie for CONV/LSTM/HYBRID, where
//--- the custom stage in between has resized things; that is exactly why CNet does not trust it.
int prevWidth = (int)(m_historyBars * m_neuronsCount);
bool result = true;
for(int i = 0; (i < m_hiddenLayersCount && result); i++)
{
int n = (i == 0)
? m_initialNeuronsCount
: MathMax(lastHidden, (int)MathRound(m_initialNeuronsCount * MathPow(taperRatio, (double)i)));
result = (AddBatchNormStage(Topology, prevWidth) && result);
if(!result)
break;
prevWidth = n;
desc = new CLayerDescription();
if(CheckPointer(desc) == POINTER_INVALID)
{
delete Topology;
return false;
}
desc.count = n;
desc.type = defNeuron;
desc.activation = HiddenLayerActivation();
desc.optimization = (ENUM_OPTIMIZATION)m_optimizationAlgo;
result = (Topology.Add(desc) && result);
}
if(!result)
{
delete Topology;
return false;
}
//--- Batch norm immediately before the head. This is the one placement that matters most: it is what
//--- keeps the logit spread from decaying as the weights below it shrink, and it is the precondition
//--- for ever running an UNBOUNDED head here (see the 2026-07-28 note on desc.activation below).
if(!AddBatchNormStage(Topology, prevWidth))
{
delete Topology;
return false;
}
//--- Output Layer
desc = new CLayerDescription();
if(CheckPointer(desc) == POINTER_INVALID)
{
delete Topology;
return false;
}
desc.count = m_outputNeuronsCount;
desc.type = defNeuron;
// Never write the activation as a literal here: this line only ever reaches a BRAND-NEW topology, so
// a change made here never touches an existing .nnw (CNeuronBaseOCL::Save persists the activation and
// Load restores it). OutputLayerActivation() is the single source of truth and EnforceTopologyContract()
// re-asserts it after every Load.
// Regression (1 output): TANH - its [-1,1] range maps straight onto the -1/0/1 Sell/Neutral/Buy
// convention, with no SIGMOID offset or clipped ReLU half.
// Classification (3 outputs): SIGMOID, deliberately NOT NONE.
// The forward head must stay BOUNDED. HiddenLayerActivation() is PRELU, so this is the only bounded
// stage in the forward path, and two downstream constants are calibrated against that: CLASS_LOGIT_SCALE
// = 6.0 is a temperature gain sized to stretch [0,1] into a usable logit span (against a free logit it
// is just a 6x amplifier), and the +-3.0 cold-start bias seed reads "sigmoid(+-3) ~= 0.95/0.05". An
// unbounded head was tried 2026-07-27 and reverted the next day: it starts at exp(6*3) vs exp(6*-3),
// saturated softmax makes the gradient input-INDEPENDENT, and all four architectures degenerated to
// one- or two-class output within two eras. Do NOT unbound the head again without simultaneously
// setting CLASS_LOGIT_SCALE to 1.0 and the bias magnitude to ~0.5.
// The BACKWARD pass is not 3 independent sigmoid deltas: CNet::backProp/backPropOCL detect the
// 3-output case and compute a joint softmax + categorical-cross-entropy gradient (softmax_i - target_i),
// which is what ties the classes together - raising one probability structurally lowers the other two.
// ApplyClassificationSoftmax() reproduces exactly that normalization at read time.
desc.activation = OutputLayerActivation();
desc.optimization = (ENUM_OPTIMIZATION)m_optimizationAlgo;
if(!Topology.Add(desc))
{
delete Topology;
return false;
}
if(CheckPointer(Net) != POINTER_INVALID)
delete Net;
Net = new CNet(Topology);
delete Topology;
if(CheckPointer(Net) == POINTER_INVALID)
return false;
// A fresh topology invalidates any existing shadow (see m_shadowNet's declaration comment) -
// its weights, if any, are shaped for the OLD Net and would either mismatch dimensionally or,
// worse, silently blend unrelated weight spaces if the shape happens to coincide. Reset to NULL
// here; EnsureShadowNet() lazily re-bootstraps a fresh clone of the new Net on first use.
if(CheckPointer(m_shadowNet) != POINTER_INVALID)
{
delete m_shadowNet;
m_shadowNet = NULL;
}
//--- Let EnsureShadowNet() re-attempt the clone bootstrap once for this new topology (see the latch's
//--- declaration comment) - the old shadow, and any prior failed-bootstrap verdict, no longer apply.
m_shadowBootstrapAttempted = false;
//--- A brand-new untrained net has NO online continual-learning history (see OnlineLearnStep): reset
//--- the watermark/guardrail/counters so a fresh start or a ResetWeights()-then-retrain never resumes
//--- from a superseded model's learned-up-to point or its stale rolling accuracy. Restored (not reset)
//--- on a normal reload of an existing model - that path loads them via LoadModelStats() and never
//--- calls BuildFreshTopology(). Harmless during the post-tune rebuild (online learning is
//--- gated off there, and the deployed final retrain rebuilds and resets again before deployment).
m_onlineLearnedUpToTime = 0;
m_onlineRollingAcc = -1.0;
m_onlineSamples = 0;
m_onlineBarsSincePersist = 0;
m_onlineBlendFrozen = false;
return true;
}
//+------------------------------------------------------------------+
//| |
//+------------------------------------------------------------------+
bool CExpertSignalAIBase::InitIndicators(CIndicators *indicators)
{
//--- Reset only the status label on (re-)init; deliberately do NOT PurgeChart() here so previously drawn
//--- signal arrows survive an EA re-init (recompile / param change / timeframe switch) instead of
//--- vanishing every time - see SIG_ARROW_PREFIX. Full cleanup still happens in the destructor.
ClearStatusLabel();
//--- NOTE: LoadChartSignals() is deliberately NOT called here any more. This method runs from
//--- InitNeuralNetwork() BEFORE the per-config fingerprint is appended to m_fileName, so at this point
//--- m_fileName is only "<folder>\<symbol>_<period>" - the load looked for e.g. "SP500_16385.arrows"
//--- while SaveChartSignals() (which only ever runs post-init, with the finished name) had written
//--- "SP500_16385_3.00000000_1.00000000_<hash>.arrows". The mismatch made the restore silently no-op on
//--- every restart from the moment the fingerprint was introduced. It is now called at the END of
//--- InitNeuralNetwork(), once m_fileName is final. Same family as the fingerprint trap documented at
//--- BuildConfigFingerprint: anything keyed on m_fileName must run AFTER it is fully built.
if(!InitOpen(indicators))
return false;
if(!InitClose(indicators))
return false;
if(!InitLow(indicators))
return false;
if(!InitHigh(indicators))
return false;
//--- label source, always created unconditionally, same as the OHLC indicators above - see
//--- m_ADZigZag's declaration comment. Optionally ALSO read as an input feature (m_useSwingContext,
//--- below) using the same already-running indicator instance - no separate init needed for that.
if(!InitADZigZag(indicators))
return false;
m_neuronsCount = 4; // (close-open)/atr, (high-open)/atr, (low-open)/atr, bullish/bearish flag
if(m_useVolumes)
{
feat(ai): widen the volume feature block from 1 value to 4 The block fed exactly one number: (v[i] - v[i-1]) / v[i-1]. That is the first difference, and it cannot express three things that matter - the LEVEL relative to a baseline (two dead bars and two frantic bars both read ~0 change), and the two volume-vs-range interactions, where heavy participation that went NOWHERE (absorption) and heavy participation that travelled (continuation) mean opposite things and currently collapse onto the same value. research/test_volume.py measures each candidate's mutual information with the triple- barrier label across 3 instruments x 2 geometries, against a BLOCK-permutation null - blocks sized to the barrier horizon, because adjacent labels share almost their entire outcome window and a free shuffle yields a null so tight that everything looks significant. Finite-sample MI bias (~7/n here) is reported alongside rather than subtracted, since the permutation null already absorbs it. Result: volLevel beats the shipped change ratio outright on 4 of 6 cells (EURUSD 2:3 +0.000118 excess at p=0.006, USDJPY 1:2 +0.000284 at p=0.002); absorption is the single strongest reading anywhere in the sweep at EURUSD 1:2 (+0.000404, p=0.002) though it is null on XAUUSD; vol x range clears on 4 of 6. The shipped change ratio is itself significant on 5 of 6, so it stays. Kept OUT: a session-relative z-score against the same hour-of-day's own recent history. It was the weakest candidate - null on both EURUSD cells - and it is the only one needing per-hour rolling bookkeeping in MQL5. Not worth the state for a reading that did not survive its own null on the primary instrument. Magnitudes, stated plainly because they are the point: the excess MI is ~2e-4 nats against a label entropy near 1.05. That is under a tenth of one percent of the label's uncertainty. It is real, it repeats across instruments, and it is nowhere near an edge - this is worth having because it costs one 50-bar loop, not because it changes the answer. Prior work stands: the whole single-series feature family measured at the noise floor. m_neuronsCount is already in the fingerprint, so the width change re-keys existing caches by itself, which is correct - the input vector genuinely changed shape. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-01 17:34:33 -04:00
// change ratio, level vs 50-bar baseline, absorption (range per unit volume), volume x range -
// see BufferTempDataCompute()'s matching block, and research/test_volume.py for the measurement
// that justified widening this from 1. m_neuronsCount is already in the config fingerprint, so
// this re-keys existing caches on its own: correct, the input vector genuinely changed shape.
m_neuronsCount += 4;
if(!InitVolumes(indicators))
return false;
}
// Unconditional, same reasoning as m_ATR/m_ADZigZag below: m_Time.GetData() is read
// unconditionally elsewhere (label-eligibility gate, cache anchor, online-learning watermark,
// arrow timestamps) regardless of whether the cyclical time-of-day/day-of-week values are also
// opted into as an explicit feature via m_useTime - so the indicator itself must always exist.
if(!InitTime(indicators))
return false;
if(m_useTime)
{
m_neuronsCount += 6;
}
if(m_useATR)
{
//already init in the base class
m_neuronsCount++;
}
if(m_useMA)
{
if(!InitMA(indicators))
return false;
m_neuronsCount += 5; // (open-MA)/atr, (high-MA)/atr, (low-MA)/atr, (close-MA)/atr, (MA-MA[1])/atr
}
if(m_useRSI)
{
if(!InitRSI(indicators))
return false;
m_neuronsCount++; // RSI/100
}
if(m_useMACD)
{
if(!InitMACDFeature(indicators))
return false;
m_neuronsCount += 3; // main/atr, signal/atr, histogram/atr
}
if(m_useIchimoku)
{
if(!InitIchimoku(indicators))
return false;
// (close-Tenkan)/atr, (close-Kijun)/atr, (Tenkan-Kijun)/atr, (close-SpanA)/atr, (close-SpanB)/atr,
// signed cloud thickness at this bar, signed PROJECTED cloud thickness, Chikou displacement
m_neuronsCount += 8;
}
if(m_useSwingContext)
m_neuronsCount += 9; // 5 confirmed-pivot features (direction, distance-since-pivot, prior-leg magnitude, retracement ratio, bars-since-pivot) + 4 recent-context features (Donchian pos 20/50, 20-bar return, 20-bar SMA extension) - see BufferTempDataCompute()'s matching block
if(m_useNews)
m_neuronsCount += 2; // NewsRecency, NewsProximity - see BufferTempDataCompute()'s matching block
feat(ai): spread as a volatility-regime feature, and fix a stale-index cache in both new blocks Adds spread/ATR and the spread change ratio as network inputs (EnableSpreadFeature, default on). Spread is the one microstructure channel that is both FX-available and genuinely historical in the Strategy Tester - "during testing, the spread is not modeled but is taken from historical data" - so unlike swap, signed tick flow or depth of market it is something a backtest can honestly validate. What it encodes, stated precisely because the raw measurement overstates it. research/test_spread.py found spr/atr the strongest single feature in this codebase, on 5 of 8 instrument/geometry cells at 2-4x any volume feature. But the barrier LABEL charges the spread inside its own barriers, so a wide-spread bar is mechanically likelier to resolve as a loss and the feature would partly be predicting its own cost model. Relabelling at zero cost and re-measuring the identical feature showed 20-40% of it WAS that tautology and the majority was not (XAUUSD retained 97%). What survives is a volatility-regime reading: spread is near-fixed while ATR is not, so the ratio runs high exactly when realised volatility is below its own ATR estimate, which genuinely predicts whether ATR-scaled barriers get reached. It is UNSIGNED - Neutral-vs-directional only, never a side. Also fixes a stale-index bug I introduced with the cross-asset panel and had just repeated in the spread series. Both cached on length alone: if(m_crossAsset.Bars() >= bars) return true; MQL5 series indices are relative to NOW, so one new closed candle shifts every index by one. Keyed only on length, the panel keeps serving its index 0 as a bar that is no longer the newest, and every cross-asset value is read one bar out of step with the price features sitting beside it in the same vector - silently, with no error and no shape change. This is the same class of defect as the dtStudied watermark behind the zero-direction backtests. Both now carry a datetime anchor on m_Time.GetData(0), the same invalidation key the label/feature bar caches already use. And a performance fix that fell out of it: with correct invalidation the panel rebuilds on every new bar, and RefreshConvergedSignal runs per bar - which in the tester would mean one full multi-symbol resample per simulated bar at training depth. Inference only reads bars 0..m_historyBars-1 plus the panel's own slow window, so it now requests exactly that. The cache check is >=, so a deeper panel left from training still satisfies it. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-01 17:42:40 -04:00
if(m_useSpreadFeature)
m_neuronsCount += 2; // spread/ATR (volatility-regime reading), spread change ratio
// - see BufferTempDataCompute()'s matching block
fix(signals): revive a dead MA model, and demote Sanyaku from state to event Two defects surfaced by research/test_classic.py, both verified fixed by re-running the transcription against 178k bars of EURUSD H1. CSignalMA model 1 could never fire. For any recursive average - and MA_TYPE_EMA is the shipped default - MA(i) = a*Close(i) + (1-a)*MA(i+1), so DiffMA(i) = a * (Close(i) - MA(i+1)) DiffCloseMA(i) = (1-a) * (Close(i) - MA(i+1)) are positive multiples of one quantity and always share a sign. Model 1 asks for a close BELOW a RISING average, which is precisely the combination that identity forbids: 0.000% of bars, either direction, any symbol. The MQL5 standard library this was ported from defaults to MODE_SMA, where the two are merely correlated - the bug arrived with the EMA default, not with the port. Reading the slope one bar back (DiffMAPrev) breaks the tie for every MA type while keeping the model's stated meaning. Now fires on 7.92% of bars. CSignalIchimoku model 11 fired on 27% of bars at weight 100. Sanyaku is three standing STATES conjoined with no transition term, so it held across long stretches - and being last in the if-chain at the top weight, the module's highest-conviction reading was also its most common one, overwriting all eight event models below it on a quarter of all bars. The old comment rejected an event form because "demanding all three flip on the same bar would fire almost never" - true, but that is not the alternative. Kouten is the TURN: the ALIGNMENT transitions, and only one role need change for it to. Testing !Sanyaku(idx+1) fires once per aligned stretch. Now 2.17%, in line with Kumo breakout (2.4%) and the strong TK cross (1.1%). DataReady() extended one bar deeper to cover the lookback. Neither pattern showed edge before or after; this is about the models meaning what they say and the vote not being dominated by a constant. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-01 17:14:34 -04:00
if(m_useCrossAsset)
m_neuronsCount += CROSSASSET_FEATURES; // FX: base/quote strength + divergence; index: denom/risk-proxy strength
fix(signals): revive a dead MA model, and demote Sanyaku from state to event Two defects surfaced by research/test_classic.py, both verified fixed by re-running the transcription against 178k bars of EURUSD H1. CSignalMA model 1 could never fire. For any recursive average - and MA_TYPE_EMA is the shipped default - MA(i) = a*Close(i) + (1-a)*MA(i+1), so DiffMA(i) = a * (Close(i) - MA(i+1)) DiffCloseMA(i) = (1-a) * (Close(i) - MA(i+1)) are positive multiples of one quantity and always share a sign. Model 1 asks for a close BELOW a RISING average, which is precisely the combination that identity forbids: 0.000% of bars, either direction, any symbol. The MQL5 standard library this was ported from defaults to MODE_SMA, where the two are merely correlated - the bug arrived with the EMA default, not with the port. Reading the slope one bar back (DiffMAPrev) breaks the tie for every MA type while keeping the model's stated meaning. Now fires on 7.92% of bars. CSignalIchimoku model 11 fired on 27% of bars at weight 100. Sanyaku is three standing STATES conjoined with no transition term, so it held across long stretches - and being last in the if-chain at the top weight, the module's highest-conviction reading was also its most common one, overwriting all eight event models below it on a quarter of all bars. The old comment rejected an event form because "demanding all three flip on the same bar would fire almost never" - true, but that is not the alternative. Kouten is the TURN: the ALIGNMENT transitions, and only one role need change for it to. Testing !Sanyaku(idx+1) fires once per aligned stretch. Now 2.17%, in line with Kumo breakout (2.4%) and the strong TK cross (1.1%). DataReady() extended one bar deeper to cover the lookback. Neither pattern showed edge before or after; this is about the models meaning what they say and the vote not being dominated by a constant. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-01 17:14:34 -04:00
// divergence, cross-sectional dispersion - System\CrossAsset.mqh
if(m_useADCumulativeDelta)
{
if(!InitADCumulativeDelta(indicators))
return false;
m_neuronsCount += 6; // Pressure, CumulativeDelta, BullishPressure, BearishPressure, Absorption, Initiative
}
if(m_useADShorteningOfThrust)
{
if(!InitADShorteningOfThrust(indicators))
return false;
m_neuronsCount += 4; // SOT, SOTEffortRegime, SOTConfirmation, SOTPushRegime
}
if(m_useADWyckoffEventStream)
{
if(!InitADWyckoffEventStream(indicators))
return false;
feat: mini-batch gradient accumulation (F4), front-end-aware capacity budget (F6), split Wyckoff categoricals (N1) Completes the 2026-08-09 training audit. FORCES A RETRAIN of every Wyckoff-enabled config (N1 re-keys the fingerprint), and BOTH DLLs must be redeployed alongside the .ex5 - they carry new exports. F4 - mini-batch accumulation, TRAIN_BATCH_SIZE=32. Training was pure online SGD (one weight update per bar), which is the mechanical source of the era-to-era whipsaw every downstream guard was built to cope with. The O(n^2) outer product is native - AccumulateWeightGrad / AccumulateWeightGradConv / AccumulateBufferInto in Network.cl, WarriorCPU and WarriorDML - while the optimizer step is host-side MQL5 shared by all tiers (ApplyAccumToBlock), so there is one Adam/SGD implementation instead of four that can drift. - the LSTM needs no outer-product kernel (WeightsGradient already holds the sample's full dW) but could NOT simply be left un-zeroed between samples: CPU_LSTMSeqBackward/DML_LSTMSeqBackward memset it on entry. Hence a separate accumulator plus an elementwise add. - batch-norm gamma/beta accumulate in host arrays, not new BatchOptions slots - BN_OPT_STRIDE is baked into every persisted .nnw. - scoped to pass 2; online learning keeps immediate updates. Every save / checkpoint / scoring boundary flushes, scaling by the real sample count. - degrades to per-sample updates (one log line) on a tier that cannot accumulate, so old devices and DLL-free builds are unaffected. - verified offline: DirectML/batch_accum_check.cpp drives the real exports against an independent reference; at B=1 the accumulator matches the shipped unbatched kernel's own gradient to 1.1e-16. Math only - the in-situ check remains the per-layer dW/W report on a real era. F6 - ComputeFirstLayerWidth budgeted against the RAW input width even where a conv/LSTM front end had already reduced it, so an LSTM's dense stack was charged for 1,280 inputs when it receives 64. Confirmed from the deployed .cfg files: CONV, LSTM and HYBRID were all pinned at the 16-unit floor. Now budgeted against the front-end output and capped at it (never fan out), with the derivation reordered so both stages settle first. N1 - EventCode/EventPhase/StructuralPhase are signed categoricals packing direction and Wyckoff stage into one scalar across a sign discontinuity. Split into direction + [0,1] magnitude, the same convention the base OHLC block uses. Information-preserving; 13 readings now occupy 16 inputs. Compiled clean (0 errors, 0 warnings); both DLLs rebuilt. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-09 11:48:03 -04:00
// 16 = 13 buffers - EventPrice (buffer 4, excluded; see BufferTempDataCompute()'s comment) with
// THREE of them split into a direction/magnitude pair each: EventCode, EventPhase and
// StructuralPhase are signed categoricals, so 13 readings now occupy 16 inputs (2026-08-09 audit,
// N1 - see the split in BufferTempDataCompute for why). No reading was added or dropped.
m_neuronsCount += 16; // eventDir, eventStage, livePhaseDir, livePhaseMag, ZoneTop, ZoneBottom, structDir, structMag, CHoCHTrendToRange, CHoCHRangeToTrend, SlopeAccumulationBullish, SlopeAccumulationBearish, SlopeDistributionBullish, SlopeDistributionBearish, Reaccumulation, Redistribution
}
if(m_useADWyckoffFailedStructure)
{
if(!InitADWyckoffFailedStructure(indicators))
return false;
m_neuronsCount += 5; // Value, BullishStructuralFailure, BearishStructuralFailure, FailedAccumulation, FailedDistribution
}
if(m_useADWyckoffSignificantBarInversion)
{
if(!InitADWyckoffSignificantBarInversion(indicators))
return false;
m_neuronsCount += 5; // SignificantBarQuality, BullishSignificantBar, BearishSignificantBar, BullishControlFlip, BearishControlFlip
}
//--- ALT DATA (2026-08-16). Externally collected, publication-stamped features (COT positioning,
//--- VIX complex, macro) exported by research/altdata/export.py into
//--- Common\Files\Warrior_EA\AltData\{SYMBOL}_{TF}.csv - see System\AltData.mqh for the
//--- lookahead/degradation contracts. The set is per-symbol: only features that survived BOTH the
//--- family-wise MI bar AND the incremental conditional-on-trailing-range test are exported, so
//--- the width varies by symbol (SP500 4, USDJPY 3, XAUUSD 1, EURUSD none as of the first export)
//--- and m_neuronsCount already re-keys the weight fingerprint on any width change.
//---
//--- ORDER MATTERS HERE: the .cfg's pinned NAME LIST is applied BEFORE the width is summed, so a
//--- resumed model keeps ITS width and ITS column meanings even if the export has since gained or
//--- reordered columns - the exact failure the cross-asset pair pin exists to prevent, arriving
//--- through a rewritten CSV instead of a changed Market Watch. Without the pre-read, a grown
//--- export would change m_neuronsCount, mismatch the .cfg compare, and silently discard a
//--- perfectly good model as a "config change".
if(m_altDataEnabled)
{
string altPin = ReadAltDataPinFromCfg();
if(altPin != "")
m_altData.SetPinnedNames(altPin);
m_altData.Load(m_symbol.Name(), (ENUM_TIMEFRAMES)m_period); // logs its own outcome; absence is normal
m_altDataNamesPinned = (altPin != "") ? altPin : m_altData.NamesCsv(); // fresh model: stamped by the first .cfg save
m_useAltData = (m_altData.FeatureCount() > 0);
if(m_useAltData)
m_neuronsCount += m_altData.FeatureCount();
}
else
{
//--- Operator opt-out (EnableAltData=false): zero width, nothing pinned. On a model trained
//--- WITH alt features this shrinks neuronsCount, mismatches the .cfg compare and correctly
//--- starts fresh - stated in the input's comment rather than silently absorbed.
m_useAltData = false;
m_altDataNamesPinned = "";
}
if(!FolderCreate(m_folderPath, FILE_COMMON))
{
if(GetLastError() != 5010) // If the error is not because the folder already exists
{
Print("Failed to create folder: " + m_folderPath);
}
else
{
ResetLastError(); // Reset the error code
}
}
return true;
}
#endif