- Implemented sqx_audit.py to audit StrategyQuant X trade lists, focusing on performance metrics and cost analysis.
- Created sqx_portfolio.py to evaluate portfolio performance based on uncorrelated components and their impact on risk and return.
- Developed swing.py to analyze cost ratios across different holding periods and assess swing trading structures.
- Introduced test_management.py to investigate the effectiveness of exit rules on random entries and their impact on expectancy.
The MI sample builder used `MathAbs(labelBarOffset)` as a padding, causing rows from offset and non-offset builds to be paired with a double shift. This broke the positive control, failed the 5× gate, and voided all reported mutual‑information figures. Replace with the fixed `MiShiftPad` constant to ensure builds enumerate the same set of bars and row-k alignment is preserved.
Add `BatchOptionsTotal()` to `CNeuronBatchNormOCL` and split the packed BN weight array in the learning report into separate norms for the outgoing dense matrix, gamma, beta, running statistics, and Adam moment buffers. This turns an ambiguous single‑norm reading into precise diagnostics that distinguish weight divergence from scaling issues.
- BufferDouble: replace hardcoded "DirectML/CPU-DLL" with dynamic backend name
and add buffer index/element count to all error prints for easier debugging.
- NetPersistence: distinguish missing file from transient lock by probing
FileIsExist before logging, eliminating false "sharing violation" warnings
when no saved model exists on first run.
- Moves CLayer neuron construction to AI/Impl/Layer.mqh to keep Network.mqh clean
- Unifies four previously duplicated architecture initialisation blocks (MLP/CONV/LSTM/HYBRID) into a single shared function
- Eliminates risk of behavioural drift where one architecture missed a setter, causing mismatched feature sets or targets
Confirmed live the first run after b4d640d made the warning specific:
"1 weight block(s) could not be blended into the EMA shadow (layer 2,
neuron type 30852)" - 30852 is defNeuronLSTMOCL - on BOTH the LSTM and
HYBRID charts. Exactly what the .nnw sizes predicted (HYBRID's shadow was
791,120 bytes short of its live net, 4 x 24,704 doubles = the LSTM block
plus its Adam moments).
Mechanism: EnsureShadowNet() clones via Net.Save() -> clone.Load(), and it
is reachable from RefreshLatestSignal(), which runs before the live net's
first forward pass. CNeuronLSTMOCL::Save writes m_iInputs = -1 and omits
every LSTM buffer in that state, so the clone came back with WeightsLSTM
== NULL. Only the LIVE net ever runs forward, so the lazy SetInputs() that
would have allocated it never fired on the shadow - permanently. The blend
skipped the layer every era and returned true.
This is a live-inference and deployment defect, not a training one: the
shadow is the net RefreshLatestSignal and the deploy path read.
Fixed by self-healing in the blend rather than by reordering the bootstrap,
so an already-bad shadow on disk repairs itself too. New
CNeuronLSTMOCL::AdoptShapeFrom copies m_iStepInputs first (SetInputs reads
it to choose between the sequence and single-timestep block shapes) then
sizes the block; the blend then copies the live weights outright rather
than blending tau of them into fresh random ones - an EMA seeds at its
first observation.
Both builds 0/0. Needs redeploy; no retrain (training reads the live net).
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
CONV and LSTM were each configured as a strictly lossier perceptron, which
is exactly what the panel showed: PAI 24% > CONV 18% > HYBRID 12% ~ LSTM
12%, monotone in how much reaches the dense stack (420 / 160 / 32 / 16).
CONV - receptive field 1 -> 3 bars, and the pool is gone.
Reading the reference kernels settled why 34d6aa4 killed CONV.
FeedForwardConv emits POSITION-MAJOR output (matrix_o[out + window_out*i]),
and FeedForwardProof is a flat contiguous max over `window` at stride
`step`. On that layout any window <= window_out maxes ACROSS FILTERS
within one position - it cannot pool over time at all. Our stage used
window = step = filterCount: one max over all 8 filters per position,
discarding 87.5% of the conv output and leaving only the argmax filter
with gradient. That is a property of the reference's layout, not a
porting bug, so there is no correct pool to swap in. Springenberg et al.
ICLR 2015 is the answer already cited in this file: no pooling, get the
hierarchy from strided convolution. The second conv went with it - its
window was counted in raw elements while its comment claimed positions,
so a "2-position" window actually spanned 2 filters of position 0.
Filter count now derives from the WINDOW (RF * features / 2) instead of
one bar, which at RF 3 was under-sizing the stage 3x.
Shape: 20 bars x 21 -> 18 positions x 16 filters = 288.
LSTM - sequence mode back on, forget bias 2.0 -> 1.0.
The forward path rules out the "no gradient" reading of the 2026-07-30
failure: CPU_LSTMSeqForward starts every sample at h_0 = c_0 = 0 and
unrolls that sample's own window, so nothing leaks between shuffled
samples. Flat IS error + Neutral:100% is equally the signature of an
output that does not vary with the input, and that is what bias 2.0
produces: c* = i*g/(1-sigmoid(b)) ~ 8.3*i*g, |c*| ~ 4.2, tanh pinned at
0.9995 with derivative 1e-3, so h_T is near-binary and set by the gate
biases rather than the bars. Choosing 2.0 off the reach sweep was a
method error - reach trades against saturation and the sweep never
measured saturation. 1.0 is the Gers/Jozefowicz/Keras default and leaves
tanh derivative ~0.1.
Both builds 0/0. DLL unchanged (CPU_LSTMSeqForward/Backward already
exported). Forces a retrain of CONV, LSTM and HYBRID - the .nnw pins
architecture.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
32880f3 warned whenever either side of the EMA blend returned no weight
block. Both-empty is normal: a dense layer whose successor owns the weight
matrix has none by design (CNeuronBaseOCL::Init only allocates Weights when
numOutputs > 0), which is the same reason LayerLearningReport prints
NOWEIGHTS for every dense layer in these topologies.
Result on the first run with it: "5 weight block(s) could not be blended"
on CONV and 6 on LSTM/HYBRID, type 30851 = defNeuronBaseOCL - all of them
the weightless dense layers. Pure noise, and it pointed at the wrong thing.
Only an ASYMMETRY is a defect: live has a block, shadow does not. That is
the HYBRID case the file sizes showed (shadow 791,120 bytes short, exactly
the LSTM weight block plus Adam moments), and it still trips the warning.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Every branch of the EMA shadow blend is `if(both sides return weights)
{ blend }` with no else. That silent degrade is deliberate - a partial
topology mismatch should not corrupt unrelated layers - but it also hid a
real defect for 343 eras on SP500 H1.
The HYBRID shadow's LSTM layer had never run a forward pass, so its
WeightsLSTM was still NULL and getWeightsLSTM() returned 0. The blend
skipped a 24,704-weight layer on every single era and returned true. The
only trace was on disk: the shadow .nnw is 791,120 bytes smaller than its
live net - 4 x 24,704 doubles, exactly the LSTM weight block plus its Adam
moments - while the CONV, LSTM and PAI shadows byte-match their live nets.
This matters beyond training: the shadow is the net live inference and
deployment read, so those layers are not tracking the trained model at all.
Counts skipped blocks and warns once per net, naming the layer index and
neuron type. Reporting only - the blend behaviour is unchanged, and the
underlying cause still needs a fix.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The per-era `dW/W` line measured the change in each layer's weight NORM.
That statistic cannot separate "this layer only shrank under weight decay"
from "this layer moved somewhere useful" - a rotation at constant norm and
pure decay can print the same number.
It matters right now: on SP500 H1 the LSTM layers print a near-constant
~1.05%/era that exactly equals their geometric norm decay over 318 eras
(HYB lstm2 12.966 -> 0.755, monotone, never once up), while a sibling conv
oscillates around a much slower drift. Norm-change can only hint at that.
Now prints norm(d|W|% / |dW|%). Under decay alone the two are equal; any
gradient component adds in quadrature to the second, so a learning layer
shows the second clearly larger. Diagnostic only - no training behaviour
changes, fingerprint untouched.
Also removes two log lines that described machinery deleted in 397b0ea:
the plateau ladder's terminal stage still claimed "after a warm restart
AND full gamma anneal", and the CONVERGED line "across a warm restart and
a full focal-gamma anneal". Both printed on every CONV/LSTM convergence
today. Same failure mode as the oversampling line 397b0ea fixed: a log
that describes what an older version would have done is confirming
evidence for a false hypothesis.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Adds "dW/W dense1:0.412(0.31%) conv1:0.088(0.000%) ..." to the era line:
each layer's weight L2 norm and its relative change since the previous era.
Why: a frozen stage and a badly-suited architecture look identical from the
outside. Both give a flat metric and a retreat to the majority class, and
neither the loss, the accuracy nor the per-class recall can tell them apart.
This session cost two full retrain cycles guessing between them - a forget-
gate bias (a real bug, measured, but not the cause of the observed failure)
and a conv receptive field (which turned out to be a regression, not a fix).
A layer sitting at ~0.000% era after era while its neighbours move is
receiving no gradient, and no amount of retraining or hyperparameter work
will change that. A net where every layer moves and the output still
collapses is a genuine architecture or objective problem. The distinction is
one glance at the log instead of a redeploy-and-wait cycle per hypothesis.
Costs one host-side buffer read per layer per era, off the training path.
Both builds compile 0 errors, 0 warnings.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
CONV's convolution used window = step = one bar, which is a per-bar
projection - a 1x1 conv with a temporal receptive field of ONE BAR. It never
mixed information across time, so "convolutional" described the layer type
and nothing about what it computed. Same finding that sank HYBRID's LSTM.
Pooling was removed on 2026-07-29 for being misconfigured against the conv
output's memory layout. That removal was right; leaving the conv at a
one-bar window was not. The two belong together: the NeuroNet_DNG reference
(references\MQL5\Experts\EDL\Trajectory.mqh layers 2-5, kernels
byte-identical to ours) pairs conv(window=2, step=1, window_out=4) with
pool(window=4, step=4), and the pool only earns its place because a conv
with a real receptive field sits above it.
The input is bar-major (BufferTempData appends m_neuronsCount contiguous
features per bar), so a flat window of k*m_neuronsCount spans exactly k
bars - the receptive field needed NO kernel change. The conv output is
position-major, so window == step == window_out is a clean
max-over-channels, which is what the reference does and what the existing
pool kernels already implement correctly.
New chain at H1 defaults (420 = 20 bars x 21):
conv1 w=42 s=21 out=8 -> 19 pos x 8 = 152
pool w=8 s=8 -> 19
conv2 w=2 s=1 out=8 -> 18 pos x 8 = 144 (effective field: 3 bars)
We deliberately stop before the reference's SECOND pool: a channel pool
emits one scalar per position, so a trailing pool would hand the dense stack
18 values and force it to fan out 18 -> 64. That is a bottleneck below every
learnable layer - the same class of mistake the 2026-07-29 removal was about.
Fixes a latent sizing bug this exposed: CNet's conv/pool position cursor
tracked sliding POSITIONS, but a conv's real width is units_count *
window_out. Any pool stacked on a conv would therefore have sized against a
width window_out times too small and silently built the wrong shape. Both
branches now read the built layer's actual Neurons(), which is what the
batch-norm branch already did for the same reason.
Also closes the architecture-pinning trap: a .nnw persists the window each
conv was built with, so an existing CONV/HYBRID model would have loaded
cleanly and gone on training under the OLD architecture. The conv weight
tensor is (window+1)*window_out, so this cannot be repaired in place -
EnforceTopologyContract now detects it, reports both shapes, and retrains.
Conv chain shape is derived in one place (ConvReceptiveFieldBars /
ConvFirstStagePositions / HasSecondConvStage / ConvOutputPositions /
ConvOutputWidth) and consumed by AddConvStage, LstmFanIn and the startup
config line, so what is built and what is logged cannot drift.
Both builds compile 0 errors, 0 warnings. Forces a CONV and HYBRID retrain.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The sequence rewrite made LSTM/HYBRID recurrences over 20 bars, but every
weight - including the gate biases - is initialized around zero. That puts
the forget gate at sigmoid(0) = 0.5, so the cell state is halved every step:
the first bar survives into the output scaled by ~0.5^20, and the gradient
reaches it scaled by the same factor.
The layer was therefore a one-bar model wearing a 20-bar interface. A one-bar
model has no signal on this task, so the head learned the base rate and
emitted Neutral everywhere - the flat 0.34 IS error across twelve eras and
OOS recall Neutral:100% seen on SP500 H1.
Measured at the shipped H1 shapes (H=64, stepInputs=21, T=20) - influence of
bar 0 on the output relative to bar 19:
bias 0.0 -> 3.0e-05 forward, 3.3e-05 backward (dead)
bias 1.0 -> 1.2e-02 forward, 1.4e-02 backward
bias 2.0 -> 2.5e-01 forward, 2.7e-01 backward (a real 20-bar field)
This is the standard fix, not a tuned knob: Gers/Schmidhuber/Cummins (2000)
introduced the forget gate with a positive bias, and Jozefowicz/Zaremba/
Sutskever (ICML 2015) recommend a bias of 1 as a default (whence Keras'
unit_forget_bias). Both 1 and 2 are standard; the sweep picks 2 because at a
20-step window a bias of 1 still leaves the oldest bar at ~1% influence.
lstm_seq_flowcheck.cpp is added as a permanent regression check and asserts
the shipped constant keeps >=5% reach in both directions. It complements
lstm_seq_gradcheck.cpp: that one proves the BPTT is CORRECT, this one proves
it is USABLE. The gradient check passed at 2.3e-10 throughout - correct math
over a recurrence that carries nothing looks exactly like a bad architecture.
Both builds compile 0 errors, 0 warnings. Initialization only, so the .nnw
format is unchanged; LSTM and HYBRID must retrain to pick it up.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Closes the gap left by 7a08197, which refused sequence mode under OpenCL. That
was defensible for a private build and not for a shipped one: the release path
includes an OpenCL laptop, and a customer with a GPU would have found LSTM and
HYBRID simply unavailable.
One launch PER TIMESTEP rather than a single kernel looping with barrier().
Every hidden unit's gates read all of h_{t-1}, OpenCL barriers only span a
work-group, and nothing here constrains how the runtime partitions the global
size - so an in-kernel loop would be correct only by luck of the partitioning.
Host-driven launches make each step an implicit global barrier: more enqueues,
correct on every device.
Backward reuses the buffers the single-timestep path leaves idle in sequence
mode - ConcatenatedGradient (4H) for gate gradients, HiddenCache (H) for dh,
Memory (2H) for dc - so BPTT costs no extra allocations. dW is zeroed once and
accumulated across steps, matching the fused DLL kernel.
Verification available on this machine has limits worth recording. The math is
the same as CPU_LSTMSeqForward/Backward, which is gradient-checked to 2.3e-10;
the kernels are syntax/type-checked offline (DirectML\opencl_seq_syntax_check.cpp,
compiled as C++ with OpenCL shims) because there is no OpenCL device or ICD
here. That check exists because a typo in Network.cl fails the WHOLE program
build, which would take the dense and conv kernels down with it - not just the
new ones. KernelCreate results are now checked and reported for these four for
the same reason; a build failure degrades to "LSTM/HYBRID unavailable on this
device" instead of an Execute error mid-training.
STILL NEEDS A RUN ON REAL OPENCL HARDWARE before release.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
CNeuronLSTMOCL consumed the whole flattened input in ONE gate computation and
back-propagated a single timestep, which its own class comment stated. Combined
with a conv stage whose window=step=neuronsCount gives it a receptive field of
exactly one bar, no stage in HYBRID mixed information across time - the bars
reached the dense stack as an unordered flat vector, the same thing the plain
MLP sees. That predicted the measured ranking (MLP 31.5%, CONV 32.5%, LSTM
30.6%, HYBRID 14.4%): each extra bottleneck cost accuracy and bought nothing.
The layer now unrolls m_historyBars timesteps, sharing one gate block across
them and carrying h/c forward, with real BPTT carrying dh and dc backward.
Per-step width comes from CLayerDescription::window, which CNet passes to the
new SetStepWidth() - previously dead metadata.
Consequences worth naming:
- Weight count drops from 4H(H+420+1) to 4H(H+21+1). Weight sharing is the
point of a recurrence, so ComputeLstmHiddenSize now budgets on the per-step
width; H goes 16 -> 64 at H1 defaults, and the model is still smaller.
- h/c start at zero per sample. The old buffers persisted across forward
passes, so under shuffled training each sample inherited an unrelated
sample's state.
- .nnw LSTM records are versioned (LSTM_SEQ_SAVE_TAG). The old weight block is
a different shape, so Load REFUSES pre-rewrite models rather than misreading
one and throwing off every later layer's offset. LSTM and HYBRID must retrain.
- Sequence mode has no Network.cl kernel, so it refuses the OpenCL tier loudly
instead of quietly running a different architecture there than on the DLL
tier - the two would train different models from one .cfg.
Legacy single-timestep path kept intact for step width <= 0.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
CNeuronLSTMOCL::Save early-returns when m_iInputs<=0, writing no LSTM
buffers at all - correct, since a layer that has never run a forward pass
has no weights to persist. But Load mirrored that early return BEFORE
allocating Memory (the c_prev cell state), and SetInputs - the lazy sizing
path that runs on the next feedForward - allocates the weight buffers but
never Memory. Both Init overloads allocate it unconditionally, so only the
save-then-load round trip could produce the gap.
Net effect: any net serialized before its first feedForward came back with
Memory==NULL. LSTMGates then failed on every call, short-circuiting the ||
before LSTMState could dereference the null buffer, so instead of crashing
the layer computed nothing forever. Observed on HYBRID after a
weights-reset-then-detach: all three class outputs pinned at exactly 1.000
(spread 0.0000), IS error stuck at 0.78, Buy/Sell recall 0%, and 636k
"Error of execution DirectML LSTM feedForward" lines in one 55MB journal.
The model looked like a converged Neutral collapse; it was a dead layer.
Allocate Memory in Load ahead of the early return, and harden SetInputs to
guarantee every buffer LSTMGates/LSTMState touch exists before it returns -
loudly, since the failure it replaces was indistinguishable from a healthy
net that simply never fires.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Menon et al. 2021 (ICLR), "Long-tail learning via logit adjustment": add
tau*log(prior_c) to each class logit inside the training gradient. Softmax
CE on adjusted logits is consistent for BALANCED error - the metric
checkpoint selection already ranks on - so the loss and the deploy decision
finally optimize the same thing.
The engine already computed a true softmax + categorical-CE gradient and
wrote it over the per-neuron sigmoid delta, so this is an offset added to
three logits in the two places that gradient is built (backProp scalar path
and backPropOCL). No backend, kernel or DLL change; the forward pass and
every inference path are untouched, which is the point - the network learns
to absorb the offset, so its raw argmax becomes the balanced-optimal
decision with nothing applied at inference.
Replaces rather than stacks. Minority replay is disabled while this is on,
and the post-hoc inference prior is forced off. Stacking is not a
theoretical worry: simulated on the measured 1118/1119/34298 distribution
in the weak-signal regime, plain CE collapses to Neutral (33.4% balanced,
Buy 0%), replay reaches 48.1%, logit adjustment 50.9% with better balance -
and BOTH together score 45.4% with Neutral recall at 0%, worse than either
alone. Buda et al. 2018 predicts exactly that.
Motivation from the six-chart run: every topology took one direction to
~50% recall and abandoned the other, the direction chosen arbitrarily (the
batch-norm control went Buy 1% / Sell 42%, the inverse of the other five).
One era in 1,301 cleared the per-class recall floor.
Fingerprinted conditionally, so the converged 60.7% models on disk keep
their filenames and stay loadable as the fallback.
Both builds compile 0 errors, 0 warnings.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Dividing a machine budget by the live chart count was wrong twice over.
The count is a snapshot taken when each net's pool is built, and charts
attach one at a time: five charts measured 10/6/5/4/4% of the same budget,
because the first only ever saw itself and the last saw all five. So the
earliest chart got several times the threads of the latest - skewing any
cross-topology comparison run on those charts, which is the exact thing
the setting existed to make fair. Nothing rebalanced afterwards either,
and rebalancing would mean tearing down a DLL context under a live trainer.
Both problems disappear once the answer stops depending on how many charts
are running. Each net now asks for a fixed 2 worker threads, converted to
the percentage the DLL wants from the detected core count.
Two is not a compromise: since the topology became data-derived the widest
dense layer is 64 units, so each ParallelFor has almost nothing to split
and per-dispatch overhead dominates. An MLP era cost ~66s at a wildly
oversubscribed 12 threads and ~80s at 1 thread - a 20% spread across a 12x
difference in thread count. Two per net also lands six concurrent charts
exactly on a 12-core box.
Removing the input costs nothing on the product side: a Market build has no
DLL tier at all, so it was already compiled out to a constant there and no
buyer could reach it.
Both builds compile 0 errors, 0 warnings.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The CPU-DLL thread pool was sized from TargetCPULoad undivided, on the
reasoning that only one pool is ever actively computing at a time. That is
true WITHIN a chart - MQL5 gives one chart's EA a single execution thread,
and every WarriorCPU.dll entry point blocks it until its ParallelFor()
completes, so the live net, the EMA shadow and HYBRID's fused pair take
turns. It does not hold ACROSS charts, which each get their own execution
thread and really do run their pools simultaneously.
At the 100% default on a 12-core box, five training charts asked for 12
threads each: 60 threads contending for 12 cores. Measured today, dropping
to ~2 threads apiece made every chart train "super fast". This had
previously been read as one architecture being mysteriously 10x slower than
another on the CPU-DLL tier while identical on OpenCL - oversubscription of
that degree degrades superlinearly and punishes whichever model issues the
most small sequential dispatches, which fits an MLP being the victim.
TargetCPULoad now means the budget for the whole machine, divided by the
number of charts running this EA. Counting charts is the correct axis:
concurrency here is one execution thread per chart, not one per CNet, and
the within-chart division that was previously removed stays removed.
Snapshot at pool-creation time on purpose - attaching another chart later
does not resize pools that already exist, because that would mean tearing
down a DLL context underneath a live training run. Skipped entirely in the
tester/optimizer, where the terminal already pins one strategy per agent.
This matters most for buyers: a Market build compiles the input out and
pins it to 100%, so they cannot reach the setting at all and would have hit
the pathological case with no way to diagnose or fix it.
The tier log line now reports the split rather than just the result.
Compiles 0 errors, 0 warnings.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
With normalization enabled a forward pass is not a pure function of its
input - it also advances the running mean/variance. ValidateCpuInference
compares the live backend net against a throwaway pure-MQL5 clone loaded
from the just-saved .nnw, so its own reference pass left the live model one
EMA step ahead of the file the clone reads. The check would then have been
measuring its own side effect, and a marginal result decides whether
buyers' backtests are allowed to run DLL-free.
Adds CNet::SetBatchNormFrozen / CNeuronBatchNormOCL::SetStatsFrozen -
classic batch-norm inference semantics, statistics used but not updated -
and freezes both sides for the duration of the comparison. Not persisted:
it is a transient evaluation mode, not model state. Default stays
adaptive, which is what the rest of the system (online continual learning)
is built around.
Compiles 0 errors, 0 warnings.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The only bounded stage in the entire forward path was the sigmoid
classification head - every hidden stage is PRELU. That is a network with
no internal scale control, and the failure ordered exactly by depth: on
SP500 H1 the shallow perceptron held ~52% balanced accuracy while the
deepest topology sat on the 33.3% one-class floor, with the per-bar logit
spread decaying monotonically (0.45 -> 0.38 over ~200 eras) until the
evidence tilt fell under the class-prior tilt. That is the signature of
internal covariate shift, which chapter 6.1 of the reference book is
entirely about and which the NeuroNet_DNG engine addresses with a layer
this project never had.
Two mechanisms make this the right fix rather than more hyperparameter
nudging:
- it decouples WEIGHT_DECAY from the learned function (van Laarhoven
2017) - with a normalized layer downstream, decay can no longer grind
the discriminative signal away, it only rescales the effective
learning rate;
- it is the precondition for ever running an unbounded logit head here.
The 2026-07-27 attempt blew up (IS error 5.6e15) precisely because
nothing upstream constrained scale.
Implementation notes:
- CNeuronBatchNormOCL computes host-side rather than as a fourth copy of
a kernel across Network.cl + WarriorCPU.cpp + WarriorDML.cpp. The math
is elementwise O(n); this way it behaves identically on all four
compute tiers, needs no DLL rebuild, and cannot drift between
backends. Same precedent as the softmax+CCE gradient and the
per-sample loss weighting, both computed in MQL5 for that reason.
- Statistics are exponential moving, not a stored mini-batch: training
is pure online SGD, one update per sample, so there is no batch to
average over. BatchNormWindow is an EMA window length.
- gamma/beta are excluded from weight decay, deliberately - decaying
gamma toward zero is the exact pathology being fixed.
- The layer self-sizes from whatever sits below it, because a conv/pool
stage's output width is derived inside the CNet constructor and is not
knowable to the topology builder.
- Checkpoint capture/restore/blend carry gamma/beta and the running
statistics alongside the dense matrix, so the plateau ladder cannot
restore a mismatched pair.
- SeedOutputLayerBias accepted only an exact defNeuronBaseOCL as the
weight-carrying penultimate layer; with normalization enabled that is
the batch-norm layer, so the cold-start bias seed would have silently
stopped being applied.
- Refuses to build, loudly, if a topology asks for normalization with no
compute backend at all - rather than quietly training a different
architecture than the one requested.
EnableBatchNorm (default on) and BatchNormWindow (1000 samples) are
inputs so the effect can be A/B'd without a recompile. Both feed the
weights-filename fingerprint, appended conditionally so existing non-BN
configs keep their fingerprints and are not forced to retrain.
Verified: analytic gradients match finite differences to 1.5e-7 relative
over 200 random cases; a faithful port of the full forward/backward chain
collapses to the 33.3% floor by era 4 without this layer and holds
36-43% with it. Compiles 0 errors, 0 warnings.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
A .nnw persists the ARCHITECTURE, not just the weights: Save writes
(int)activation per neuron and Load reads it straight back. The activation
chosen in BuildFreshTopology() therefore only ever reached a brand-new
topology - every reload restored the file's value and the next save wrote it
back out, so a wrong value could never heal while the source read as though
it were already fixed.
That is how five models kept training with an unbounded NONE classification
head for a full day after the 07-28 revert to SIGMOID. Confirmed by parsing
the binaries: 848cb42c.nnw / 2e754b43.nnw carry `act=NONE` on the 3-neuron
output layer, while a genuinely reset model of the same config carries
act=SIGMOID. In the log it showed as negative "OOS raw out" values -
impossible under sigmoid - escalating to a 4.14e13 logit spread with all
three classes numerically identical (input-independent output) and balanced
accuracy pinned on the 33.3% one-class floor.
- OutputLayerActivation() is now the single source of truth, called by both
BuildFreshTopology() and the new load-time repair, so the two can no
longer diverge the way a duplicated literal did.
- CNet::EnforceOutputActivation() re-asserts it after Load and reports the
stale value; CExpertSignalAIBase::EnforceTopologyContract() logs the
repair loudly, since weights learned under the old head may not be worth
keeping even once the head is corrected.
- Hidden layers are deliberately left alone: they legitimately differ per
stage (PRELU dense/conv, NONE pool, TANH LSTM).
Compiles 0 errors, 0 warnings.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
DRY - topology construction
---------------------------
CSignalCONV and CSignalHYBRID each built the Conv+Pool front-end from scratch;
CSignalLSTM and CSignalHYBRID each built the LSTM stage from scratch. The
duplicates had already drifted: HYBRID guarded the LSTM step with
MathMax(1, historyBars/2), CSignalLSTM divided unguarded, so a historyBars of 1
gave two different steps for what is documented as the same layer.
Extracted AddConvPoolStage() and AddLstmStage() onto CExpertSignalAIBase. The
three overrides are now compositions:
CONV = AddConvPoolStage
LSTM = AddLstmStage
HYBRID = AddConvPoolStage && AddLstmStage
HYBRID's "matches the standalone CONV front-end exactly, then adds LSTM" is
enforced by construction instead of by comment. Took the guarded step for both.
Also fixed a descriptor leak the duplicates shared: on a failed topology.Add()
the CLayerDescription was neither owned by the array nor deleted.
Dead code
---------
- CNet::SaveCheckpoint / CNet::LoadCheckpoint (123 lines). Superseded by the
in-memory CaptureWeights/RestoreWeights pair; Network.mqh:1312 already said so
("This replaces the file-based SaveCheckpoint/LoadCheckpoint"). Zero call
sites - every remaining mention was a comment. The five comments that
referenced them have been reworded rather than left dangling.
- CExpertSignalCustom::CheckForDuplicateTrade / FindLastTradeIndex /
UpdateTradeStatusAndExit: declared, never defined anywhere, never called.
They only made it look as though duplicate-trade detection existed.
Compiles 0 errors, 0 warnings.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
FileOpen(FILE_WRITE) truncates its target on open. CNet::Save already staged
the .nnw through a temp file + rename for that reason, but the three sidecars
written beside it did not:
.stats ExpertSignalAIBase.mqh:5918
.arrows ExpertSignalAIBase.mqh:6224
.cfg ExpertSignalAIBase.mqh:7329
Two defects followed.
1. An interrupted write published a truncated sidecar. For .cfg that is the
worst case: LoadAndCompareTopologyConfiguration() reads a short file as a
mismatch, which discards the trained model and restarts from era 0.
2. Windows file sharing is a mutual contract - a writer opened with no
FILE_SHARE_* blocks every concurrent open regardless of the reader's flags.
All three read paths carry FILE_SHARE_READ|FILE_SHARE_WRITE specifically so
a tester agent can read them while a live chart runs; an exclusive writer on
the same path defeated that.
Extracted CNet::Save's proven pattern into System\AtomicFile.mqh
(AtomicWriteBegin/AtomicWriteEnd) and routed all four writers through it. This
also encodes the FileMove gotcha once instead of per call site: the destination
location comes from FILE_COMMON inside the 4th arg, NOT inherited from the
source, and getting it wrong moves the file to the wrong sandbox silently.
Also fixed while in these functions:
- SaveTopologyConfiguration had 13 copy-pasted 6-line error blocks that each
returned WITHOUT FileClose(handle), leaking the handle on every write
failure. Collapsed to one ok-chain that closes exactly once. The on-disk
field order and types are unchanged (asserted during the rewrite) so existing
.cfg files still load.
- SaveChartSignals documented that pruning runs only after a successful write
("a failed write above leaves both the file AND the chart untouched") but
never checked any write result, so a partial write still deleted the chart
objects. Results are checked now, making the existing comment true.
Compiles 0 errors, 0 warnings.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The DFA (Direct Feedback Alignment) option was never a correct implementation:
it deterministically flipped the sign of half of all gradients based on
connection index parity, causing permanent gradient ascent for those weights
and guaranteed divergence. The backward pass was also incompatible with the
OpenCL/DirectML neuron model (layer.Total() == 1). This change removes all DFA
logic, including the enum value and `DfaFeedbackSignal` method, and replaces it
with plain gradient descent in all momentum update kernels. The `optimizer`
kernel argument is retained for binary compatibility but is no longer used.
The DFA optimizer was never real Direct Feedback Alignment — it flipped
gradient signs for half of each weight tensor deterministically, causing
permanent gradient ascent and divergence. Its backward pass was also
incompatible with the OpenCL/DirectML layer model. Remove the enum
entry, all related logic in the OpenCL kernel weight update functions,
and the `DfaFeedbackSignal` method. The `optimizer` kernel argument is
retained (unused) to preserve binary compatibility with existing DLLs.
Add type checks to correctly cast `CNeuronBaseOCL` objects in gradient clipping,
softmax evaluation, and `backProp`/`backPropDfa` loops. Previously, all neurons
were assumed to be `CNeuronBase`, causing invalid pointer casts and incorrect
gradient/output access for OpenCL layers. This ensures proper support for both
CPU and OCL neuron implementations.
Replace dynamic_cast with explicit type checks using obj.Type() and static casts in CaptureOptimizerSnapshot, RestoreOptimizerSnapshot, and SetOptimizerForAllNeurons. This avoids the runtime overhead of dynamic_cast inside critical loops, improving performance when capturing/restoring optimizer snapshots and setting optimizer for all neurons.
Changed updateInputWeights methods to accept pointer arguments by value (e.g., CLayer* instead of CLayer*&) to prevent unintended pointer reassignment and improve safety. Added overloads in CNeuronBase and CNeuronBaseOCL to support both generic CObject* and derived type pointers, unifying the virtual interface and resolving inconsistencies in network weight update logic.
Changed updateInputWeights signature from accepting `CObject*` to `CObject*&` to allow the method to modify the caller's pointer, preventing potential object copying or pointer invalidation. A large block of outdated commentary in ExpertSignalAIBase.mqh was also removed, cleaning up documentation no longer applicable after previous refactoring.
Introduce Direct Feedback Alignment (DFA) backward pass with gradient clipping, feedback matrix initialization, and a dedicated backPropDfa method. Add optimizer snapshot/restore hooks (CaptureOptimizerSnapshot, RestoreOptimizerSnapshot, SetOptimizerForAllNeurons) to temporarily switch the entire network's optimizer for replay-only updates during pass 2, preserving the original optimizer state. Support all neuron types including dropout, deconv, LSTM, and softmax in the snapshot logic.
Introduce an `optimizer` parameter to UpdateWeightsMomentum, UpdateWeightsConvMomentum, and UpdateWeightsAdam kernels. When set to a non-zero value, the gradient used for weight updates is multiplied by ±1 based on the parity of the weight index, implementing a basic feedback alignment signal for experimentation. When zero, the standard gradient is used unchanged. This allows A/B testing of alternative learning signals without modifying the rest of the training pipeline.
In AI/Network.mqh, return early from InitDirectML during
tester/optimization/forward runs to prevent agent-side file-lock
failures caused by rapid stop/restart cycles accessing DLL imports.
In Expert/ExpertSignalAIBase.mqh, add MathIsValidNumber checks in
CalibratedConfidenceMagnitude and SignaledConfidence to safely handle
NaN values, and refactor ShutdownChartCleanup to accept a preserve
flag, avoiding unnecessary chart purges during tester runs for faster
shutdowns. Also add m_purgeChartOnDestruct member.
In AI/NeuronDirectML.mqh, clean up a minor comment formatting issue.
Raise MIN_ACTIVATION_DERIVATIVE from 1e-4 to 1e-3 in both Network.cl and Network.mqh
to strengthen the escape signal through saturated hidden neurons. This provides a 10x
stronger safety net against fp32 OpenCL-specific saturation, complementing the earlier
output-layer fix (logit activation instead of sigmoid) that resolved the primary neutral
collapse bug.
Add MathSrand(GetTickCount()) before BuildFreshTopology() in ExpertSignalAIBase.mqh
to guarantee genuinely random weight initialization after genetic tuner evaluations,
matching the final-retrain path and Warrior_EA.mq5's OnInit. This prevents the previous
deterministic RNG state from dominating weight init.
Also remove UTF‑8 BOM from ExpertSignalAIBase.mqh and Network.mqh for cleaner encoding.
BlendWeightsFrom now uses CObject::Type() to correctly identify neuron classes, avoiding undefined behavior when neurons are from the plain-CPU hierarchy (CNeuron, CNeuronConv, CNeuronPool, CNeuronLSTM). Weight blending for those legacy classes is also implemented.
Add FILE_SHARE_READ and FILE_SHARE_WRITE to all FileOpen calls in CNet::Load,
CNet::LoadCheckpoint, and in ExpertSignalAIBase (LoadModelStats, LoadChartSignals)
so that reading model files does not fail with error 5004 when another process
(e.g., a live chart) holds the file. Replace FileCopy with the new CopySharedFile
function for sidecar files (.cfg, .stats, _shadow.nnw) to perform share-aware
copying, preventing the same failure during tester seeding. This ensures the EA
can read and copy model artifacts even while a deployed instance is running.
Add retry logic (CopyFileWithRetry, LoadNetWithRetry) to the tester's seeding path in ExpertSignalAIBase.mqh, and print diagnostic error messages in CNet::Load (Network.mqh) when FileOpen fails. This addresses silent failures caused by transient Windows sharing violations that occur when a live chart's atomic Save() overlaps with a Strategy Tester agent's FileCopy or FileOpen on the same production file. The new helpers retry the operation a few times with a short pause instead of silently falling back to an untrained model.
The Market build has no DLL tier (WarriorCPU.dll is compiled out per MQL5 Market rules), so the TargetCPULoad input would have sat unused in the Inputs tab. Now it is conditionally compiled as a const when WARRIOR_MARKET_BUILD is defined, keeping the call site intact while avoiding a dead input.
The Market_Description.html was rewritten to better explain the EA's neural network, its self-training on unseen data, and the dual classic/AI signal paths, along with installation details and screenshots.
A single EA run instantiates several CNet objects (main net, EMA shadow, OOS clones, etc.). Each previously ran its own OpenCL/DirectML probe, printing redundant error banners (e.g., "OpenCL not found") and compute tier messages.
Added process-wide static flags `s_openclUnavailable` and `s_computeTierLogged` to skip subsequent probes and log once per process. This eliminates log clutter and avoids unnecessary probe overhead on hosts without OpenCL.
Rename the original `CreateElement` with a defaulted `weighScale` parameter to `CreateElementScaled` and provide a proper virtual override that matches the base `CArrayObj::CreateElement` signature exactly. This ensures `CArrayObj::Load()` dispatches correctly, fixing a bug where every saved model load failed at the first layer. Update all call sites in `CNeuronBase::Init` and `CNeuronPool::Init`. Additionally, enhance error diagnostics in `CNet::Load` to distinguish between file truncation and code faults (such as the signature mismatch).
The `Load` method now accepts an optional `quiet` flag that suppresses diagnostic
`Print` statements when set to `true`. This is used by callers that anticipate
a load failure and handle it gracefully — for example, the EMA shadow-net
bootstrap on a CPU-DLL box, which cannot hold a second full network. The main
model load keeps `quiet=false` so actual failures remain visible.
- In Network.mqh: Add upfront file size check to distinguish truncated files from backend allocation failures during model load, improving error diagnosis.
- In ExpertSignalAIBase.mqh: Remove redundant shadow net save on shutdown to avoid doubling shutdown cost and exceeding MT5's deinit budget, preventing abnormal termination and subsequent retrain.
- In Warrior_EA.mq5: Reduce training timer interval from 5s to 250ms to allow more frequent training cycles instead of sitting idle ~98% of the time.
The previous save directly wrote to the target file, which could leave a truncated/partial file if the process was force-killed (e.g., MT5 deinit timeout). Now the save writes to a temporary file (.savetmp) and only renames it over the real file after a successful write. This ensures that an interrupted save never corrupts the last good model. Additionally, improved diagnostics in CNet::Load to distinguish corrupt/incompatible files from compute errors.
Add CaptureWeights and RestoreWeights methods that snapshot every neuron's weights into host arrays (CArrayDouble per neuron) and restore them in-place via setWeights. This replaces the file-based SaveCheckpoint/LoadCheckpoint for the mid-run best-era rollback, because the file path re-creates neurons (CLayer+Init) which fails on the multithreaded CPU-DLL backend (CDirectMLMy/WarriorCPU.dll) that cannot allocate a second full set of neuron tensors while the live set exists. In-memory weight copy uses only getWeights/setWeights, already proven by the per-era shadow blend. Snapshots weights only (not Adam moments); the regression handler decays eta on restore and clips per-step deltas to prevent stale-moment overshoot. Snapshot is valid only within a single Train() run. Also adds HaveWeightSnapshot() query and the m_weightSnapshot / m_haveWeightSnapshot member variables.
SaveCheckpoint and LoadCheckpoint in Network.mqh now accept a `bool common` flag. When true, the ephemeral checkpoint is stored in FILE_COMMON (co-located with the model's .nnw/.cfg/.stats/_shadow.nnw), preventing a parallel folder tree under the terminal-local Files directory. ExpertSignalAIBase.mqh is updated to pass the same flag to file existence checks and deletions, ensuring all sidecar files use the correct location.
