CaclHiddenGradient computed this layer's gradient as matrix_w[(outputs+1)*i + k]
against a buffer whose actual layout (one row per NEXT-layer neuron, stride
inputs+1) makes the correct read matrix_w[k*(inputs+1) + i]: the transpose for
square layers, and for the non-square boundaries this EA actually builds
(tapered stacks, the 3-neuron head) a mis-strided walk that ran past the buffer
end - garbage on OpenCL, zeroed reads on the CPU DLL, so the tiers did not even
agree with each other. Every gradient crossing a dense boundary on its way down
- the entire learning signal reaching the BN/conv/LSTM front ends - passed
through a fixed wrong matrix: feedback-alignment dynamics, not backprop, which
is why nets still "learned something" and this survived. The book reference
(NeuroNet_DNG) fixed this in a later article version; our kernel descended from
the earlier one. Confounds every model-based negative verdict to date.
Also in this commit, same root cause family:
- per-sample UpdateWeightsAdam (OpenCL): input for slot group j was read at
matrix_i[j] instead of matrix_i[j*4] (corrupted outer product past group 0),
and dispatch dim 1 sized on ceil(inputs/4) left the bias column unreachable
whenever inputs%4==0 - dense biases never trained on OpenCL. Rewritten as a
lane-guarded scalar loop keeping our Adam conventions (sqrt-stored v,
decoupled decay, both clamps, no sign gate). The batched accum path never had
either bug; this kernel is what SetBatchSize(1) runs - including online
continual learning on client machines, where OpenCL is the only tier.
- conv backward passed raw (int)Activation() where the kernels expect
NativeActivationCode(): NONE took the tanh branch (clamping a BN layer's
unbounded z-scores), TANH took sigmoid, PRELU took none. Dormant only because
the conv sits at layer 1 today.
- hidden-gradient dispatch over Neurons()+1 dropped to Neurons(): biases get no
backprop gradient and the extra work-item only ever read past matrix_o.
All three backends (Network.cl, WarriorCPU.cpp, WarriorDML.cpp HLSL) changed in
lockstep; DML gained an `inputs` constant to derive the row stride. New
dense_backprop_check.cpp proves the CPU kernel is central-finite-difference
consistent with the real forward kernel on 8x8, 64x3, 33x64, 5x3 (max diff
3e-9) and that all three activation branches match transcription. All 16 checks
pass. Offline math check only - the in-situ proof remains the per-layer dW/W
report on a real era.
FORCES FULL RETRAIN. Both DLLs rebuilt and redeployed to MQL5\Libraries.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Market builds cannot import a DLL, so OpenCL is the tier paying clients run.
It was several times slower than the CPU DLL, and the dominant reason was a
host-side optimizer step I shipped with the mini-batch work in 274630f.
ApplyAccumToBlock read the weights, the accumulator and both Adam moments back
over the bus, stepped them in MQL5, and wrote four buffers out - eight full
weight-matrix transfers per batch PER WEIGHT BLOCK, each a blocking sync. At
TRAIN_BATCH_SIZE 8 that is roughly one entire weight matrix crossing the bus
per training sample. It was host-side for a good reason (one optimizer
implementation shared by all four tiers instead of four that can drift), and
that reason turned out to cost the product's own compute tier.
- ApplyAccumAdam / ApplyAccumMomentum in Network.cl: flat, elementwise, and
they zero the accumulator themselves so there is no separate clear dispatch
and no way to leave it dirty via an early return
- ApplyAccumOnDevice dispatches them; the host step stays as the reference and
as the implementation for DirectML, the CPU DLL and pure-MQL5
- failure latches OFF process-wide with one warning rather than a failed
Execute per batch, since a kernel that did not build will not build later
- m_applyKernelsOk is tracked SEPARATELY from m_batchKernelsOk: without the
accumulation kernels a device cannot batch and must drop to per-sample
updates, whereas without these it batches normally and merely pays the
transfers. Conflating them would turn a missing optimisation into a changed
optimizer
The kernel runs fp32 where the host step ran fp64. That is the OpenCL tier
becoming self-consistent, not a regression: its device buffers are already
fp32 (CBufferDouble::m_data_f) and its unbatched optimizer already ran in
fp32, so the batched path was the odd one out. DLL tiers keep fp64 throughout.
Validated: no OpenCL platform exists on this box, so the kernel source is
syntax/type checked as C against a shim and driven for 4000 steps. It clears
the accumulator, and it is scale-invariant - displacement 1.199336 at |g|=1
versus 1.199333 at |g|=1e-4, matching the figure DirectML/batch_accum_check
produced for the fixed CPU_UpdateWeightsAdam. It reproduces the corrected Adam,
not the pre-371f8aa one.
Both build variants compile 0 errors, 0 warnings.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Root cause of the B=32 regression, and it predates F4 entirely. Every Adam
kernel stored v already square-rooted and then fed that stored value back in
as if it were the variance:
v_new = sqrt(b2 * v_old + (1 - b2) * g^2)
That recursion has a fixed point at v ~= b2 = 0.999 for ANY gradient below
unit scale, so the denominator stops tracking the gradient and Adam degrades
into plain SGD with lr = lt. Measured against the shipped WarriorCPU.dll
(batch_accum_check.cpp, TestOptimizerScaleInvariance), 4000 steps of a
constant gradient: 3285x less displacement at |g|=1e-5 than at |g|=1, where
a scale-invariant optimizer gives the same distance for both. After the fix
all six magnitudes read 1.199 and v tracks |g| exactly.
It hit conv/LSTM specifically because they sit behind a batch-norm with
running variance ~2.6e+05, so their gradients arrive divided by ~500 - deep
in the degraded regime - while the dense stack near the loss stayed in the
working one. In situ on SP500 H1: lstm1 dW/W 2.62/10.0/7.14% -> 0.024/0.022/
0.003%, conv1 decaying to 0.000% by era 30. NeuronBatchNorm.mqh already
squared v back for gamma/beta and its comment named the kernels as wrong,
which is exactly why gamma/beta kept training while the stages behind froze.
Persisted .nnw needs no migration - v keeps its std-dev meaning.
Also, the two ways F4 exposed it, both mine:
- No LR compensation for B fewer steps per era. sqrt(B) for adaptive methods
(Krizhevsky 2014; Granziol et al. 2022), applied once in
InitialEtaForOptimizer(). Linear scaling (Goyal et al. 2017) is for SGD.
- Plateau patience denominated in eras, so raising B made the ladder 32x more
impatient in its only unit. PAI converged at era 41 on ~49k updates where
the same config had been finding new bests at era 1028.
TrainPlateauPatienceEras() stretches it by the same sqrt(B).
TRAIN_BATCH_SIZE 32 -> 8 so the patience stretch stays affordable (8 -> 23
eras per stage, not 8 -> 45). Both helpers are identities at B=1.
Deploy gate: DEPLOY_MIN_SIDE_RECALL_PCT (10%) folded into tradeableOK. The
perceptron reported Sell:0% recall in all 41 eras, cleared the floor on Buy
alone at 36.6% vs 34% chance, deployed, and sprayed buy arrows. Folded into
the ranking key rather than checked at deploy time so a one-sided era cannot
become best-so-far in the first place.
Deinit: the arrow purge now runs BEFORE ExtPanel.Destroy(), an unbounded
CAppDialog teardown that sat ahead of it - the same ordering inversion the
rule there exists to prevent. CONV was force-terminated 4.8 s into OnDeinit
(vs ~1.1 s for the three that finished) having reached none of its cleanup,
so its arrows stayed on the chart. Steps are now timed in the log.
PurgeChart's verification rescan filtered on OBJ_ARROW, the same blind spot
as the bulk delete, so "persisted 10 ... cleared 0" passed silently. It now
walks every object type and reports the object counts when both are zero.
Both build variants compile 0 errors / 0 warnings; both DLLs rebuilt.
FORCES A RETRAIN (already forced by N1) and both DLLs must ship with the .ex5.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Completes the 2026-08-09 training audit. FORCES A RETRAIN of every
Wyckoff-enabled config (N1 re-keys the fingerprint), and BOTH DLLs must be
redeployed alongside the .ex5 - they carry new exports.
F4 - mini-batch accumulation, TRAIN_BATCH_SIZE=32. Training was pure online
SGD (one weight update per bar), which is the mechanical source of the
era-to-era whipsaw every downstream guard was built to cope with. The O(n^2)
outer product is native - AccumulateWeightGrad / AccumulateWeightGradConv /
AccumulateBufferInto in Network.cl, WarriorCPU and WarriorDML - while the
optimizer step is host-side MQL5 shared by all tiers (ApplyAccumToBlock), so
there is one Adam/SGD implementation instead of four that can drift.
- the LSTM needs no outer-product kernel (WeightsGradient already holds the
sample's full dW) but could NOT simply be left un-zeroed between samples:
CPU_LSTMSeqBackward/DML_LSTMSeqBackward memset it on entry. Hence a
separate accumulator plus an elementwise add.
- batch-norm gamma/beta accumulate in host arrays, not new BatchOptions
slots - BN_OPT_STRIDE is baked into every persisted .nnw.
- scoped to pass 2; online learning keeps immediate updates. Every save /
checkpoint / scoring boundary flushes, scaling by the real sample count.
- degrades to per-sample updates (one log line) on a tier that cannot
accumulate, so old devices and DLL-free builds are unaffected.
- verified offline: DirectML/batch_accum_check.cpp drives the real exports
against an independent reference; at B=1 the accumulator matches the
shipped unbatched kernel's own gradient to 1.1e-16. Math only - the
in-situ check remains the per-layer dW/W report on a real era.
F6 - ComputeFirstLayerWidth budgeted against the RAW input width even where a
conv/LSTM front end had already reduced it, so an LSTM's dense stack was
charged for 1,280 inputs when it receives 64. Confirmed from the deployed
.cfg files: CONV, LSTM and HYBRID were all pinned at the 16-unit floor. Now
budgeted against the front-end output and capped at it (never fan out), with
the derivation reordered so both stages settle first.
N1 - EventCode/EventPhase/StructuralPhase are signed categoricals packing
direction and Wyckoff stage into one scalar across a sign discontinuity. Split
into direction + [0,1] magnitude, the same convention the base OHLC block uses.
Information-preserving; 13 readings now occupy 16 inputs.
Compiled clean (0 errors, 0 warnings); both DLLs rebuilt.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
- BufferDouble: replace hardcoded "DirectML/CPU-DLL" with dynamic backend name
and add buffer index/element count to all error prints for easier debugging.
- NetPersistence: distinguish missing file from transient lock by probing
FileIsExist before logging, eliminating false "sharing violation" warnings
when no saved model exists on first run.
- Moves CLayer neuron construction to AI/Impl/Layer.mqh to keep Network.mqh clean
- Unifies four previously duplicated architecture initialisation blocks (MLP/CONV/LSTM/HYBRID) into a single shared function
- Eliminates risk of behavioural drift where one architecture missed a setter, causing mismatched feature sets or targets
