Completes the 2026-08-09 training audit. FORCES A RETRAIN of every
Wyckoff-enabled config (N1 re-keys the fingerprint), and BOTH DLLs must be
redeployed alongside the .ex5 - they carry new exports.
F4 - mini-batch accumulation, TRAIN_BATCH_SIZE=32. Training was pure online
SGD (one weight update per bar), which is the mechanical source of the
era-to-era whipsaw every downstream guard was built to cope with. The O(n^2)
outer product is native - AccumulateWeightGrad / AccumulateWeightGradConv /
AccumulateBufferInto in Network.cl, WarriorCPU and WarriorDML - while the
optimizer step is host-side MQL5 shared by all tiers (ApplyAccumToBlock), so
there is one Adam/SGD implementation instead of four that can drift.
- the LSTM needs no outer-product kernel (WeightsGradient already holds the
sample's full dW) but could NOT simply be left un-zeroed between samples:
CPU_LSTMSeqBackward/DML_LSTMSeqBackward memset it on entry. Hence a
separate accumulator plus an elementwise add.
- batch-norm gamma/beta accumulate in host arrays, not new BatchOptions
slots - BN_OPT_STRIDE is baked into every persisted .nnw.
- scoped to pass 2; online learning keeps immediate updates. Every save /
checkpoint / scoring boundary flushes, scaling by the real sample count.
- degrades to per-sample updates (one log line) on a tier that cannot
accumulate, so old devices and DLL-free builds are unaffected.
- verified offline: DirectML/batch_accum_check.cpp drives the real exports
against an independent reference; at B=1 the accumulator matches the
shipped unbatched kernel's own gradient to 1.1e-16. Math only - the
in-situ check remains the per-layer dW/W report on a real era.
F6 - ComputeFirstLayerWidth budgeted against the RAW input width even where a
conv/LSTM front end had already reduced it, so an LSTM's dense stack was
charged for 1,280 inputs when it receives 64. Confirmed from the deployed
.cfg files: CONV, LSTM and HYBRID were all pinned at the 16-unit floor. Now
budgeted against the front-end output and capped at it (never fan out), with
the derivation reordered so both stages settle first.
N1 - EventCode/EventPhase/StructuralPhase are signed categoricals packing
direction and Wyckoff stage into one scalar across a sign discontinuity. Split
into direction + [0,1] magnitude, the same convention the base OHLC block uses.
Information-preserving; 13 readings now occupy 16 inputs.
Compiled clean (0 errors, 0 warnings); both DLLs rebuilt.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The per-step entry points cannot express a sequence model. CPU_LSTMGates takes
the ENTIRE flattened input as one timestep, and CPU_LSTMGateGradient has no
parameter for dc arriving from the following step - so the recurrent gradient
path does not exist and cannot be assembled from these primitives at any call
pattern. The layer built on them is a gated dense layer that the class comment
already described honestly: "single-timestep-truncated BPTT".
Adds CPU_LSTMSeqForward / CPU_LSTMSeqBackward: the whole unrolled sequence in
one call each, weights shared across timesteps, dW accumulated over all of them
(the per-step CPU_LSTMWeightsGradient assigns rather than accumulates, so it
could not have been reused even with the dc term). Fused rather than dispatched
per step because the recurrence is sequential - T round trips would serialise T
lock/dispatch pairs for a few thousand FLOPs each.
h_{-1} and c_{-1} are zero per sample. The old layer carried its cell state
across forward passes, so under shuffled training every sample inherited the
state of an unrelated one.
DirectML gets the same math host-side (readback, compute in double, upload)
rather than HLSL: the recurrence needs a barrier per timestep, the GPU buffers
are float and BPTT accumulation is where that hurts most, and no D3D12 device
exists on this machine to test a shader against. Documented at the definition.
Verified with lstm_seq_gradcheck.cpp - central-difference check of dW and dX
against an asymmetric loss over the final hidden state. Max relative error
2.3e-10 on both, with a non-trivial gradient magnitude asserted so the check
cannot pass on an all-zero result.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Introduce an `optimizer` parameter to UpdateWeightsMomentum, UpdateWeightsConvMomentum, and UpdateWeightsAdam kernels. When set to a non-zero value, the gradient used for weight updates is multiplied by ±1 based on the parity of the weight index, implementing a basic feedback alignment signal for experimentation. When zero, the standard gradient is used unchanged. This allows A/B testing of alternative learning signals without modifying the rest of the training pipeline.
Replace hardcoded lr and momentum with new input variables for Adam and
SGD+momentum. Add OpenCL kernel LSTM_UpdateWeightsMomentum alongside the
existing Adam kernel. Update comments and revert beta1 to book default 0.9.
- In CaclOutputGradient kernel:
- Case 1 (sigmoid classification): removed erroneous multiplication by out*(1-out) which dampened gradients – the binary cross-entropy loss already cancels the sigmoid derivative, so direct (target-out) is correct.
- Added default case for NONE activation (softmax classification) to compute plain error (target-out); previously unhandled, resulting in zero gradients that froze the entire network when OpenCL was active.
- In UpdateWeightsMomentum and UpdateWeightsAdam kernels: added clamp to MAX_WEIGHT when updating weights to prevent gradient spikes from producing ±Infinity and subsequent NaN propagation through dense layers (e.g., classification output head).
- Define MAX_WEIGHT constant (1.0e6) for weight limits in clusters
- Remove redundant barrier from FeedForward kernel (prevents sync issues)
- Port FeedForwardProof and CalcInputGradientProof kernels for max-pooling (no weights, sliding max)
- Port FeedForwardConv kernel for convolution layers (shared weights, multiple output channels)
- Remove unused code and refactor signal condition logic (CSignalPAI)
