THE OPTIMIZER ("0.1% an hour per agent", 0 of 39 passes in 78 min,
12 agents): the tester fires OnTimer on SIMULATED time, so the live
chart's 500ms EventSetMillisecondTimer over a 2016-2026 pass is ~600
MILLION OnTimer calls - each walking 4x PollTraining, the vote
readout's string build, the overlay advance and the deployed census.
None of it serves an inference-only pass: training never runs, per-bar
inference is driven by OnTickHandler off the tick stream, the risk
budget re-checks in OnTick, and there is no chart to keep fresh.
StepSetTimer now arms EventSetTimer(3600) in tester/optimizer/forward
(~2,600 calls per pass) and keeps the 500ms timer for live charts.
Plus a TESTER PASS SELF-PROFILE: per-tick buckets (pre / Expert.OnTick
/ journal) and the timer total, printed once at the pass's OnDeinit -
so if a pass is still slow it names its own consumer instead of being
diagnosed from outside.
OFFLOAD (operator: "as much calculation as possible to DLL/OpenCL"):
batch norm was the ONE stage still host-side on the DLL tier - the
device path was OpenCL-only, so every sample crossed the bus twice per
BN layer and normalized in interpreted MQL5 (and every model runs
batchnorm ON). Four new exports mirror AI\Network.cl's BatchNorm*
kernels 1:1 in DOUBLE precision (closer to the host reference than
the float OpenCL kernels): forward with running stats + frozen flag,
hidden gradient with the clamp derivative, gamma/beta accumulate, and
the batch-mean apply (no weight decay, moments-before-skip ordering,
sqrt-stored v). BnDeviceEligible/EnsureBnDeviceBuffers/all four
Dispatch* now route by backend; the EXISTING in-situ self-checks
(host-vs-device on the first real sample, latch-off + host fallback on
mismatch) verify the DLL kernels exactly as they verified OpenCL ones.
batch_accum_check regression: ALL CHECKS PASSED on the rebuilt DLL.
Same deployment coupling as bd46374: the .ex5 imports the new exports
- copy DirectML\WarriorCPU.dll into MQL5\Libraries (terminal closed)
together with the new .ex5, and re-copy it to the tester agents (or
just run DirectML\build_cpu.bat once with everything closed - it
deploys to every discovered Libraries folder).
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
"Hundreds of times slower than a regular EA" decomposed into two
multiplied factors, both measured:
1. THE OPTIMIZER STEP RAN IN INTERPRETED MQL5. The CPU tier shipped
the F4 accumulate exports with deliberately no matching apply
(WarriorCPU.h said so), so on the DLL backend - this box - every
TRAIN_BATCH_SIZE=8 batch fell to the host loop in ApplyAccumToBlock:
a per-weight MQL5 pass through CBufferDouble.At()/Update() plus four
full weight-matrix BufferRead/Write round trips. The 2026-07-26
profile had already shown the per-sample Adam step at 81% of ALL
runtime (feedForward: 8%; feature building: 0.35%) - sqrt+divide
per weight vs one multiply-add; moving it into MQL5 made it worse.
New CPU_ApplyAccumAdam / CPU_ApplyAccumMomentum: one element-wise
ParallelFor takes the batch-mean step and zeroes the accumulator
DLL-side, generic over any flat block (dense/conv/LSTM/batch-norm -
all apply paths funnel through ApplyAccumToBlock, which now tries
the DLL first, with the same one-warning failure latch as the
OpenCL fast path). Math is the shipped step to the last clamp:
sqrt-stored v, ClampDelta, AdamW decay, ClampWeight.
batch_accum_check extended (check 6) and ALL PASS: apply == host
reference at B=8/B=4, accumulator zeroed, and B=1 accumulate+apply
== the unbatched Adam kernel BIT-EXACTLY (kernel-vs-kernel, no
transcription). DLL rebuilt with the shipped /fp:fast recipe.
2. A 24% DUTY CYCLE. Train sliced 120ms per 500ms timer period
(30ms/member x4), leaving the chart thread idle 76% of the time.
Now 300ms total (75ms/member): ~60% duty, ~2.5x, click latency
bounded at ~300ms while training runs - between the fully-reactive
120 and the documented "sticky drag" 480.
DEPLOYMENT COUPLING: the new .ex5 #imports the new exports, so it will
NOT LOAD against the old WarriorCPU.dll ("cannot find function"). Copy
DirectML\WarriorCPU.dll into MQL5\Libraries (terminal closed) in the
same step as deploying the new .ex5.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Completes the 2026-08-09 training audit. FORCES A RETRAIN of every
Wyckoff-enabled config (N1 re-keys the fingerprint), and BOTH DLLs must be
redeployed alongside the .ex5 - they carry new exports.
F4 - mini-batch accumulation, TRAIN_BATCH_SIZE=32. Training was pure online
SGD (one weight update per bar), which is the mechanical source of the
era-to-era whipsaw every downstream guard was built to cope with. The O(n^2)
outer product is native - AccumulateWeightGrad / AccumulateWeightGradConv /
AccumulateBufferInto in Network.cl, WarriorCPU and WarriorDML - while the
optimizer step is host-side MQL5 shared by all tiers (ApplyAccumToBlock), so
there is one Adam/SGD implementation instead of four that can drift.
- the LSTM needs no outer-product kernel (WeightsGradient already holds the
sample's full dW) but could NOT simply be left un-zeroed between samples:
CPU_LSTMSeqBackward/DML_LSTMSeqBackward memset it on entry. Hence a
separate accumulator plus an elementwise add.
- batch-norm gamma/beta accumulate in host arrays, not new BatchOptions
slots - BN_OPT_STRIDE is baked into every persisted .nnw.
- scoped to pass 2; online learning keeps immediate updates. Every save /
checkpoint / scoring boundary flushes, scaling by the real sample count.
- degrades to per-sample updates (one log line) on a tier that cannot
accumulate, so old devices and DLL-free builds are unaffected.
- verified offline: DirectML/batch_accum_check.cpp drives the real exports
against an independent reference; at B=1 the accumulator matches the
shipped unbatched kernel's own gradient to 1.1e-16. Math only - the
in-situ check remains the per-layer dW/W report on a real era.
F6 - ComputeFirstLayerWidth budgeted against the RAW input width even where a
conv/LSTM front end had already reduced it, so an LSTM's dense stack was
charged for 1,280 inputs when it receives 64. Confirmed from the deployed
.cfg files: CONV, LSTM and HYBRID were all pinned at the 16-unit floor. Now
budgeted against the front-end output and capped at it (never fan out), with
the derivation reordered so both stages settle first.
N1 - EventCode/EventPhase/StructuralPhase are signed categoricals packing
direction and Wyckoff stage into one scalar across a sign discontinuity. Split
into direction + [0,1] magnitude, the same convention the base OHLC block uses.
Information-preserving; 13 readings now occupy 16 inputs.
Compiled clean (0 errors, 0 warnings); both DLLs rebuilt.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The per-step entry points cannot express a sequence model. CPU_LSTMGates takes
the ENTIRE flattened input as one timestep, and CPU_LSTMGateGradient has no
parameter for dc arriving from the following step - so the recurrent gradient
path does not exist and cannot be assembled from these primitives at any call
pattern. The layer built on them is a gated dense layer that the class comment
already described honestly: "single-timestep-truncated BPTT".
Adds CPU_LSTMSeqForward / CPU_LSTMSeqBackward: the whole unrolled sequence in
one call each, weights shared across timesteps, dW accumulated over all of them
(the per-step CPU_LSTMWeightsGradient assigns rather than accumulates, so it
could not have been reused even with the dc term). Fused rather than dispatched
per step because the recurrence is sequential - T round trips would serialise T
lock/dispatch pairs for a few thousand FLOPs each.
h_{-1} and c_{-1} are zero per sample. The old layer carried its cell state
across forward passes, so under shuffled training every sample inherited the
state of an unrelated one.
DirectML gets the same math host-side (readback, compute in double, upload)
rather than HLSL: the recurrence needs a barrier per timestep, the GPU buffers
are float and BPTT accumulation is where that hurts most, and no D3D12 device
exists on this machine to test a shader against. Documented at the definition.
Verified with lstm_seq_gradcheck.cpp - central-difference check of dW and dX
against an asymmetric loss over the final hidden state. Max relative error
2.3e-10 on both, with a non-trivial gradient magnitude asserted so the check
cannot pass on an all-zero result.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Introduce an `optimizer` parameter to UpdateWeightsMomentum, UpdateWeightsConvMomentum, and UpdateWeightsAdam kernels. When set to a non-zero value, the gradient used for weight updates is multiplied by ±1 based on the parity of the weight index, implementing a basic feedback alignment signal for experimentation. When zero, the standard gradient is used unchanged. This allows A/B testing of alternative learning signals without modifying the rest of the training pipeline.
Replace hardcoded lr and momentum with new input variables for Adam and
SGD+momentum. Add OpenCL kernel LSTM_UpdateWeightsMomentum alongside the
existing Adam kernel. Update comments and revert beta1 to book default 0.9.
- In CaclOutputGradient kernel:
- Case 1 (sigmoid classification): removed erroneous multiplication by out*(1-out) which dampened gradients – the binary cross-entropy loss already cancels the sigmoid derivative, so direct (target-out) is correct.
- Added default case for NONE activation (softmax classification) to compute plain error (target-out); previously unhandled, resulting in zero gradients that froze the entire network when OpenCL was active.
- In UpdateWeightsMomentum and UpdateWeightsAdam kernels: added clamp to MAX_WEIGHT when updating weights to prevent gradient spikes from producing ±Infinity and subsequent NaN propagation through dense layers (e.g., classification output head).
Replace the absolute thread count input (CpuDllThreads) with a percentage-based CPU_LOAD_PRESET enum (TargetCPULoad). This allows users to specify a percentage of detected cores to use when the CPU DLL fallback is active, improving flexibility and preventing issues when multiple instances share the same CPU DLL pool. Also adds CPU_GetHardwareConcurrency() for accurate core detection.
- Define MAX_WEIGHT constant (1.0e6) for weight limits in clusters
- Remove redundant barrier from FeedForward kernel (prevents sync issues)
- Port FeedForwardProof and CalcInputGradientProof kernels for max-pooling (no weights, sliding max)
- Port FeedForwardConv kernel for convolution layers (shared weights, multiple output channels)
- Remove unused code and refactor signal condition logic (CSignalPAI)
