- BufferDouble: replace hardcoded "DirectML/CPU-DLL" with dynamic backend name
and add buffer index/element count to all error prints for easier debugging.
- NetPersistence: distinguish missing file from transient lock by probing
FileIsExist before logging, eliminating false "sharing violation" warnings
when no saved model exists on first run.
The sequence rewrite made LSTM/HYBRID recurrences over 20 bars, but every
weight - including the gate biases - is initialized around zero. That puts
the forget gate at sigmoid(0) = 0.5, so the cell state is halved every step:
the first bar survives into the output scaled by ~0.5^20, and the gradient
reaches it scaled by the same factor.
The layer was therefore a one-bar model wearing a 20-bar interface. A one-bar
model has no signal on this task, so the head learned the base rate and
emitted Neutral everywhere - the flat 0.34 IS error across twelve eras and
OOS recall Neutral:100% seen on SP500 H1.
Measured at the shipped H1 shapes (H=64, stepInputs=21, T=20) - influence of
bar 0 on the output relative to bar 19:
bias 0.0 -> 3.0e-05 forward, 3.3e-05 backward (dead)
bias 1.0 -> 1.2e-02 forward, 1.4e-02 backward
bias 2.0 -> 2.5e-01 forward, 2.7e-01 backward (a real 20-bar field)
This is the standard fix, not a tuned knob: Gers/Schmidhuber/Cummins (2000)
introduced the forget gate with a positive bias, and Jozefowicz/Zaremba/
Sutskever (ICML 2015) recommend a bias of 1 as a default (whence Keras'
unit_forget_bias). Both 1 and 2 are standard; the sweep picks 2 because at a
20-step window a bias of 1 still leaves the oldest bar at ~1% influence.
lstm_seq_flowcheck.cpp is added as a permanent regression check and asserts
the shipped constant keeps >=5% reach in both directions. It complements
lstm_seq_gradcheck.cpp: that one proves the BPTT is CORRECT, this one proves
it is USABLE. The gradient check passed at 2.3e-10 throughout - correct math
over a recurrence that carries nothing looks exactly like a bad architecture.
Both builds compile 0 errors, 0 warnings. Initialization only, so the .nnw
format is unchanged; LSTM and HYBRID must retrain to pick it up.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Closes the gap left by 7a08197, which refused sequence mode under OpenCL. That
was defensible for a private build and not for a shipped one: the release path
includes an OpenCL laptop, and a customer with a GPU would have found LSTM and
HYBRID simply unavailable.
One launch PER TIMESTEP rather than a single kernel looping with barrier().
Every hidden unit's gates read all of h_{t-1}, OpenCL barriers only span a
work-group, and nothing here constrains how the runtime partitions the global
size - so an in-kernel loop would be correct only by luck of the partitioning.
Host-driven launches make each step an implicit global barrier: more enqueues,
correct on every device.
Backward reuses the buffers the single-timestep path leaves idle in sequence
mode - ConcatenatedGradient (4H) for gate gradients, HiddenCache (H) for dh,
Memory (2H) for dc - so BPTT costs no extra allocations. dW is zeroed once and
accumulated across steps, matching the fused DLL kernel.
Verification available on this machine has limits worth recording. The math is
the same as CPU_LSTMSeqForward/Backward, which is gradient-checked to 2.3e-10;
the kernels are syntax/type-checked offline (DirectML\opencl_seq_syntax_check.cpp,
compiled as C++ with OpenCL shims) because there is no OpenCL device or ICD
here. That check exists because a typo in Network.cl fails the WHOLE program
build, which would take the dense and conv kernels down with it - not just the
new ones. KernelCreate results are now checked and reported for these four for
the same reason; a build failure degrades to "LSTM/HYBRID unavailable on this
device" instead of an Execute error mid-training.
STILL NEEDS A RUN ON REAL OPENCL HARDWARE before release.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The per-step entry points cannot express a sequence model. CPU_LSTMGates takes
the ENTIRE flattened input as one timestep, and CPU_LSTMGateGradient has no
parameter for dc arriving from the following step - so the recurrent gradient
path does not exist and cannot be assembled from these primitives at any call
pattern. The layer built on them is a gated dense layer that the class comment
already described honestly: "single-timestep-truncated BPTT".
Adds CPU_LSTMSeqForward / CPU_LSTMSeqBackward: the whole unrolled sequence in
one call each, weights shared across timesteps, dW accumulated over all of them
(the per-step CPU_LSTMWeightsGradient assigns rather than accumulates, so it
could not have been reused even with the dc term). Fused rather than dispatched
per step because the recurrence is sequential - T round trips would serialise T
lock/dispatch pairs for a few thousand FLOPs each.
h_{-1} and c_{-1} are zero per sample. The old layer carried its cell state
across forward passes, so under shuffled training every sample inherited the
state of an unrelated one.
DirectML gets the same math host-side (readback, compute in double, upload)
rather than HLSL: the recurrence needs a barrier per timestep, the GPU buffers
are float and BPTT accumulation is where that hurts most, and no D3D12 device
exists on this machine to test a shader against. Documented at the definition.
Verified with lstm_seq_gradcheck.cpp - central-difference check of dW and dX
against an asymmetric loss over the final hidden state. Max relative error
2.3e-10 on both, with a non-trivial gradient magnitude asserted so the check
cannot pass on an all-zero result.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The DFA (Direct Feedback Alignment) option was never a correct implementation:
it deterministically flipped the sign of half of all gradients based on
connection index parity, causing permanent gradient ascent for those weights
and guaranteed divergence. The backward pass was also incompatible with the
OpenCL/DirectML neuron model (layer.Total() == 1). This change removes all DFA
logic, including the enum value and `DfaFeedbackSignal` method, and replaces it
with plain gradient descent in all momentum update kernels. The `optimizer`
kernel argument is retained for binary compatibility but is no longer used.
Add type checks to correctly cast `CNeuronBaseOCL` objects in gradient clipping,
softmax evaluation, and `backProp`/`backPropDfa` loops. Previously, all neurons
were assumed to be `CNeuronBase`, causing invalid pointer casts and incorrect
gradient/output access for OpenCL layers. This ensures proper support for both
CPU and OCL neuron implementations.
Replace dynamic_cast with explicit type checks using obj.Type() and static casts in CaptureOptimizerSnapshot, RestoreOptimizerSnapshot, and SetOptimizerForAllNeurons. This avoids the runtime overhead of dynamic_cast inside critical loops, improving performance when capturing/restoring optimizer snapshots and setting optimizer for all neurons.
Introduce an `optimizer` parameter to UpdateWeightsMomentum, UpdateWeightsConvMomentum, and UpdateWeightsAdam kernels. When set to a non-zero value, the gradient used for weight updates is multiplied by ±1 based on the parity of the weight index, implementing a basic feedback alignment signal for experimentation. When zero, the standard gradient is used unchanged. This allows A/B testing of alternative learning signals without modifying the rest of the training pipeline.
Reduce weight decay from 0.01 to 0.001 across all backends (Network.cl, Network.mqh, WarriorCPU.cpp) to fix a training collapse issue. The original 0.01 AdamW default caused discriminative weights to decay below the calibration-capped class-prior offsets, resulting in a monotonically shrinking per-bar logit spread and eventual constant Neutral predictions (argmax degenerated once evidence tilt dropped under the prior tilt). The new value 0.001 lifts the evidence ceiling 10× while still bounding long-run weight growth, restoring effective discrimination. Note: this change must remain in sync across all four backends.
The sign-agreement gate in `UpdateWeightsAdam`, `UpdateWeightsConvAdam`, and `LSTM_UpdateWeightsAdam` caused a gradient ratchet effect under one-hot softmax with categorical cross-entropy, leading to an all-Neutral collapse. Removing this gate aligns all four backends (CPU, GPU, OpenCL, MQL) and restores correct gradient flow.
Replace hardcoded lr and momentum with new input variables for Adam and
SGD+momentum. Add OpenCL kernel LSTM_UpdateWeightsMomentum alongside the
existing Adam kernel. Update comments and revert beta1 to book default 0.9.
- Ensure `tick_volume` array is set as series in ADShorteningOfThrust.mq5 to prevent future-data leak in volume calculations.
- Ensure `open` array is set as series in ADWyckoffFailedStructure.mq5 to prevent future-data leak in structure detection.
- Add missing PReLU gradient scaling (multiply by 0.01 for negative outputs) in CPU_CalcHiddenGradient and DirectML shader to match expected derivative behavior across all backends.
- Add inline threshold (512) in WarriorCPU to avoid thread-pool overhead for small dispatches; run small workloads inline on the calling thread.
- Reduce training time budget from 500ms to 120ms in ExpertSignalAIBase to keep the UI reactive while training.
- Tighten MAX_WEIGHT from 1.0e6 to 100.0 to prevent unbounded weight growth.
- Add MIN_ACTIVATION_DERIVATIVE (1e-4) to avoid zero gradients for saturated units.
- Introduce WEIGHT_DECAY (0.01) to decouple decay from Adam updates.
- Add MAX_WEIGHT_DELTA (0.1) to clamp per-step Adam updates and prevent overshoot.
- Apply sign-agreement gate on weight updates in Adam kernel to only apply steps aligned with current gradient direction.
- Fix tanh/sigmoid derivative calculations to use the new floor instead of hard-coded edge-case values.
- In CaclOutputGradient kernel:
- Case 1 (sigmoid classification): removed erroneous multiplication by out*(1-out) which dampened gradients – the binary cross-entropy loss already cancels the sigmoid derivative, so direct (target-out) is correct.
- Added default case for NONE activation (softmax classification) to compute plain error (target-out); previously unhandled, resulting in zero gradients that froze the entire network when OpenCL was active.
- In UpdateWeightsMomentum and UpdateWeightsAdam kernels: added clamp to MAX_WEIGHT when updating weights to prevent gradient spikes from producing ±Infinity and subsequent NaN propagation through dense layers (e.g., classification output head).
- Introduce per-bar feature cache to avoid redundant recomputation of input vectors during training.
- Rename EnsureLabelCacheCapacity to EnsureBarCachesCapacity to reflect management of both label and feature caches.
- Fix oversampling logic to maintain balanced representation among minority classes, replacing independent 5x caps that caused relative bias.
Warrior_EA.mq5: clear Comment() and destroy the control panel before
Expert.Deinit()'s object-purge cascade runs out from under it; stop
re-registering the same signal filters on every DB retry (was causing a
double-delete of the same pointer on shutdown).
DirectML/WarriorCPU.cpp: bound the worker-thread join in ThreadPool::Stop()
instead of blocking forever - CPU_Shutdown() held g_mutex across an unbounded
join, so a watchdog-killed calling thread could leave it locked forever,
poisoning every future call into the DLL (matches reports of the EA getting
stuck on "initializing" after being removed and re-added to a chart).
Also includes prior era-0 label-cache prebuild and pullback/reversal
label-quality work in AI/Network.mqh, Expert/ExpertSignalAIBase.mqh, and
Variables/Inputs.mqh.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Replace the absolute thread count input (CpuDllThreads) with a percentage-based CPU_LOAD_PRESET enum (TargetCPULoad). This allows users to specify a percentage of detected cores to use when the CPU DLL fallback is active, improving flexibility and preventing issues when multiple instances share the same CPU DLL pool. Also adds CPU_GetHardwareConcurrency() for accurate core detection.
The TANH activation's derivative (1-out^2) approaches zero when the output nears ±1, stalling training exactly where convergence to extreme values (e.g., buy/sell signals) is needed. Removing the multiplication by this derivative in the output gradient calculation (both CPU OpenCL and MQL4 paths) prevents this saturation, analogous to using cross-entropy with sigmoid.
Additionally, extend `Save()` and `Load()` to include a new `era` field, enabling tracking of training generations across sessions.
- Define MAX_WEIGHT constant (1.0e6) for weight limits in clusters
- Remove redundant barrier from FeedForward kernel (prevents sync issues)
- Port FeedForwardProof and CalcInputGradientProof kernels for max-pooling (no weights, sliding max)
- Port FeedForwardConv kernel for convolution layers (shared weights, multiple output channels)
- Remove unused code and refactor signal condition logic (CSignalPAI)
