- BufferDouble: replace hardcoded "DirectML/CPU-DLL" with dynamic backend name
and add buffer index/element count to all error prints for easier debugging.
- NetPersistence: distinguish missing file from transient lock by probing
FileIsExist before logging, eliminating false "sharing violation" warnings
when no saved model exists on first run.
The per-step entry points cannot express a sequence model. CPU_LSTMGates takes
the ENTIRE flattened input as one timestep, and CPU_LSTMGateGradient has no
parameter for dc arriving from the following step - so the recurrent gradient
path does not exist and cannot be assembled from these primitives at any call
pattern. The layer built on them is a gated dense layer that the class comment
already described honestly: "single-timestep-truncated BPTT".
Adds CPU_LSTMSeqForward / CPU_LSTMSeqBackward: the whole unrolled sequence in
one call each, weights shared across timesteps, dW accumulated over all of them
(the per-step CPU_LSTMWeightsGradient assigns rather than accumulates, so it
could not have been reused even with the dc term). Fused rather than dispatched
per step because the recurrence is sequential - T round trips would serialise T
lock/dispatch pairs for a few thousand FLOPs each.
h_{-1} and c_{-1} are zero per sample. The old layer carried its cell state
across forward passes, so under shuffled training every sample inherited the
state of an unrelated one.
DirectML gets the same math host-side (readback, compute in double, upload)
rather than HLSL: the recurrence needs a barrier per timestep, the GPU buffers
are float and BPTT accumulation is where that hurts most, and no D3D12 device
exists on this machine to test a shader against. Documented at the definition.
Verified with lstm_seq_gradcheck.cpp - central-difference check of dW and dX
against an asymmetric loss over the final hidden state. Max relative error
2.3e-10 on both, with a non-trivial gradient magnitude asserted so the check
cannot pass on an all-zero result.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The DFA (Direct Feedback Alignment) option was never a correct implementation:
it deterministically flipped the sign of half of all gradients based on
connection index parity, causing permanent gradient ascent for those weights
and guaranteed divergence. The backward pass was also incompatible with the
OpenCL/DirectML neuron model (layer.Total() == 1). This change removes all DFA
logic, including the enum value and `DfaFeedbackSignal` method, and replaces it
with plain gradient descent in all momentum update kernels. The `optimizer`
kernel argument is retained for binary compatibility but is no longer used.
Add type checks to correctly cast `CNeuronBaseOCL` objects in gradient clipping,
softmax evaluation, and `backProp`/`backPropDfa` loops. Previously, all neurons
were assumed to be `CNeuronBase`, causing invalid pointer casts and incorrect
gradient/output access for OpenCL layers. This ensures proper support for both
CPU and OCL neuron implementations.
Replace dynamic_cast with explicit type checks using obj.Type() and static casts in CaptureOptimizerSnapshot, RestoreOptimizerSnapshot, and SetOptimizerForAllNeurons. This avoids the runtime overhead of dynamic_cast inside critical loops, improving performance when capturing/restoring optimizer snapshots and setting optimizer for all neurons.
Reduce weight decay from 0.01 to 0.001 across all backends (Network.cl, Network.mqh, WarriorCPU.cpp) to fix a training collapse issue. The original 0.01 AdamW default caused discriminative weights to decay below the calibration-capped class-prior offsets, resulting in a monotonically shrinking per-bar logit spread and eventual constant Neutral predictions (argmax degenerated once evidence tilt dropped under the prior tilt). The new value 0.001 lifts the evidence ceiling 10× while still bounding long-run weight growth, restoring effective discrimination. Note: this change must remain in sync across all four backends.
The sign-agreement gate in `UpdateWeightsAdam`, `UpdateWeightsConvAdam`, and `LSTM_UpdateWeightsAdam` caused a gradient ratchet effect under one-hot softmax with categorical cross-entropy, leading to an all-Neutral collapse. Removing this gate aligns all four backends (CPU, GPU, OpenCL, MQL) and restores correct gradient flow.
Replace hardcoded lr and momentum with new input variables for Adam and
SGD+momentum. Add OpenCL kernel LSTM_UpdateWeightsMomentum alongside the
existing Adam kernel. Update comments and revert beta1 to book default 0.9.
- Ensure `tick_volume` array is set as series in ADShorteningOfThrust.mq5 to prevent future-data leak in volume calculations.
- Ensure `open` array is set as series in ADWyckoffFailedStructure.mq5 to prevent future-data leak in structure detection.
- Add missing PReLU gradient scaling (multiply by 0.01 for negative outputs) in CPU_CalcHiddenGradient and DirectML shader to match expected derivative behavior across all backends.
- Add inline threshold (512) in WarriorCPU to avoid thread-pool overhead for small dispatches; run small workloads inline on the calling thread.
- Reduce training time budget from 500ms to 120ms in ExpertSignalAIBase to keep the UI reactive while training.
- Tighten MAX_WEIGHT from 1.0e6 to 100.0 to prevent unbounded weight growth.
- Add MIN_ACTIVATION_DERIVATIVE (1e-4) to avoid zero gradients for saturated units.
- Introduce WEIGHT_DECAY (0.01) to decouple decay from Adam updates.
- Add MAX_WEIGHT_DELTA (0.1) to clamp per-step Adam updates and prevent overshoot.
- Apply sign-agreement gate on weight updates in Adam kernel to only apply steps aligned with current gradient direction.
- Fix tanh/sigmoid derivative calculations to use the new floor instead of hard-coded edge-case values.
- In CaclOutputGradient kernel:
- Case 1 (sigmoid classification): removed erroneous multiplication by out*(1-out) which dampened gradients – the binary cross-entropy loss already cancels the sigmoid derivative, so direct (target-out) is correct.
- Added default case for NONE activation (softmax classification) to compute plain error (target-out); previously unhandled, resulting in zero gradients that froze the entire network when OpenCL was active.
- In UpdateWeightsMomentum and UpdateWeightsAdam kernels: added clamp to MAX_WEIGHT when updating weights to prevent gradient spikes from producing ±Infinity and subsequent NaN propagation through dense layers (e.g., classification output head).
Warrior_EA.mq5: clear Comment() and destroy the control panel before
Expert.Deinit()'s object-purge cascade runs out from under it; stop
re-registering the same signal filters on every DB retry (was causing a
double-delete of the same pointer on shutdown).
DirectML/WarriorCPU.cpp: bound the worker-thread join in ThreadPool::Stop()
instead of blocking forever - CPU_Shutdown() held g_mutex across an unbounded
join, so a watchdog-killed calling thread could leave it locked forever,
poisoning every future call into the DLL (matches reports of the EA getting
stuck on "initializing" after being removed and re-added to a chart).
Also includes prior era-0 label-cache prebuild and pullback/reversal
label-quality work in AI/Network.mqh, Expert/ExpertSignalAIBase.mqh, and
Variables/Inputs.mqh.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Replace the absolute thread count input (CpuDllThreads) with a percentage-based CPU_LOAD_PRESET enum (TargetCPULoad). This allows users to specify a percentage of detected cores to use when the CPU DLL fallback is active, improving flexibility and preventing issues when multiple instances share the same CPU DLL pool. Also adds CPU_GetHardwareConcurrency() for accurate core detection.
The TANH activation's derivative (1-out^2) approaches zero when the output nears ±1, stalling training exactly where convergence to extreme values (e.g., buy/sell signals) is needed. Removing the multiplication by this derivative in the output gradient calculation (both CPU OpenCL and MQL4 paths) prevents this saturation, analogous to using cross-entropy with sigmoid.
Additionally, extend `Save()` and `Load()` to include a new `era` field, enabling tracking of training generations across sessions.
- Define MAX_WEIGHT constant (1.0e6) for weight limits in clusters
- Remove redundant barrier from FeedForward kernel (prevents sync issues)
- Port FeedForwardProof and CalcInputGradientProof kernels for max-pooling (no weights, sliding max)
- Port FeedForwardConv kernel for convolution layers (shared weights, multiple output channels)
- Remove unused code and refactor signal condition logic (CSignalPAI)
