The DFA (Direct Feedback Alignment) option was never a correct implementation:
it deterministically flipped the sign of half of all gradients based on
connection index parity, causing permanent gradient ascent for those weights
and guaranteed divergence. The backward pass was also incompatible with the
OpenCL/DirectML neuron model (layer.Total() == 1). This change removes all DFA
logic, including the enum value and `DfaFeedbackSignal` method, and replaces it
with plain gradient descent in all momentum update kernels. The `optimizer`
kernel argument is retained for binary compatibility but is no longer used.
Introduce an `optimizer` parameter to UpdateWeightsMomentum, UpdateWeightsConvMomentum, and UpdateWeightsAdam kernels. When set to a non-zero value, the gradient used for weight updates is multiplied by ±1 based on the parity of the weight index, implementing a basic feedback alignment signal for experimentation. When zero, the standard gradient is used unchanged. This allows A/B testing of alternative learning signals without modifying the rest of the training pipeline.
Raise MIN_ACTIVATION_DERIVATIVE from 1e-4 to 1e-3 in both Network.cl and Network.mqh
to strengthen the escape signal through saturated hidden neurons. This provides a 10x
stronger safety net against fp32 OpenCL-specific saturation, complementing the earlier
output-layer fix (logit activation instead of sigmoid) that resolved the primary neutral
collapse bug.
Add MathSrand(GetTickCount()) before BuildFreshTopology() in ExpertSignalAIBase.mqh
to guarantee genuinely random weight initialization after genetic tuner evaluations,
matching the final-retrain path and Warrior_EA.mq5's OnInit. This prevents the previous
deterministic RNG state from dominating weight init.
Also remove UTF‑8 BOM from ExpertSignalAIBase.mqh and Network.mqh for cleaner encoding.
Reduce weight decay from 0.01 to 0.001 across all backends (Network.cl, Network.mqh, WarriorCPU.cpp) to fix a training collapse issue. The original 0.01 AdamW default caused discriminative weights to decay below the calibration-capped class-prior offsets, resulting in a monotonically shrinking per-bar logit spread and eventual constant Neutral predictions (argmax degenerated once evidence tilt dropped under the prior tilt). The new value 0.001 lifts the evidence ceiling 10× while still bounding long-run weight growth, restoring effective discrimination. Note: this change must remain in sync across all four backends.
The sign-agreement gate in `UpdateWeightsAdam`, `UpdateWeightsConvAdam`, and `LSTM_UpdateWeightsAdam` caused a gradient ratchet effect under one-hot softmax with categorical cross-entropy, leading to an all-Neutral collapse. Removing this gate aligns all four backends (CPU, GPU, OpenCL, MQL) and restores correct gradient flow.
Replace hardcoded lr and momentum with new input variables for Adam and
SGD+momentum. Add OpenCL kernel LSTM_UpdateWeightsMomentum alongside the
existing Adam kernel. Update comments and revert beta1 to book default 0.9.
All __global buffers, local vectors, and constants (MAX_WEIGHT, MIN_ACTIVATION_DERIVATIVE, WEIGHT_DECAY, MAX_WEIGHT_DELTA) now use `float` instead of `double`. This avoids the 1/16th throughput penalty on consumer GPUs (e.g. AMD Polaris/RX 580) while remaining numerically safe: weight/delta clamps stay within ±100, gradients and activations fit float32, and the Adam epsilon guard is not sensitive to underflow. The removed `cl_khr_fp64` directive now causes an explicit compile failure if any double slips back in.
- Tighten MAX_WEIGHT from 1.0e6 to 100.0 to prevent unbounded weight growth.
- Add MIN_ACTIVATION_DERIVATIVE (1e-4) to avoid zero gradients for saturated units.
- Introduce WEIGHT_DECAY (0.01) to decouple decay from Adam updates.
- Add MAX_WEIGHT_DELTA (0.1) to clamp per-step Adam updates and prevent overshoot.
- Apply sign-agreement gate on weight updates in Adam kernel to only apply steps aligned with current gradient direction.
- Fix tanh/sigmoid derivative calculations to use the new floor instead of hard-coded edge-case values.
- In CaclOutputGradient kernel:
- Case 1 (sigmoid classification): removed erroneous multiplication by out*(1-out) which dampened gradients – the binary cross-entropy loss already cancels the sigmoid derivative, so direct (target-out) is correct.
- Added default case for NONE activation (softmax classification) to compute plain error (target-out); previously unhandled, resulting in zero gradients that froze the entire network when OpenCL was active.
- In UpdateWeightsMomentum and UpdateWeightsAdam kernels: added clamp to MAX_WEIGHT when updating weights to prevent gradient spikes from producing ±Infinity and subsequent NaN propagation through dense layers (e.g., classification output head).
Warrior_EA.mq5: clear Comment() and destroy the control panel before
Expert.Deinit()'s object-purge cascade runs out from under it; stop
re-registering the same signal filters on every DB retry (was causing a
double-delete of the same pointer on shutdown).
DirectML/WarriorCPU.cpp: bound the worker-thread join in ThreadPool::Stop()
instead of blocking forever - CPU_Shutdown() held g_mutex across an unbounded
join, so a watchdog-killed calling thread could leave it locked forever,
poisoning every future call into the DLL (matches reports of the EA getting
stuck on "initializing" after being removed and re-added to a chart).
Also includes prior era-0 label-cache prebuild and pullback/reversal
label-quality work in AI/Network.mqh, Expert/ExpertSignalAIBase.mqh, and
Variables/Inputs.mqh.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
The TANH activation's derivative (1-out^2) approaches zero when the output nears ±1, stalling training exactly where convergence to extreme values (e.g., buy/sell signals) is needed. Removing the multiplication by this derivative in the output gradient calculation (both CPU OpenCL and MQL4 paths) prevents this saturation, analogous to using cross-entropy with sigmoid.
Additionally, extend `Save()` and `Load()` to include a new `era` field, enabling tracking of training generations across sessions.
- Define MAX_WEIGHT constant (1.0e6) for weight limits in clusters
- Remove redundant barrier from FeedForward kernel (prevents sync issues)
- Port FeedForwardProof and CalcInputGradientProof kernels for max-pooling (no weights, sliding max)
- Port FeedForwardConv kernel for convolution layers (shared weights, multiple output channels)
- Remove unused code and refactor signal condition logic (CSignalPAI)
