The DFA (Direct Feedback Alignment) option was never a correct implementation:
it deterministically flipped the sign of half of all gradients based on
connection index parity, causing permanent gradient ascent for those weights
and guaranteed divergence. The backward pass was also incompatible with the
OpenCL/DirectML neuron model (layer.Total() == 1). This change removes all DFA
logic, including the enum value and `DfaFeedbackSignal` method, and replaces it
with plain gradient descent in all momentum update kernels. The `optimizer`
kernel argument is retained for binary compatibility but is no longer used.
Add type checks to correctly cast `CNeuronBaseOCL` objects in gradient clipping,
softmax evaluation, and `backProp`/`backPropDfa` loops. Previously, all neurons
were assumed to be `CNeuronBase`, causing invalid pointer casts and incorrect
gradient/output access for OpenCL layers. This ensures proper support for both
CPU and OCL neuron implementations.
Replace dynamic_cast with explicit type checks using obj.Type() and static casts in CaptureOptimizerSnapshot, RestoreOptimizerSnapshot, and SetOptimizerForAllNeurons. This avoids the runtime overhead of dynamic_cast inside critical loops, improving performance when capturing/restoring optimizer snapshots and setting optimizer for all neurons.
Reduce weight decay from 0.01 to 0.001 across all backends (Network.cl, Network.mqh, WarriorCPU.cpp) to fix a training collapse issue. The original 0.01 AdamW default caused discriminative weights to decay below the calibration-capped class-prior offsets, resulting in a monotonically shrinking per-bar logit spread and eventual constant Neutral predictions (argmax degenerated once evidence tilt dropped under the prior tilt). The new value 0.001 lifts the evidence ceiling 10× while still bounding long-run weight growth, restoring effective discrimination. Note: this change must remain in sync across all four backends.
Replace hardcoded lr and momentum with new input variables for Adam and
SGD+momentum. Add OpenCL kernel LSTM_UpdateWeightsMomentum alongside the
existing Adam kernel. Update comments and revert beta1 to book default 0.9.
Replace the absolute thread count input (CpuDllThreads) with a percentage-based CPU_LOAD_PRESET enum (TargetCPULoad). This allows users to specify a percentage of detected cores to use when the CPU DLL fallback is active, improving flexibility and preventing issues when multiple instances share the same CPU DLL pool. Also adds CPU_GetHardwareConcurrency() for accurate core detection.
- Define MAX_WEIGHT constant (1.0e6) for weight limits in clusters
- Remove redundant barrier from FeedForward kernel (prevents sync issues)
- Port FeedForwardProof and CalcInputGradientProof kernels for max-pooling (no weights, sliding max)
- Port FeedForwardConv kernel for convolution layers (shared weights, multiple output channels)
- Remove unused code and refactor signal condition logic (CSignalPAI)
