Reduce weight decay from 0.01 to 0.001 across all backends (Network.cl, Network.mqh, WarriorCPU.cpp) to fix a training collapse issue. The original 0.01 AdamW default caused discriminative weights to decay below the calibration-capped class-prior offsets, resulting in a monotonically shrinking per-bar logit spread and eventual constant Neutral predictions (argmax degenerated once evidence tilt dropped under the prior tilt). The new value 0.001 lifts the evidence ceiling 10× while still bounding long-run weight growth, restoring effective discrimination. Note: this change must remain in sync across all four backends.
The sign-agreement gate in `UpdateWeightsAdam`, `UpdateWeightsConvAdam`, and `LSTM_UpdateWeightsAdam` caused a gradient ratchet effect under one-hot softmax with categorical cross-entropy, leading to an all-Neutral collapse. Removing this gate aligns all four backends (CPU, GPU, OpenCL, MQL) and restores correct gradient flow.
Remove overly verbose explanations in Network.mqh comments and InputEnums.mqh enum values. Shorten descriptions to improve readability without losing essential information.
AI/Network.mqh was 5,805 lines / 18 classes in one file. Investigation
found method implementations for several classes (CNeuronBase/Pool/Conv,
CNeuronBaseOCL) hand-interleaved across thousands of lines - not safe to
split without risky manual reassembly. But 8 classes turned out to be
genuinely self-contained (declaration + every method body physically
contiguous, and only ever depended upon, never depending on anything
declared later): CConnection/CArrayCon, CNeuron, CDirectMLMy (+ its
WarriorDML.dll/WarriorCPU.dll #import blocks), CArrayLayer,
CLayerDescription, CBufferDouble, and CNeuronConvOCL/CNeuronPoolOCL.
Extracted each verbatim, via exact line-range extraction (not manual
retyping) to eliminate transcription risk, into its own AI/*.mqh file,
included from Network.mqh at the exact point each class used to sit -
preserving original declaration order exactly. Mathematically verified
byte-for-byte: reconstructing the original file from the 7 new files'
bodies + Network.mqh's remaining segments is line-for-line identical to
the pre-split git history. Compiled clean (MetaEditor, 0 errors/0
warnings) both before and after.
The remaining tangled classes (CNeuronBase, CNeuronPool, CNeuronConv,
CNet, CNeuronLSTM, CNeuronBaseOCL, CNeuronConvOCL/PoolOCL's shared base,
CNeuronLSTMOCL, COpenCLMy) stay in Network.mqh (now ~4,450 lines) -
splitting those safely needs deliberate per-method surgery, deferred to
a future dedicated pass rather than rushed into this one.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Replace hardcoded lr and momentum with new input variables for Adam and
SGD+momentum. Add OpenCL kernel LSTM_UpdateWeightsMomentum alongside the
existing Adam kernel. Update comments and revert beta1 to book default 0.9.
- Lowered Adam momentum parameter b1 from 0.9 to 0.8, shortening the effective memory window (~5 steps) to reduce multi-era bias streaks observed in training runs.
- Added FOCAL_GAMMA_PRESET enum and corresponding member variable m_focalGamma to apply focal loss modulation (Lin et al. 2017) on top of class-balance weighting, targeting the same streak issue by downweighting already-confident examples.
- Updated comments to document the rationale and experimental nature of both changes.
All __global buffers, local vectors, and constants (MAX_WEIGHT, MIN_ACTIVATION_DERIVATIVE, WEIGHT_DECAY, MAX_WEIGHT_DELTA) now use `float` instead of `double`. This avoids the 1/16th throughput penalty on consumer GPUs (e.g. AMD Polaris/RX 580) while remaining numerically safe: weight/delta clamps stay within ±100, gradients and activations fit float32, and the Adam epsilon guard is not sensitive to underflow. The removed `cl_khr_fp64` directive now causes an explicit compile failure if any double slips back in.
The network uses PReLU (leaky ReLU) activations for all hidden layers, but the previous initialization used LeCun-uniform scaling (1/sqrt(fan_in+1)), which is tuned for saturating activations like tanh/sigmoid. The new He-uniform scaling (sqrt(2/fan_in)) better accounts for the half-zeroing effect of ReLU variants, reducing the risk of early saturation and improving gradient flow during training. Applied uniformly across all layers via the shared Init() method, as the output layer (3 neurons) is negligibly affected and the more precise alternative would require risky threading of activation awareness through every neuron subtype.
- Extend `CNet::getResults` and `CLayer::Load` to support `defNeuronConvOCL`, `defNeuronPoolOCL`, and `defNeuronLSTMOCL` types (previously only `defNeuronBaseOCL` was handled), preventing potential crashes or incorrect outputs with those neuron types.
- Increase `ind_Periods` default from `PERIOD_5` to `PERIOD_20` because `PERIOD_5` was smaller than ADZigZag's `InpDepth=12`, making it structurally impossible for the model to see enough bars to recognize swing pivots, likely causing the erratic Buy/Sell OOS recall observed during training runs.
A freshly-initialized network's weights are plain random init with no
informed prior, so its argmax is close to uniform noise across the 3
classes - on this project's typically heavily Neutral-skewed label
distribution, that means era 0 fires far more spurious Buy/Sell calls
on the very first (oldest) bars than the true base rate warrants,
until enough backProp steps correct it.
The label-cache prebuild already computes the real upfront Buy/Sell/
Neutral tally before era 0 starts. Added CNet::SeedOutputLayerBias()
(AI/Network.mqh) to overwrite just the output layer's bias term (the
per-input weights stay randomly initialized and still carry the real
learning signal) with a fixed +-3.0 push toward whichever class
actually dominates - reaches one layer back from the output layer,
since that's where the weight block producing it is stored. Wired in
at the end of AdvanceLabelCachePrebuild(), fresh-network-only.
The training log showed the model had genuinely collapsed to always-predict-
Neutral: 100+ consecutive eras with Buy/Sell OOS recall flat at 0% and IS
error frozen exactly at 0.14, not a "still early" transient. Root cause is
the 33:1 Buy/Sell-vs-Neutral label imbalance combined with the oversample
replay cap - capped at 3x specifically because more identical back-to-back
backProp() calls fed Adam highly-correlated gradients and caused runaway
momentum (a past incident: OOS accuracy diving from 90%+ to single digits
within ~20 eras). That cap meant minority classes never got enough gradient
influence to matter once the model settled into all-Neutral.
Replaced the N-times replay with a single backProp() call per example, with
its output-layer gradient scaled by inverse class frequency (maxCount/
trueCount from the previous era's true label distribution, uncapped - the
existing MAX_WEIGHT_DELTA per-step clip already bounds how far any single
step can move a weight regardless of gradient magnitude, so there's no
repeated-gradient momentum risk left to cap against).
AI/Network.mqh: CNet::backProp()/backPropOCL() take a new optional
sampleWeight parameter (default 1.0, so every other caller is unaffected).
For the OCL/DirectML path, since WarriorCPU.dll/WarriorDML.dll/Network.cl
have no notion of per-sample weighting, the raw gradient computed by the
native CalcOutputGradient call is read back into MQL5, scaled, and written
back via a new CNeuronBaseOCL::setGradient() before the hidden layers read
it - no changes needed to any of the 3 compute backends themselves.
Also fixed: a run that "converged" at era 63 only because that specific
era's small OOS sample happened to contain zero true Buy/Sell examples
(recall shows n/a and auto-passes the gate when a class is absent from an
era's sample) - the model had already fully collapsed several eras earlier;
this was a lucky/unlucky sampling fluke, not real convergence. Not fixed in
this commit (separate, narrower issue - the gate's n/a auto-pass exists to
avoid deadlocking on a genuinely rare class, and distinguishing that from a
collapsed model needs its own follow-up).
Needs a fresh retrain like the prior structural fixes.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
AI/Network.mqh: CNet::feedForward() was adding sin(i)/cos(i) (alternating by
index parity) directly onto every single input value, on both the CPU and
GPU/DirectML paths - an undocumented, uncontrolled +/-1 offset baked into
already-normalized features that mostly live in a much smaller range. No
comment anywhere explained it, unlike the rest of this codebase; removed.
Expert/ExpertSignalAIBase.mqh, data normalization cleanup:
- Volume relative-change feature is now clamped to +/-5 - unlike the
ATR-normalized price features, it had no ceiling, so a thin previous bar
(e.g. 1 tick) followed by a normal one could feed the network an outlier
input an order of magnitude past every other feature's range.
- Dropped ADCumulativeDelta's raw CumulativeDelta buffer (1) from the feature
set - it's an unbounded, tick-volume-scale running sum, unlike every sibling
buffer in that same indicator (all explicitly clamped +/-2). Pressure
(buffer 0) is this same signal already normalized, so nothing is lost.
- Dropped ADWyckoffEventStream's EventPhase buffer (1) - confirmed via source
it's a byte-for-byte duplicate of EventCode (same underlying variable), not
a distinct phase reading; StructuralPhase (buffer 5) is the real phase
signal and stays.
m_neuronsCount updated accordingly (topology changes, needs a fresh retrain
same as the earlier window-offset fix).
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
TargetCPULoad=max was only ever getting ~half the cores (e.g. 6 of 12)
even outside HYBRID mode, because every CNet (including the live net's
own EMA shadow net, which every signal has) counted as a "peer" that
divided the requested CPU budget.
That division was solving a problem that can't actually happen: MQL5
gives one chart's EA a single execution thread, and every WarriorCPU.dll
entry point blocks that thread until its own ParallelFor() completes,
so only one CNet's worker pool is ever actively computing regardless of
how many are alive (live, shadow, or - under HYBRID - PAI/CONV/LSTM's
own pairs). The idle ones sit blocked on a condvar at ~0% CPU. Dividing
the budget across "peers" that never contend just capped whichever pool
happened to be running at a fraction of the cores actually available.
Removes the now-pointless g_netPeerCount/EffectiveCpuLoadPercent()/
PeerNetworkCount() machinery entirely; SetCpuLoadPercent() now just
takes TargetCPULoad directly.
Two issues compounded to keep the network oscillating for 350+ eras
without ever reaching m_objectiveMet, despite good OOS accuracy:
- CNet::backPropOCL() zeroed the diagnostic error contribution from
any output neuron whose target was exactly 0, unlike backProp()'s
CPU-fallback path. For the one-hot 3-class classification head this
meant dError only ever reflected the true class's own calibration,
never whether the other two classes were correctly suppressed -
purely a reporting bug, the real gradient (CPU_CalcOutputGradient
in WarriorCPU.cpp) was already correct and unaffected.
- Train()'s convergence gate required dError < 0.1 unconditionally.
That RMS-error floor is reachable for the single-neuron regression
head, but for 3 one-hot outputs it demands near-perfect confident
calibration on every bar - unreachable under normal market label
noise, and redundant with the classification-native gates already
in place (dOosForecast >= target, per-class directionalRecallOK).
Now skipped when m_outputNeuronsCount == 3.
- Replace the hand-rolled fractal/deviation-%/ATR-trend-context ZigZag
approximation with the actual MQL5 ZigZag indicator (rebranded as
CustomIndicators/ADZigZag.mq5, logic untouched) as the training label
source; bump the settle/confirmation window from 20 to 100 bars so a
proper leg can form before being trusted, and drop the now-dead
ZigZagDeviationPct/MinTrendATRMultiple/TrendContextBars inputs.
- Rebrand the stock Volumes indicator the same way (ADVolume.mq5).
- Fix HYBRID mode (AIType=HYBRID) oversubscribing the CPU fallback tier:
every concurrent CNet instance was independently sizing its worker
pool off the same global TargetCPULoad input. g_netPeerCount/
PeerNetworkCount() now split it across however many CNet instances
(live+shadow x active signals) are actually sharing the CPU.
- Fix NEURONS_REDUCTION_FACTOR's confusing retention-vs-reduction
semantics so RF_70 means an actual 70% reduction; default to 4 hidden
layers / RF_70.
- Migrate remaining free-form training/indicator inputs to enums for UI
consistency; default news filter lookback to 1h, disable every-tick.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
- Tighten MAX_WEIGHT from 1.0e6 to 100.0 to prevent unbounded weight growth.
- Add MIN_ACTIVATION_DERIVATIVE (1e-4) to avoid zero gradients for saturated units.
- Introduce WEIGHT_DECAY (0.01) to decouple decay from Adam updates.
- Add MAX_WEIGHT_DELTA (0.1) to clamp per-step Adam updates and prevent overshoot.
- Apply sign-agreement gate on weight updates in Adam kernel to only apply steps aligned with current gradient direction.
- Fix tanh/sigmoid derivative calculations to use the new floor instead of hard-coded edge-case values.
- In CaclOutputGradient kernel:
- Case 1 (sigmoid classification): removed erroneous multiplication by out*(1-out) which dampened gradients – the binary cross-entropy loss already cancels the sigmoid derivative, so direct (target-out) is correct.
- Added default case for NONE activation (softmax classification) to compute plain error (target-out); previously unhandled, resulting in zero gradients that froze the entire network when OpenCL was active.
- In UpdateWeightsMomentum and UpdateWeightsAdam kernels: added clamp to MAX_WEIGHT when updating weights to prevent gradient spikes from producing ±Infinity and subsequent NaN propagation through dense layers (e.g., classification output head).
Warrior_EA.mq5: clear Comment() and destroy the control panel before
Expert.Deinit()'s object-purge cascade runs out from under it; stop
re-registering the same signal filters on every DB retry (was causing a
double-delete of the same pointer on shutdown).
DirectML/WarriorCPU.cpp: bound the worker-thread join in ThreadPool::Stop()
instead of blocking forever - CPU_Shutdown() held g_mutex across an unbounded
join, so a watchdog-killed calling thread could leave it locked forever,
poisoning every future call into the DLL (matches reports of the EA getting
stuck on "initializing" after being removed and re-added to a chart).
Also includes prior era-0 label-cache prebuild and pullback/reversal
label-quality work in AI/Network.mqh, Expert/ExpertSignalAIBase.mqh, and
Variables/Inputs.mqh.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Replace the absolute thread count input (CpuDllThreads) with a percentage-based CPU_LOAD_PRESET enum (TargetCPULoad). This allows users to specify a percentage of detected cores to use when the CPU DLL fallback is active, improving flexibility and preventing issues when multiple instances share the same CPU DLL pool. Also adds CPU_GetHardwareConcurrency() for accurate core detection.
MQL5 refuses to compile `#resource` directives pointing to PE executables ("unsupported resource type ... executable file prohibited"). This commit removes the `#resource` lines and the `ExtractComputeDlls()` function from `AI/Network.mqh`, and updates comments in `Warrior_EA.mq5` to explain that the DLLs must be manually placed in `MQL5\Libraries\`. The call to `ExtractComputeDlls()` in `OnInit()` is also removed.
Add #resource directives for WarriorDML.dll and WarriorCPU.dll, and ExtractComputeDlls() to extract them on first run, enabling MQL Cloud Protector builds to ship DLLs without manual copy. Extend CNet::Save and Load with trainingComplete and indicatorParams arrays to persist AutoTune indicator param values.
The TANH activation's derivative (1-out^2) approaches zero when the output nears ±1, stalling training exactly where convergence to extreme values (e.g., buy/sell signals) is needed. Removing the multiplication by this derivative in the output gradient calculation (both CPU OpenCL and MQL4 paths) prevents this saturation, analogous to using cross-entropy with sigmoid.
Additionally, extend `Save()` and `Load()` to include a new `era` field, enabling tracking of training generations across sessions.
- Define MAX_WEIGHT constant (1.0e6) for weight limits in clusters
- Remove redundant barrier from FeedForward kernel (prevents sync issues)
- Port FeedForwardProof and CalcInputGradientProof kernels for max-pooling (no weights, sliding max)
- Port FeedForwardConv kernel for convolution layers (shared weights, multiple output channels)
- Remove unused code and refactor signal condition logic (CSignalPAI)
- Updated README.md with project overview, key features, directory structure, getting started guide, and modernization roadmap.
- Added AI_NETWORK.md detailing the neural network and AI/ML infrastructure, including architecture, components, usage patterns, and next steps.
- Introduced DATABASE.md for the Database module, outlining key components, design highlights, usage patterns, and future enhancements.
- Created README.md files for Enumerations, Expert, Money, Signals, Structures, System, Trailing, Variables directories, detailing their purpose, key components, and integration notes.
- Documented the Signals subsystem, emphasizing modularity, extensibility, and AI/ML readiness.
- Added comprehensive descriptions for individual signal modules in Signals/ directory.
- Established clear integration notes and recommendations for future improvements across all modules.
