Menon et al. 2021 (ICLR), "Long-tail learning via logit adjustment": add
tau*log(prior_c) to each class logit inside the training gradient. Softmax
CE on adjusted logits is consistent for BALANCED error - the metric
checkpoint selection already ranks on - so the loss and the deploy decision
finally optimize the same thing.
The engine already computed a true softmax + categorical-CE gradient and
wrote it over the per-neuron sigmoid delta, so this is an offset added to
three logits in the two places that gradient is built (backProp scalar path
and backPropOCL). No backend, kernel or DLL change; the forward pass and
every inference path are untouched, which is the point - the network learns
to absorb the offset, so its raw argmax becomes the balanced-optimal
decision with nothing applied at inference.
Replaces rather than stacks. Minority replay is disabled while this is on,
and the post-hoc inference prior is forced off. Stacking is not a
theoretical worry: simulated on the measured 1118/1119/34298 distribution
in the weak-signal regime, plain CE collapses to Neutral (33.4% balanced,
Buy 0%), replay reaches 48.1%, logit adjustment 50.9% with better balance -
and BOTH together score 45.4% with Neutral recall at 0%, worse than either
alone. Buda et al. 2018 predicts exactly that.
Motivation from the six-chart run: every topology took one direction to
~50% recall and abandoned the other, the direction chosen arbitrarily (the
batch-norm control went Buy 1% / Sell 42%, the inverse of the other five).
One era in 1,301 cleared the per-class recall floor.
Fingerprinted conditionally, so the converged 60.7% models on disk keep
their filenames and stay loadable as the fallback.
Both builds compile 0 errors, 0 warnings.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Dividing a machine budget by the live chart count was wrong twice over.
The count is a snapshot taken when each net's pool is built, and charts
attach one at a time: five charts measured 10/6/5/4/4% of the same budget,
because the first only ever saw itself and the last saw all five. So the
earliest chart got several times the threads of the latest - skewing any
cross-topology comparison run on those charts, which is the exact thing
the setting existed to make fair. Nothing rebalanced afterwards either,
and rebalancing would mean tearing down a DLL context under a live trainer.
Both problems disappear once the answer stops depending on how many charts
are running. Each net now asks for a fixed 2 worker threads, converted to
the percentage the DLL wants from the detected core count.
Two is not a compromise: since the topology became data-derived the widest
dense layer is 64 units, so each ParallelFor has almost nothing to split
and per-dispatch overhead dominates. An MLP era cost ~66s at a wildly
oversubscribed 12 threads and ~80s at 1 thread - a 20% spread across a 12x
difference in thread count. Two per net also lands six concurrent charts
exactly on a 12-core box.
Removing the input costs nothing on the product side: a Market build has no
DLL tier at all, so it was already compiled out to a constant there and no
buyer could reach it.
Both builds compile 0 errors, 0 warnings.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The CPU-DLL thread pool was sized from TargetCPULoad undivided, on the
reasoning that only one pool is ever actively computing at a time. That is
true WITHIN a chart - MQL5 gives one chart's EA a single execution thread,
and every WarriorCPU.dll entry point blocks it until its ParallelFor()
completes, so the live net, the EMA shadow and HYBRID's fused pair take
turns. It does not hold ACROSS charts, which each get their own execution
thread and really do run their pools simultaneously.
At the 100% default on a 12-core box, five training charts asked for 12
threads each: 60 threads contending for 12 cores. Measured today, dropping
to ~2 threads apiece made every chart train "super fast". This had
previously been read as one architecture being mysteriously 10x slower than
another on the CPU-DLL tier while identical on OpenCL - oversubscription of
that degree degrades superlinearly and punishes whichever model issues the
most small sequential dispatches, which fits an MLP being the victim.
TargetCPULoad now means the budget for the whole machine, divided by the
number of charts running this EA. Counting charts is the correct axis:
concurrency here is one execution thread per chart, not one per CNet, and
the within-chart division that was previously removed stays removed.
Snapshot at pool-creation time on purpose - attaching another chart later
does not resize pools that already exist, because that would mean tearing
down a DLL context underneath a live training run. Skipped entirely in the
tester/optimizer, where the terminal already pins one strategy per agent.
This matters most for buyers: a Market build compiles the input out and
pins it to 100%, so they cannot reach the setting at all and would have hit
the pathological case with no way to diagnose or fix it.
The tier log line now reports the split rather than just the result.
Compiles 0 errors, 0 warnings.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
With normalization enabled a forward pass is not a pure function of its
input - it also advances the running mean/variance. ValidateCpuInference
compares the live backend net against a throwaway pure-MQL5 clone loaded
from the just-saved .nnw, so its own reference pass left the live model one
EMA step ahead of the file the clone reads. The check would then have been
measuring its own side effect, and a marginal result decides whether
buyers' backtests are allowed to run DLL-free.
Adds CNet::SetBatchNormFrozen / CNeuronBatchNormOCL::SetStatsFrozen -
classic batch-norm inference semantics, statistics used but not updated -
and freezes both sides for the duration of the comparison. Not persisted:
it is a transient evaluation mode, not model state. Default stays
adaptive, which is what the rest of the system (online continual learning)
is built around.
Compiles 0 errors, 0 warnings.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The only bounded stage in the entire forward path was the sigmoid
classification head - every hidden stage is PRELU. That is a network with
no internal scale control, and the failure ordered exactly by depth: on
SP500 H1 the shallow perceptron held ~52% balanced accuracy while the
deepest topology sat on the 33.3% one-class floor, with the per-bar logit
spread decaying monotonically (0.45 -> 0.38 over ~200 eras) until the
evidence tilt fell under the class-prior tilt. That is the signature of
internal covariate shift, which chapter 6.1 of the reference book is
entirely about and which the NeuroNet_DNG engine addresses with a layer
this project never had.
Two mechanisms make this the right fix rather than more hyperparameter
nudging:
- it decouples WEIGHT_DECAY from the learned function (van Laarhoven
2017) - with a normalized layer downstream, decay can no longer grind
the discriminative signal away, it only rescales the effective
learning rate;
- it is the precondition for ever running an unbounded logit head here.
The 2026-07-27 attempt blew up (IS error 5.6e15) precisely because
nothing upstream constrained scale.
Implementation notes:
- CNeuronBatchNormOCL computes host-side rather than as a fourth copy of
a kernel across Network.cl + WarriorCPU.cpp + WarriorDML.cpp. The math
is elementwise O(n); this way it behaves identically on all four
compute tiers, needs no DLL rebuild, and cannot drift between
backends. Same precedent as the softmax+CCE gradient and the
per-sample loss weighting, both computed in MQL5 for that reason.
- Statistics are exponential moving, not a stored mini-batch: training
is pure online SGD, one update per sample, so there is no batch to
average over. BatchNormWindow is an EMA window length.
- gamma/beta are excluded from weight decay, deliberately - decaying
gamma toward zero is the exact pathology being fixed.
- The layer self-sizes from whatever sits below it, because a conv/pool
stage's output width is derived inside the CNet constructor and is not
knowable to the topology builder.
- Checkpoint capture/restore/blend carry gamma/beta and the running
statistics alongside the dense matrix, so the plateau ladder cannot
restore a mismatched pair.
- SeedOutputLayerBias accepted only an exact defNeuronBaseOCL as the
weight-carrying penultimate layer; with normalization enabled that is
the batch-norm layer, so the cold-start bias seed would have silently
stopped being applied.
- Refuses to build, loudly, if a topology asks for normalization with no
compute backend at all - rather than quietly training a different
architecture than the one requested.
EnableBatchNorm (default on) and BatchNormWindow (1000 samples) are
inputs so the effect can be A/B'd without a recompile. Both feed the
weights-filename fingerprint, appended conditionally so existing non-BN
configs keep their fingerprints and are not forced to retrain.
Verified: analytic gradients match finite differences to 1.5e-7 relative
over 200 random cases; a faithful port of the full forward/backward chain
collapses to the 33.3% floor by era 4 without this layer and holds
36-43% with it. Compiles 0 errors, 0 warnings.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
A .nnw persists the ARCHITECTURE, not just the weights: Save writes
(int)activation per neuron and Load reads it straight back. The activation
chosen in BuildFreshTopology() therefore only ever reached a brand-new
topology - every reload restored the file's value and the next save wrote it
back out, so a wrong value could never heal while the source read as though
it were already fixed.
That is how five models kept training with an unbounded NONE classification
head for a full day after the 07-28 revert to SIGMOID. Confirmed by parsing
the binaries: 848cb42c.nnw / 2e754b43.nnw carry `act=NONE` on the 3-neuron
output layer, while a genuinely reset model of the same config carries
act=SIGMOID. In the log it showed as negative "OOS raw out" values -
impossible under sigmoid - escalating to a 4.14e13 logit spread with all
three classes numerically identical (input-independent output) and balanced
accuracy pinned on the 33.3% one-class floor.
- OutputLayerActivation() is now the single source of truth, called by both
BuildFreshTopology() and the new load-time repair, so the two can no
longer diverge the way a duplicated literal did.
- CNet::EnforceOutputActivation() re-asserts it after Load and reports the
stale value; CExpertSignalAIBase::EnforceTopologyContract() logs the
repair loudly, since weights learned under the old head may not be worth
keeping even once the head is corrected.
- Hidden layers are deliberately left alone: they legitimately differ per
stage (PRELU dense/conv, NONE pool, TANH LSTM).
Compiles 0 errors, 0 warnings.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
DRY - topology construction
---------------------------
CSignalCONV and CSignalHYBRID each built the Conv+Pool front-end from scratch;
CSignalLSTM and CSignalHYBRID each built the LSTM stage from scratch. The
duplicates had already drifted: HYBRID guarded the LSTM step with
MathMax(1, historyBars/2), CSignalLSTM divided unguarded, so a historyBars of 1
gave two different steps for what is documented as the same layer.
Extracted AddConvPoolStage() and AddLstmStage() onto CExpertSignalAIBase. The
three overrides are now compositions:
CONV = AddConvPoolStage
LSTM = AddLstmStage
HYBRID = AddConvPoolStage && AddLstmStage
HYBRID's "matches the standalone CONV front-end exactly, then adds LSTM" is
enforced by construction instead of by comment. Took the guarded step for both.
Also fixed a descriptor leak the duplicates shared: on a failed topology.Add()
the CLayerDescription was neither owned by the array nor deleted.
Dead code
---------
- CNet::SaveCheckpoint / CNet::LoadCheckpoint (123 lines). Superseded by the
in-memory CaptureWeights/RestoreWeights pair; Network.mqh:1312 already said so
("This replaces the file-based SaveCheckpoint/LoadCheckpoint"). Zero call
sites - every remaining mention was a comment. The five comments that
referenced them have been reworded rather than left dangling.
- CExpertSignalCustom::CheckForDuplicateTrade / FindLastTradeIndex /
UpdateTradeStatusAndExit: declared, never defined anywhere, never called.
They only made it look as though duplicate-trade detection existed.
Compiles 0 errors, 0 warnings.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
FileOpen(FILE_WRITE) truncates its target on open. CNet::Save already staged
the .nnw through a temp file + rename for that reason, but the three sidecars
written beside it did not:
.stats ExpertSignalAIBase.mqh:5918
.arrows ExpertSignalAIBase.mqh:6224
.cfg ExpertSignalAIBase.mqh:7329
Two defects followed.
1. An interrupted write published a truncated sidecar. For .cfg that is the
worst case: LoadAndCompareTopologyConfiguration() reads a short file as a
mismatch, which discards the trained model and restarts from era 0.
2. Windows file sharing is a mutual contract - a writer opened with no
FILE_SHARE_* blocks every concurrent open regardless of the reader's flags.
All three read paths carry FILE_SHARE_READ|FILE_SHARE_WRITE specifically so
a tester agent can read them while a live chart runs; an exclusive writer on
the same path defeated that.
Extracted CNet::Save's proven pattern into System\AtomicFile.mqh
(AtomicWriteBegin/AtomicWriteEnd) and routed all four writers through it. This
also encodes the FileMove gotcha once instead of per call site: the destination
location comes from FILE_COMMON inside the 4th arg, NOT inherited from the
source, and getting it wrong moves the file to the wrong sandbox silently.
Also fixed while in these functions:
- SaveTopologyConfiguration had 13 copy-pasted 6-line error blocks that each
returned WITHOUT FileClose(handle), leaking the handle on every write
failure. Collapsed to one ok-chain that closes exactly once. The on-disk
field order and types are unchanged (asserted during the rewrite) so existing
.cfg files still load.
- SaveChartSignals documented that pruning runs only after a successful write
("a failed write above leaves both the file AND the chart untouched") but
never checked any write result, so a partial write still deleted the chart
objects. Results are checked now, making the existing comment true.
Compiles 0 errors, 0 warnings.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The DFA (Direct Feedback Alignment) option was never a correct implementation:
it deterministically flipped the sign of half of all gradients based on
connection index parity, causing permanent gradient ascent for those weights
and guaranteed divergence. The backward pass was also incompatible with the
OpenCL/DirectML neuron model (layer.Total() == 1). This change removes all DFA
logic, including the enum value and `DfaFeedbackSignal` method, and replaces it
with plain gradient descent in all momentum update kernels. The `optimizer`
kernel argument is retained for binary compatibility but is no longer used.
The DFA optimizer was never real Direct Feedback Alignment — it flipped
gradient signs for half of each weight tensor deterministically, causing
permanent gradient ascent and divergence. Its backward pass was also
incompatible with the OpenCL/DirectML layer model. Remove the enum
entry, all related logic in the OpenCL kernel weight update functions,
and the `DfaFeedbackSignal` method. The `optimizer` kernel argument is
retained (unused) to preserve binary compatibility with existing DLLs.
Add type checks to correctly cast `CNeuronBaseOCL` objects in gradient clipping,
softmax evaluation, and `backProp`/`backPropDfa` loops. Previously, all neurons
were assumed to be `CNeuronBase`, causing invalid pointer casts and incorrect
gradient/output access for OpenCL layers. This ensures proper support for both
CPU and OCL neuron implementations.
Replace dynamic_cast with explicit type checks using obj.Type() and static casts in CaptureOptimizerSnapshot, RestoreOptimizerSnapshot, and SetOptimizerForAllNeurons. This avoids the runtime overhead of dynamic_cast inside critical loops, improving performance when capturing/restoring optimizer snapshots and setting optimizer for all neurons.
Changed updateInputWeights methods to accept pointer arguments by value (e.g., CLayer* instead of CLayer*&) to prevent unintended pointer reassignment and improve safety. Added overloads in CNeuronBase and CNeuronBaseOCL to support both generic CObject* and derived type pointers, unifying the virtual interface and resolving inconsistencies in network weight update logic.
Changed updateInputWeights signature from accepting `CObject*` to `CObject*&` to allow the method to modify the caller's pointer, preventing potential object copying or pointer invalidation. A large block of outdated commentary in ExpertSignalAIBase.mqh was also removed, cleaning up documentation no longer applicable after previous refactoring.
Introduce Direct Feedback Alignment (DFA) backward pass with gradient clipping, feedback matrix initialization, and a dedicated backPropDfa method. Add optimizer snapshot/restore hooks (CaptureOptimizerSnapshot, RestoreOptimizerSnapshot, SetOptimizerForAllNeurons) to temporarily switch the entire network's optimizer for replay-only updates during pass 2, preserving the original optimizer state. Support all neuron types including dropout, deconv, LSTM, and softmax in the snapshot logic.
Introduce an `optimizer` parameter to UpdateWeightsMomentum, UpdateWeightsConvMomentum, and UpdateWeightsAdam kernels. When set to a non-zero value, the gradient used for weight updates is multiplied by ±1 based on the parity of the weight index, implementing a basic feedback alignment signal for experimentation. When zero, the standard gradient is used unchanged. This allows A/B testing of alternative learning signals without modifying the rest of the training pipeline.
In AI/Network.mqh, return early from InitDirectML during
tester/optimization/forward runs to prevent agent-side file-lock
failures caused by rapid stop/restart cycles accessing DLL imports.
In Expert/ExpertSignalAIBase.mqh, add MathIsValidNumber checks in
CalibratedConfidenceMagnitude and SignaledConfidence to safely handle
NaN values, and refactor ShutdownChartCleanup to accept a preserve
flag, avoiding unnecessary chart purges during tester runs for faster
shutdowns. Also add m_purgeChartOnDestruct member.
In AI/NeuronDirectML.mqh, clean up a minor comment formatting issue.
Raise MIN_ACTIVATION_DERIVATIVE from 1e-4 to 1e-3 in both Network.cl and Network.mqh
to strengthen the escape signal through saturated hidden neurons. This provides a 10x
stronger safety net against fp32 OpenCL-specific saturation, complementing the earlier
output-layer fix (logit activation instead of sigmoid) that resolved the primary neutral
collapse bug.
Add MathSrand(GetTickCount()) before BuildFreshTopology() in ExpertSignalAIBase.mqh
to guarantee genuinely random weight initialization after genetic tuner evaluations,
matching the final-retrain path and Warrior_EA.mq5's OnInit. This prevents the previous
deterministic RNG state from dominating weight init.
Also remove UTF‑8 BOM from ExpertSignalAIBase.mqh and Network.mqh for cleaner encoding.
BlendWeightsFrom now uses CObject::Type() to correctly identify neuron classes, avoiding undefined behavior when neurons are from the plain-CPU hierarchy (CNeuron, CNeuronConv, CNeuronPool, CNeuronLSTM). Weight blending for those legacy classes is also implemented.
Add FILE_SHARE_READ and FILE_SHARE_WRITE to all FileOpen calls in CNet::Load,
CNet::LoadCheckpoint, and in ExpertSignalAIBase (LoadModelStats, LoadChartSignals)
so that reading model files does not fail with error 5004 when another process
(e.g., a live chart) holds the file. Replace FileCopy with the new CopySharedFile
function for sidecar files (.cfg, .stats, _shadow.nnw) to perform share-aware
copying, preventing the same failure during tester seeding. This ensures the EA
can read and copy model artifacts even while a deployed instance is running.
Add retry logic (CopyFileWithRetry, LoadNetWithRetry) to the tester's seeding path in ExpertSignalAIBase.mqh, and print diagnostic error messages in CNet::Load (Network.mqh) when FileOpen fails. This addresses silent failures caused by transient Windows sharing violations that occur when a live chart's atomic Save() overlaps with a Strategy Tester agent's FileCopy or FileOpen on the same production file. The new helpers retry the operation a few times with a short pause instead of silently falling back to an untrained model.
The Market build has no DLL tier (WarriorCPU.dll is compiled out per MQL5 Market rules), so the TargetCPULoad input would have sat unused in the Inputs tab. Now it is conditionally compiled as a const when WARRIOR_MARKET_BUILD is defined, keeping the call site intact while avoiding a dead input.
The Market_Description.html was rewritten to better explain the EA's neural network, its self-training on unseen data, and the dual classic/AI signal paths, along with installation details and screenshots.
A single EA run instantiates several CNet objects (main net, EMA shadow, OOS clones, etc.). Each previously ran its own OpenCL/DirectML probe, printing redundant error banners (e.g., "OpenCL not found") and compute tier messages.
Added process-wide static flags `s_openclUnavailable` and `s_computeTierLogged` to skip subsequent probes and log once per process. This eliminates log clutter and avoids unnecessary probe overhead on hosts without OpenCL.
Rename the original `CreateElement` with a defaulted `weighScale` parameter to `CreateElementScaled` and provide a proper virtual override that matches the base `CArrayObj::CreateElement` signature exactly. This ensures `CArrayObj::Load()` dispatches correctly, fixing a bug where every saved model load failed at the first layer. Update all call sites in `CNeuronBase::Init` and `CNeuronPool::Init`. Additionally, enhance error diagnostics in `CNet::Load` to distinguish between file truncation and code faults (such as the signature mismatch).
The `Load` method now accepts an optional `quiet` flag that suppresses diagnostic
`Print` statements when set to `true`. This is used by callers that anticipate
a load failure and handle it gracefully — for example, the EMA shadow-net
bootstrap on a CPU-DLL box, which cannot hold a second full network. The main
model load keeps `quiet=false` so actual failures remain visible.
- In Network.mqh: Add upfront file size check to distinguish truncated files from backend allocation failures during model load, improving error diagnosis.
- In ExpertSignalAIBase.mqh: Remove redundant shadow net save on shutdown to avoid doubling shutdown cost and exceeding MT5's deinit budget, preventing abnormal termination and subsequent retrain.
- In Warrior_EA.mq5: Reduce training timer interval from 5s to 250ms to allow more frequent training cycles instead of sitting idle ~98% of the time.
The previous save directly wrote to the target file, which could leave a truncated/partial file if the process was force-killed (e.g., MT5 deinit timeout). Now the save writes to a temporary file (.savetmp) and only renames it over the real file after a successful write. This ensures that an interrupted save never corrupts the last good model. Additionally, improved diagnostics in CNet::Load to distinguish corrupt/incompatible files from compute errors.
Add CaptureWeights and RestoreWeights methods that snapshot every neuron's weights into host arrays (CArrayDouble per neuron) and restore them in-place via setWeights. This replaces the file-based SaveCheckpoint/LoadCheckpoint for the mid-run best-era rollback, because the file path re-creates neurons (CLayer+Init) which fails on the multithreaded CPU-DLL backend (CDirectMLMy/WarriorCPU.dll) that cannot allocate a second full set of neuron tensors while the live set exists. In-memory weight copy uses only getWeights/setWeights, already proven by the per-era shadow blend. Snapshots weights only (not Adam moments); the regression handler decays eta on restore and clips per-step deltas to prevent stale-moment overshoot. Snapshot is valid only within a single Train() run. Also adds HaveWeightSnapshot() query and the m_weightSnapshot / m_haveWeightSnapshot member variables.
SaveCheckpoint and LoadCheckpoint in Network.mqh now accept a `bool common` flag. When true, the ephemeral checkpoint is stored in FILE_COMMON (co-located with the model's .nnw/.cfg/.stats/_shadow.nnw), preventing a parallel folder tree under the terminal-local Files directory. ExpertSignalAIBase.mqh is updated to pass the same flag to file existence checks and deletions, ensuring all sidecar files use the correct location.
Previously, LoadCheckpoint cleared the live layers before reading, so any read failure (truncated/corrupt file, OCL init hiccup) left the network with zero layers. This collapsed predictions to neutral, caused Save to refuse writing 0-layer stubs, and wiped chart arrows. Now the checkpoint is loaded into a temporary layer set that is swapped in only on full success; on failure the current weights are preserved and a diagnostic message is printed.
In CNet::Load and CNet::LoadCheckpoint, when a CLayer object is allocated but subsequent Load() or layers.Add() fails, the layer was not freed, causing a memory leak (reported as "1 object of class 'CLayer'" on MT5 unload). Added a check for invalid pointer and explicit deletion of the layer before breaking from the loop, ensuring proper cleanup.
- Guard CNet::Save to refuse writing a 0-layer network (prevents overwriting ~18MB model with empty stub)
- In CNet::Load and LoadCheckpoint, treat 0-layer files as load failure (older stubs still on disk)
- Introduce LOGIT_PRIOR_STRENGTH_PRESETS enum (0–100%) to control logit adjustment tau
- Prepare member variables and AdjustedSignalFromSoftmax for prior-corrected posterior at inference
- Ensure raw argmax scoring for recall/convergence remains unchanged; correction only affects live signal
- Added optional `weighScale` parameter (default -1.0) to `CNeuronBase::Init` and `CLayer::CreateElement`.
- Updated `CNeuronPool::Init` to use LeCun-uniform scaling (1/sqrt(window+1)) for its base initialization.
- Updated `CNet::CNet` to use He-scaled initialization (sqrt(2/neurons)) for dense layers.
- These changes enable more flexible and statistically sound weight initialization, matching the rationale used in OCL-based implementations, leading to better training stability and convergence.
Remove verbose book references from input parameter comments in
Network.mqh for clarity. Add #ifndef guard around ENUM_OPTIMIZATION
to allow inclusion from multiple headers without redefinition.
Document the MQL5 Market DLL restriction in NeuronDirectML.mqh and
introduce WARRIOR_MARKET_BUILD macro to conditionally compile out
DirectML DLL imports for Market-compliant builds.
Reduce weight decay from 0.01 to 0.001 across all backends (Network.cl, Network.mqh, WarriorCPU.cpp) to fix a training collapse issue. The original 0.01 AdamW default caused discriminative weights to decay below the calibration-capped class-prior offsets, resulting in a monotonically shrinking per-bar logit spread and eventual constant Neutral predictions (argmax degenerated once evidence tilt dropped under the prior tilt). The new value 0.001 lifts the evidence ceiling 10× while still bounding long-run weight growth, restoring effective discrimination. Note: this change must remain in sync across all four backends.
- Adam: replace raw neuron gradient with per-weight gradient `g = gradient * outputVal` so that each input weight receives its own mt/vt, fixing a bug where all weights of a neuron were updated identically (uniform scaling).
- `calcHiddenGradients`: compute `gradient = sumDOW * activationDerivative` directly instead of routing through `calcOutputGradients` with a clamped pseudo-target, which corrupted hidden-layer error signals (especially for PRELU activations).
The sign-agreement gate in `UpdateWeightsAdam`, `UpdateWeightsConvAdam`, and `LSTM_UpdateWeightsAdam` caused a gradient ratchet effect under one-hot softmax with categorical cross-entropy, leading to an all-Neutral collapse. Removing this gate aligns all four backends (CPU, GPU, OpenCL, MQL) and restores correct gradient flow.
Remove overly verbose explanations in Network.mqh comments and InputEnums.mqh enum values. Shorten descriptions to improve readability without losing essential information.
AI/Network.mqh was 5,805 lines / 18 classes in one file. Investigation
found method implementations for several classes (CNeuronBase/Pool/Conv,
CNeuronBaseOCL) hand-interleaved across thousands of lines - not safe to
split without risky manual reassembly. But 8 classes turned out to be
genuinely self-contained (declaration + every method body physically
contiguous, and only ever depended upon, never depending on anything
declared later): CConnection/CArrayCon, CNeuron, CDirectMLMy (+ its
WarriorDML.dll/WarriorCPU.dll #import blocks), CArrayLayer,
CLayerDescription, CBufferDouble, and CNeuronConvOCL/CNeuronPoolOCL.
Extracted each verbatim, via exact line-range extraction (not manual
retyping) to eliminate transcription risk, into its own AI/*.mqh file,
included from Network.mqh at the exact point each class used to sit -
preserving original declaration order exactly. Mathematically verified
byte-for-byte: reconstructing the original file from the 7 new files'
bodies + Network.mqh's remaining segments is line-for-line identical to
the pre-split git history. Compiled clean (MetaEditor, 0 errors/0
warnings) both before and after.
The remaining tangled classes (CNeuronBase, CNeuronPool, CNeuronConv,
CNet, CNeuronLSTM, CNeuronBaseOCL, CNeuronConvOCL/PoolOCL's shared base,
CNeuronLSTMOCL, COpenCLMy) stay in Network.mqh (now ~4,450 lines) -
splitting those safely needs deliberate per-method surgery, deferred to
a future dedicated pass rather than rushed into this one.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Replace hardcoded lr and momentum with new input variables for Adam and
SGD+momentum. Add OpenCL kernel LSTM_UpdateWeightsMomentum alongside the
existing Adam kernel. Update comments and revert beta1 to book default 0.9.
- Lowered Adam momentum parameter b1 from 0.9 to 0.8, shortening the effective memory window (~5 steps) to reduce multi-era bias streaks observed in training runs.
- Added FOCAL_GAMMA_PRESET enum and corresponding member variable m_focalGamma to apply focal loss modulation (Lin et al. 2017) on top of class-balance weighting, targeting the same streak issue by downweighting already-confident examples.
- Updated comments to document the rationale and experimental nature of both changes.
All __global buffers, local vectors, and constants (MAX_WEIGHT, MIN_ACTIVATION_DERIVATIVE, WEIGHT_DECAY, MAX_WEIGHT_DELTA) now use `float` instead of `double`. This avoids the 1/16th throughput penalty on consumer GPUs (e.g. AMD Polaris/RX 580) while remaining numerically safe: weight/delta clamps stay within ±100, gradients and activations fit float32, and the Adam epsilon guard is not sensitive to underflow. The removed `cl_khr_fp64` directive now causes an explicit compile failure if any double slips back in.
The network uses PReLU (leaky ReLU) activations for all hidden layers, but the previous initialization used LeCun-uniform scaling (1/sqrt(fan_in+1)), which is tuned for saturating activations like tanh/sigmoid. The new He-uniform scaling (sqrt(2/fan_in)) better accounts for the half-zeroing effect of ReLU variants, reducing the risk of early saturation and improving gradient flow during training. Applied uniformly across all layers via the shared Init() method, as the output layer (3 neurons) is negligibly affected and the more precise alternative would require risky threading of activation awareness through every neuron subtype.
