forked from animatedread/Warrior_EA
30206cbabc
The only bounded stage in the entire forward path was the sigmoid
classification head - every hidden stage is PRELU. That is a network with
no internal scale control, and the failure ordered exactly by depth: on
SP500 H1 the shallow perceptron held ~52% balanced accuracy while the
deepest topology sat on the 33.3% one-class floor, with the per-bar logit
spread decaying monotonically (0.45 -> 0.38 over ~200 eras) until the
evidence tilt fell under the class-prior tilt. That is the signature of
internal covariate shift, which chapter 6.1 of the reference book is
entirely about and which the NeuroNet_DNG engine addresses with a layer
this project never had.
Two mechanisms make this the right fix rather than more hyperparameter
nudging:
- it decouples WEIGHT_DECAY from the learned function (van Laarhoven
2017) - with a normalized layer downstream, decay can no longer grind
the discriminative signal away, it only rescales the effective
learning rate;
- it is the precondition for ever running an unbounded logit head here.
The 2026-07-27 attempt blew up (IS error 5.6e15) precisely because
nothing upstream constrained scale.
Implementation notes:
- CNeuronBatchNormOCL computes host-side rather than as a fourth copy of
a kernel across Network.cl + WarriorCPU.cpp + WarriorDML.cpp. The math
is elementwise O(n); this way it behaves identically on all four
compute tiers, needs no DLL rebuild, and cannot drift between
backends. Same precedent as the softmax+CCE gradient and the
per-sample loss weighting, both computed in MQL5 for that reason.
- Statistics are exponential moving, not a stored mini-batch: training
is pure online SGD, one update per sample, so there is no batch to
average over. BatchNormWindow is an EMA window length.
- gamma/beta are excluded from weight decay, deliberately - decaying
gamma toward zero is the exact pathology being fixed.
- The layer self-sizes from whatever sits below it, because a conv/pool
stage's output width is derived inside the CNet constructor and is not
knowable to the topology builder.
- Checkpoint capture/restore/blend carry gamma/beta and the running
statistics alongside the dense matrix, so the plateau ladder cannot
restore a mismatched pair.
- SeedOutputLayerBias accepted only an exact defNeuronBaseOCL as the
weight-carrying penultimate layer; with normalization enabled that is
the batch-norm layer, so the cold-start bias seed would have silently
stopped being applied.
- Refuses to build, loudly, if a topology asks for normalization with no
compute backend at all - rather than quietly training a different
architecture than the one requested.
EnableBatchNorm (default on) and BatchNormWindow (1000 samples) are
inputs so the effect can be A/B'd without a recompile. Both feed the
weights-filename fingerprint, appended conditionally so existing non-BN
configs keep their fingerprints and are not forced to retrain.
Verified: analytic gradients match finite differences to 1.5e-7 relative
over 200 random cases; a faithful port of the full forward/backward chain
collapses to the 33.3% floor by era 4 without this layer and holds
36-43% with it. Compiles 0 errors, 0 warnings.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
39 lines
1.8 KiB
MQL5
39 lines
1.8 KiB
MQL5
//+------------------------------------------------------------------+
|
|
//| LayerDescription.mqh|
|
|
//| AnimateDread |
|
|
//| https://www.mql5.com |
|
|
//+------------------------------------------------------------------+
|
|
//| CLayerDescription - plain topology-spec data holder consumed by |
|
|
//| CNet's topology-build code (AI\Network.mqh). Self-contained (no |
|
|
//| class-type dependencies, just enum/int defaults). Extracted |
|
|
//| verbatim (SOLID cleanup) - no logic changes. |
|
|
//+------------------------------------------------------------------+
|
|
class CLayerDescription : public CObject
|
|
{
|
|
public:
|
|
CLayerDescription(void);
|
|
~CLayerDescription(void) {};
|
|
//---
|
|
int type;
|
|
int count;
|
|
int window;
|
|
int step;
|
|
//--- Batch-normalization layers only (defNeuronBatchNorm): the EMA window length over which the
|
|
//--- running mean/variance are estimated. Ignored by every other layer type. Not folded into
|
|
//--- `window`, which already means "input window width" for conv/pool and would read as a
|
|
//--- completely different quantity here.
|
|
int batch;
|
|
ENUM_ACTIVATION activation;
|
|
ENUM_OPTIMIZATION optimization;
|
|
};
|
|
//+------------------------------------------------------------------+
|
|
//| |
|
|
//+------------------------------------------------------------------+
|
|
CLayerDescription::CLayerDescription(void) : type(defNeuron),
|
|
count(0),
|
|
window(1),
|
|
step(1),
|
|
batch(1), // 1 == normalization disabled; only batch-norm descriptors ever raise it
|
|
activation(TANH),
|
|
optimization(SGD)
|
|
{}
|