mnbvc188199/Warrior_EA
Watch
1
0
Fork
You've already forked Warrior_EA
 1
forked from animatedread/Warrior_EA
Code Pull requests Activity
Warrior_EA/AI/NeuronBatchNorm.mqh

1220 lines
59 KiB
MQL5
Raw Permalink Normal View History

feat(ai): batch normalization between dense layers The only bounded stage in the entire forward path was the sigmoid classification head - every hidden stage is PRELU. That is a network with no internal scale control, and the failure ordered exactly by depth: on SP500 H1 the shallow perceptron held ~52% balanced accuracy while the deepest topology sat on the 33.3% one-class floor, with the per-bar logit spread decaying monotonically (0.45 -> 0.38 over ~200 eras) until the evidence tilt fell under the class-prior tilt. That is the signature of internal covariate shift, which chapter 6.1 of the reference book is entirely about and which the NeuroNet_DNG engine addresses with a layer this project never had. Two mechanisms make this the right fix rather than more hyperparameter nudging: - it decouples WEIGHT_DECAY from the learned function (van Laarhoven 2017) - with a normalized layer downstream, decay can no longer grind the discriminative signal away, it only rescales the effective learning rate; - it is the precondition for ever running an unbounded logit head here. The 2026-07-27 attempt blew up (IS error 5.6e15) precisely because nothing upstream constrained scale. Implementation notes: - CNeuronBatchNormOCL computes host-side rather than as a fourth copy of a kernel across Network.cl + WarriorCPU.cpp + WarriorDML.cpp. The math is elementwise O(n); this way it behaves identically on all four compute tiers, needs no DLL rebuild, and cannot drift between backends. Same precedent as the softmax+CCE gradient and the per-sample loss weighting, both computed in MQL5 for that reason. - Statistics are exponential moving, not a stored mini-batch: training is pure online SGD, one update per sample, so there is no batch to average over. BatchNormWindow is an EMA window length. - gamma/beta are excluded from weight decay, deliberately - decaying gamma toward zero is the exact pathology being fixed. - The layer self-sizes from whatever sits below it, because a conv/pool stage's output width is derived inside the CNet constructor and is not knowable to the topology builder. - Checkpoint capture/restore/blend carry gamma/beta and the running statistics alongside the dense matrix, so the plateau ladder cannot restore a mismatched pair. - SeedOutputLayerBias accepted only an exact defNeuronBaseOCL as the weight-carrying penultimate layer; with normalization enabled that is the batch-norm layer, so the cold-start bias seed would have silently stopped being applied. - Refuses to build, loudly, if a topology asks for normalization with no compute backend at all - rather than quietly training a different architecture than the one requested. EnableBatchNorm (default on) and BatchNormWindow (1000 samples) are inputs so the effect can be A/B'd without a recompile. Both feed the weights-filename fingerprint, appended conditionally so existing non-BN configs keep their fingerprints and are not forced to retrain. Verified: analytic gradients match finite differences to 1.5e-7 relative over 200 random cases; a faithful port of the full forward/backward chain collapses to the 33.3% floor by era 4 without this layer and holds 36-43% with it. Compiles 0 errors, 0 warnings. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-29 12:34:29 -04:00
//+------------------------------------------------------------------+
//| NeuronBatchNorm.mqh |
//| AnimateDread |
//| https://www.mql5.com |
refactor(comments): box headers to stdlib length The //| box blocks were excluded from 0b06f8e and 5efdb48 and were what remained: 160 of them ran to 10+ lines, the longest to 88. Compressed to their leading topic sentences - 5 lines for a function header, 8 for a file header - keeping the box format and the standard MQL5 name/author lines verbatim. Verified at the BYTE level this time, across every in-scope file: the list of non-comment lines is byte-identical to HEAD and braces balance. The first check compared a locale-decoded 'git show' against a UTF-8 read and flagged 25 files that had not changed at all - every BOM and every non-ASCII line mismatched. 47,696 -> 40,665 lines in scope; comment share 38% -> 26%. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-22 00:30:14 -04:00
//| CNeuronBatchNormOCL - batch normalization (Ioffe & Szegedy |
//| 2015). Needs CNeuronBaseOCL (AI\Network.mqh) already declared; |
//| included from there, not standalone. |
feat(ai): batch normalization between dense layers The only bounded stage in the entire forward path was the sigmoid classification head - every hidden stage is PRELU. That is a network with no internal scale control, and the failure ordered exactly by depth: on SP500 H1 the shallow perceptron held ~52% balanced accuracy while the deepest topology sat on the 33.3% one-class floor, with the per-bar logit spread decaying monotonically (0.45 -> 0.38 over ~200 eras) until the evidence tilt fell under the class-prior tilt. That is the signature of internal covariate shift, which chapter 6.1 of the reference book is entirely about and which the NeuroNet_DNG engine addresses with a layer this project never had. Two mechanisms make this the right fix rather than more hyperparameter nudging: - it decouples WEIGHT_DECAY from the learned function (van Laarhoven 2017) - with a normalized layer downstream, decay can no longer grind the discriminative signal away, it only rescales the effective learning rate; - it is the precondition for ever running an unbounded logit head here. The 2026-07-27 attempt blew up (IS error 5.6e15) precisely because nothing upstream constrained scale. Implementation notes: - CNeuronBatchNormOCL computes host-side rather than as a fourth copy of a kernel across Network.cl + WarriorCPU.cpp + WarriorDML.cpp. The math is elementwise O(n); this way it behaves identically on all four compute tiers, needs no DLL rebuild, and cannot drift between backends. Same precedent as the softmax+CCE gradient and the per-sample loss weighting, both computed in MQL5 for that reason. - Statistics are exponential moving, not a stored mini-batch: training is pure online SGD, one update per sample, so there is no batch to average over. BatchNormWindow is an EMA window length. - gamma/beta are excluded from weight decay, deliberately - decaying gamma toward zero is the exact pathology being fixed. - The layer self-sizes from whatever sits below it, because a conv/pool stage's output width is derived inside the CNet constructor and is not knowable to the topology builder. - Checkpoint capture/restore/blend carry gamma/beta and the running statistics alongside the dense matrix, so the plateau ladder cannot restore a mismatched pair. - SeedOutputLayerBias accepted only an exact defNeuronBaseOCL as the weight-carrying penultimate layer; with normalization enabled that is the batch-norm layer, so the cold-start bias seed would have silently stopped being applied. - Refuses to build, loudly, if a topology asks for normalization with no compute backend at all - rather than quietly training a different architecture than the one requested. EnableBatchNorm (default on) and BatchNormWindow (1000 samples) are inputs so the effect can be A/B'd without a recompile. Both feed the weights-filename fingerprint, appended conditionally so existing non-BN configs keep their fingerprints and are not forced to retrain. Verified: analytic gradients match finite differences to 1.5e-7 relative over 200 random cases; a faithful port of the full forward/backward chain collapses to the 33.3% floor by era 4 without this layer and holds 36-43% with it. Compiles 0 errors, 0 warnings. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-29 12:34:29 -04:00
//+------------------------------------------------------------------+
#ifndef WARRIOR_NEURON_BATCHNORM_MQH
#define WARRIOR_NEURON_BATCHNORM_MQH
refactor(comments): stdlib comment style across the remaining in-scope files Same pass as 0b06f8e, applied file by file: comment runs of 4+ lines compressed to their leading topic sentences, capped at 4 lines, whole sentences only. Warning sentences (NEVER / MUST / trap / would-have) survive the budget. Every file was checked the same way before committing: the list of non-comment lines is byte-identical to HEAD, and braces balance. No code was touched. Panel/, Enumerations/ and the already-terse System headers needed little or nothing - PooledGate, TradeChecks, BinomialStats and Random came through with no blocks over the threshold at all. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-22 00:25:52 -04:00
//--- Per-neuron slot layout inside BatchOptions.
feat(ai): batch normalization between dense layers The only bounded stage in the entire forward path was the sigmoid classification head - every hidden stage is PRELU. That is a network with no internal scale control, and the failure ordered exactly by depth: on SP500 H1 the shallow perceptron held ~52% balanced accuracy while the deepest topology sat on the 33.3% one-class floor, with the per-bar logit spread decaying monotonically (0.45 -> 0.38 over ~200 eras) until the evidence tilt fell under the class-prior tilt. That is the signature of internal covariate shift, which chapter 6.1 of the reference book is entirely about and which the NeuroNet_DNG engine addresses with a layer this project never had. Two mechanisms make this the right fix rather than more hyperparameter nudging: - it decouples WEIGHT_DECAY from the learned function (van Laarhoven 2017) - with a normalized layer downstream, decay can no longer grind the discriminative signal away, it only rescales the effective learning rate; - it is the precondition for ever running an unbounded logit head here. The 2026-07-27 attempt blew up (IS error 5.6e15) precisely because nothing upstream constrained scale. Implementation notes: - CNeuronBatchNormOCL computes host-side rather than as a fourth copy of a kernel across Network.cl + WarriorCPU.cpp + WarriorDML.cpp. The math is elementwise O(n); this way it behaves identically on all four compute tiers, needs no DLL rebuild, and cannot drift between backends. Same precedent as the softmax+CCE gradient and the per-sample loss weighting, both computed in MQL5 for that reason. - Statistics are exponential moving, not a stored mini-batch: training is pure online SGD, one update per sample, so there is no batch to average over. BatchNormWindow is an EMA window length. - gamma/beta are excluded from weight decay, deliberately - decaying gamma toward zero is the exact pathology being fixed. - The layer self-sizes from whatever sits below it, because a conv/pool stage's output width is derived inside the CNet constructor and is not knowable to the topology builder. - Checkpoint capture/restore/blend carry gamma/beta and the running statistics alongside the dense matrix, so the plateau ladder cannot restore a mismatched pair. - SeedOutputLayerBias accepted only an exact defNeuronBaseOCL as the weight-carrying penultimate layer; with normalization enabled that is the batch-norm layer, so the cold-start bias seed would have silently stopped being applied. - Refuses to build, loudly, if a topology asks for normalization with no compute backend at all - rather than quietly training a different architecture than the one requested. EnableBatchNorm (default on) and BatchNormWindow (1000 samples) are inputs so the effect can be A/B'd without a recompile. Both feed the weights-filename fingerprint, appended conditionally so existing non-BN configs keep their fingerprints and are not forced to retrain. Verified: analytic gradients match finite differences to 1.5e-7 relative over 200 random cases; a faithful port of the full forward/backward chain collapses to the 33.3% floor by era 4 without this layer and holds 36-43% with it. Compiles 0 errors, 0 warnings. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-29 12:34:29 -04:00
#define BN_OPT_STRIDE 9
#define BN_OPT_MEAN 0 // running mean
#define BN_OPT_VAR 1 // running variance
#define BN_OPT_NX 2 // normalized input, cached from the forward pass for the backward pass
#define BN_OPT_GAMMA 3 // learned scale, init 1
#define BN_OPT_BETA 4 // learned shift, init 0
#define BN_OPT_MG 5 // gamma: Adam first momentum, or SGD previous delta
#define BN_OPT_MB 6 // beta: Adam first momentum, or SGD previous delta
#define BN_OPT_VG 7 // gamma: Adam second momentum (stored already square-rooted, as the
#define BN_OPT_VB 8 // beta: ... UpdateWeightsAdam kernels in this engine also do)
refactor(comments): stdlib comment style across the remaining in-scope files Same pass as 0b06f8e, applied file by file: comment runs of 4+ lines compressed to their leading topic sentences, capped at 4 lines, whole sentences only. Warning sentences (NEVER / MUST / trap / would-have) survive the budget. Every file was checked the same way before committing: the list of non-comment lines is byte-identical to HEAD, and braces balance. No code was touched. Panel/, Enumerations/ and the already-terse System headers needed little or nothing - PooledGate, TradeChecks, BinomialStats and Random came through with no blocks over the threshold at all. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-22 00:25:52 -04:00
//--- Variance floor. Applied to the standard deviation (not the variance) so it reads as "no unit
//--- is amplified by more than 1e4", which is the property that actually matters.
feat(ai): batch normalization between dense layers The only bounded stage in the entire forward path was the sigmoid classification head - every hidden stage is PRELU. That is a network with no internal scale control, and the failure ordered exactly by depth: on SP500 H1 the shallow perceptron held ~52% balanced accuracy while the deepest topology sat on the 33.3% one-class floor, with the per-bar logit spread decaying monotonically (0.45 -> 0.38 over ~200 eras) until the evidence tilt fell under the class-prior tilt. That is the signature of internal covariate shift, which chapter 6.1 of the reference book is entirely about and which the NeuroNet_DNG engine addresses with a layer this project never had. Two mechanisms make this the right fix rather than more hyperparameter nudging: - it decouples WEIGHT_DECAY from the learned function (van Laarhoven 2017) - with a normalized layer downstream, decay can no longer grind the discriminative signal away, it only rescales the effective learning rate; - it is the precondition for ever running an unbounded logit head here. The 2026-07-27 attempt blew up (IS error 5.6e15) precisely because nothing upstream constrained scale. Implementation notes: - CNeuronBatchNormOCL computes host-side rather than as a fourth copy of a kernel across Network.cl + WarriorCPU.cpp + WarriorDML.cpp. The math is elementwise O(n); this way it behaves identically on all four compute tiers, needs no DLL rebuild, and cannot drift between backends. Same precedent as the softmax+CCE gradient and the per-sample loss weighting, both computed in MQL5 for that reason. - Statistics are exponential moving, not a stored mini-batch: training is pure online SGD, one update per sample, so there is no batch to average over. BatchNormWindow is an EMA window length. - gamma/beta are excluded from weight decay, deliberately - decaying gamma toward zero is the exact pathology being fixed. - The layer self-sizes from whatever sits below it, because a conv/pool stage's output width is derived inside the CNet constructor and is not knowable to the topology builder. - Checkpoint capture/restore/blend carry gamma/beta and the running statistics alongside the dense matrix, so the plateau ladder cannot restore a mismatched pair. - SeedOutputLayerBias accepted only an exact defNeuronBaseOCL as the weight-carrying penultimate layer; with normalization enabled that is the batch-norm layer, so the cold-start bias seed would have silently stopped being applied. - Refuses to build, loudly, if a topology asks for normalization with no compute backend at all - rather than quietly training a different architecture than the one requested. EnableBatchNorm (default on) and BatchNormWindow (1000 samples) are inputs so the effect can be A/B'd without a recompile. Both feed the weights-filename fingerprint, appended conditionally so existing non-BN configs keep their fingerprints and are not forced to retrain. Verified: analytic gradients match finite differences to 1.5e-7 relative over 200 random cases; a faithful port of the full forward/backward chain collapses to the 33.3% floor by era 4 without this layer and holds 36-43% with it. Compiles 0 errors, 0 warnings. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-29 12:34:29 -04:00
#define BN_EPSILON 1.0e-10
#define BN_MIN_STD 1.0e-4
fix(batchnorm): bound the normalized value - a constant input feature was amplified 1e4x and pinned PAI's head to its rails BN_MIN_STD = 1e-4 caps the per-unit gain at 1/1e-4 = 1e4, and the comment above it states that as though it were a safety property. It is not. A unit whose running variance is ~0 is a CONSTANT feature carrying no information, and dividing its rounding noise by 1e-4 hands the next layer an activation of several hundred. BN's contract is "output has ~unit variance"; a unit that cannot supply that must contribute nothing, not the largest signal in the layer. MEASURED, 2026-08-17 SP500 H4, four topologies on identical separate charts: model spread Neutral CHOSE Neutral TIED rail CONV 0.386 0.68% 0.10% 0.48% LSTM 0.392 0.63% 0.00% 0.00% HYB 0.376 1.79% 0.00% 0.01% PAI 0.192 0.09% 80.63% 99.99% bn1's cached nx normed 1.38e4 over 800 units. PAI's SIGMOID head was on its rails on 99.99% of bars, with Buy and Sell landing on the SAME rail so they compared exactly equal, and ApplyClassificationSoftmax()'s strict-majority rule reported that tie as Neutral on ~80% of bars. So the long-running "PAI is heavily biased toward Neutral" was never a class-prior problem: the net CHOSE Neutral on 0.09% of bars. It was float equality on a saturated head. The 331ab29 counters answered it on their first run. PAI-only because it is the one topology whose FIRST batch norm sits on the raw 800-dim input vector - CONV/LSTM/CONVLSTM all have a conv or LSTM stage in front, so their first BN sees a learned representation with no degenerate units. That asymmetry was already on file as a suspicion; this is the mechanism. FIX, mirrored in both backends (host NeuronBatchNorm.mqh and device Network.cl): forward nx = clamp(delta/sd, -BN_MAX_NX, +BN_MAX_NX), BN_MAX_NX = 8 backward if the forward bound this unit, the output stopped depending on the input, so d(nx)/dx = 0 and NO gradient passes The backward half is not optional. g is divided by the same sd the forward multiplies by, so a degenerate unit gets its GRADIENT amplified 1e4x too - the "receives gradients divided by sqrt(var) ~ 500" pathology already noted in Network.cl's Adam kernel. Bounding only the forward would move the explosion downstream. 8 sigma is inert on anything healthy (|nx| > 8 is a ~1e-15 event under normality); it binds only on degenerate units, which is the entire point. Same clamp-to-range idiom the activation derivatives beside it already use. SelfCheckBnForward/SelfCheckBnHiddenGrad already prove host against kernel, and BN_OPT_NX was already consumed in the backward for the gamma gradient, so the new read adds no lifetime assumption. Expect PAI to change behaviour and CONV/LSTM/CONVLSTM not to (their rail rate is ~0%, so the clamp never binds). No .nnw format or fingerprint change. ALSO: print the zero-skill reference on the era line. m_oosWinLongTotal and m_oosWinShortTotal have been accumulated for a long time and NEVER printed, which is why three separate topologies all sitting at 62% read as a mysterious coincidence rather than the obvious base rate. It is not a coincidence: with the target (1.70 ATR) nearer than the stop (3.07 ATR), BOTH sides win on 24.5% of bars, so winLong+winShort covers ~124% of them and a no-edge caller collects 62.1% whichever way it calls - against a 64.3% break-even. Derived from this run's own label counts: (11329 - 2776 + 2*2776) / (2*11350) = 62.14%. Every win rate on that line must be read against this, not against 50%. NOT COMPILED - user compiles. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-17 00:21:51 -04:00
//--- HARD BOUND ON THE NORMALIZED VALUE. BN_MIN_STD alone caps the per-unit gain at 1/1e-4 = 1e4,
refactor(comments): stdlib comment style across the remaining in-scope files Same pass as 0b06f8e, applied file by file: comment runs of 4+ lines compressed to their leading topic sentences, capped at 4 lines, whole sentences only. Warning sentences (NEVER / MUST / trap / would-have) survive the budget. Every file was checked the same way before committing: the list of non-comment lines is byte-identical to HEAD, and braces balance. No code was touched. Panel/, Enumerations/ and the already-terse System headers needed little or nothing - PooledGate, TradeChecks, BinomialStats and Random came through with no blocks over the threshold at all. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-22 00:25:52 -04:00
//--- and the comment above states that as if it were a safety property.
fix(batchnorm): bound the normalized value - a constant input feature was amplified 1e4x and pinned PAI's head to its rails BN_MIN_STD = 1e-4 caps the per-unit gain at 1/1e-4 = 1e4, and the comment above it states that as though it were a safety property. It is not. A unit whose running variance is ~0 is a CONSTANT feature carrying no information, and dividing its rounding noise by 1e-4 hands the next layer an activation of several hundred. BN's contract is "output has ~unit variance"; a unit that cannot supply that must contribute nothing, not the largest signal in the layer. MEASURED, 2026-08-17 SP500 H4, four topologies on identical separate charts: model spread Neutral CHOSE Neutral TIED rail CONV 0.386 0.68% 0.10% 0.48% LSTM 0.392 0.63% 0.00% 0.00% HYB 0.376 1.79% 0.00% 0.01% PAI 0.192 0.09% 80.63% 99.99% bn1's cached nx normed 1.38e4 over 800 units. PAI's SIGMOID head was on its rails on 99.99% of bars, with Buy and Sell landing on the SAME rail so they compared exactly equal, and ApplyClassificationSoftmax()'s strict-majority rule reported that tie as Neutral on ~80% of bars. So the long-running "PAI is heavily biased toward Neutral" was never a class-prior problem: the net CHOSE Neutral on 0.09% of bars. It was float equality on a saturated head. The 331ab29 counters answered it on their first run. PAI-only because it is the one topology whose FIRST batch norm sits on the raw 800-dim input vector - CONV/LSTM/CONVLSTM all have a conv or LSTM stage in front, so their first BN sees a learned representation with no degenerate units. That asymmetry was already on file as a suspicion; this is the mechanism. FIX, mirrored in both backends (host NeuronBatchNorm.mqh and device Network.cl): forward nx = clamp(delta/sd, -BN_MAX_NX, +BN_MAX_NX), BN_MAX_NX = 8 backward if the forward bound this unit, the output stopped depending on the input, so d(nx)/dx = 0 and NO gradient passes The backward half is not optional. g is divided by the same sd the forward multiplies by, so a degenerate unit gets its GRADIENT amplified 1e4x too - the "receives gradients divided by sqrt(var) ~ 500" pathology already noted in Network.cl's Adam kernel. Bounding only the forward would move the explosion downstream. 8 sigma is inert on anything healthy (|nx| > 8 is a ~1e-15 event under normality); it binds only on degenerate units, which is the entire point. Same clamp-to-range idiom the activation derivatives beside it already use. SelfCheckBnForward/SelfCheckBnHiddenGrad already prove host against kernel, and BN_OPT_NX was already consumed in the backward for the gamma gradient, so the new read adds no lifetime assumption. Expect PAI to change behaviour and CONV/LSTM/CONVLSTM not to (their rail rate is ~0%, so the clamp never binds). No .nnw format or fingerprint change. ALSO: print the zero-skill reference on the era line. m_oosWinLongTotal and m_oosWinShortTotal have been accumulated for a long time and NEVER printed, which is why three separate topologies all sitting at 62% read as a mysterious coincidence rather than the obvious base rate. It is not a coincidence: with the target (1.70 ATR) nearer than the stop (3.07 ATR), BOTH sides win on 24.5% of bars, so winLong+winShort covers ~124% of them and a no-edge caller collects 62.1% whichever way it calls - against a 64.3% break-even. Derived from this run's own label counts: (11329 - 2776 + 2*2776) / (2*11350) = 62.14%. Every win rate on that line must be read against this, not against 50%. NOT COMPILED - user compiles. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-17 00:21:51 -04:00
#define BN_MAX_NX 8.0
refactor(comments): stdlib comment style across the remaining in-scope files Same pass as 0b06f8e, applied file by file: comment runs of 4+ lines compressed to their leading topic sentences, capped at 4 lines, whole sentences only. Warning sentences (NEVER / MUST / trap / would-have) survive the budget. Every file was checked the same way before committing: the list of non-comment lines is byte-identical to HEAD, and braces balance. No code was touched. Panel/, Enumerations/ and the already-terse System headers needed little or nothing - PooledGate, TradeChecks, BinomialStats and Random came through with no blocks over the threshold at all. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-22 00:25:52 -04:00
//--- Sanity ceiling on a single input value, applied before it can touch the running statistics
//--- below. See NormalizeHost() for the failure this exists to stop.
fix: improve error messages and suppress false sharing-violation logs - BufferDouble: replace hardcoded "DirectML/CPU-DLL" with dynamic backend name and add buffer index/element count to all error prints for easier debugging. - NetPersistence: distinguish missing file from transient lock by probing FileIsExist before logging, eliminating false "sharing violation" warnings when no saved model exists on first run.
2026-08-02 01:09:18 -04:00
#define BN_MAX_INPUT 1.0e6
feat(ai): batch normalization between dense layers The only bounded stage in the entire forward path was the sigmoid classification head - every hidden stage is PRELU. That is a network with no internal scale control, and the failure ordered exactly by depth: on SP500 H1 the shallow perceptron held ~52% balanced accuracy while the deepest topology sat on the 33.3% one-class floor, with the per-bar logit spread decaying monotonically (0.45 -> 0.38 over ~200 eras) until the evidence tilt fell under the class-prior tilt. That is the signature of internal covariate shift, which chapter 6.1 of the reference book is entirely about and which the NeuroNet_DNG engine addresses with a layer this project never had. Two mechanisms make this the right fix rather than more hyperparameter nudging: - it decouples WEIGHT_DECAY from the learned function (van Laarhoven 2017) - with a normalized layer downstream, decay can no longer grind the discriminative signal away, it only rescales the effective learning rate; - it is the precondition for ever running an unbounded logit head here. The 2026-07-27 attempt blew up (IS error 5.6e15) precisely because nothing upstream constrained scale. Implementation notes: - CNeuronBatchNormOCL computes host-side rather than as a fourth copy of a kernel across Network.cl + WarriorCPU.cpp + WarriorDML.cpp. The math is elementwise O(n); this way it behaves identically on all four compute tiers, needs no DLL rebuild, and cannot drift between backends. Same precedent as the softmax+CCE gradient and the per-sample loss weighting, both computed in MQL5 for that reason. - Statistics are exponential moving, not a stored mini-batch: training is pure online SGD, one update per sample, so there is no batch to average over. BatchNormWindow is an EMA window length. - gamma/beta are excluded from weight decay, deliberately - decaying gamma toward zero is the exact pathology being fixed. - The layer self-sizes from whatever sits below it, because a conv/pool stage's output width is derived inside the CNet constructor and is not knowable to the topology builder. - Checkpoint capture/restore/blend carry gamma/beta and the running statistics alongside the dense matrix, so the plateau ladder cannot restore a mismatched pair. - SeedOutputLayerBias accepted only an exact defNeuronBaseOCL as the weight-carrying penultimate layer; with normalization enabled that is the batch-norm layer, so the cold-start bias seed would have silently stopped being applied. - Refuses to build, loudly, if a topology asks for normalization with no compute backend at all - rather than quietly training a different architecture than the one requested. EnableBatchNorm (default on) and BatchNormWindow (1000 samples) are inputs so the effect can be A/B'd without a recompile. Both feed the weights-filename fingerprint, appended conditionally so existing non-BN configs keep their fingerprints and are not forced to retrain. Verified: analytic gradients match finite differences to 1.5e-7 relative over 200 random cases; a faithful port of the full forward/backward chain collapses to the 33.3% floor by era 4 without this layer and holds 36-43% with it. Compiles 0 errors, 0 warnings. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-29 12:34:29 -04:00
//+------------------------------------------------------------------+
//| |
//+------------------------------------------------------------------+
class CNeuronBatchNormOCL : public CNeuronBaseOCL
{
protected:
int iBatchSize; // EMA window length; <=1 disables normalization entirely
refactor(comments): stdlib comment style across the remaining in-scope files Same pass as 0b06f8e, applied file by file: comment runs of 4+ lines compressed to their leading topic sentences, capped at 4 lines, whole sentences only. Warning sentences (NEVER / MUST / trap / would-have) survive the budget. Every file was checked the same way before committing: the list of non-comment lines is byte-identical to HEAD, and braces balance. No code was touched. Panel/, Enumerations/ and the already-terse System headers needed little or nothing - PooledGate, TradeChecks, BinomialStats and Random came through with no blocks over the threshold at all. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-22 00:25:52 -04:00
//--- When true the running statistics are USED but not UPDATED, i.e. classic batch-norm
//--- inference semantics. Off by default (see the note on adaptation in the class header).
fix(ai): freeze batch-norm statistics when comparing two forward passes With normalization enabled a forward pass is not a pure function of its input - it also advances the running mean/variance. ValidateCpuInference compares the live backend net against a throwaway pure-MQL5 clone loaded from the just-saved .nnw, so its own reference pass left the live model one EMA step ahead of the file the clone reads. The check would then have been measuring its own side effect, and a marginal result decides whether buyers' backtests are allowed to run DLL-free. Adds CNet::SetBatchNormFrozen / CNeuronBatchNormOCL::SetStatsFrozen - classic batch-norm inference semantics, statistics used but not updated - and freezes both sides for the duration of the comparison. Not persisted: it is a transient evaluation mode, not model state. Default stays adaptive, which is what the rest of the system (online continual learning) is built around. Compiles 0 errors, 0 warnings. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-29 12:37:50 -04:00
bool bStatsFrozen;
feat(ai): batch normalization between dense layers The only bounded stage in the entire forward path was the sigmoid classification head - every hidden stage is PRELU. That is a network with no internal scale control, and the failure ordered exactly by depth: on SP500 H1 the shallow perceptron held ~52% balanced accuracy while the deepest topology sat on the 33.3% one-class floor, with the per-bar logit spread decaying monotonically (0.45 -> 0.38 over ~200 eras) until the evidence tilt fell under the class-prior tilt. That is the signature of internal covariate shift, which chapter 6.1 of the reference book is entirely about and which the NeuroNet_DNG engine addresses with a layer this project never had. Two mechanisms make this the right fix rather than more hyperparameter nudging: - it decouples WEIGHT_DECAY from the learned function (van Laarhoven 2017) - with a normalized layer downstream, decay can no longer grind the discriminative signal away, it only rescales the effective learning rate; - it is the precondition for ever running an unbounded logit head here. The 2026-07-27 attempt blew up (IS error 5.6e15) precisely because nothing upstream constrained scale. Implementation notes: - CNeuronBatchNormOCL computes host-side rather than as a fourth copy of a kernel across Network.cl + WarriorCPU.cpp + WarriorDML.cpp. The math is elementwise O(n); this way it behaves identically on all four compute tiers, needs no DLL rebuild, and cannot drift between backends. Same precedent as the softmax+CCE gradient and the per-sample loss weighting, both computed in MQL5 for that reason. - Statistics are exponential moving, not a stored mini-batch: training is pure online SGD, one update per sample, so there is no batch to average over. BatchNormWindow is an EMA window length. - gamma/beta are excluded from weight decay, deliberately - decaying gamma toward zero is the exact pathology being fixed. - The layer self-sizes from whatever sits below it, because a conv/pool stage's output width is derived inside the CNet constructor and is not knowable to the topology builder. - Checkpoint capture/restore/blend carry gamma/beta and the running statistics alongside the dense matrix, so the plateau ladder cannot restore a mismatched pair. - SeedOutputLayerBias accepted only an exact defNeuronBaseOCL as the weight-carrying penultimate layer; with normalization enabled that is the batch-norm layer, so the cold-start bias seed would have silently stopped being applied. - Refuses to build, loudly, if a topology asks for normalization with no compute backend at all - rather than quietly training a different architecture than the one requested. EnableBatchNorm (default on) and BatchNormWindow (1000 samples) are inputs so the effect can be A/B'd without a recompile. Both feed the weights-filename fingerprint, appended conditionally so existing non-BN configs keep their fingerprints and are not forced to retrain. Verified: analytic gradients match finite differences to 1.5e-7 relative over 200 random cases; a faithful port of the full forward/backward chain collapses to the 33.3% floor by era 4 without this layer and holds 36-43% with it. Compiles 0 errors, 0 warnings. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-29 12:34:29 -04:00
//--- Number of samples seen. Used only to ramp the effective window up from 1 to iBatchSize over
refactor(comments): stdlib comment style across the remaining in-scope files Same pass as 0b06f8e, applied file by file: comment runs of 4+ lines compressed to their leading topic sentences, capped at 4 lines, whole sentences only. Warning sentences (NEVER / MUST / trap / would-have) survive the budget. Every file was checked the same way before committing: the list of non-comment lines is byte-identical to HEAD, and braces balance. No code was touched. Panel/, Enumerations/ and the already-terse System headers needed little or nothing - PooledGate, TradeChecks, BinomialStats and Random came through with no blocks over the threshold at all. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-22 00:25:52 -04:00
//--- the first iBatchSize samples (standard EMA bias correction).
feat(ai): batch normalization between dense layers The only bounded stage in the entire forward path was the sigmoid classification head - every hidden stage is PRELU. That is a network with no internal scale control, and the failure ordered exactly by depth: on SP500 H1 the shallow perceptron held ~52% balanced accuracy while the deepest topology sat on the 33.3% one-class floor, with the per-bar logit spread decaying monotonically (0.45 -> 0.38 over ~200 eras) until the evidence tilt fell under the class-prior tilt. That is the signature of internal covariate shift, which chapter 6.1 of the reference book is entirely about and which the NeuroNet_DNG engine addresses with a layer this project never had. Two mechanisms make this the right fix rather than more hyperparameter nudging: - it decouples WEIGHT_DECAY from the learned function (van Laarhoven 2017) - with a normalized layer downstream, decay can no longer grind the discriminative signal away, it only rescales the effective learning rate; - it is the precondition for ever running an unbounded logit head here. The 2026-07-27 attempt blew up (IS error 5.6e15) precisely because nothing upstream constrained scale. Implementation notes: - CNeuronBatchNormOCL computes host-side rather than as a fourth copy of a kernel across Network.cl + WarriorCPU.cpp + WarriorDML.cpp. The math is elementwise O(n); this way it behaves identically on all four compute tiers, needs no DLL rebuild, and cannot drift between backends. Same precedent as the softmax+CCE gradient and the per-sample loss weighting, both computed in MQL5 for that reason. - Statistics are exponential moving, not a stored mini-batch: training is pure online SGD, one update per sample, so there is no batch to average over. BatchNormWindow is an EMA window length. - gamma/beta are excluded from weight decay, deliberately - decaying gamma toward zero is the exact pathology being fixed. - The layer self-sizes from whatever sits below it, because a conv/pool stage's output width is derived inside the CNet constructor and is not knowable to the topology builder. - Checkpoint capture/restore/blend carry gamma/beta and the running statistics alongside the dense matrix, so the plateau ladder cannot restore a mismatched pair. - SeedOutputLayerBias accepted only an exact defNeuronBaseOCL as the weight-carrying penultimate layer; with normalization enabled that is the batch-norm layer, so the cold-start bias seed would have silently stopped being applied. - Refuses to build, loudly, if a topology asks for normalization with no compute backend at all - rather than quietly training a different architecture than the one requested. EnableBatchNorm (default on) and BatchNormWindow (1000 samples) are inputs so the effect can be A/B'd without a recompile. Both feed the weights-filename fingerprint, appended conditionally so existing non-BN configs keep their fingerprints and are not forced to retrain. Verified: analytic gradients match finite differences to 1.5e-7 relative over 200 random cases; a faithful port of the full forward/backward chain collapses to the 33.3% floor by era 4 without this layer and holds 36-43% with it. Compiles 0 errors, 0 warnings. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-29 12:34:29 -04:00
int iSamplesSeen;
refactor(comments): stdlib comment style across the remaining in-scope files Same pass as 0b06f8e, applied file by file: comment runs of 4+ lines compressed to their leading topic sentences, capped at 4 lines, whole sentences only. Warning sentences (NEVER / MUST / trap / would-have) survive the budget. Every file was checked the same way before committing: the list of non-comment lines is byte-identical to HEAD, and braces balance. No code was touched. Panel/, Enumerations/ and the already-terse System headers needed little or nothing - PooledGate, TradeChecks, BinomialStats and Random came through with no blocks over the threshold at all. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-22 00:25:52 -04:00
//--- Persisted through Save/Load like any other parameter - gamma/beta are learned, and the
//--- running statistics ARE the layer's inference behaviour, so a model that loses them is not
//--- the model that was trained.
fix: live trades now use the geometry the gate certifies; perf: BN kernels Three changes, one theme: the trade placed, the trade graded, and the trade computed are now the same trade. 1) GEOMETRY WIRE (correctness, the ranked #1 open issue). The measured barrier pair reached the LABELS only - OpenParams still placed orders at the enum geometry (2*ATR/6*ATR), so the deploy gate certified "reaches 1.62*ATR before 3.33*ATR above break-even" about trades the EA never placed. Published via g_DerivedSlAtrMult/g_DerivedTpAtrMult (ConfidenceBridge, same same-tick contract as the confidence globals, because OpenParams runs on the root signal which has no pointer to the AI filter). Two writers: DeriveBarrierGeometry at era 0, and the .cfg adoption a deployed model takes. Overrides both legs and both Intelligent modes - the certificate is exact or it is nothing. TP is ATR-anchored like the label, NOT risk-relative, so a floor-widened stop cannot reshape the certified target. 2) BATCH NORM RUNS DEVICE-SIDE ON OPENCL. Four kernels in Network.cl - forward, hidden gradient, gamma/beta accumulate, gamma/beta apply - each a line-for-line transcription of the host implementation (NormalizeHost / HiddenGradHost / StepGammaBeta) including every NaN guard, clamp, and the exact moment-write ordering. The host copies remain the runtime for the DLL and pure-MQL5 tiers and the reference the kernels must match. Because this box has no OpenCL platform, the safety story is layered: - shim validation: kernels compiled as C and driven against a fp64 host transcription over NaN-poisoned stats, NaN gamma, over-clamp inputs, the frozen path, both optimizers, 3 batches - ALL PASS, worst normalized diff 0.132 vs tolerance 1.0 - in-situ self-check: each kernel is compared against its host twin ON FIRST USE on the real device (SelfCheckBn*), covering what the shim cannot - arg indices and buffer bindings. Any disagreement resyncs from the good copy, latches all BN kernels off process-wide, and training continues host-side. A transcription bug costs a warning and some speed, never a poisoned .nnw. - sync discipline: BatchOptions is now a CBufferDouble with explicit authority tracking (m_bnDeviceAuthoritative). Checkpoints/saves pull read-only; restores/loads/resets push; a mid-batch handover drains the device gamma/beta accumulator into the host arrays so no sample is lost. 3) SMALL FIXES. Apply-kernel build failure now latches the dispatch path at init (one warning instead of warning + failed Execute). Build tag bumped to win-scoring-gpu-v1 - first tag change since expectancy-stop-v1 despite five binary-changing commits. Both build variants compile 0 errors, 0 warnings. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-09 17:51:40 -04:00
CBufferDouble *BatchOptions;
feat: mini-batch gradient accumulation (F4), front-end-aware capacity budget (F6), split Wyckoff categoricals (N1) Completes the 2026-08-09 training audit. FORCES A RETRAIN of every Wyckoff-enabled config (N1 re-keys the fingerprint), and BOTH DLLs must be redeployed alongside the .ex5 - they carry new exports. F4 - mini-batch accumulation, TRAIN_BATCH_SIZE=32. Training was pure online SGD (one weight update per bar), which is the mechanical source of the era-to-era whipsaw every downstream guard was built to cope with. The O(n^2) outer product is native - AccumulateWeightGrad / AccumulateWeightGradConv / AccumulateBufferInto in Network.cl, WarriorCPU and WarriorDML - while the optimizer step is host-side MQL5 shared by all tiers (ApplyAccumToBlock), so there is one Adam/SGD implementation instead of four that can drift. - the LSTM needs no outer-product kernel (WeightsGradient already holds the sample's full dW) but could NOT simply be left un-zeroed between samples: CPU_LSTMSeqBackward/DML_LSTMSeqBackward memset it on entry. Hence a separate accumulator plus an elementwise add. - batch-norm gamma/beta accumulate in host arrays, not new BatchOptions slots - BN_OPT_STRIDE is baked into every persisted .nnw. - scoped to pass 2; online learning keeps immediate updates. Every save / checkpoint / scoring boundary flushes, scaling by the real sample count. - degrades to per-sample updates (one log line) on a tier that cannot accumulate, so old devices and DLL-free builds are unaffected. - verified offline: DirectML/batch_accum_check.cpp drives the real exports against an independent reference; at B=1 the accumulator matches the shipped unbatched kernel's own gradient to 1.1e-16. Math only - the in-situ check remains the per-layer dW/W report on a real era. F6 - ComputeFirstLayerWidth budgeted against the RAW input width even where a conv/LSTM front end had already reduced it, so an LSTM's dense stack was charged for 1,280 inputs when it receives 64. Confirmed from the deployed .cfg files: CONV, LSTM and HYBRID were all pinned at the 16-unit floor. Now budgeted against the front-end output and capped at it (never fan out), with the derivation reordered so both stages settle first. N1 - EventCode/EventPhase/StructuralPhase are signed categoricals packing direction and Wyckoff stage into one scalar across a sign discontinuity. Split into direction + [0,1] magnitude, the same convention the base OHLC block uses. Information-preserving; 13 readings now occupy 16 inputs. Compiled clean (0 errors, 0 warnings); both DLLs rebuilt. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-09 11:48:03 -04:00
//--- Mini-batch running sums of dL/dgamma and dL/dbeta, one entry per unit. Host-only and NOT
//--- persisted: transient within a batch, and every save point flushes first. See
//--- accumulateInputWeightGrads for why they are not extra BatchOptions slots.
double m_accGamma[];
double m_accBeta[];
refactor(comments): stdlib comment style across the remaining in-scope files Same pass as 0b06f8e, applied file by file: comment runs of 4+ lines compressed to their leading topic sentences, capped at 4 lines, whole sentences only. Warning sentences (NEVER / MUST / trap / would-have) survive the budget. Every file was checked the same way before committing: the list of non-comment lines is byte-identical to HEAD, and braces balance. No code was touched. Panel/, Enumerations/ and the already-terse System headers needed little or nothing - PooledGate, TradeChecks, BinomialStats and Random came through with no blocks over the threshold at all. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-22 00:25:52 -04:00
//--- PER-SAMPLE TRANSFER CACHES.
perf: batch norm stops re-reading buffers it already has Batch norm computes host-side while its neighbours are device-resident, so every value it touches crosses the bus - and the cost is the BLOCKING SYNC per crossing, not the bytes. Per sample it did four reads, two of which were exact duplicates: feedForward -> previous layer's Output calcInputGradients -> own Gradient, and the previous layer's Output AGAIN update/accumulate -> own Gradient AGAIN Nothing writes the previous layer's Output between the forward and backward passes, and nothing writes this layer's Gradient between the gradient pass and the weight-update pass - backPropOCL runs those as two separate top-to-bottom loops and only the first writes gradients. So the repeats are removable and the result is bit-exact: the same values, read once instead of twice. Each cache is armed by its producer and disarmed by feedForward, so a consumer whose producer did not run this sample falls back to reading the buffer rather than using the previous sample's data. That is not hypothetical - a batch norm at layer 1 never gets calcInputGradients called at all, the same asymmetry backPropOCL already documents for a layer-1 LSTM, so its gradient cache is never armed and it takes the fallback every time. Both build variants compile 0 errors, 0 warnings. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-09 17:16:08 -04:00
double m_fwdInputCache[];
bool m_fwdInputCached;
double m_gradCache[];
bool m_gradCached;
fix: live trades now use the geometry the gate certifies; perf: BN kernels Three changes, one theme: the trade placed, the trade graded, and the trade computed are now the same trade. 1) GEOMETRY WIRE (correctness, the ranked #1 open issue). The measured barrier pair reached the LABELS only - OpenParams still placed orders at the enum geometry (2*ATR/6*ATR), so the deploy gate certified "reaches 1.62*ATR before 3.33*ATR above break-even" about trades the EA never placed. Published via g_DerivedSlAtrMult/g_DerivedTpAtrMult (ConfidenceBridge, same same-tick contract as the confidence globals, because OpenParams runs on the root signal which has no pointer to the AI filter). Two writers: DeriveBarrierGeometry at era 0, and the .cfg adoption a deployed model takes. Overrides both legs and both Intelligent modes - the certificate is exact or it is nothing. TP is ATR-anchored like the label, NOT risk-relative, so a floor-widened stop cannot reshape the certified target. 2) BATCH NORM RUNS DEVICE-SIDE ON OPENCL. Four kernels in Network.cl - forward, hidden gradient, gamma/beta accumulate, gamma/beta apply - each a line-for-line transcription of the host implementation (NormalizeHost / HiddenGradHost / StepGammaBeta) including every NaN guard, clamp, and the exact moment-write ordering. The host copies remain the runtime for the DLL and pure-MQL5 tiers and the reference the kernels must match. Because this box has no OpenCL platform, the safety story is layered: - shim validation: kernels compiled as C and driven against a fp64 host transcription over NaN-poisoned stats, NaN gamma, over-clamp inputs, the frozen path, both optimizers, 3 batches - ALL PASS, worst normalized diff 0.132 vs tolerance 1.0 - in-situ self-check: each kernel is compared against its host twin ON FIRST USE on the real device (SelfCheckBn*), covering what the shim cannot - arg indices and buffer bindings. Any disagreement resyncs from the good copy, latches all BN kernels off process-wide, and training continues host-side. A transcription bug costs a warning and some speed, never a poisoned .nnw. - sync discipline: BatchOptions is now a CBufferDouble with explicit authority tracking (m_bnDeviceAuthoritative). Checkpoints/saves pull read-only; restores/loads/resets push; a mid-batch handover drains the device gamma/beta accumulator into the host arrays so no sample is lost. 3) SMALL FIXES. Apply-kernel build failure now latches the dispatch path at init (one warning instead of warning + failed Execute). Build tag bumped to win-scoring-gpu-v1 - first tag change since expectancy-stop-v1 despite five binary-changing commits. Both build variants compile 0 errors, 0 warnings. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-09 17:51:40 -04:00
//--- DEVICE PATH (OpenCL only, 2026-08-09). m_bnAcc holds the mini-batch gamma/beta gradient sums
//--- on the device (2 floats per unit: gamma then beta), the kernel twin of m_accGamma/m_accBeta.
//--- m_bnDeviceAuthoritative says the DEVICE copy of BatchOptions is the truth (kernels have
//--- written it since the last host sync); the m_bnChecked* flags latch each kernel's one-time
//--- self-check against its host twin. The checks are the whole safety story for shipping kernels
//--- that could not be built on the dev machine: a transcription or dispatch-binding error is
//--- caught on its first use, the layer resyncs from the good copy, latches the kernels off
//--- process-wide, and training continues host-side - a warning and some speed, never a poisoned
//--- .nnw.
CBufferDouble *m_bnAcc;
bool m_bnDeviceAuthoritative;
bool m_bnCheckedFwd;
bool m_bnCheckedGrad;
bool m_bnCheckedAccum;
bool m_bnCheckedApply;
//--- eligibility + buffer management for the kernel path
bool BnDeviceEligible(void);
bool EnsureBnDeviceBuffers(void);
//--- read-only pull of the device statistics into the host mirror (checkpoints/saves mid-training);
//--- device stays authoritative
void SyncOptionsToHost(void);
//--- full handover to the host path: pull statistics, drain the device accumulator into
//--- m_accGamma/m_accBeta so a mid-batch handover loses nothing, clear the flag
void EnsureHostAuthoritative(void);
//--- one-way process-wide latch + this layer's handover, with the reason printed once
void LatchBnKernelsOff(const string reason);
//--- kernel dispatches (return false on any SetArgument/Execute failure, no logging - the caller
//--- decides between latching and falling back)
bool DispatchBnForward(CNeuronBaseOCL *NeuronOCL, double w);
bool DispatchBnHiddenGrad(CNeuronBaseOCL *NeuronOCL);
bool DispatchBnAccum(void);
bool DispatchBnApply(double scale, double lt);
//--- one-time kernel-vs-host comparisons; each returns the OPERATION's result (true = the work got
//--- done correctly, by whichever path survived), never "the kernel matched"
bool SelfCheckBnForward(CNeuronBaseOCL *NeuronOCL);
bool SelfCheckBnHiddenGrad(CNeuronBaseOCL *NeuronOCL);
bool SelfCheckBnAccum(void);
bool SelfCheckBnApply(double scale, double lt);
refactor(comments): stdlib comment style across the remaining in-scope files Same pass as 0b06f8e, applied file by file: comment runs of 4+ lines compressed to their leading topic sentences, capped at 4 lines, whole sentences only. Warning sentences (NEVER / MUST / trap / would-have) survive the budget. Every file was checked the same way before committing: the list of non-comment lines is byte-identical to HEAD, and braces balance. No code was touched. Panel/, Enumerations/ and the already-terse System headers needed little or nothing - PooledGate, TradeChecks, BinomialStats and Random came through with no blocks over the threshold at all. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-22 00:25:52 -04:00
//--- normalized disagreement: |got-ref| / (1e-3 * max(1,|ref|)), <=1 passes. DBL_MAX when
//--- exactly one side is non-finite.
fix: live trades now use the geometry the gate certifies; perf: BN kernels Three changes, one theme: the trade placed, the trade graded, and the trade computed are now the same trade. 1) GEOMETRY WIRE (correctness, the ranked #1 open issue). The measured barrier pair reached the LABELS only - OpenParams still placed orders at the enum geometry (2*ATR/6*ATR), so the deploy gate certified "reaches 1.62*ATR before 3.33*ATR above break-even" about trades the EA never placed. Published via g_DerivedSlAtrMult/g_DerivedTpAtrMult (ConfidenceBridge, same same-tick contract as the confidence globals, because OpenParams runs on the root signal which has no pointer to the AI filter). Two writers: DeriveBarrierGeometry at era 0, and the .cfg adoption a deployed model takes. Overrides both legs and both Intelligent modes - the certificate is exact or it is nothing. TP is ATR-anchored like the label, NOT risk-relative, so a floor-widened stop cannot reshape the certified target. 2) BATCH NORM RUNS DEVICE-SIDE ON OPENCL. Four kernels in Network.cl - forward, hidden gradient, gamma/beta accumulate, gamma/beta apply - each a line-for-line transcription of the host implementation (NormalizeHost / HiddenGradHost / StepGammaBeta) including every NaN guard, clamp, and the exact moment-write ordering. The host copies remain the runtime for the DLL and pure-MQL5 tiers and the reference the kernels must match. Because this box has no OpenCL platform, the safety story is layered: - shim validation: kernels compiled as C and driven against a fp64 host transcription over NaN-poisoned stats, NaN gamma, over-clamp inputs, the frozen path, both optimizers, 3 batches - ALL PASS, worst normalized diff 0.132 vs tolerance 1.0 - in-situ self-check: each kernel is compared against its host twin ON FIRST USE on the real device (SelfCheckBn*), covering what the shim cannot - arg indices and buffer bindings. Any disagreement resyncs from the good copy, latches all BN kernels off process-wide, and training continues host-side. A transcription bug costs a warning and some speed, never a poisoned .nnw. - sync discipline: BatchOptions is now a CBufferDouble with explicit authority tracking (m_bnDeviceAuthoritative). Checkpoints/saves pull read-only; restores/loads/resets push; a mid-batch handover drains the device gamma/beta accumulator into the host arrays so no sample is lost. 3) SMALL FIXES. Apply-kernel build failure now latches the dispatch path at init (one warning instead of warning + failed Execute). Build tag bumped to win-scoring-gpu-v1 - first tag change since expectancy-stop-v1 despite five binary-changing commits. Both build variants compile 0 errors, 0 warnings. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-09 17:51:40 -04:00
double BnDiffScore(double ref, double got);
//--- host twin of the backward elementwise math, factored out of calcInputGradients so the
//--- self-check compares against literally the same code the host path runs
void HiddenGradHost(const double &grad[], const double &prevOut[],
ENUM_ACTIVATION act, double &ig[], int n);
feat(ai): batch normalization between dense layers The only bounded stage in the entire forward path was the sigmoid classification head - every hidden stage is PRELU. That is a network with no internal scale control, and the failure ordered exactly by depth: on SP500 H1 the shallow perceptron held ~52% balanced accuracy while the deepest topology sat on the 33.3% one-class floor, with the per-bar logit spread decaying monotonically (0.45 -> 0.38 over ~200 eras) until the evidence tilt fell under the class-prior tilt. That is the signature of internal covariate shift, which chapter 6.1 of the reference book is entirely about and which the NeuroNet_DNG engine addresses with a layer this project never had. Two mechanisms make this the right fix rather than more hyperparameter nudging: - it decouples WEIGHT_DECAY from the learned function (van Laarhoven 2017) - with a normalized layer downstream, decay can no longer grind the discriminative signal away, it only rescales the effective learning rate; - it is the precondition for ever running an unbounded logit head here. The 2026-07-27 attempt blew up (IS error 5.6e15) precisely because nothing upstream constrained scale. Implementation notes: - CNeuronBatchNormOCL computes host-side rather than as a fourth copy of a kernel across Network.cl + WarriorCPU.cpp + WarriorDML.cpp. The math is elementwise O(n); this way it behaves identically on all four compute tiers, needs no DLL rebuild, and cannot drift between backends. Same precedent as the softmax+CCE gradient and the per-sample loss weighting, both computed in MQL5 for that reason. - Statistics are exponential moving, not a stored mini-batch: training is pure online SGD, one update per sample, so there is no batch to average over. BatchNormWindow is an EMA window length. - gamma/beta are excluded from weight decay, deliberately - decaying gamma toward zero is the exact pathology being fixed. - The layer self-sizes from whatever sits below it, because a conv/pool stage's output width is derived inside the CNet constructor and is not knowable to the topology builder. - Checkpoint capture/restore/blend carry gamma/beta and the running statistics alongside the dense matrix, so the plateau ladder cannot restore a mismatched pair. - SeedOutputLayerBias accepted only an exact defNeuronBaseOCL as the weight-carrying penultimate layer; with normalization enabled that is the batch-norm layer, so the cold-start bias seed would have silently stopped being applied. - Refuses to build, loudly, if a topology asks for normalization with no compute backend at all - rather than quietly training a different architecture than the one requested. EnableBatchNorm (default on) and BatchNormWindow (1000 samples) are inputs so the effect can be A/B'd without a recompile. Both feed the weights-filename fingerprint, appended conditionally so existing non-BN configs keep their fingerprints and are not forced to retrain. Verified: analytic gradients match finite differences to 1.5e-7 relative over 200 random cases; a faithful port of the full forward/backward chain collapses to the 33.3% floor by era 4 without this layer and holds 36-43% with it. Compiles 0 errors, 0 warnings. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-29 12:34:29 -04:00
//---
virtual bool feedForward(CNeuronBaseOCL *NeuronOCL);
virtual bool feedForwardCPU(CNeuronBaseOCL *NeuronOCL);
virtual bool updateInputWeights(CNeuronBaseOCL *NeuronOCL);
feat: mini-batch gradient accumulation (F4), front-end-aware capacity budget (F6), split Wyckoff categoricals (N1) Completes the 2026-08-09 training audit. FORCES A RETRAIN of every Wyckoff-enabled config (N1 re-keys the fingerprint), and BOTH DLLs must be redeployed alongside the .ex5 - they carry new exports. F4 - mini-batch accumulation, TRAIN_BATCH_SIZE=32. Training was pure online SGD (one weight update per bar), which is the mechanical source of the era-to-era whipsaw every downstream guard was built to cope with. The O(n^2) outer product is native - AccumulateWeightGrad / AccumulateWeightGradConv / AccumulateBufferInto in Network.cl, WarriorCPU and WarriorDML - while the optimizer step is host-side MQL5 shared by all tiers (ApplyAccumToBlock), so there is one Adam/SGD implementation instead of four that can drift. - the LSTM needs no outer-product kernel (WeightsGradient already holds the sample's full dW) but could NOT simply be left un-zeroed between samples: CPU_LSTMSeqBackward/DML_LSTMSeqBackward memset it on entry. Hence a separate accumulator plus an elementwise add. - batch-norm gamma/beta accumulate in host arrays, not new BatchOptions slots - BN_OPT_STRIDE is baked into every persisted .nnw. - scoped to pass 2; online learning keeps immediate updates. Every save / checkpoint / scoring boundary flushes, scaling by the real sample count. - degrades to per-sample updates (one log line) on a tier that cannot accumulate, so old devices and DLL-free builds are unaffected. - verified offline: DirectML/batch_accum_check.cpp drives the real exports against an independent reference; at B=1 the accumulator matches the shipped unbatched kernel's own gradient to 1.1e-16. Math only - the in-situ check remains the per-layer dW/W report on a real era. F6 - ComputeFirstLayerWidth budgeted against the RAW input width even where a conv/LSTM front end had already reduced it, so an LSTM's dense stack was charged for 1,280 inputs when it receives 64. Confirmed from the deployed .cfg files: CONV, LSTM and HYBRID were all pinned at the 16-unit floor. Now budgeted against the front-end output and capped at it (never fan out), with the derivation reordered so both stages settle first. N1 - EventCode/EventPhase/StructuralPhase are signed categoricals packing direction and Wyckoff stage into one scalar across a sign discontinuity. Split into direction + [0,1] magnitude, the same convention the base OHLC block uses. Information-preserving; 13 readings now occupy 16 inputs. Compiled clean (0 errors, 0 warnings); both DLLs rebuilt. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-09 11:48:03 -04:00
virtual bool accumulateInputWeightGrads(CNeuronBaseOCL *NeuronOCL);
//--- one unit's gamma/beta step, shared by the per-sample and per-batch paths
bool StepGammaBeta(int shift, double gGamma, double gBeta, double lt);
public:
virtual bool BeginGradAccum(void);
virtual bool ApplyAccumulatedGradients(double scale);
protected:
feat(ai): batch normalization between dense layers The only bounded stage in the entire forward path was the sigmoid classification head - every hidden stage is PRELU. That is a network with no internal scale control, and the failure ordered exactly by depth: on SP500 H1 the shallow perceptron held ~52% balanced accuracy while the deepest topology sat on the 33.3% one-class floor, with the per-bar logit spread decaying monotonically (0.45 -> 0.38 over ~200 eras) until the evidence tilt fell under the class-prior tilt. That is the signature of internal covariate shift, which chapter 6.1 of the reference book is entirely about and which the NeuroNet_DNG engine addresses with a layer this project never had. Two mechanisms make this the right fix rather than more hyperparameter nudging: - it decouples WEIGHT_DECAY from the learned function (van Laarhoven 2017) - with a normalized layer downstream, decay can no longer grind the discriminative signal away, it only rescales the effective learning rate; - it is the precondition for ever running an unbounded logit head here. The 2026-07-27 attempt blew up (IS error 5.6e15) precisely because nothing upstream constrained scale. Implementation notes: - CNeuronBatchNormOCL computes host-side rather than as a fourth copy of a kernel across Network.cl + WarriorCPU.cpp + WarriorDML.cpp. The math is elementwise O(n); this way it behaves identically on all four compute tiers, needs no DLL rebuild, and cannot drift between backends. Same precedent as the softmax+CCE gradient and the per-sample loss weighting, both computed in MQL5 for that reason. - Statistics are exponential moving, not a stored mini-batch: training is pure online SGD, one update per sample, so there is no batch to average over. BatchNormWindow is an EMA window length. - gamma/beta are excluded from weight decay, deliberately - decaying gamma toward zero is the exact pathology being fixed. - The layer self-sizes from whatever sits below it, because a conv/pool stage's output width is derived inside the CNet constructor and is not knowable to the topology builder. - Checkpoint capture/restore/blend carry gamma/beta and the running statistics alongside the dense matrix, so the plateau ladder cannot restore a mismatched pair. - SeedOutputLayerBias accepted only an exact defNeuronBaseOCL as the weight-carrying penultimate layer; with normalization enabled that is the batch-norm layer, so the cold-start bias seed would have silently stopped being applied. - Refuses to build, loudly, if a topology asks for normalization with no compute backend at all - rather than quietly training a different architecture than the one requested. EnableBatchNorm (default on) and BatchNormWindow (1000 samples) are inputs so the effect can be A/B'd without a recompile. Both feed the weights-filename fingerprint, appended conditionally so existing non-BN configs keep their fingerprints and are not forced to retrain. Verified: analytic gradients match finite differences to 1.5e-7 relative over 200 random cases; a faithful port of the full forward/backward chain collapses to the 33.3% floor by era 4 without this layer and holds 36-43% with it. Compiles 0 errors, 0 warnings. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-29 12:34:29 -04:00
//--- shared by feedForward/feedForwardCPU: the whole forward transform for one already-read input
//--- vector, writing straight into the host mirror of Output.
bool NormalizeHost(const double &inputs[], int count);
//--- ensures BatchOptions exists and is sized/seeded for `neurons` units
bool InitOptions(int neurons);
public:
CNeuronBatchNormOCL(void);
~CNeuronBatchNormOCL(void);
//---
virtual bool Init(uint numOutputs, uint myIndex, COpenCLMy *open_cl, uint numNeurons, uint batchSize, ENUM_OPTIMIZATION optimization_type);
virtual bool Init(uint numOutputs, uint myIndex, CDirectMLMy *direct_ml, uint numNeurons, uint batchSize, ENUM_OPTIMIZATION optimization_type);
virtual bool calcInputGradients(CNeuronBaseOCL *NeuronOCL);
fix(ai): freeze batch-norm statistics when comparing two forward passes With normalization enabled a forward pass is not a pure function of its input - it also advances the running mean/variance. ValidateCpuInference compares the live backend net against a throwaway pure-MQL5 clone loaded from the just-saved .nnw, so its own reference pass left the live model one EMA step ahead of the file the clone reads. The check would then have been measuring its own side effect, and a marginal result decides whether buyers' backtests are allowed to run DLL-free. Adds CNet::SetBatchNormFrozen / CNeuronBatchNormOCL::SetStatsFrozen - classic batch-norm inference semantics, statistics used but not updated - and freezes both sides for the duration of the comparison. Not persisted: it is a transient evaluation mode, not model state. Default stays adaptive, which is what the rest of the system (online continual learning) is built around. Compiles 0 errors, 0 warnings. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-29 12:37:50 -04:00
//--- see bStatsFrozen. Deliberately NOT persisted: it is a transient evaluation mode, not model state.
void SetStatsFrozen(bool v) { bStatsFrozen = v; }
feat(hud): per-member neuron lines + a vote label that moves as the nets learn Both 2026-08-19 reports were the same staleness: every source behind the label was an ERA artifact (live cache refills at pass-3 completion, the snapshot copies once per era, dPrevSignal is the frozen purge-band edge bar) - so the readout stepped at era cadence at best, stayed glued to one direction, and lagged the era counter. DisplayInference(): throttled (4s, 1s across an era boundary), SIDE-EFFECT-FREE forward of the current decision bar (window ending on bar 1, same question the live path asks) through the LEARNER net. Batch-norm running stats are bracketed frozen/RESTORED via the new CNet::GetBatchNormFrozen() + CNeuronBatchNormOCL::StatsFrozen() - restore, not unfreeze, because a display tick can land between pass-3 chunks whose whole scan holds them frozen. Writes nothing a trading or training path reads (dPrevSignal, NMS state, tallies, watermarks all untouched; RefreshLatestSignal is not reusable here precisely because it writes all of them). LSTM safe by construction: h/c zeroed per forward. ProspectiveVote() reads the fresh forward as its FIRST source; the era-artifact chain becomes the fallback (meta head, warm-up, window holes). DisplayHudLine(): the reference library's training label, per ensemble member - name, output activations (softmax probs or raw scalar), the decision, its weighted vote (the exact consensus numerator term), era, recent average error, "(trn)" while not vote-capable. Rendered under the vote line in RefreshVoteReadout BEFORE the live-vote defer (member lines are telemetry, not tradable readings), coloured by the member's own direction in muted tones - the vote line's strict green-only-when-it-would-trade rule is untouched. NOT COMPILED - user compiles in MetaEditor. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-19 08:48:34 -04:00
//--- Read-back for save/restore bracketing (CNet::GetBatchNormFrozen): a display-only forward
//--- must put the flag back the way it FOUND it, not assume the trainer wanted it unfrozen -
//--- pass 3 and the online-learning probes hold it frozen across their whole scan.
bool StatsFrozen(void) const { return bStatsFrozen; }
refactor(comments): stdlib comment style across the remaining in-scope files Same pass as 0b06f8e, applied file by file: comment runs of 4+ lines compressed to their leading topic sentences, capped at 4 lines, whole sentences only. Warning sentences (NEVER / MUST / trap / would-have) survive the budget. Every file was checked the same way before committing: the list of non-comment lines is byte-identical to HEAD, and braces balance. No code was touched. Panel/, Enumerations/ and the already-terse System headers needed little or nothing - PooledGate, TradeChecks, BinomialStats and Random came through with no blocks over the threshold at all. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-22 00:25:52 -04:00
//--- Checkpoint/blend support. Without this the plateau ladder's "restore best checkpoint" would
//--- put the dense weights back while leaving this layer's parameters at whatever the diverged
//--- era left behind - a silently mismatched pair.
feat(ai): batch normalization between dense layers The only bounded stage in the entire forward path was the sigmoid classification head - every hidden stage is PRELU. That is a network with no internal scale control, and the failure ordered exactly by depth: on SP500 H1 the shallow perceptron held ~52% balanced accuracy while the deepest topology sat on the 33.3% one-class floor, with the per-bar logit spread decaying monotonically (0.45 -> 0.38 over ~200 eras) until the evidence tilt fell under the class-prior tilt. That is the signature of internal covariate shift, which chapter 6.1 of the reference book is entirely about and which the NeuroNet_DNG engine addresses with a layer this project never had. Two mechanisms make this the right fix rather than more hyperparameter nudging: - it decouples WEIGHT_DECAY from the learned function (van Laarhoven 2017) - with a normalized layer downstream, decay can no longer grind the discriminative signal away, it only rescales the effective learning rate; - it is the precondition for ever running an unbounded logit head here. The 2026-07-27 attempt blew up (IS error 5.6e15) precisely because nothing upstream constrained scale. Implementation notes: - CNeuronBatchNormOCL computes host-side rather than as a fourth copy of a kernel across Network.cl + WarriorCPU.cpp + WarriorDML.cpp. The math is elementwise O(n); this way it behaves identically on all four compute tiers, needs no DLL rebuild, and cannot drift between backends. Same precedent as the softmax+CCE gradient and the per-sample loss weighting, both computed in MQL5 for that reason. - Statistics are exponential moving, not a stored mini-batch: training is pure online SGD, one update per sample, so there is no batch to average over. BatchNormWindow is an EMA window length. - gamma/beta are excluded from weight decay, deliberately - decaying gamma toward zero is the exact pathology being fixed. - The layer self-sizes from whatever sits below it, because a conv/pool stage's output width is derived inside the CNet constructor and is not knowable to the topology builder. - Checkpoint capture/restore/blend carry gamma/beta and the running statistics alongside the dense matrix, so the plateau ladder cannot restore a mismatched pair. - SeedOutputLayerBias accepted only an exact defNeuronBaseOCL as the weight-carrying penultimate layer; with normalization enabled that is the batch-norm layer, so the cold-start bias seed would have silently stopped being applied. - Refuses to build, loudly, if a topology asks for normalization with no compute backend at all - rather than quietly training a different architecture than the one requested. EnableBatchNorm (default on) and BatchNormWindow (1000 samples) are inputs so the effect can be A/B'd without a recompile. Both feed the weights-filename fingerprint, appended conditionally so existing non-BN configs keep their fingerprints and are not forced to retrain. Verified: analytic gradients match finite differences to 1.5e-7 relative over 200 random cases; a faithful port of the full forward/backward chain collapses to the 33.3% floor by era 4 without this layer and holds 36-43% with it. Compiles 0 errors, 0 warnings. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-29 12:34:29 -04:00
virtual int getWeightsBN(double &values[]);
virtual bool setWeightsBN(double &values[]);
fix: correct MI sample alignment and improve BN weight diagnostic report The MI sample builder used `MathAbs(labelBarOffset)` as a padding, causing rows from offset and non-offset builds to be paired with a double shift. This broke the positive control, failed the 5× gate, and voided all reported mutual‑information figures. Replace with the fixed `MiShiftPad` constant to ensure builds enumerate the same set of bars and row-k alignment is preserved. Add `BatchOptionsTotal()` to `CNeuronBatchNormOCL` and split the packed BN weight array in the learning report into separate norms for the outgoing dense matrix, gamma, beta, running statistics, and Adam moment buffers. This turns an ambiguous single‑norm reading into precise diagnostics that distinguish weight divergence from scaling issues.
2026-08-02 08:12:47 -04:00
//--- How many of the trailing entries in getWeightsBN's array are BatchOptions rather than the
refactor(comments): stdlib comment style across the remaining in-scope files Same pass as 0b06f8e, applied file by file: comment runs of 4+ lines compressed to their leading topic sentences, capped at 4 lines, whole sentences only. Warning sentences (NEVER / MUST / trap / would-have) survive the budget. Every file was checked the same way before committing: the list of non-comment lines is byte-identical to HEAD, and braces balance. No code was touched. Panel/, Enumerations/ and the already-terse System headers needed little or nothing - PooledGate, TradeChecks, BinomialStats and Random came through with no blocks over the threshold at all. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-22 00:25:52 -04:00
//--- outgoing dense matrix.
fix: correct MI sample alignment and improve BN weight diagnostic report The MI sample builder used `MathAbs(labelBarOffset)` as a padding, causing rows from offset and non-offset builds to be paired with a double shift. This broke the positive control, failed the 5× gate, and voided all reported mutual‑information figures. Replace with the fixed `MiShiftPad` constant to ensure builds enumerate the same set of bars and row-k alignment is preserved. Add `BatchOptionsTotal()` to `CNeuronBatchNormOCL` and split the packed BN weight array in the learning report into separate norms for the outgoing dense matrix, gamma, beta, running statistics, and Adam moment buffers. This turns an ambiguous single‑norm reading into precise diagnostics that distinguish weight divergence from scaling issues.
2026-08-02 08:12:47 -04:00
int BatchOptionsTotal(void) const
{
return (CheckPointer(BatchOptions) == POINTER_INVALID) ? 0 : BatchOptions.Total();
}
refactor(comments): stdlib comment style across the remaining in-scope files Same pass as 0b06f8e, applied file by file: comment runs of 4+ lines compressed to their leading topic sentences, capped at 4 lines, whole sentences only. Warning sentences (NEVER / MUST / trap / would-have) survive the budget. Every file was checked the same way before committing: the list of non-comment lines is byte-identical to HEAD, and braces balance. No code was touched. Panel/, Enumerations/ and the already-terse System headers needed little or nothing - PooledGate, TradeChecks, BinomialStats and Random came through with no blocks over the threshold at all. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-22 00:25:52 -04:00
//--- Zero ONLY the gamma/beta moment slots (BN_OPT_MG/MB/VG/VB) plus the base class's buffers
//--- for the outgoing dense matrix. See CNet::ResetOptimizerState.
fix: training-stability audit fixes F1/F2/F3/F5 - unbiased shuffle, real plateau escapes, fresh optimizer state on restore, pure OOS metric Four of the six findings from research/training_pipeline_audit_2026-08-09.md (F4 mini-batching and F6 feature re-encode deliberately deferred - see the report's implementation-status section for why): - F1: pass-2 Fisher-Yates (and AutoTune's MI block shuffle) used MathRand()%, which is 15-bit - provably non-uniform on every full-history era over 32,768 queued samples. New 30-bit ShuffleRandomIndex(). - F2: plateau warm restarts were a no-op whenever eta already sat at its ceiling (the normal state of a non-regressing plateau) - the ladder was just a 24-era countdown. Restarts now overshoot to 5x the ceiling (PLATEAU_RESTART_BOOST) and anneal geometrically back over the patience window, SGDR-style; ETA_MIN widened 1e-4 -> 1e-5 so the decay schedule has real range. - F3: checkpoint restores put weights back but kept the rejected trajectory's Adam moments, so the optimizer immediately pushed back toward the rolled-back state (the restore->regress->restore oscillation). CNet::ResetOptimizerState() zeroes moments/momentum/step counters (weights, BN statistics, gamma/beta untouched) on every mid-run restore, every boosted restart, and the deploy-time restore that online learning continues from. - F5: batch-norm running statistics now freeze for the pass-3 OOS scoring walk, so the selection metric the checkpoint ranking and deploy gate read is a pure function of the checkpoint instead of partly measuring BN drift. Defensive unfreeze in FinalizeTrainRun covers stop-mid-pass; live/online adaptation and the OOS continual-learning simulation stay adaptive by design. Compiled clean (0 errors, 0 warnings) via the staged-tree recipe. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-09 10:54:09 -04:00
virtual bool ResetOptimizerState(void)
{
bool ok = CNeuronBaseOCL::ResetOptimizerState();
if(CheckPointer(BatchOptions) != POINTER_INVALID)
{
fix: live trades now use the geometry the gate certifies; perf: BN kernels Three changes, one theme: the trade placed, the trade graded, and the trade computed are now the same trade. 1) GEOMETRY WIRE (correctness, the ranked #1 open issue). The measured barrier pair reached the LABELS only - OpenParams still placed orders at the enum geometry (2*ATR/6*ATR), so the deploy gate certified "reaches 1.62*ATR before 3.33*ATR above break-even" about trades the EA never placed. Published via g_DerivedSlAtrMult/g_DerivedTpAtrMult (ConfidenceBridge, same same-tick contract as the confidence globals, because OpenParams runs on the root signal which has no pointer to the AI filter). Two writers: DeriveBarrierGeometry at era 0, and the .cfg adoption a deployed model takes. Overrides both legs and both Intelligent modes - the certificate is exact or it is nothing. TP is ATR-anchored like the label, NOT risk-relative, so a floor-widened stop cannot reshape the certified target. 2) BATCH NORM RUNS DEVICE-SIDE ON OPENCL. Four kernels in Network.cl - forward, hidden gradient, gamma/beta accumulate, gamma/beta apply - each a line-for-line transcription of the host implementation (NormalizeHost / HiddenGradHost / StepGammaBeta) including every NaN guard, clamp, and the exact moment-write ordering. The host copies remain the runtime for the DLL and pure-MQL5 tiers and the reference the kernels must match. Because this box has no OpenCL platform, the safety story is layered: - shim validation: kernels compiled as C and driven against a fp64 host transcription over NaN-poisoned stats, NaN gamma, over-clamp inputs, the frozen path, both optimizers, 3 batches - ALL PASS, worst normalized diff 0.132 vs tolerance 1.0 - in-situ self-check: each kernel is compared against its host twin ON FIRST USE on the real device (SelfCheckBn*), covering what the shim cannot - arg indices and buffer bindings. Any disagreement resyncs from the good copy, latches all BN kernels off process-wide, and training continues host-side. A transcription bug costs a warning and some speed, never a poisoned .nnw. - sync discipline: BatchOptions is now a CBufferDouble with explicit authority tracking (m_bnDeviceAuthoritative). Checkpoints/saves pull read-only; restores/loads/resets push; a mid-batch handover drains the device gamma/beta accumulator into the host arrays so no sample is lost. 3) SMALL FIXES. Apply-kernel build failure now latches the dispatch path at init (one warning instead of warning + failed Execute). Build tag bumped to win-scoring-gpu-v1 - first tag change since expectancy-stop-v1 despite five binary-changing commits. Both build variants compile 0 errors, 0 warnings. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-09 17:51:40 -04:00
//--- Kernel-mode discipline: this zeroes SOME slots of a block whose truth may live on the
//--- device, so pull first (or the untouched slots would be written back stale), zero, push.
SyncOptionsToHost();
fix: training-stability audit fixes F1/F2/F3/F5 - unbiased shuffle, real plateau escapes, fresh optimizer state on restore, pure OOS metric Four of the six findings from research/training_pipeline_audit_2026-08-09.md (F4 mini-batching and F6 feature re-encode deliberately deferred - see the report's implementation-status section for why): - F1: pass-2 Fisher-Yates (and AutoTune's MI block shuffle) used MathRand()%, which is 15-bit - provably non-uniform on every full-history era over 32,768 queued samples. New 30-bit ShuffleRandomIndex(). - F2: plateau warm restarts were a no-op whenever eta already sat at its ceiling (the normal state of a non-regressing plateau) - the ladder was just a 24-era countdown. Restarts now overshoot to 5x the ceiling (PLATEAU_RESTART_BOOST) and anneal geometrically back over the patience window, SGDR-style; ETA_MIN widened 1e-4 -> 1e-5 so the decay schedule has real range. - F3: checkpoint restores put weights back but kept the rejected trajectory's Adam moments, so the optimizer immediately pushed back toward the rolled-back state (the restore->regress->restore oscillation). CNet::ResetOptimizerState() zeroes moments/momentum/step counters (weights, BN statistics, gamma/beta untouched) on every mid-run restore, every boosted restart, and the deploy-time restore that online learning continues from. - F5: batch-norm running statistics now freeze for the pass-3 OOS scoring walk, so the selection metric the checkpoint ranking and deploy gate read is a pure function of the checkpoint instead of partly measuring BN drift. Defensive unfreeze in FinalizeTrainRun covers stop-mid-pass; live/online adaptation and the OOS continual-learning simulation stay adaptive by design. Compiled clean (0 errors, 0 warnings) via the staged-tree recipe. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-09 10:54:09 -04:00
int totalSlots = BatchOptions.Total();
for(int shift = 0; shift + BN_OPT_VB < totalSlots; shift += BN_OPT_STRIDE)
{
ok = BatchOptions.Update(shift + BN_OPT_MG, 0.0) && ok;
ok = BatchOptions.Update(shift + BN_OPT_MB, 0.0) && ok;
ok = BatchOptions.Update(shift + BN_OPT_VG, 0.0) && ok;
ok = BatchOptions.Update(shift + BN_OPT_VB, 0.0) && ok;
}
fix: live trades now use the geometry the gate certifies; perf: BN kernels Three changes, one theme: the trade placed, the trade graded, and the trade computed are now the same trade. 1) GEOMETRY WIRE (correctness, the ranked #1 open issue). The measured barrier pair reached the LABELS only - OpenParams still placed orders at the enum geometry (2*ATR/6*ATR), so the deploy gate certified "reaches 1.62*ATR before 3.33*ATR above break-even" about trades the EA never placed. Published via g_DerivedSlAtrMult/g_DerivedTpAtrMult (ConfidenceBridge, same same-tick contract as the confidence globals, because OpenParams runs on the root signal which has no pointer to the AI filter). Two writers: DeriveBarrierGeometry at era 0, and the .cfg adoption a deployed model takes. Overrides both legs and both Intelligent modes - the certificate is exact or it is nothing. TP is ATR-anchored like the label, NOT risk-relative, so a floor-widened stop cannot reshape the certified target. 2) BATCH NORM RUNS DEVICE-SIDE ON OPENCL. Four kernels in Network.cl - forward, hidden gradient, gamma/beta accumulate, gamma/beta apply - each a line-for-line transcription of the host implementation (NormalizeHost / HiddenGradHost / StepGammaBeta) including every NaN guard, clamp, and the exact moment-write ordering. The host copies remain the runtime for the DLL and pure-MQL5 tiers and the reference the kernels must match. Because this box has no OpenCL platform, the safety story is layered: - shim validation: kernels compiled as C and driven against a fp64 host transcription over NaN-poisoned stats, NaN gamma, over-clamp inputs, the frozen path, both optimizers, 3 batches - ALL PASS, worst normalized diff 0.132 vs tolerance 1.0 - in-situ self-check: each kernel is compared against its host twin ON FIRST USE on the real device (SelfCheckBn*), covering what the shim cannot - arg indices and buffer bindings. Any disagreement resyncs from the good copy, latches all BN kernels off process-wide, and training continues host-side. A transcription bug costs a warning and some speed, never a poisoned .nnw. - sync discipline: BatchOptions is now a CBufferDouble with explicit authority tracking (m_bnDeviceAuthoritative). Checkpoints/saves pull read-only; restores/loads/resets push; a mid-batch handover drains the device gamma/beta accumulator into the host arrays so no sample is lost. 3) SMALL FIXES. Apply-kernel build failure now latches the dispatch path at init (one warning instead of warning + failed Execute). Build tag bumped to win-scoring-gpu-v1 - first tag change since expectancy-stop-v1 despite five binary-changing commits. Both build variants compile 0 errors, 0 warnings. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-09 17:51:40 -04:00
if(BatchOptions.GetIndex() >= 0)
ok = BatchOptions.BufferWrite() && ok;
fix: training-stability audit fixes F1/F2/F3/F5 - unbiased shuffle, real plateau escapes, fresh optimizer state on restore, pure OOS metric Four of the six findings from research/training_pipeline_audit_2026-08-09.md (F4 mini-batching and F6 feature re-encode deliberately deferred - see the report's implementation-status section for why): - F1: pass-2 Fisher-Yates (and AutoTune's MI block shuffle) used MathRand()%, which is 15-bit - provably non-uniform on every full-history era over 32,768 queued samples. New 30-bit ShuffleRandomIndex(). - F2: plateau warm restarts were a no-op whenever eta already sat at its ceiling (the normal state of a non-regressing plateau) - the ladder was just a 24-era countdown. Restarts now overshoot to 5x the ceiling (PLATEAU_RESTART_BOOST) and anneal geometrically back over the patience window, SGDR-style; ETA_MIN widened 1e-4 -> 1e-5 so the decay schedule has real range. - F3: checkpoint restores put weights back but kept the rejected trajectory's Adam moments, so the optimizer immediately pushed back toward the rolled-back state (the restore->regress->restore oscillation). CNet::ResetOptimizerState() zeroes moments/momentum/step counters (weights, BN statistics, gamma/beta untouched) on every mid-run restore, every boosted restart, and the deploy-time restore that online learning continues from. - F5: batch-norm running statistics now freeze for the pass-3 OOS scoring walk, so the selection metric the checkpoint ranking and deploy gate read is a pure function of the checkpoint instead of partly measuring BN drift. Defensive unfreeze in FinalizeTrainRun covers stop-mid-pass; live/online adaptation and the OOS continual-learning simulation stay adaptive by design. Compiled clean (0 errors, 0 warnings) via the staged-tree recipe. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-09 10:54:09 -04:00
}
return ok;
}
feat(ai): batch normalization between dense layers The only bounded stage in the entire forward path was the sigmoid classification head - every hidden stage is PRELU. That is a network with no internal scale control, and the failure ordered exactly by depth: on SP500 H1 the shallow perceptron held ~52% balanced accuracy while the deepest topology sat on the 33.3% one-class floor, with the per-bar logit spread decaying monotonically (0.45 -> 0.38 over ~200 eras) until the evidence tilt fell under the class-prior tilt. That is the signature of internal covariate shift, which chapter 6.1 of the reference book is entirely about and which the NeuroNet_DNG engine addresses with a layer this project never had. Two mechanisms make this the right fix rather than more hyperparameter nudging: - it decouples WEIGHT_DECAY from the learned function (van Laarhoven 2017) - with a normalized layer downstream, decay can no longer grind the discriminative signal away, it only rescales the effective learning rate; - it is the precondition for ever running an unbounded logit head here. The 2026-07-27 attempt blew up (IS error 5.6e15) precisely because nothing upstream constrained scale. Implementation notes: - CNeuronBatchNormOCL computes host-side rather than as a fourth copy of a kernel across Network.cl + WarriorCPU.cpp + WarriorDML.cpp. The math is elementwise O(n); this way it behaves identically on all four compute tiers, needs no DLL rebuild, and cannot drift between backends. Same precedent as the softmax+CCE gradient and the per-sample loss weighting, both computed in MQL5 for that reason. - Statistics are exponential moving, not a stored mini-batch: training is pure online SGD, one update per sample, so there is no batch to average over. BatchNormWindow is an EMA window length. - gamma/beta are excluded from weight decay, deliberately - decaying gamma toward zero is the exact pathology being fixed. - The layer self-sizes from whatever sits below it, because a conv/pool stage's output width is derived inside the CNet constructor and is not knowable to the topology builder. - Checkpoint capture/restore/blend carry gamma/beta and the running statistics alongside the dense matrix, so the plateau ladder cannot restore a mismatched pair. - SeedOutputLayerBias accepted only an exact defNeuronBaseOCL as the weight-carrying penultimate layer; with normalization enabled that is the batch-norm layer, so the cold-start bias seed would have silently stopped being applied. - Refuses to build, loudly, if a topology asks for normalization with no compute backend at all - rather than quietly training a different architecture than the one requested. EnableBatchNorm (default on) and BatchNormWindow (1000 samples) are inputs so the effect can be A/B'd without a recompile. Both feed the weights-filename fingerprint, appended conditionally so existing non-BN configs keep their fingerprints and are not forced to retrain. Verified: analytic gradients match finite differences to 1.5e-7 relative over 200 random cases; a faithful port of the full forward/backward chain collapses to the 33.3% floor by era 4 without this layer and holds 36-43% with it. Compiles 0 errors, 0 warnings. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-29 12:34:29 -04:00
//---
virtual bool Save(int const file_handle);
virtual bool Load(int const file_handle);
//---
virtual int Type(void) const { return defNeuronBatchNormOCL; }
};
//+------------------------------------------------------------------+
//| |
//+------------------------------------------------------------------+
perf: batch norm stops re-reading buffers it already has Batch norm computes host-side while its neighbours are device-resident, so every value it touches crosses the bus - and the cost is the BLOCKING SYNC per crossing, not the bytes. Per sample it did four reads, two of which were exact duplicates: feedForward -> previous layer's Output calcInputGradients -> own Gradient, and the previous layer's Output AGAIN update/accumulate -> own Gradient AGAIN Nothing writes the previous layer's Output between the forward and backward passes, and nothing writes this layer's Gradient between the gradient pass and the weight-update pass - backPropOCL runs those as two separate top-to-bottom loops and only the first writes gradients. So the repeats are removable and the result is bit-exact: the same values, read once instead of twice. Each cache is armed by its producer and disarmed by feedForward, so a consumer whose producer did not run this sample falls back to reading the buffer rather than using the previous sample's data. That is not hypothetical - a batch norm at layer 1 never gets calcInputGradients called at all, the same asymmetry backPropOCL already documents for a layer-1 LSTM, so its gradient cache is never armed and it takes the fallback every time. Both build variants compile 0 errors, 0 warnings. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-09 17:16:08 -04:00
CNeuronBatchNormOCL::CNeuronBatchNormOCL(void) : iBatchSize(1), bStatsFrozen(false), iSamplesSeen(0),
fix: live trades now use the geometry the gate certifies; perf: BN kernels Three changes, one theme: the trade placed, the trade graded, and the trade computed are now the same trade. 1) GEOMETRY WIRE (correctness, the ranked #1 open issue). The measured barrier pair reached the LABELS only - OpenParams still placed orders at the enum geometry (2*ATR/6*ATR), so the deploy gate certified "reaches 1.62*ATR before 3.33*ATR above break-even" about trades the EA never placed. Published via g_DerivedSlAtrMult/g_DerivedTpAtrMult (ConfidenceBridge, same same-tick contract as the confidence globals, because OpenParams runs on the root signal which has no pointer to the AI filter). Two writers: DeriveBarrierGeometry at era 0, and the .cfg adoption a deployed model takes. Overrides both legs and both Intelligent modes - the certificate is exact or it is nothing. TP is ATR-anchored like the label, NOT risk-relative, so a floor-widened stop cannot reshape the certified target. 2) BATCH NORM RUNS DEVICE-SIDE ON OPENCL. Four kernels in Network.cl - forward, hidden gradient, gamma/beta accumulate, gamma/beta apply - each a line-for-line transcription of the host implementation (NormalizeHost / HiddenGradHost / StepGammaBeta) including every NaN guard, clamp, and the exact moment-write ordering. The host copies remain the runtime for the DLL and pure-MQL5 tiers and the reference the kernels must match. Because this box has no OpenCL platform, the safety story is layered: - shim validation: kernels compiled as C and driven against a fp64 host transcription over NaN-poisoned stats, NaN gamma, over-clamp inputs, the frozen path, both optimizers, 3 batches - ALL PASS, worst normalized diff 0.132 vs tolerance 1.0 - in-situ self-check: each kernel is compared against its host twin ON FIRST USE on the real device (SelfCheckBn*), covering what the shim cannot - arg indices and buffer bindings. Any disagreement resyncs from the good copy, latches all BN kernels off process-wide, and training continues host-side. A transcription bug costs a warning and some speed, never a poisoned .nnw. - sync discipline: BatchOptions is now a CBufferDouble with explicit authority tracking (m_bnDeviceAuthoritative). Checkpoints/saves pull read-only; restores/loads/resets push; a mid-batch handover drains the device gamma/beta accumulator into the host arrays so no sample is lost. 3) SMALL FIXES. Apply-kernel build failure now latches the dispatch path at init (one warning instead of warning + failed Execute). Build tag bumped to win-scoring-gpu-v1 - first tag change since expectancy-stop-v1 despite five binary-changing commits. Both build variants compile 0 errors, 0 warnings. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-09 17:51:40 -04:00
m_fwdInputCached(false), m_gradCached(false), m_bnDeviceAuthoritative(false),
m_bnCheckedFwd(false), m_bnCheckedGrad(false), m_bnCheckedAccum(false), m_bnCheckedApply(false)
feat(ai): batch normalization between dense layers The only bounded stage in the entire forward path was the sigmoid classification head - every hidden stage is PRELU. That is a network with no internal scale control, and the failure ordered exactly by depth: on SP500 H1 the shallow perceptron held ~52% balanced accuracy while the deepest topology sat on the 33.3% one-class floor, with the per-bar logit spread decaying monotonically (0.45 -> 0.38 over ~200 eras) until the evidence tilt fell under the class-prior tilt. That is the signature of internal covariate shift, which chapter 6.1 of the reference book is entirely about and which the NeuroNet_DNG engine addresses with a layer this project never had. Two mechanisms make this the right fix rather than more hyperparameter nudging: - it decouples WEIGHT_DECAY from the learned function (van Laarhoven 2017) - with a normalized layer downstream, decay can no longer grind the discriminative signal away, it only rescales the effective learning rate; - it is the precondition for ever running an unbounded logit head here. The 2026-07-27 attempt blew up (IS error 5.6e15) precisely because nothing upstream constrained scale. Implementation notes: - CNeuronBatchNormOCL computes host-side rather than as a fourth copy of a kernel across Network.cl + WarriorCPU.cpp + WarriorDML.cpp. The math is elementwise O(n); this way it behaves identically on all four compute tiers, needs no DLL rebuild, and cannot drift between backends. Same precedent as the softmax+CCE gradient and the per-sample loss weighting, both computed in MQL5 for that reason. - Statistics are exponential moving, not a stored mini-batch: training is pure online SGD, one update per sample, so there is no batch to average over. BatchNormWindow is an EMA window length. - gamma/beta are excluded from weight decay, deliberately - decaying gamma toward zero is the exact pathology being fixed. - The layer self-sizes from whatever sits below it, because a conv/pool stage's output width is derived inside the CNet constructor and is not knowable to the topology builder. - Checkpoint capture/restore/blend carry gamma/beta and the running statistics alongside the dense matrix, so the plateau ladder cannot restore a mismatched pair. - SeedOutputLayerBias accepted only an exact defNeuronBaseOCL as the weight-carrying penultimate layer; with normalization enabled that is the batch-norm layer, so the cold-start bias seed would have silently stopped being applied. - Refuses to build, loudly, if a topology asks for normalization with no compute backend at all - rather than quietly training a different architecture than the one requested. EnableBatchNorm (default on) and BatchNormWindow (1000 samples) are inputs so the effect can be A/B'd without a recompile. Both feed the weights-filename fingerprint, appended conditionally so existing non-BN configs keep their fingerprints and are not forced to retrain. Verified: analytic gradients match finite differences to 1.5e-7 relative over 200 random cases; a faithful port of the full forward/backward chain collapses to the 33.3% floor by era 4 without this layer and holds 36-43% with it. Compiles 0 errors, 0 warnings. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-29 12:34:29 -04:00
{
BatchOptions = NULL;
fix: live trades now use the geometry the gate certifies; perf: BN kernels Three changes, one theme: the trade placed, the trade graded, and the trade computed are now the same trade. 1) GEOMETRY WIRE (correctness, the ranked #1 open issue). The measured barrier pair reached the LABELS only - OpenParams still placed orders at the enum geometry (2*ATR/6*ATR), so the deploy gate certified "reaches 1.62*ATR before 3.33*ATR above break-even" about trades the EA never placed. Published via g_DerivedSlAtrMult/g_DerivedTpAtrMult (ConfidenceBridge, same same-tick contract as the confidence globals, because OpenParams runs on the root signal which has no pointer to the AI filter). Two writers: DeriveBarrierGeometry at era 0, and the .cfg adoption a deployed model takes. Overrides both legs and both Intelligent modes - the certificate is exact or it is nothing. TP is ATR-anchored like the label, NOT risk-relative, so a floor-widened stop cannot reshape the certified target. 2) BATCH NORM RUNS DEVICE-SIDE ON OPENCL. Four kernels in Network.cl - forward, hidden gradient, gamma/beta accumulate, gamma/beta apply - each a line-for-line transcription of the host implementation (NormalizeHost / HiddenGradHost / StepGammaBeta) including every NaN guard, clamp, and the exact moment-write ordering. The host copies remain the runtime for the DLL and pure-MQL5 tiers and the reference the kernels must match. Because this box has no OpenCL platform, the safety story is layered: - shim validation: kernels compiled as C and driven against a fp64 host transcription over NaN-poisoned stats, NaN gamma, over-clamp inputs, the frozen path, both optimizers, 3 batches - ALL PASS, worst normalized diff 0.132 vs tolerance 1.0 - in-situ self-check: each kernel is compared against its host twin ON FIRST USE on the real device (SelfCheckBn*), covering what the shim cannot - arg indices and buffer bindings. Any disagreement resyncs from the good copy, latches all BN kernels off process-wide, and training continues host-side. A transcription bug costs a warning and some speed, never a poisoned .nnw. - sync discipline: BatchOptions is now a CBufferDouble with explicit authority tracking (m_bnDeviceAuthoritative). Checkpoints/saves pull read-only; restores/loads/resets push; a mid-batch handover drains the device gamma/beta accumulator into the host arrays so no sample is lost. 3) SMALL FIXES. Apply-kernel build failure now latches the dispatch path at init (one warning instead of warning + failed Execute). Build tag bumped to win-scoring-gpu-v1 - first tag change since expectancy-stop-v1 despite five binary-changing commits. Both build variants compile 0 errors, 0 warnings. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-09 17:51:40 -04:00
m_bnAcc = NULL;
feat(ai): batch normalization between dense layers The only bounded stage in the entire forward path was the sigmoid classification head - every hidden stage is PRELU. That is a network with no internal scale control, and the failure ordered exactly by depth: on SP500 H1 the shallow perceptron held ~52% balanced accuracy while the deepest topology sat on the 33.3% one-class floor, with the per-bar logit spread decaying monotonically (0.45 -> 0.38 over ~200 eras) until the evidence tilt fell under the class-prior tilt. That is the signature of internal covariate shift, which chapter 6.1 of the reference book is entirely about and which the NeuroNet_DNG engine addresses with a layer this project never had. Two mechanisms make this the right fix rather than more hyperparameter nudging: - it decouples WEIGHT_DECAY from the learned function (van Laarhoven 2017) - with a normalized layer downstream, decay can no longer grind the discriminative signal away, it only rescales the effective learning rate; - it is the precondition for ever running an unbounded logit head here. The 2026-07-27 attempt blew up (IS error 5.6e15) precisely because nothing upstream constrained scale. Implementation notes: - CNeuronBatchNormOCL computes host-side rather than as a fourth copy of a kernel across Network.cl + WarriorCPU.cpp + WarriorDML.cpp. The math is elementwise O(n); this way it behaves identically on all four compute tiers, needs no DLL rebuild, and cannot drift between backends. Same precedent as the softmax+CCE gradient and the per-sample loss weighting, both computed in MQL5 for that reason. - Statistics are exponential moving, not a stored mini-batch: training is pure online SGD, one update per sample, so there is no batch to average over. BatchNormWindow is an EMA window length. - gamma/beta are excluded from weight decay, deliberately - decaying gamma toward zero is the exact pathology being fixed. - The layer self-sizes from whatever sits below it, because a conv/pool stage's output width is derived inside the CNet constructor and is not knowable to the topology builder. - Checkpoint capture/restore/blend carry gamma/beta and the running statistics alongside the dense matrix, so the plateau ladder cannot restore a mismatched pair. - SeedOutputLayerBias accepted only an exact defNeuronBaseOCL as the weight-carrying penultimate layer; with normalization enabled that is the batch-norm layer, so the cold-start bias seed would have silently stopped being applied. - Refuses to build, loudly, if a topology asks for normalization with no compute backend at all - rather than quietly training a different architecture than the one requested. EnableBatchNorm (default on) and BatchNormWindow (1000 samples) are inputs so the effect can be A/B'd without a recompile. Both feed the weights-filename fingerprint, appended conditionally so existing non-BN configs keep their fingerprints and are not forced to retrain. Verified: analytic gradients match finite differences to 1.5e-7 relative over 200 random cases; a faithful port of the full forward/backward chain collapses to the 33.3% floor by era 4 without this layer and holds 36-43% with it. Compiles 0 errors, 0 warnings. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-29 12:34:29 -04:00
}
//+------------------------------------------------------------------+
//| |
//+------------------------------------------------------------------+
CNeuronBatchNormOCL::~CNeuronBatchNormOCL(void)
{
if(CheckPointer(BatchOptions) != POINTER_INVALID)
delete BatchOptions;
fix: live trades now use the geometry the gate certifies; perf: BN kernels Three changes, one theme: the trade placed, the trade graded, and the trade computed are now the same trade. 1) GEOMETRY WIRE (correctness, the ranked #1 open issue). The measured barrier pair reached the LABELS only - OpenParams still placed orders at the enum geometry (2*ATR/6*ATR), so the deploy gate certified "reaches 1.62*ATR before 3.33*ATR above break-even" about trades the EA never placed. Published via g_DerivedSlAtrMult/g_DerivedTpAtrMult (ConfidenceBridge, same same-tick contract as the confidence globals, because OpenParams runs on the root signal which has no pointer to the AI filter). Two writers: DeriveBarrierGeometry at era 0, and the .cfg adoption a deployed model takes. Overrides both legs and both Intelligent modes - the certificate is exact or it is nothing. TP is ATR-anchored like the label, NOT risk-relative, so a floor-widened stop cannot reshape the certified target. 2) BATCH NORM RUNS DEVICE-SIDE ON OPENCL. Four kernels in Network.cl - forward, hidden gradient, gamma/beta accumulate, gamma/beta apply - each a line-for-line transcription of the host implementation (NormalizeHost / HiddenGradHost / StepGammaBeta) including every NaN guard, clamp, and the exact moment-write ordering. The host copies remain the runtime for the DLL and pure-MQL5 tiers and the reference the kernels must match. Because this box has no OpenCL platform, the safety story is layered: - shim validation: kernels compiled as C and driven against a fp64 host transcription over NaN-poisoned stats, NaN gamma, over-clamp inputs, the frozen path, both optimizers, 3 batches - ALL PASS, worst normalized diff 0.132 vs tolerance 1.0 - in-situ self-check: each kernel is compared against its host twin ON FIRST USE on the real device (SelfCheckBn*), covering what the shim cannot - arg indices and buffer bindings. Any disagreement resyncs from the good copy, latches all BN kernels off process-wide, and training continues host-side. A transcription bug costs a warning and some speed, never a poisoned .nnw. - sync discipline: BatchOptions is now a CBufferDouble with explicit authority tracking (m_bnDeviceAuthoritative). Checkpoints/saves pull read-only; restores/loads/resets push; a mid-batch handover drains the device gamma/beta accumulator into the host arrays so no sample is lost. 3) SMALL FIXES. Apply-kernel build failure now latches the dispatch path at init (one warning instead of warning + failed Execute). Build tag bumped to win-scoring-gpu-v1 - first tag change since expectancy-stop-v1 despite five binary-changing commits. Both build variants compile 0 errors, 0 warnings. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-09 17:51:40 -04:00
if(CheckPointer(m_bnAcc) != POINTER_INVALID)
delete m_bnAcc;
feat(ai): batch normalization between dense layers The only bounded stage in the entire forward path was the sigmoid classification head - every hidden stage is PRELU. That is a network with no internal scale control, and the failure ordered exactly by depth: on SP500 H1 the shallow perceptron held ~52% balanced accuracy while the deepest topology sat on the 33.3% one-class floor, with the per-bar logit spread decaying monotonically (0.45 -> 0.38 over ~200 eras) until the evidence tilt fell under the class-prior tilt. That is the signature of internal covariate shift, which chapter 6.1 of the reference book is entirely about and which the NeuroNet_DNG engine addresses with a layer this project never had. Two mechanisms make this the right fix rather than more hyperparameter nudging: - it decouples WEIGHT_DECAY from the learned function (van Laarhoven 2017) - with a normalized layer downstream, decay can no longer grind the discriminative signal away, it only rescales the effective learning rate; - it is the precondition for ever running an unbounded logit head here. The 2026-07-27 attempt blew up (IS error 5.6e15) precisely because nothing upstream constrained scale. Implementation notes: - CNeuronBatchNormOCL computes host-side rather than as a fourth copy of a kernel across Network.cl + WarriorCPU.cpp + WarriorDML.cpp. The math is elementwise O(n); this way it behaves identically on all four compute tiers, needs no DLL rebuild, and cannot drift between backends. Same precedent as the softmax+CCE gradient and the per-sample loss weighting, both computed in MQL5 for that reason. - Statistics are exponential moving, not a stored mini-batch: training is pure online SGD, one update per sample, so there is no batch to average over. BatchNormWindow is an EMA window length. - gamma/beta are excluded from weight decay, deliberately - decaying gamma toward zero is the exact pathology being fixed. - The layer self-sizes from whatever sits below it, because a conv/pool stage's output width is derived inside the CNet constructor and is not knowable to the topology builder. - Checkpoint capture/restore/blend carry gamma/beta and the running statistics alongside the dense matrix, so the plateau ladder cannot restore a mismatched pair. - SeedOutputLayerBias accepted only an exact defNeuronBaseOCL as the weight-carrying penultimate layer; with normalization enabled that is the batch-norm layer, so the cold-start bias seed would have silently stopped being applied. - Refuses to build, loudly, if a topology asks for normalization with no compute backend at all - rather than quietly training a different architecture than the one requested. EnableBatchNorm (default on) and BatchNormWindow (1000 samples) are inputs so the effect can be A/B'd without a recompile. Both feed the weights-filename fingerprint, appended conditionally so existing non-BN configs keep their fingerprints and are not forced to retrain. Verified: analytic gradients match finite differences to 1.5e-7 relative over 200 random cases; a faithful port of the full forward/backward chain collapses to the 33.3% floor by era 4 without this layer and holds 36-43% with it. Compiles 0 errors, 0 warnings. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-29 12:34:29 -04:00
}
//+------------------------------------------------------------------+
//| gamma=1 / beta=0 / zeroed statistics for every unit. |
//+------------------------------------------------------------------+
bool CNeuronBatchNormOCL::InitOptions(int neurons)
{
if(neurons <= 0)
return false;
if(CheckPointer(BatchOptions) == POINTER_INVALID)
{
fix: live trades now use the geometry the gate certifies; perf: BN kernels Three changes, one theme: the trade placed, the trade graded, and the trade computed are now the same trade. 1) GEOMETRY WIRE (correctness, the ranked #1 open issue). The measured barrier pair reached the LABELS only - OpenParams still placed orders at the enum geometry (2*ATR/6*ATR), so the deploy gate certified "reaches 1.62*ATR before 3.33*ATR above break-even" about trades the EA never placed. Published via g_DerivedSlAtrMult/g_DerivedTpAtrMult (ConfidenceBridge, same same-tick contract as the confidence globals, because OpenParams runs on the root signal which has no pointer to the AI filter). Two writers: DeriveBarrierGeometry at era 0, and the .cfg adoption a deployed model takes. Overrides both legs and both Intelligent modes - the certificate is exact or it is nothing. TP is ATR-anchored like the label, NOT risk-relative, so a floor-widened stop cannot reshape the certified target. 2) BATCH NORM RUNS DEVICE-SIDE ON OPENCL. Four kernels in Network.cl - forward, hidden gradient, gamma/beta accumulate, gamma/beta apply - each a line-for-line transcription of the host implementation (NormalizeHost / HiddenGradHost / StepGammaBeta) including every NaN guard, clamp, and the exact moment-write ordering. The host copies remain the runtime for the DLL and pure-MQL5 tiers and the reference the kernels must match. Because this box has no OpenCL platform, the safety story is layered: - shim validation: kernels compiled as C and driven against a fp64 host transcription over NaN-poisoned stats, NaN gamma, over-clamp inputs, the frozen path, both optimizers, 3 batches - ALL PASS, worst normalized diff 0.132 vs tolerance 1.0 - in-situ self-check: each kernel is compared against its host twin ON FIRST USE on the real device (SelfCheckBn*), covering what the shim cannot - arg indices and buffer bindings. Any disagreement resyncs from the good copy, latches all BN kernels off process-wide, and training continues host-side. A transcription bug costs a warning and some speed, never a poisoned .nnw. - sync discipline: BatchOptions is now a CBufferDouble with explicit authority tracking (m_bnDeviceAuthoritative). Checkpoints/saves pull read-only; restores/loads/resets push; a mid-batch handover drains the device gamma/beta accumulator into the host arrays so no sample is lost. 3) SMALL FIXES. Apply-kernel build failure now latches the dispatch path at init (one warning instead of warning + failed Execute). Build tag bumped to win-scoring-gpu-v1 - first tag change since expectancy-stop-v1 despite five binary-changing commits. Both build variants compile 0 errors, 0 warnings. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-09 17:51:40 -04:00
BatchOptions = new CBufferDouble();
feat(ai): batch normalization between dense layers The only bounded stage in the entire forward path was the sigmoid classification head - every hidden stage is PRELU. That is a network with no internal scale control, and the failure ordered exactly by depth: on SP500 H1 the shallow perceptron held ~52% balanced accuracy while the deepest topology sat on the 33.3% one-class floor, with the per-bar logit spread decaying monotonically (0.45 -> 0.38 over ~200 eras) until the evidence tilt fell under the class-prior tilt. That is the signature of internal covariate shift, which chapter 6.1 of the reference book is entirely about and which the NeuroNet_DNG engine addresses with a layer this project never had. Two mechanisms make this the right fix rather than more hyperparameter nudging: - it decouples WEIGHT_DECAY from the learned function (van Laarhoven 2017) - with a normalized layer downstream, decay can no longer grind the discriminative signal away, it only rescales the effective learning rate; - it is the precondition for ever running an unbounded logit head here. The 2026-07-27 attempt blew up (IS error 5.6e15) precisely because nothing upstream constrained scale. Implementation notes: - CNeuronBatchNormOCL computes host-side rather than as a fourth copy of a kernel across Network.cl + WarriorCPU.cpp + WarriorDML.cpp. The math is elementwise O(n); this way it behaves identically on all four compute tiers, needs no DLL rebuild, and cannot drift between backends. Same precedent as the softmax+CCE gradient and the per-sample loss weighting, both computed in MQL5 for that reason. - Statistics are exponential moving, not a stored mini-batch: training is pure online SGD, one update per sample, so there is no batch to average over. BatchNormWindow is an EMA window length. - gamma/beta are excluded from weight decay, deliberately - decaying gamma toward zero is the exact pathology being fixed. - The layer self-sizes from whatever sits below it, because a conv/pool stage's output width is derived inside the CNet constructor and is not knowable to the topology builder. - Checkpoint capture/restore/blend carry gamma/beta and the running statistics alongside the dense matrix, so the plateau ladder cannot restore a mismatched pair. - SeedOutputLayerBias accepted only an exact defNeuronBaseOCL as the weight-carrying penultimate layer; with normalization enabled that is the batch-norm layer, so the cold-start bias seed would have silently stopped being applied. - Refuses to build, loudly, if a topology asks for normalization with no compute backend at all - rather than quietly training a different architecture than the one requested. EnableBatchNorm (default on) and BatchNormWindow (1000 samples) are inputs so the effect can be A/B'd without a recompile. Both feed the weights-filename fingerprint, appended conditionally so existing non-BN configs keep their fingerprints and are not forced to retrain. Verified: analytic gradients match finite differences to 1.5e-7 relative over 200 random cases; a faithful port of the full forward/backward chain collapses to the 33.3% floor by era 4 without this layer and holds 36-43% with it. Compiles 0 errors, 0 warnings. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-29 12:34:29 -04:00
if(CheckPointer(BatchOptions) == POINTER_INVALID)
return false;
}
BatchOptions.Clear();
if(!BatchOptions.Reserve(neurons * BN_OPT_STRIDE))
return false;
for(int n = 0; n < neurons; n++)
for(int s = 0; s < BN_OPT_STRIDE; s++)
if(!BatchOptions.Add(s == BN_OPT_GAMMA ? 1.0 : 0.0))
return false;
iSamplesSeen = 0;
fix: live trades now use the geometry the gate certifies; perf: BN kernels Three changes, one theme: the trade placed, the trade graded, and the trade computed are now the same trade. 1) GEOMETRY WIRE (correctness, the ranked #1 open issue). The measured barrier pair reached the LABELS only - OpenParams still placed orders at the enum geometry (2*ATR/6*ATR), so the deploy gate certified "reaches 1.62*ATR before 3.33*ATR above break-even" about trades the EA never placed. Published via g_DerivedSlAtrMult/g_DerivedTpAtrMult (ConfidenceBridge, same same-tick contract as the confidence globals, because OpenParams runs on the root signal which has no pointer to the AI filter). Two writers: DeriveBarrierGeometry at era 0, and the .cfg adoption a deployed model takes. Overrides both legs and both Intelligent modes - the certificate is exact or it is nothing. TP is ATR-anchored like the label, NOT risk-relative, so a floor-widened stop cannot reshape the certified target. 2) BATCH NORM RUNS DEVICE-SIDE ON OPENCL. Four kernels in Network.cl - forward, hidden gradient, gamma/beta accumulate, gamma/beta apply - each a line-for-line transcription of the host implementation (NormalizeHost / HiddenGradHost / StepGammaBeta) including every NaN guard, clamp, and the exact moment-write ordering. The host copies remain the runtime for the DLL and pure-MQL5 tiers and the reference the kernels must match. Because this box has no OpenCL platform, the safety story is layered: - shim validation: kernels compiled as C and driven against a fp64 host transcription over NaN-poisoned stats, NaN gamma, over-clamp inputs, the frozen path, both optimizers, 3 batches - ALL PASS, worst normalized diff 0.132 vs tolerance 1.0 - in-situ self-check: each kernel is compared against its host twin ON FIRST USE on the real device (SelfCheckBn*), covering what the shim cannot - arg indices and buffer bindings. Any disagreement resyncs from the good copy, latches all BN kernels off process-wide, and training continues host-side. A transcription bug costs a warning and some speed, never a poisoned .nnw. - sync discipline: BatchOptions is now a CBufferDouble with explicit authority tracking (m_bnDeviceAuthoritative). Checkpoints/saves pull read-only; restores/loads/resets push; a mid-batch handover drains the device gamma/beta accumulator into the host arrays so no sample is lost. 3) SMALL FIXES. Apply-kernel build failure now latches the dispatch path at init (one warning instead of warning + failed Execute). Build tag bumped to win-scoring-gpu-v1 - first tag change since expectancy-stop-v1 despite five binary-changing commits. Both build variants compile 0 errors, 0 warnings. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-09 17:51:40 -04:00
//--- Full host overwrite: if a device copy exists, push it so the two cannot disagree. The host is
//--- authoritative after a re-init by definition.
if(BatchOptions.GetIndex() >= 0)
{
if(!BatchOptions.BufferWrite())
return false;
m_bnDeviceAuthoritative = false;
}
feat(ai): batch normalization between dense layers The only bounded stage in the entire forward path was the sigmoid classification head - every hidden stage is PRELU. That is a network with no internal scale control, and the failure ordered exactly by depth: on SP500 H1 the shallow perceptron held ~52% balanced accuracy while the deepest topology sat on the 33.3% one-class floor, with the per-bar logit spread decaying monotonically (0.45 -> 0.38 over ~200 eras) until the evidence tilt fell under the class-prior tilt. That is the signature of internal covariate shift, which chapter 6.1 of the reference book is entirely about and which the NeuroNet_DNG engine addresses with a layer this project never had. Two mechanisms make this the right fix rather than more hyperparameter nudging: - it decouples WEIGHT_DECAY from the learned function (van Laarhoven 2017) - with a normalized layer downstream, decay can no longer grind the discriminative signal away, it only rescales the effective learning rate; - it is the precondition for ever running an unbounded logit head here. The 2026-07-27 attempt blew up (IS error 5.6e15) precisely because nothing upstream constrained scale. Implementation notes: - CNeuronBatchNormOCL computes host-side rather than as a fourth copy of a kernel across Network.cl + WarriorCPU.cpp + WarriorDML.cpp. The math is elementwise O(n); this way it behaves identically on all four compute tiers, needs no DLL rebuild, and cannot drift between backends. Same precedent as the softmax+CCE gradient and the per-sample loss weighting, both computed in MQL5 for that reason. - Statistics are exponential moving, not a stored mini-batch: training is pure online SGD, one update per sample, so there is no batch to average over. BatchNormWindow is an EMA window length. - gamma/beta are excluded from weight decay, deliberately - decaying gamma toward zero is the exact pathology being fixed. - The layer self-sizes from whatever sits below it, because a conv/pool stage's output width is derived inside the CNet constructor and is not knowable to the topology builder. - Checkpoint capture/restore/blend carry gamma/beta and the running statistics alongside the dense matrix, so the plateau ladder cannot restore a mismatched pair. - SeedOutputLayerBias accepted only an exact defNeuronBaseOCL as the weight-carrying penultimate layer; with normalization enabled that is the batch-norm layer, so the cold-start bias seed would have silently stopped being applied. - Refuses to build, loudly, if a topology asks for normalization with no compute backend at all - rather than quietly training a different architecture than the one requested. EnableBatchNorm (default on) and BatchNormWindow (1000 samples) are inputs so the effect can be A/B'd without a recompile. Both feed the weights-filename fingerprint, appended conditionally so existing non-BN configs keep their fingerprints and are not forced to retrain. Verified: analytic gradients match finite differences to 1.5e-7 relative over 200 random cases; a faithful port of the full forward/backward chain collapses to the 33.3% floor by era 4 without this layer and holds 36-43% with it. Compiles 0 errors, 0 warnings. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-29 12:34:29 -04:00
return true;
}
//+------------------------------------------------------------------+
//| |
//+------------------------------------------------------------------+
bool CNeuronBatchNormOCL::Init(uint numOutputs, uint myIndex, COpenCLMy *open_cl, uint numNeurons, uint batchSize, ENUM_OPTIMIZATION optimization_type)
{
if(!CNeuronBaseOCL::Init(numOutputs, myIndex, open_cl, numNeurons, optimization_type))
return false;
//--- Identity forward transform. The activation belongs to the layer AFTER this one; normalizing
//--- and then squashing in the same step would defeat the point (Ioffe & Szegedy place the
//--- normalization immediately BEFORE the non-linearity, not around it).
activation = NONE;
iBatchSize = (int)batchSize;
return InitOptions((int)numNeurons);
}
//+------------------------------------------------------------------+
//| |
//+------------------------------------------------------------------+
bool CNeuronBatchNormOCL::Init(uint numOutputs, uint myIndex, CDirectMLMy *direct_ml, uint numNeurons, uint batchSize, ENUM_OPTIMIZATION optimization_type)
{
if(!CNeuronBaseOCL::Init(numOutputs, myIndex, direct_ml, numNeurons, optimization_type))
return false;
activation = NONE;
iBatchSize = (int)batchSize;
return InitOptions((int)numNeurons);
}
//+------------------------------------------------------------------+
//| The forward transform, host-side. Mirrors the NeuroNet_DNG |
//| BatchFeedForward kernel, plus the bias-corrected warm-up window. |
//+------------------------------------------------------------------+
bool CNeuronBatchNormOCL::NormalizeHost(const double &inputs[], int count)
{
if(CheckPointer(Output) == POINTER_INVALID || CheckPointer(BatchOptions) == POINTER_INVALID)
return false;
if(count > Output.Total())
count = Output.Total();
if(BatchOptions.Total() < count * BN_OPT_STRIDE)
return false;
//--- A window of 1 makes mean==x and variance==0 for every sample, i.e. a constant-zero output.
//--- Treat it as "normalization off" and pass the input through untouched rather than emit zeros.
if(iBatchSize <= 1)
{
for(int i = 0; i < count; i++)
if(!Output.Update(i, inputs[i]))
return false;
return true;
}
fix(ai): freeze batch-norm statistics when comparing two forward passes With normalization enabled a forward pass is not a pure function of its input - it also advances the running mean/variance. ValidateCpuInference compares the live backend net against a throwaway pure-MQL5 clone loaded from the just-saved .nnw, so its own reference pass left the live model one EMA step ahead of the file the clone reads. The check would then have been measuring its own side effect, and a marginal result decides whether buyers' backtests are allowed to run DLL-free. Adds CNet::SetBatchNormFrozen / CNeuronBatchNormOCL::SetStatsFrozen - classic batch-norm inference semantics, statistics used but not updated - and freezes both sides for the duration of the comparison. Not persisted: it is a transient evaluation mode, not model state. Default stays adaptive, which is what the rest of the system (online continual learning) is built around. Compiles 0 errors, 0 warnings. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-29 12:37:50 -04:00
if(!bStatsFrozen && iSamplesSeen < iBatchSize)
feat(ai): batch normalization between dense layers The only bounded stage in the entire forward path was the sigmoid classification head - every hidden stage is PRELU. That is a network with no internal scale control, and the failure ordered exactly by depth: on SP500 H1 the shallow perceptron held ~52% balanced accuracy while the deepest topology sat on the 33.3% one-class floor, with the per-bar logit spread decaying monotonically (0.45 -> 0.38 over ~200 eras) until the evidence tilt fell under the class-prior tilt. That is the signature of internal covariate shift, which chapter 6.1 of the reference book is entirely about and which the NeuroNet_DNG engine addresses with a layer this project never had. Two mechanisms make this the right fix rather than more hyperparameter nudging: - it decouples WEIGHT_DECAY from the learned function (van Laarhoven 2017) - with a normalized layer downstream, decay can no longer grind the discriminative signal away, it only rescales the effective learning rate; - it is the precondition for ever running an unbounded logit head here. The 2026-07-27 attempt blew up (IS error 5.6e15) precisely because nothing upstream constrained scale. Implementation notes: - CNeuronBatchNormOCL computes host-side rather than as a fourth copy of a kernel across Network.cl + WarriorCPU.cpp + WarriorDML.cpp. The math is elementwise O(n); this way it behaves identically on all four compute tiers, needs no DLL rebuild, and cannot drift between backends. Same precedent as the softmax+CCE gradient and the per-sample loss weighting, both computed in MQL5 for that reason. - Statistics are exponential moving, not a stored mini-batch: training is pure online SGD, one update per sample, so there is no batch to average over. BatchNormWindow is an EMA window length. - gamma/beta are excluded from weight decay, deliberately - decaying gamma toward zero is the exact pathology being fixed. - The layer self-sizes from whatever sits below it, because a conv/pool stage's output width is derived inside the CNet constructor and is not knowable to the topology builder. - Checkpoint capture/restore/blend carry gamma/beta and the running statistics alongside the dense matrix, so the plateau ladder cannot restore a mismatched pair. - SeedOutputLayerBias accepted only an exact defNeuronBaseOCL as the weight-carrying penultimate layer; with normalization enabled that is the batch-norm layer, so the cold-start bias seed would have silently stopped being applied. - Refuses to build, loudly, if a topology asks for normalization with no compute backend at all - rather than quietly training a different architecture than the one requested. EnableBatchNorm (default on) and BatchNormWindow (1000 samples) are inputs so the effect can be A/B'd without a recompile. Both feed the weights-filename fingerprint, appended conditionally so existing non-BN configs keep their fingerprints and are not forced to retrain. Verified: analytic gradients match finite differences to 1.5e-7 relative over 200 random cases; a faithful port of the full forward/backward chain collapses to the 33.3% floor by era 4 without this layer and holds 36-43% with it. Compiles 0 errors, 0 warnings. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-29 12:34:29 -04:00
iSamplesSeen++;
//--- Effective window: ramps 1,2,3... up to iBatchSize, so the first samples produce an honest
//--- running mean instead of one biased toward the zero initializer.
fix(ai): freeze batch-norm statistics when comparing two forward passes With normalization enabled a forward pass is not a pure function of its input - it also advances the running mean/variance. ValidateCpuInference compares the live backend net against a throwaway pure-MQL5 clone loaded from the just-saved .nnw, so its own reference pass left the live model one EMA step ahead of the file the clone reads. The check would then have been measuring its own side effect, and a marginal result decides whether buyers' backtests are allowed to run DLL-free. Adds CNet::SetBatchNormFrozen / CNeuronBatchNormOCL::SetStatsFrozen - classic batch-norm inference semantics, statistics used but not updated - and freezes both sides for the duration of the comparison. Not persisted: it is a transient evaluation mode, not model state. Default stays adaptive, which is what the rest of the system (online continual learning) is built around. Compiles 0 errors, 0 warnings. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-29 12:37:50 -04:00
double w = (double)MathMax(1, iSamplesSeen);
feat(ai): batch normalization between dense layers The only bounded stage in the entire forward path was the sigmoid classification head - every hidden stage is PRELU. That is a network with no internal scale control, and the failure ordered exactly by depth: on SP500 H1 the shallow perceptron held ~52% balanced accuracy while the deepest topology sat on the 33.3% one-class floor, with the per-bar logit spread decaying monotonically (0.45 -> 0.38 over ~200 eras) until the evidence tilt fell under the class-prior tilt. That is the signature of internal covariate shift, which chapter 6.1 of the reference book is entirely about and which the NeuroNet_DNG engine addresses with a layer this project never had. Two mechanisms make this the right fix rather than more hyperparameter nudging: - it decouples WEIGHT_DECAY from the learned function (van Laarhoven 2017) - with a normalized layer downstream, decay can no longer grind the discriminative signal away, it only rescales the effective learning rate; - it is the precondition for ever running an unbounded logit head here. The 2026-07-27 attempt blew up (IS error 5.6e15) precisely because nothing upstream constrained scale. Implementation notes: - CNeuronBatchNormOCL computes host-side rather than as a fourth copy of a kernel across Network.cl + WarriorCPU.cpp + WarriorDML.cpp. The math is elementwise O(n); this way it behaves identically on all four compute tiers, needs no DLL rebuild, and cannot drift between backends. Same precedent as the softmax+CCE gradient and the per-sample loss weighting, both computed in MQL5 for that reason. - Statistics are exponential moving, not a stored mini-batch: training is pure online SGD, one update per sample, so there is no batch to average over. BatchNormWindow is an EMA window length. - gamma/beta are excluded from weight decay, deliberately - decaying gamma toward zero is the exact pathology being fixed. - The layer self-sizes from whatever sits below it, because a conv/pool stage's output width is derived inside the CNet constructor and is not knowable to the topology builder. - Checkpoint capture/restore/blend carry gamma/beta and the running statistics alongside the dense matrix, so the plateau ladder cannot restore a mismatched pair. - SeedOutputLayerBias accepted only an exact defNeuronBaseOCL as the weight-carrying penultimate layer; with normalization enabled that is the batch-norm layer, so the cold-start bias seed would have silently stopped being applied. - Refuses to build, loudly, if a topology asks for normalization with no compute backend at all - rather than quietly training a different architecture than the one requested. EnableBatchNorm (default on) and BatchNormWindow (1000 samples) are inputs so the effect can be A/B'd without a recompile. Both feed the weights-filename fingerprint, appended conditionally so existing non-BN configs keep their fingerprints and are not forced to retrain. Verified: analytic gradients match finite differences to 1.5e-7 relative over 200 random cases; a faithful port of the full forward/backward chain collapses to the 33.3% floor by era 4 without this layer and holds 36-43% with it. Compiles 0 errors, 0 warnings. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-29 12:34:29 -04:00
for(int i = 0; i < count; i++)
{
int shift = i * BN_OPT_STRIDE;
double x = inputs[i];
refactor(comments): stdlib comment style across the remaining in-scope files Same pass as 0b06f8e, applied file by file: comment runs of 4+ lines compressed to their leading topic sentences, capped at 4 lines, whole sentences only. Warning sentences (NEVER / MUST / trap / would-have) survive the budget. Every file was checked the same way before committing: the list of non-comment lines is byte-identical to HEAD, and braces balance. No code was touched. Panel/, Enumerations/ and the already-terse System headers needed little or nothing - PooledGate, TradeChecks, BinomialStats and Random came through with no blocks over the threshold at all. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-22 00:25:52 -04:00
//--- The running mean/variance below are PERSISTENT: they live in BatchOptions, are carried
//--- into the .nnw by getWeightsBN, and every later sample normalizes against them. That
//--- makes them a LATCH.
fix: improve error messages and suppress false sharing-violation logs - BufferDouble: replace hardcoded "DirectML/CPU-DLL" with dynamic backend name and add buffer index/element count to all error prints for easier debugging. - NetPersistence: distinguish missing file from transient lock by probing FileIsExist before logging, eliminating false "sharing violation" warnings when no saved model exists on first run.
2026-08-02 01:09:18 -04:00
if(!MathIsValidNumber(x))
x = 0.0;
x = MathMax(-BN_MAX_INPUT, MathMin(BN_MAX_INPUT, x));
fix(ai): freeze batch-norm statistics when comparing two forward passes With normalization enabled a forward pass is not a pure function of its input - it also advances the running mean/variance. ValidateCpuInference compares the live backend net against a throwaway pure-MQL5 clone loaded from the just-saved .nnw, so its own reference pass left the live model one EMA step ahead of the file the clone reads. The check would then have been measuring its own side effect, and a marginal result decides whether buyers' backtests are allowed to run DLL-free. Adds CNet::SetBatchNormFrozen / CNeuronBatchNormOCL::SetStatsFrozen - classic batch-norm inference semantics, statistics used but not updated - and freezes both sides for the duration of the comparison. Not persisted: it is a transient evaluation mode, not model state. Default stays adaptive, which is what the rest of the system (online continual learning) is built around. Compiles 0 errors, 0 warnings. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-29 12:37:50 -04:00
double mean = BatchOptions.At(shift + BN_OPT_MEAN);
double variance = BatchOptions.At(shift + BN_OPT_VAR);
fix: improve error messages and suppress false sharing-violation logs - BufferDouble: replace hardcoded "DirectML/CPU-DLL" with dynamic backend name and add buffer index/element count to all error prints for easier debugging. - NetPersistence: distinguish missing file from transient lock by probing FileIsExist before logging, eliminating false "sharing violation" warnings when no saved model exists on first run.
2026-08-02 01:09:18 -04:00
//--- Self-heal statistics that were already poisoned before this guard existed. Without it an
//--- affected .nnw stays dead across restarts, because Load faithfully restores the NaN.
if(!MathIsValidNumber(mean))
mean = x;
if(!MathIsValidNumber(variance) || variance < 0.0)
variance = 0.0;
fix(ai): freeze batch-norm statistics when comparing two forward passes With normalization enabled a forward pass is not a pure function of its input - it also advances the running mean/variance. ValidateCpuInference compares the live backend net against a throwaway pure-MQL5 clone loaded from the just-saved .nnw, so its own reference pass left the live model one EMA step ahead of the file the clone reads. The check would then have been measuring its own side effect, and a marginal result decides whether buyers' backtests are allowed to run DLL-free. Adds CNet::SetBatchNormFrozen / CNeuronBatchNormOCL::SetStatsFrozen - classic batch-norm inference semantics, statistics used but not updated - and freezes both sides for the duration of the comparison. Not persisted: it is a transient evaluation mode, not model state. Default stays adaptive, which is what the rest of the system (online continual learning) is built around. Compiles 0 errors, 0 warnings. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-29 12:37:50 -04:00
if(!bStatsFrozen)
{
mean = (mean * (w - 1.0) + x) / w;
variance = (variance * (w - 1.0) + (x - mean) * (x - mean)) / w;
}
feat(ai): batch normalization between dense layers The only bounded stage in the entire forward path was the sigmoid classification head - every hidden stage is PRELU. That is a network with no internal scale control, and the failure ordered exactly by depth: on SP500 H1 the shallow perceptron held ~52% balanced accuracy while the deepest topology sat on the 33.3% one-class floor, with the per-bar logit spread decaying monotonically (0.45 -> 0.38 over ~200 eras) until the evidence tilt fell under the class-prior tilt. That is the signature of internal covariate shift, which chapter 6.1 of the reference book is entirely about and which the NeuroNet_DNG engine addresses with a layer this project never had. Two mechanisms make this the right fix rather than more hyperparameter nudging: - it decouples WEIGHT_DECAY from the learned function (van Laarhoven 2017) - with a normalized layer downstream, decay can no longer grind the discriminative signal away, it only rescales the effective learning rate; - it is the precondition for ever running an unbounded logit head here. The 2026-07-27 attempt blew up (IS error 5.6e15) precisely because nothing upstream constrained scale. Implementation notes: - CNeuronBatchNormOCL computes host-side rather than as a fourth copy of a kernel across Network.cl + WarriorCPU.cpp + WarriorDML.cpp. The math is elementwise O(n); this way it behaves identically on all four compute tiers, needs no DLL rebuild, and cannot drift between backends. Same precedent as the softmax+CCE gradient and the per-sample loss weighting, both computed in MQL5 for that reason. - Statistics are exponential moving, not a stored mini-batch: training is pure online SGD, one update per sample, so there is no batch to average over. BatchNormWindow is an EMA window length. - gamma/beta are excluded from weight decay, deliberately - decaying gamma toward zero is the exact pathology being fixed. - The layer self-sizes from whatever sits below it, because a conv/pool stage's output width is derived inside the CNet constructor and is not knowable to the topology builder. - Checkpoint capture/restore/blend carry gamma/beta and the running statistics alongside the dense matrix, so the plateau ladder cannot restore a mismatched pair. - SeedOutputLayerBias accepted only an exact defNeuronBaseOCL as the weight-carrying penultimate layer; with normalization enabled that is the batch-norm layer, so the cold-start bias seed would have silently stopped being applied. - Refuses to build, loudly, if a topology asks for normalization with no compute backend at all - rather than quietly training a different architecture than the one requested. EnableBatchNorm (default on) and BatchNormWindow (1000 samples) are inputs so the effect can be A/B'd without a recompile. Both feed the weights-filename fingerprint, appended conditionally so existing non-BN configs keep their fingerprints and are not forced to retrain. Verified: analytic gradients match finite differences to 1.5e-7 relative over 200 random cases; a faithful port of the full forward/backward chain collapses to the 33.3% floor by era 4 without this layer and holds 36-43% with it. Compiles 0 errors, 0 warnings. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-29 12:34:29 -04:00
double delta = x - mean;
double sd = MathMax(MathSqrt(variance + BN_EPSILON), BN_MIN_STD);
fix(batchnorm): bound the normalized value - a constant input feature was amplified 1e4x and pinned PAI's head to its rails BN_MIN_STD = 1e-4 caps the per-unit gain at 1/1e-4 = 1e4, and the comment above it states that as though it were a safety property. It is not. A unit whose running variance is ~0 is a CONSTANT feature carrying no information, and dividing its rounding noise by 1e-4 hands the next layer an activation of several hundred. BN's contract is "output has ~unit variance"; a unit that cannot supply that must contribute nothing, not the largest signal in the layer. MEASURED, 2026-08-17 SP500 H4, four topologies on identical separate charts: model spread Neutral CHOSE Neutral TIED rail CONV 0.386 0.68% 0.10% 0.48% LSTM 0.392 0.63% 0.00% 0.00% HYB 0.376 1.79% 0.00% 0.01% PAI 0.192 0.09% 80.63% 99.99% bn1's cached nx normed 1.38e4 over 800 units. PAI's SIGMOID head was on its rails on 99.99% of bars, with Buy and Sell landing on the SAME rail so they compared exactly equal, and ApplyClassificationSoftmax()'s strict-majority rule reported that tie as Neutral on ~80% of bars. So the long-running "PAI is heavily biased toward Neutral" was never a class-prior problem: the net CHOSE Neutral on 0.09% of bars. It was float equality on a saturated head. The 331ab29 counters answered it on their first run. PAI-only because it is the one topology whose FIRST batch norm sits on the raw 800-dim input vector - CONV/LSTM/CONVLSTM all have a conv or LSTM stage in front, so their first BN sees a learned representation with no degenerate units. That asymmetry was already on file as a suspicion; this is the mechanism. FIX, mirrored in both backends (host NeuronBatchNorm.mqh and device Network.cl): forward nx = clamp(delta/sd, -BN_MAX_NX, +BN_MAX_NX), BN_MAX_NX = 8 backward if the forward bound this unit, the output stopped depending on the input, so d(nx)/dx = 0 and NO gradient passes The backward half is not optional. g is divided by the same sd the forward multiplies by, so a degenerate unit gets its GRADIENT amplified 1e4x too - the "receives gradients divided by sqrt(var) ~ 500" pathology already noted in Network.cl's Adam kernel. Bounding only the forward would move the explosion downstream. 8 sigma is inert on anything healthy (|nx| > 8 is a ~1e-15 event under normality); it binds only on degenerate units, which is the entire point. Same clamp-to-range idiom the activation derivatives beside it already use. SelfCheckBnForward/SelfCheckBnHiddenGrad already prove host against kernel, and BN_OPT_NX was already consumed in the backward for the gamma gradient, so the new read adds no lifetime assumption. Expect PAI to change behaviour and CONV/LSTM/CONVLSTM not to (their rail rate is ~0%, so the clamp never binds). No .nnw format or fingerprint change. ALSO: print the zero-skill reference on the era line. m_oosWinLongTotal and m_oosWinShortTotal have been accumulated for a long time and NEVER printed, which is why three separate topologies all sitting at 62% read as a mysterious coincidence rather than the obvious base rate. It is not a coincidence: with the target (1.70 ATR) nearer than the stop (3.07 ATR), BOTH sides win on 24.5% of bars, so winLong+winShort covers ~124% of them and a no-edge caller collects 62.1% whichever way it calls - against a 64.3% break-even. Derived from this run's own label counts: (11329 - 2776 + 2*2776) / (2*11350) = 62.14%. Every win rate on that line must be read against this, not against 50%. NOT COMPILED - user compiles. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-17 00:21:51 -04:00
//--- Bounded - see BN_MAX_NX. The clamped value is what gets CACHED below, which is what makes
//--- the backward pass able to see that it bound (HiddenGradHost reads BN_OPT_NX).
double nx = MathMax(-BN_MAX_NX, MathMin(BN_MAX_NX, delta / sd));
fix: improve error messages and suppress false sharing-violation logs - BufferDouble: replace hardcoded "DirectML/CPU-DLL" with dynamic backend name and add buffer index/element count to all error prints for easier debugging. - NetPersistence: distinguish missing file from transient lock by probing FileIsExist before logging, eliminating false "sharing violation" warnings when no saved model exists on first run.
2026-08-02 01:09:18 -04:00
//--- gamma/beta cannot go non-finite going forward (updateInputWeights validates and clamps every
//--- step), but a model SAVED before that guard existed can carry NaN in here through Load. Same
//--- self-heal as the statistics above: fall back to the identity transform for that unit.
double gamma = BatchOptions.At(shift + BN_OPT_GAMMA);
double beta = BatchOptions.At(shift + BN_OPT_BETA);
if(!MathIsValidNumber(gamma))
gamma = 1.0;
if(!MathIsValidNumber(beta))
beta = 0.0;
double y = gamma * nx + beta;
fix(ai): freeze batch-norm statistics when comparing two forward passes With normalization enabled a forward pass is not a pure function of its input - it also advances the running mean/variance. ValidateCpuInference compares the live backend net against a throwaway pure-MQL5 clone loaded from the just-saved .nnw, so its own reference pass left the live model one EMA step ahead of the file the clone reads. The check would then have been measuring its own side effect, and a marginal result decides whether buyers' backtests are allowed to run DLL-free. Adds CNet::SetBatchNormFrozen / CNeuronBatchNormOCL::SetStatsFrozen - classic batch-norm inference semantics, statistics used but not updated - and freezes both sides for the duration of the comparison. Not persisted: it is a transient evaluation mode, not model state. Default stays adaptive, which is what the rest of the system (online continual learning) is built around. Compiles 0 errors, 0 warnings. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-29 12:37:50 -04:00
//--- nx is still cached when frozen: it costs nothing and keeps the buffer consistent with the
//--- output just produced. There is no backward pass while frozen, so nothing reads it.
if(!bStatsFrozen &&
(!BatchOptions.Update(shift + BN_OPT_MEAN, mean) ||
!BatchOptions.Update(shift + BN_OPT_VAR, variance)))
return false;
if(!BatchOptions.Update(shift + BN_OPT_NX, nx))
feat(ai): batch normalization between dense layers The only bounded stage in the entire forward path was the sigmoid classification head - every hidden stage is PRELU. That is a network with no internal scale control, and the failure ordered exactly by depth: on SP500 H1 the shallow perceptron held ~52% balanced accuracy while the deepest topology sat on the 33.3% one-class floor, with the per-bar logit spread decaying monotonically (0.45 -> 0.38 over ~200 eras) until the evidence tilt fell under the class-prior tilt. That is the signature of internal covariate shift, which chapter 6.1 of the reference book is entirely about and which the NeuroNet_DNG engine addresses with a layer this project never had. Two mechanisms make this the right fix rather than more hyperparameter nudging: - it decouples WEIGHT_DECAY from the learned function (van Laarhoven 2017) - with a normalized layer downstream, decay can no longer grind the discriminative signal away, it only rescales the effective learning rate; - it is the precondition for ever running an unbounded logit head here. The 2026-07-27 attempt blew up (IS error 5.6e15) precisely because nothing upstream constrained scale. Implementation notes: - CNeuronBatchNormOCL computes host-side rather than as a fourth copy of a kernel across Network.cl + WarriorCPU.cpp + WarriorDML.cpp. The math is elementwise O(n); this way it behaves identically on all four compute tiers, needs no DLL rebuild, and cannot drift between backends. Same precedent as the softmax+CCE gradient and the per-sample loss weighting, both computed in MQL5 for that reason. - Statistics are exponential moving, not a stored mini-batch: training is pure online SGD, one update per sample, so there is no batch to average over. BatchNormWindow is an EMA window length. - gamma/beta are excluded from weight decay, deliberately - decaying gamma toward zero is the exact pathology being fixed. - The layer self-sizes from whatever sits below it, because a conv/pool stage's output width is derived inside the CNet constructor and is not knowable to the topology builder. - Checkpoint capture/restore/blend carry gamma/beta and the running statistics alongside the dense matrix, so the plateau ladder cannot restore a mismatched pair. - SeedOutputLayerBias accepted only an exact defNeuronBaseOCL as the weight-carrying penultimate layer; with normalization enabled that is the batch-norm layer, so the cold-start bias seed would have silently stopped being applied. - Refuses to build, loudly, if a topology asks for normalization with no compute backend at all - rather than quietly training a different architecture than the one requested. EnableBatchNorm (default on) and BatchNormWindow (1000 samples) are inputs so the effect can be A/B'd without a recompile. Both feed the weights-filename fingerprint, appended conditionally so existing non-BN configs keep their fingerprints and are not forced to retrain. Verified: analytic gradients match finite differences to 1.5e-7 relative over 200 random cases; a faithful port of the full forward/backward chain collapses to the 33.3% floor by era 4 without this layer and holds 36-43% with it. Compiles 0 errors, 0 warnings. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-29 12:34:29 -04:00
return false;
if(!Output.Update(i, y))
return false;
}
return true;
}
//+------------------------------------------------------------------+
fix: live trades now use the geometry the gate certifies; perf: BN kernels Three changes, one theme: the trade placed, the trade graded, and the trade computed are now the same trade. 1) GEOMETRY WIRE (correctness, the ranked #1 open issue). The measured barrier pair reached the LABELS only - OpenParams still placed orders at the enum geometry (2*ATR/6*ATR), so the deploy gate certified "reaches 1.62*ATR before 3.33*ATR above break-even" about trades the EA never placed. Published via g_DerivedSlAtrMult/g_DerivedTpAtrMult (ConfidenceBridge, same same-tick contract as the confidence globals, because OpenParams runs on the root signal which has no pointer to the AI filter). Two writers: DeriveBarrierGeometry at era 0, and the .cfg adoption a deployed model takes. Overrides both legs and both Intelligent modes - the certificate is exact or it is nothing. TP is ATR-anchored like the label, NOT risk-relative, so a floor-widened stop cannot reshape the certified target. 2) BATCH NORM RUNS DEVICE-SIDE ON OPENCL. Four kernels in Network.cl - forward, hidden gradient, gamma/beta accumulate, gamma/beta apply - each a line-for-line transcription of the host implementation (NormalizeHost / HiddenGradHost / StepGammaBeta) including every NaN guard, clamp, and the exact moment-write ordering. The host copies remain the runtime for the DLL and pure-MQL5 tiers and the reference the kernels must match. Because this box has no OpenCL platform, the safety story is layered: - shim validation: kernels compiled as C and driven against a fp64 host transcription over NaN-poisoned stats, NaN gamma, over-clamp inputs, the frozen path, both optimizers, 3 batches - ALL PASS, worst normalized diff 0.132 vs tolerance 1.0 - in-situ self-check: each kernel is compared against its host twin ON FIRST USE on the real device (SelfCheckBn*), covering what the shim cannot - arg indices and buffer bindings. Any disagreement resyncs from the good copy, latches all BN kernels off process-wide, and training continues host-side. A transcription bug costs a warning and some speed, never a poisoned .nnw. - sync discipline: BatchOptions is now a CBufferDouble with explicit authority tracking (m_bnDeviceAuthoritative). Checkpoints/saves pull read-only; restores/loads/resets push; a mid-batch handover drains the device gamma/beta accumulator into the host arrays so no sample is lost. 3) SMALL FIXES. Apply-kernel build failure now latches the dispatch path at init (one warning instead of warning + failed Execute). Build tag bumped to win-scoring-gpu-v1 - first tag change since expectancy-stop-v1 despite five binary-changing commits. Both build variants compile 0 errors, 0 warnings. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-09 17:51:40 -04:00
//| Forward. On OpenCL this is a KERNEL since 2026-08-09 (verified |
//| against NormalizeHost on first use, see SelfCheckBnForward); on |
//| every other backend it pulls the previous layer's output to the |
//| host, runs the transform, and pushes its output back. |
feat(ai): batch normalization between dense layers The only bounded stage in the entire forward path was the sigmoid classification head - every hidden stage is PRELU. That is a network with no internal scale control, and the failure ordered exactly by depth: on SP500 H1 the shallow perceptron held ~52% balanced accuracy while the deepest topology sat on the 33.3% one-class floor, with the per-bar logit spread decaying monotonically (0.45 -> 0.38 over ~200 eras) until the evidence tilt fell under the class-prior tilt. That is the signature of internal covariate shift, which chapter 6.1 of the reference book is entirely about and which the NeuroNet_DNG engine addresses with a layer this project never had. Two mechanisms make this the right fix rather than more hyperparameter nudging: - it decouples WEIGHT_DECAY from the learned function (van Laarhoven 2017) - with a normalized layer downstream, decay can no longer grind the discriminative signal away, it only rescales the effective learning rate; - it is the precondition for ever running an unbounded logit head here. The 2026-07-27 attempt blew up (IS error 5.6e15) precisely because nothing upstream constrained scale. Implementation notes: - CNeuronBatchNormOCL computes host-side rather than as a fourth copy of a kernel across Network.cl + WarriorCPU.cpp + WarriorDML.cpp. The math is elementwise O(n); this way it behaves identically on all four compute tiers, needs no DLL rebuild, and cannot drift between backends. Same precedent as the softmax+CCE gradient and the per-sample loss weighting, both computed in MQL5 for that reason. - Statistics are exponential moving, not a stored mini-batch: training is pure online SGD, one update per sample, so there is no batch to average over. BatchNormWindow is an EMA window length. - gamma/beta are excluded from weight decay, deliberately - decaying gamma toward zero is the exact pathology being fixed. - The layer self-sizes from whatever sits below it, because a conv/pool stage's output width is derived inside the CNet constructor and is not knowable to the topology builder. - Checkpoint capture/restore/blend carry gamma/beta and the running statistics alongside the dense matrix, so the plateau ladder cannot restore a mismatched pair. - SeedOutputLayerBias accepted only an exact defNeuronBaseOCL as the weight-carrying penultimate layer; with normalization enabled that is the batch-norm layer, so the cold-start bias seed would have silently stopped being applied. - Refuses to build, loudly, if a topology asks for normalization with no compute backend at all - rather than quietly training a different architecture than the one requested. EnableBatchNorm (default on) and BatchNormWindow (1000 samples) are inputs so the effect can be A/B'd without a recompile. Both feed the weights-filename fingerprint, appended conditionally so existing non-BN configs keep their fingerprints and are not forced to retrain. Verified: analytic gradients match finite differences to 1.5e-7 relative over 200 random cases; a faithful port of the full forward/backward chain collapses to the 33.3% floor by era 4 without this layer and holds 36-43% with it. Compiles 0 errors, 0 warnings. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-29 12:34:29 -04:00
//+------------------------------------------------------------------+
bool CNeuronBatchNormOCL::feedForward(CNeuronBaseOCL *NeuronOCL)
{
if(CheckPointer(NeuronOCL) == POINTER_INVALID)
return false;
perf: batch norm stops re-reading buffers it already has Batch norm computes host-side while its neighbours are device-resident, so every value it touches crosses the bus - and the cost is the BLOCKING SYNC per crossing, not the bytes. Per sample it did four reads, two of which were exact duplicates: feedForward -> previous layer's Output calcInputGradients -> own Gradient, and the previous layer's Output AGAIN update/accumulate -> own Gradient AGAIN Nothing writes the previous layer's Output between the forward and backward passes, and nothing writes this layer's Gradient between the gradient pass and the weight-update pass - backPropOCL runs those as two separate top-to-bottom loops and only the first writes gradients. So the repeats are removable and the result is bit-exact: the same values, read once instead of twice. Each cache is armed by its producer and disarmed by feedForward, so a consumer whose producer did not run this sample falls back to reading the buffer rather than using the previous sample's data. That is not hypothetical - a batch norm at layer 1 never gets calcInputGradients called at all, the same asymmetry backPropOCL already documents for a layer-1 LSTM, so its gradient cache is never armed and it takes the fallback every time. Both build variants compile 0 errors, 0 warnings. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-09 17:16:08 -04:00
//--- A new sample starts here, so last sample's gradient is no longer valid to reuse. Disarmed
//--- BEFORE anything can fail below, so an aborted forward pass cannot leave a consumer holding a
//--- cache that looks current - see the declaration comment.
m_gradCached = false;
m_fwdInputCached = false;
fix: live trades now use the geometry the gate certifies; perf: BN kernels Three changes, one theme: the trade placed, the trade graded, and the trade computed are now the same trade. 1) GEOMETRY WIRE (correctness, the ranked #1 open issue). The measured barrier pair reached the LABELS only - OpenParams still placed orders at the enum geometry (2*ATR/6*ATR), so the deploy gate certified "reaches 1.62*ATR before 3.33*ATR above break-even" about trades the EA never placed. Published via g_DerivedSlAtrMult/g_DerivedTpAtrMult (ConfidenceBridge, same same-tick contract as the confidence globals, because OpenParams runs on the root signal which has no pointer to the AI filter). Two writers: DeriveBarrierGeometry at era 0, and the .cfg adoption a deployed model takes. Overrides both legs and both Intelligent modes - the certificate is exact or it is nothing. TP is ATR-anchored like the label, NOT risk-relative, so a floor-widened stop cannot reshape the certified target. 2) BATCH NORM RUNS DEVICE-SIDE ON OPENCL. Four kernels in Network.cl - forward, hidden gradient, gamma/beta accumulate, gamma/beta apply - each a line-for-line transcription of the host implementation (NormalizeHost / HiddenGradHost / StepGammaBeta) including every NaN guard, clamp, and the exact moment-write ordering. The host copies remain the runtime for the DLL and pure-MQL5 tiers and the reference the kernels must match. Because this box has no OpenCL platform, the safety story is layered: - shim validation: kernels compiled as C and driven against a fp64 host transcription over NaN-poisoned stats, NaN gamma, over-clamp inputs, the frozen path, both optimizers, 3 batches - ALL PASS, worst normalized diff 0.132 vs tolerance 1.0 - in-situ self-check: each kernel is compared against its host twin ON FIRST USE on the real device (SelfCheckBn*), covering what the shim cannot - arg indices and buffer bindings. Any disagreement resyncs from the good copy, latches all BN kernels off process-wide, and training continues host-side. A transcription bug costs a warning and some speed, never a poisoned .nnw. - sync discipline: BatchOptions is now a CBufferDouble with explicit authority tracking (m_bnDeviceAuthoritative). Checkpoints/saves pull read-only; restores/loads/resets push; a mid-batch handover drains the device gamma/beta accumulator into the host arrays so no sample is lost. 3) SMALL FIXES. Apply-kernel build failure now latches the dispatch path at init (one warning instead of warning + failed Execute). Build tag bumped to win-scoring-gpu-v1 - first tag change since expectancy-stop-v1 despite five binary-changing commits. Both build variants compile 0 errors, 0 warnings. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-09 17:51:40 -04:00
//--- DEVICE PATH: nothing crosses the bus at all. The self-check runs the first time through and
//--- proves the kernel against NormalizeHost before the device is trusted with the persisted
//--- statistics; after that it is one dispatch per sample.
if(BnDeviceEligible() && NeuronOCL.getOutputIndex() >= 0 && getOutputIndex() >= 0 &&
EnsureBnDeviceBuffers())
{
if(!m_bnCheckedFwd)
return SelfCheckBnForward(NeuronOCL);
//--- Ramp bookkeeping stays host-side with iSamplesSeen; committed only on success so a failed
//--- dispatch that falls through to NormalizeHost (which increments itself) cannot double-count.
int seen = iSamplesSeen;
if(!bStatsFrozen && seen < iBatchSize)
seen++;
if(DispatchBnForward(NeuronOCL, (double)MathMax(1, seen)))
{
iSamplesSeen = seen;
return true;
}
LatchBnKernelsOff("BatchNormForward dispatch failed (error " +
IntegerToString(GetLastError()) + ")");
}
//--- HOST PATH (every non-OpenCL tier, and OpenCL after a latch). If the device had been
//--- authoritative, pull its state first - NormalizeHost below must advance the REAL statistics,
//--- not a stale mirror.
EnsureHostAuthoritative();
feat(ai): batch normalization between dense layers The only bounded stage in the entire forward path was the sigmoid classification head - every hidden stage is PRELU. That is a network with no internal scale control, and the failure ordered exactly by depth: on SP500 H1 the shallow perceptron held ~52% balanced accuracy while the deepest topology sat on the 33.3% one-class floor, with the per-bar logit spread decaying monotonically (0.45 -> 0.38 over ~200 eras) until the evidence tilt fell under the class-prior tilt. That is the signature of internal covariate shift, which chapter 6.1 of the reference book is entirely about and which the NeuroNet_DNG engine addresses with a layer this project never had. Two mechanisms make this the right fix rather than more hyperparameter nudging: - it decouples WEIGHT_DECAY from the learned function (van Laarhoven 2017) - with a normalized layer downstream, decay can no longer grind the discriminative signal away, it only rescales the effective learning rate; - it is the precondition for ever running an unbounded logit head here. The 2026-07-27 attempt blew up (IS error 5.6e15) precisely because nothing upstream constrained scale. Implementation notes: - CNeuronBatchNormOCL computes host-side rather than as a fourth copy of a kernel across Network.cl + WarriorCPU.cpp + WarriorDML.cpp. The math is elementwise O(n); this way it behaves identically on all four compute tiers, needs no DLL rebuild, and cannot drift between backends. Same precedent as the softmax+CCE gradient and the per-sample loss weighting, both computed in MQL5 for that reason. - Statistics are exponential moving, not a stored mini-batch: training is pure online SGD, one update per sample, so there is no batch to average over. BatchNormWindow is an EMA window length. - gamma/beta are excluded from weight decay, deliberately - decaying gamma toward zero is the exact pathology being fixed. - The layer self-sizes from whatever sits below it, because a conv/pool stage's output width is derived inside the CNet constructor and is not knowable to the topology builder. - Checkpoint capture/restore/blend carry gamma/beta and the running statistics alongside the dense matrix, so the plateau ladder cannot restore a mismatched pair. - SeedOutputLayerBias accepted only an exact defNeuronBaseOCL as the weight-carrying penultimate layer; with normalization enabled that is the batch-norm layer, so the cold-start bias seed would have silently stopped being applied. - Refuses to build, loudly, if a topology asks for normalization with no compute backend at all - rather than quietly training a different architecture than the one requested. EnableBatchNorm (default on) and BatchNormWindow (1000 samples) are inputs so the effect can be A/B'd without a recompile. Both feed the weights-filename fingerprint, appended conditionally so existing non-BN configs keep their fingerprints and are not forced to retrain. Verified: analytic gradients match finite differences to 1.5e-7 relative over 200 random cases; a faithful port of the full forward/backward chain collapses to the 33.3% floor by era 4 without this layer and holds 36-43% with it. Compiles 0 errors, 0 warnings. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-29 12:34:29 -04:00
double inputs[];
int count = NeuronOCL.getOutputVal(inputs);
if(count <= 0)
return false;
if(!NormalizeHost(inputs, count))
return false;
perf: batch norm stops re-reading buffers it already has Batch norm computes host-side while its neighbours are device-resident, so every value it touches crosses the bus - and the cost is the BLOCKING SYNC per crossing, not the bytes. Per sample it did four reads, two of which were exact duplicates: feedForward -> previous layer's Output calcInputGradients -> own Gradient, and the previous layer's Output AGAIN update/accumulate -> own Gradient AGAIN Nothing writes the previous layer's Output between the forward and backward passes, and nothing writes this layer's Gradient between the gradient pass and the weight-update pass - backPropOCL runs those as two separate top-to-bottom loops and only the first writes gradients. So the repeats are removable and the result is bit-exact: the same values, read once instead of twice. Each cache is armed by its producer and disarmed by feedForward, so a consumer whose producer did not run this sample falls back to reading the buffer rather than using the previous sample's data. That is not hypothetical - a batch norm at layer 1 never gets calcInputGradients called at all, the same asymmetry backPropOCL already documents for a layer-1 LSTM, so its gradient cache is never armed and it takes the fallback every time. Both build variants compile 0 errors, 0 warnings. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-09 17:16:08 -04:00
//--- Armed for calcInputGradients, which needs exactly these values and would otherwise read the
//--- same buffer back a second time this sample.
if(ArrayCopy(m_fwdInputCache, inputs, 0, 0, count) == count)
m_fwdInputCached = true;
feat(ai): batch normalization between dense layers The only bounded stage in the entire forward path was the sigmoid classification head - every hidden stage is PRELU. That is a network with no internal scale control, and the failure ordered exactly by depth: on SP500 H1 the shallow perceptron held ~52% balanced accuracy while the deepest topology sat on the 33.3% one-class floor, with the per-bar logit spread decaying monotonically (0.45 -> 0.38 over ~200 eras) until the evidence tilt fell under the class-prior tilt. That is the signature of internal covariate shift, which chapter 6.1 of the reference book is entirely about and which the NeuroNet_DNG engine addresses with a layer this project never had. Two mechanisms make this the right fix rather than more hyperparameter nudging: - it decouples WEIGHT_DECAY from the learned function (van Laarhoven 2017) - with a normalized layer downstream, decay can no longer grind the discriminative signal away, it only rescales the effective learning rate; - it is the precondition for ever running an unbounded logit head here. The 2026-07-27 attempt blew up (IS error 5.6e15) precisely because nothing upstream constrained scale. Implementation notes: - CNeuronBatchNormOCL computes host-side rather than as a fourth copy of a kernel across Network.cl + WarriorCPU.cpp + WarriorDML.cpp. The math is elementwise O(n); this way it behaves identically on all four compute tiers, needs no DLL rebuild, and cannot drift between backends. Same precedent as the softmax+CCE gradient and the per-sample loss weighting, both computed in MQL5 for that reason. - Statistics are exponential moving, not a stored mini-batch: training is pure online SGD, one update per sample, so there is no batch to average over. BatchNormWindow is an EMA window length. - gamma/beta are excluded from weight decay, deliberately - decaying gamma toward zero is the exact pathology being fixed. - The layer self-sizes from whatever sits below it, because a conv/pool stage's output width is derived inside the CNet constructor and is not knowable to the topology builder. - Checkpoint capture/restore/blend carry gamma/beta and the running statistics alongside the dense matrix, so the plateau ladder cannot restore a mismatched pair. - SeedOutputLayerBias accepted only an exact defNeuronBaseOCL as the weight-carrying penultimate layer; with normalization enabled that is the batch-norm layer, so the cold-start bias seed would have silently stopped being applied. - Refuses to build, loudly, if a topology asks for normalization with no compute backend at all - rather than quietly training a different architecture than the one requested. EnableBatchNorm (default on) and BatchNormWindow (1000 samples) are inputs so the effect can be A/B'd without a recompile. Both feed the weights-filename fingerprint, appended conditionally so existing non-BN configs keep their fingerprints and are not forced to retrain. Verified: analytic gradients match finite differences to 1.5e-7 relative over 200 random cases; a faithful port of the full forward/backward chain collapses to the 33.3% floor by era 4 without this layer and holds 36-43% with it. Compiles 0 errors, 0 warnings. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-29 12:34:29 -04:00
//--- Unlike the dense/conv kernels this layer's Output does NOT stay device-resident by itself -
//--- it was just written host-side, so it has to be pushed before the next layer's kernel reads
//--- it through getOutputIndex().
return Output.BufferWrite();
}
//+------------------------------------------------------------------+
//| Pure-MQL5 inference path (no backend at all): the previous |
//| layer's values live only in its host mirror, so read them there. |
//+------------------------------------------------------------------+
bool CNeuronBatchNormOCL::feedForwardCPU(CNeuronBaseOCL *NeuronOCL)
{
if(CheckPointer(NeuronOCL) == POINTER_INVALID)
return false;
int count = NeuronOCL.Neurons();
if(count <= 0)
return false;
double inputs[];
if(ArrayResize(inputs, count) != count)
return false;
for(int i = 0; i < count; i++)
inputs[i] = NeuronOCL.OutputHost(i);
//--- No BufferWrite: there is no device buffer on this path, and Output's host mirror is what
//--- the next layer's feedForwardCPU (and GetOutputsCPU) read.
return NormalizeHost(inputs, count);
}
//+------------------------------------------------------------------+
fix: live trades now use the geometry the gate certifies; perf: BN kernels Three changes, one theme: the trade placed, the trade graded, and the trade computed are now the same trade. 1) GEOMETRY WIRE (correctness, the ranked #1 open issue). The measured barrier pair reached the LABELS only - OpenParams still placed orders at the enum geometry (2*ATR/6*ATR), so the deploy gate certified "reaches 1.62*ATR before 3.33*ATR above break-even" about trades the EA never placed. Published via g_DerivedSlAtrMult/g_DerivedTpAtrMult (ConfidenceBridge, same same-tick contract as the confidence globals, because OpenParams runs on the root signal which has no pointer to the AI filter). Two writers: DeriveBarrierGeometry at era 0, and the .cfg adoption a deployed model takes. Overrides both legs and both Intelligent modes - the certificate is exact or it is nothing. TP is ATR-anchored like the label, NOT risk-relative, so a floor-widened stop cannot reshape the certified target. 2) BATCH NORM RUNS DEVICE-SIDE ON OPENCL. Four kernels in Network.cl - forward, hidden gradient, gamma/beta accumulate, gamma/beta apply - each a line-for-line transcription of the host implementation (NormalizeHost / HiddenGradHost / StepGammaBeta) including every NaN guard, clamp, and the exact moment-write ordering. The host copies remain the runtime for the DLL and pure-MQL5 tiers and the reference the kernels must match. Because this box has no OpenCL platform, the safety story is layered: - shim validation: kernels compiled as C and driven against a fp64 host transcription over NaN-poisoned stats, NaN gamma, over-clamp inputs, the frozen path, both optimizers, 3 batches - ALL PASS, worst normalized diff 0.132 vs tolerance 1.0 - in-situ self-check: each kernel is compared against its host twin ON FIRST USE on the real device (SelfCheckBn*), covering what the shim cannot - arg indices and buffer bindings. Any disagreement resyncs from the good copy, latches all BN kernels off process-wide, and training continues host-side. A transcription bug costs a warning and some speed, never a poisoned .nnw. - sync discipline: BatchOptions is now a CBufferDouble with explicit authority tracking (m_bnDeviceAuthoritative). Checkpoints/saves pull read-only; restores/loads/resets push; a mid-batch handover drains the device gamma/beta accumulator into the host arrays so no sample is lost. 3) SMALL FIXES. Apply-kernel build failure now latches the dispatch path at init (one warning instead of warning + failed Execute). Build tag bumped to win-scoring-gpu-v1 - first tag change since expectancy-stop-v1 despite five binary-changing commits. Both build variants compile 0 errors, 0 warnings. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-09 17:51:40 -04:00
//| DEVICE PATH plumbing - see the m_bnAcc declaration comment for |
//| the design. Everything below is OpenCL-only. |
//+------------------------------------------------------------------+
bool CNeuronBatchNormOCL::BnDeviceEligible(void)
{
return g_bnKernelUsable && iBatchSize > 1 &&
CheckPointer(OpenCL) != POINTER_INVALID &&
CheckPointer(BatchOptions) != POINTER_INVALID;
}
//+------------------------------------------------------------------+
bool CNeuronBatchNormOCL::EnsureBnDeviceBuffers(void)
{
//--- BufferCreate pushes the current host contents, so a freshly-created device copy is exactly
//--- the state the host was authoritative over - the invariant every self-check relies on.
if(BatchOptions.GetIndex() < 0 && !BatchOptions.BufferCreate(OpenCL))
return false;
if(CheckPointer(m_bnAcc) == POINTER_INVALID)
{
m_bnAcc = new CBufferDouble();
if(CheckPointer(m_bnAcc) == POINTER_INVALID)
return false;
if(!m_bnAcc.BufferInit(Neurons() * 2, 0.0))
return false;
}
if(m_bnAcc.GetIndex() < 0 && !m_bnAcc.BufferCreate(OpenCL))
return false;
return true;
}
//+------------------------------------------------------------------+
void CNeuronBatchNormOCL::SyncOptionsToHost(void)
{
//--- Read-only pull for checkpoints/saves taken mid-training; the device REMAINS authoritative.
//--- Guarded by the flag, because an unguarded BufferRead on a host-authoritative layer would
//--- clobber good host state with whatever stale bytes the device still holds.
if(m_bnDeviceAuthoritative && CheckPointer(BatchOptions) != POINTER_INVALID &&
BatchOptions.GetIndex() >= 0)
BatchOptions.BufferRead();
}
//+------------------------------------------------------------------+
void CNeuronBatchNormOCL::EnsureHostAuthoritative(void)
{
if(!m_bnDeviceAuthoritative)
return;
if(CheckPointer(BatchOptions) != POINTER_INVALID && BatchOptions.GetIndex() >= 0)
BatchOptions.BufferRead();
//--- Drain the device-side batch accumulator into the host arrays, so a handover in the MIDDLE of
//--- a batch keeps the samples the kernels already accumulated - without this, latching off after
//--- sample 3 of 8 would silently drop three samples' gradients from the batch.
if(CheckPointer(m_bnAcc) != POINTER_INVALID && m_bnAcc.GetIndex() >= 0 && m_bnAcc.BufferRead())
{
int units = Neurons();
if(ArraySize(m_accGamma) != units || ArraySize(m_accBeta) != units)
{
ArrayResize(m_accGamma, units);
ArrayResize(m_accBeta, units);
ArrayInitialize(m_accGamma, 0.0);
ArrayInitialize(m_accBeta, 0.0);
}
for(int i = 0; i < units && 2 * i + 1 < m_bnAcc.Total(); i++)
{
m_accGamma[i] += m_bnAcc.At(2 * i);
m_accBeta[i] += m_bnAcc.At(2 * i + 1);
}
ZeroOptimizerBuffer(m_bnAcc);
}
m_bnDeviceAuthoritative = false;
}
//+------------------------------------------------------------------+
void CNeuronBatchNormOCL::LatchBnKernelsOff(const string reason)
{
if(g_bnKernelUsable)
{
g_bnKernelUsable = false;
Print(__FUNCTION__ + ": BATCH-NORM KERNELS DISABLED for the rest of this run - " + reason +
". Every batch-norm layer falls back to the host implementation: results stay correct "
"(the host path is the reference the kernels were transcribed from), each layer just pays "
"its device round-trips again. State was resynced from the good copy before anything "
"could persist.");
}
EnsureHostAuthoritative();
}
//+------------------------------------------------------------------+
double CNeuronBatchNormOCL::BnDiffScore(double ref, double got)
{
bool fRef = MathIsValidNumber(ref), fGot = MathIsValidNumber(got);
if(!fRef || !fGot)
return (fRef == fGot) ? 0.0 : DBL_MAX; // both poisoned the same way is agreement
return MathAbs(got - ref) / (1.0e-3 * MathMax(1.0, MathAbs(ref)));
}
//+------------------------------------------------------------------+
bool CNeuronBatchNormOCL::DispatchBnForward(CNeuronBaseOCL *NeuronOCL, double w)
{
uint offset[1] = {0};
uint size[1];
size[0] = (uint)Neurons();
if(!OpenCL.SetArgumentBuffer(def_k_BatchNormForward, def_k_bnf_matrix_i, NeuronOCL.getOutputIndex()) ||
!OpenCL.SetArgumentBuffer(def_k_BatchNormForward, def_k_bnf_matrix_o, getOutputIndex()) ||
!OpenCL.SetArgumentBuffer(def_k_BatchNormForward, def_k_bnf_options, BatchOptions.GetIndex()) ||
!OpenCL.SetArgument(def_k_BatchNormForward, def_k_bnf_w, (float)w) ||
!OpenCL.SetArgument(def_k_BatchNormForward, def_k_bnf_frozen, bStatsFrozen ? 1 : 0))
return false;
ResetLastError();
if(!OpenCL.Execute(def_k_BatchNormForward, 1, offset, size))
return false;
//--- The kernel just wrote the running statistics on the device: it is now the authority.
m_bnDeviceAuthoritative = true;
return true;
}
//+------------------------------------------------------------------+
bool CNeuronBatchNormOCL::DispatchBnHiddenGrad(CNeuronBaseOCL *NeuronOCL)
{
uint offset[1] = {0};
uint size[1];
size[0] = (uint)Neurons();
if(!OpenCL.SetArgumentBuffer(def_k_BatchNormHiddenGrad, def_k_bnh_matrix_g, getGradientIndex()) ||
!OpenCL.SetArgumentBuffer(def_k_BatchNormHiddenGrad, def_k_bnh_prev_o, NeuronOCL.getOutputIndex()) ||
!OpenCL.SetArgumentBuffer(def_k_BatchNormHiddenGrad, def_k_bnh_prev_g, NeuronOCL.getGradientIndex()) ||
!OpenCL.SetArgumentBuffer(def_k_BatchNormHiddenGrad, def_k_bnh_options, BatchOptions.GetIndex()) ||
!OpenCL.SetArgument(def_k_BatchNormHiddenGrad, def_k_bnh_activation, NativeActivationCode(NeuronOCL.Activation())))
return false;
ResetLastError();
return OpenCL.Execute(def_k_BatchNormHiddenGrad, 1, offset, size);
}
//+------------------------------------------------------------------+
bool CNeuronBatchNormOCL::DispatchBnAccum(void)
{
uint offset[1] = {0};
uint size[1];
size[0] = (uint)Neurons();
if(!OpenCL.SetArgumentBuffer(def_k_BatchNormAccumGammaBeta, def_k_bna_matrix_g, getGradientIndex()) ||
!OpenCL.SetArgumentBuffer(def_k_BatchNormAccumGammaBeta, def_k_bna_options, BatchOptions.GetIndex()) ||
!OpenCL.SetArgumentBuffer(def_k_BatchNormAccumGammaBeta, def_k_bna_acc, m_bnAcc.GetIndex()))
return false;
ResetLastError();
return OpenCL.Execute(def_k_BatchNormAccumGammaBeta, 1, offset, size);
}
//+------------------------------------------------------------------+
bool CNeuronBatchNormOCL::DispatchBnApply(double scale, double lt)
{
uint offset[1] = {0};
uint size[1];
size[0] = (uint)Neurons();
if(!OpenCL.SetArgumentBuffer(def_k_BatchNormApplyGammaBeta, def_k_bnp_options, BatchOptions.GetIndex()) ||
!OpenCL.SetArgumentBuffer(def_k_BatchNormApplyGammaBeta, def_k_bnp_acc, m_bnAcc.GetIndex()) ||
!OpenCL.SetArgument(def_k_BatchNormApplyGammaBeta, def_k_bnp_scale, (float)scale) ||
!OpenCL.SetArgument(def_k_BatchNormApplyGammaBeta, def_k_bnp_lt, (float)lt) ||
refactor(stdlib): adopt Math\Stat for the deploy gate's normal tail; retire the b1/b2/lr/momentum macros The gate's NormalUpperTail was a hand-rolled Abramowitz & Stegun 26.2.17 approximation. Its own comment gave the reason - "drags a chain of headers behind it" - and that turned out to be one file: Math\Stat\Normal.mqh includes only Math.mqh, which includes nothing. Swapped for Cody's rational approximation in the library (~18 significant digits vs |error| < 7.5e-8). No past verdict changes: at the z the gate operates on, the difference is orders of magnitude below DEPLOY_FAMILY_WISE_ALPHA. Adopting it needed the four bare macros in AI\Network.mqh gone first. "#define b1 AdamBeta1" collides with an identifier in Math.mqh, so the include would have macro-expanded the library's own local and failed to compile - the same landmine that made the original author rename the approximation's coefficients to ntB1..ntB5 rather than use the reference's b1..b5. lr, b2 and momentum are the same class of hazard: single-token global macros in a 52k-line codebase. All four now resolve to the input names they always aliased, which is a pure textual identity - verified zero bare occurrences remain. Also: - SelectionSort over the buffered signals was O(n^2) with an O(n^2) count of StructToTime calls, because the comparison rebuilt both datetimes from the six int date fields every time. Now materialises the keys once and does an insertion sort; ArraySort cannot permute a struct array. IsEarlier goes with it, MakeDateTime becomes SignalTime. - Seven FileOpen sites lacked FILE_SHARE_READ|FILE_SHARE_WRITE, including AtomicWriteBegin, which stages every model save. All 43 sites now carry them - an exclusive open fails outright when another process holds the path, which here has meant a silently skipped save. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-19 19:31:36 -04:00
!OpenCL.SetArgument(def_k_BatchNormApplyGammaBeta, def_k_bnp_b1, (float)AdamBeta1) ||
!OpenCL.SetArgument(def_k_BatchNormApplyGammaBeta, def_k_bnp_b2, (float)AdamBeta2) ||
fix(build): g_eta - the learning rate global no longer shadows a stdlib local MetaEditor: "declaration of 'eta' hides global variable" (Math.mqh:792 vs Network.mqh:80). The standard library's Math\Stat\Math.mqh declares a local `double eta` in its incomplete-gamma branch, and our bare global of the same name is in scope there. Same fault as the b1/b2/lr/momentum macros retired in ea2552e: a single-token global name living in a header that library code gets compiled beside. The library cannot move, so ours does - 112 references across 13 files, whole-word only. Named g_eta rather than g_learningRate to stay inside the vocabulary already around it (ETA_DECAY_FACTOR, ETA_MIN, m_etaCeiling, etaBefore), all of which are untouched and none of which shadow anything. One log line said "continuing to explore without decaying eta", where the word was prose rather than a symbol reference; that reads "the learning rate" now instead of naming a variable at the trader. Scanned for the next occurrence rather than waiting for it: the only other bare lowercase globals in the tree are eaName and tableschema, both distinctive enough not to collide with a library local. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-20 07:00:09 -04:00
!OpenCL.SetArgument(def_k_BatchNormApplyGammaBeta, def_k_bnp_lr, (float)g_eta) ||
fix: live trades now use the geometry the gate certifies; perf: BN kernels Three changes, one theme: the trade placed, the trade graded, and the trade computed are now the same trade. 1) GEOMETRY WIRE (correctness, the ranked #1 open issue). The measured barrier pair reached the LABELS only - OpenParams still placed orders at the enum geometry (2*ATR/6*ATR), so the deploy gate certified "reaches 1.62*ATR before 3.33*ATR above break-even" about trades the EA never placed. Published via g_DerivedSlAtrMult/g_DerivedTpAtrMult (ConfidenceBridge, same same-tick contract as the confidence globals, because OpenParams runs on the root signal which has no pointer to the AI filter). Two writers: DeriveBarrierGeometry at era 0, and the .cfg adoption a deployed model takes. Overrides both legs and both Intelligent modes - the certificate is exact or it is nothing. TP is ATR-anchored like the label, NOT risk-relative, so a floor-widened stop cannot reshape the certified target. 2) BATCH NORM RUNS DEVICE-SIDE ON OPENCL. Four kernels in Network.cl - forward, hidden gradient, gamma/beta accumulate, gamma/beta apply - each a line-for-line transcription of the host implementation (NormalizeHost / HiddenGradHost / StepGammaBeta) including every NaN guard, clamp, and the exact moment-write ordering. The host copies remain the runtime for the DLL and pure-MQL5 tiers and the reference the kernels must match. Because this box has no OpenCL platform, the safety story is layered: - shim validation: kernels compiled as C and driven against a fp64 host transcription over NaN-poisoned stats, NaN gamma, over-clamp inputs, the frozen path, both optimizers, 3 batches - ALL PASS, worst normalized diff 0.132 vs tolerance 1.0 - in-situ self-check: each kernel is compared against its host twin ON FIRST USE on the real device (SelfCheckBn*), covering what the shim cannot - arg indices and buffer bindings. Any disagreement resyncs from the good copy, latches all BN kernels off process-wide, and training continues host-side. A transcription bug costs a warning and some speed, never a poisoned .nnw. - sync discipline: BatchOptions is now a CBufferDouble with explicit authority tracking (m_bnDeviceAuthoritative). Checkpoints/saves pull read-only; restores/loads/resets push; a mid-batch handover drains the device gamma/beta accumulator into the host arrays so no sample is lost. 3) SMALL FIXES. Apply-kernel build failure now latches the dispatch path at init (one warning instead of warning + failed Execute). Build tag bumped to win-scoring-gpu-v1 - first tag change since expectancy-stop-v1 despite five binary-changing commits. Both build variants compile 0 errors, 0 warnings. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-09 17:51:40 -04:00
!OpenCL.SetArgument(def_k_BatchNormApplyGammaBeta, def_k_bnp_momentum, (float)alpha) ||
!OpenCL.SetArgument(def_k_BatchNormApplyGammaBeta, def_k_bnp_optimizer, (optimization == SGD) ? 0 : 1))
return false;
ResetLastError();
if(!OpenCL.Execute(def_k_BatchNormApplyGammaBeta, 1, offset, size))
return false;
m_bnDeviceAuthoritative = true;
return true;
}
//+------------------------------------------------------------------+
//| SELF-CHECK: forward. Runs the HOST transform first (on the same |
//| pre-state the device holds), dispatches the kernel, then compares |
//| output + statistics elementwise. Pass -> the device is verified |
//| and becomes authoritative. Fail -> the host result is restored to |
//| both copies and the kernels latch off. Cost: one extra input read |
//| and one output+options read, ONCE per layer per process. |
//+------------------------------------------------------------------+
bool CNeuronBatchNormOCL::SelfCheckBnForward(CNeuronBaseOCL *NeuronOCL)
{
int units = Neurons();
double inputs[];
int count = NeuronOCL.getOutputVal(inputs);
if(count <= 0)
return false;
//--- Host reference. NormalizeHost mutates the host mirrors (Output, BatchOptions, iSamplesSeen);
//--- the device still holds the untouched pre-state, which is exactly what the kernel must see.
if(!NormalizeHost(inputs, count))
return false;
int oTotal = BatchOptions.Total();
double refY[], refOpt[];
ArrayResize(refY, units);
ArrayResize(refOpt, oTotal);
for(int i = 0; i < units; i++)
refY[i] = Output.At(i);
for(int i = 0; i < oTotal; i++)
refOpt[i] = BatchOptions.At(i);
//--- Kernel on the same sample. NormalizeHost already advanced iSamplesSeen, so the ramped window
//--- it used is exactly MathMax(1, iSamplesSeen) now.
bool ok = DispatchBnForward(NeuronOCL, (double)MathMax(1, iSamplesSeen));
if(ok)
ok = Output.BufferRead() && BatchOptions.BufferRead();
double worst = 0.0;
int worstAt = -1;
if(ok)
{
for(int i = 0; i < units; i++)
{
double s = BnDiffScore(refY[i], Output.At(i));
if(s > worst) { worst = s; worstAt = i; }
}
for(int i = 0; i < oTotal; i++)
{
double s = BnDiffScore(refOpt[i], BatchOptions.At(i));
if(s > worst) { worst = s; worstAt = units + i; }
}
}
if(ok && worst <= 1.0)
{
m_bnCheckedFwd = true;
m_bnDeviceAuthoritative = true;
Print(__FUNCTION__ + StringFormat(": BatchNormForward kernel VERIFIED against the host math on "
"%d units (worst normalized diff %.2e) - this layer's forward "
"pass now runs device-side.", units, worst));
return true;
}
//--- Kernel wrong or unreachable: the host result is the answer. Put it back in both copies.
for(int i = 0; i < units; i++)
Output.Update(i, refY[i]);
for(int i = 0; i < oTotal; i++)
BatchOptions.Update(i, refOpt[i]);
if(BatchOptions.GetIndex() >= 0)
BatchOptions.BufferWrite();
m_bnDeviceAuthoritative = false;
LatchBnKernelsOff(ok ? StringFormat("BatchNormForward disagrees with the host math (worst "
"normalized diff %.2e at element %d)", worst, worstAt)
: "BatchNormForward dispatch/readback failed (error " +
IntegerToString(GetLastError()) + ")");
return Output.BufferWrite();
}
//+------------------------------------------------------------------+
//| SELF-CHECK: backward. Same pattern; the reference is |
//| HiddenGradHost, which IS the host path's own loop. |
//+------------------------------------------------------------------+
bool CNeuronBatchNormOCL::SelfCheckBnHiddenGrad(CNeuronBaseOCL *NeuronOCL)
{
int units = Neurons();
SyncOptionsToHost(); // the host reference must see the statistics the kernels have been updating
double grad[];
if(getGradient(grad) < units)
return false;
double prevOut[];
if(NeuronOCL.getOutputVal(prevOut) < units)
return false;
double ig[];
if(ArrayResize(ig, units) != units)
return false;
HiddenGradHost(grad, prevOut, NeuronOCL.Activation(), ig, units);
bool ok = DispatchBnHiddenGrad(NeuronOCL);
double got[];
if(ok)
ok = (NeuronOCL.getGradient(got) >= units);
double worst = 0.0;
int worstAt = -1;
if(ok)
for(int i = 0; i < units; i++)
{
double s = BnDiffScore(ig[i], got[i]);
if(s > worst) { worst = s; worstAt = i; }
}
if(ok && worst <= 1.0)
{
m_bnCheckedGrad = true;
Print(__FUNCTION__ + StringFormat(": BatchNormHiddenGrad kernel VERIFIED against the host math "
"on %d units (worst normalized diff %.2e).", units, worst));
return true;
}
LatchBnKernelsOff(ok ? StringFormat("BatchNormHiddenGrad disagrees with the host math (worst "
"normalized diff %.2e at unit %d)", worst, worstAt)
: "BatchNormHiddenGrad dispatch/readback failed (error " +
IntegerToString(GetLastError()) + ")");
//--- The host result is the answer either way; setGradient writes host and device copies.
return NeuronOCL.setGradient(ig);
}
//+------------------------------------------------------------------+
//| SELF-CHECK: gamma/beta accumulate. First use ever, so the device |
//| accumulator holds the zeros it was created with. |
//+------------------------------------------------------------------+
bool CNeuronBatchNormOCL::SelfCheckBnAccum(void)
{
int units = Neurons();
SyncOptionsToHost(); // NX is device-fresh in kernel mode
double grad[];
if(getGradient(grad) < units)
return false;
double preAcc[];
if(m_bnAcc.GetIndex() >= 0 && !m_bnAcc.BufferRead())
{
//--- Cannot even read the accumulator: latch (which drains whatever earlier samples the kernels
//--- put there) before reporting the failure, so the batch is not silently truncated.
LatchBnKernelsOff("BatchNormAccumGammaBeta pre-check read failed (error " +
IntegerToString(GetLastError()) + ")");
return false;
}
ArrayResize(preAcc, m_bnAcc.Total());
for(int i = 0; i < m_bnAcc.Total(); i++)
preAcc[i] = m_bnAcc.At(i);
bool ok = DispatchBnAccum();
if(ok)
ok = m_bnAcc.BufferRead();
double worst = 0.0;
int worstAt = -1;
if(ok)
for(int i = 0; i < units; i++)
{
double expG = preAcc[2 * i] + grad[i] * BatchOptions.At(i * BN_OPT_STRIDE + BN_OPT_NX);
double expB = preAcc[2 * i + 1] + grad[i];
double s = MathMax(BnDiffScore(expG, m_bnAcc.At(2 * i)), BnDiffScore(expB, m_bnAcc.At(2 * i + 1)));
if(s > worst) { worst = s; worstAt = i; }
}
if(ok && worst <= 1.0)
{
m_bnCheckedAccum = true;
Print(__FUNCTION__ + StringFormat(": BatchNormAccumGammaBeta kernel VERIFIED against the host "
"math on %d units (worst normalized diff %.2e).", units, worst));
return true;
}
//--- Restore the truth (pre-state plus this sample's host-computed contribution) into the device
//--- accumulator BEFORE latching, so the drain inside the latch hands the host arrays exactly the
//--- right sums.
for(int i = 0; i < units; i++)
{
m_bnAcc.Update(2 * i, preAcc[2 * i] + grad[i] * BatchOptions.At(i * BN_OPT_STRIDE + BN_OPT_NX));
m_bnAcc.Update(2 * i + 1, preAcc[2 * i + 1] + grad[i]);
}
if(m_bnAcc.GetIndex() >= 0)
m_bnAcc.BufferWrite();
LatchBnKernelsOff(ok ? StringFormat("BatchNormAccumGammaBeta disagrees with the host math (worst "
"normalized diff %.2e at unit %d)", worst, worstAt)
: "BatchNormAccumGammaBeta dispatch/readback failed (error " +
IntegerToString(GetLastError()) + ")");
return true;
}
//+------------------------------------------------------------------+
//| SELF-CHECK: gamma/beta apply. The host reference is StepGammaBeta |
//| itself, run on the synced host mirror - literally the code the |
//| host path executes, so the comparison cannot drift from it. |
//+------------------------------------------------------------------+
bool CNeuronBatchNormOCL::SelfCheckBnApply(double scale, double lt)
{
int units = Neurons();
SyncOptionsToHost();
if(m_bnAcc.GetIndex() >= 0 && !m_bnAcc.BufferRead())
{
//--- Same reasoning as the accumulate pre-check: latch drains the device sums into the host
//--- arrays, so the caller's host fallback steps the REAL batch rather than an empty one.
LatchBnKernelsOff("BatchNormApplyGammaBeta pre-check read failed (error " +
IntegerToString(GetLastError()) + ")");
return false;
}
//--- Host reference: run the real host step on the host mirror (currently the pre-state).
double accG[], accB[];
ArrayResize(accG, units);
ArrayResize(accB, units);
for(int i = 0; i < units; i++)
{
accG[i] = m_bnAcc.At(2 * i);
accB[i] = m_bnAcc.At(2 * i + 1);
if(!StepGammaBeta(i * BN_OPT_STRIDE, accG[i] * scale, accB[i] * scale, lt))
return false;
}
int oTotal = BatchOptions.Total();
double refOpt[];
ArrayResize(refOpt, oTotal);
for(int i = 0; i < oTotal; i++)
refOpt[i] = BatchOptions.At(i);
bool ok = DispatchBnApply(scale, lt);
if(ok)
ok = BatchOptions.BufferRead() && m_bnAcc.BufferRead();
double worst = 0.0;
int worstAt = -1;
if(ok)
{
for(int i = 0; i < oTotal; i++)
{
double s = BnDiffScore(refOpt[i], BatchOptions.At(i));
if(s > worst) { worst = s; worstAt = i; }
}
//--- and the kernel must have zeroed the accumulator
for(int i = 0; i < m_bnAcc.Total(); i++)
if(MathAbs(m_bnAcc.At(i)) > 1.0e-12)
{ worst = DBL_MAX; worstAt = oTotal + i; break; }
}
if(ok && worst <= 1.0)
{
m_bnCheckedApply = true;
m_bnDeviceAuthoritative = true;
Print(__FUNCTION__ + StringFormat(": BatchNormApplyGammaBeta kernel VERIFIED against the host "
"math on %d units (worst normalized diff %.2e).", units, worst));
return true;
}
//--- The host step already produced the correct post-state in the host mirror - push it, zero the
//--- accumulator everywhere, and latch.
for(int i = 0; i < oTotal; i++)
BatchOptions.Update(i, refOpt[i]);
if(BatchOptions.GetIndex() >= 0)
BatchOptions.BufferWrite();
ZeroOptimizerBuffer(m_bnAcc);
m_bnDeviceAuthoritative = false;
LatchBnKernelsOff(ok ? StringFormat("BatchNormApplyGammaBeta disagrees with the host math (worst "
"normalized diff %.2e at slot %d)", worst, worstAt)
: "BatchNormApplyGammaBeta dispatch/readback failed (error " +
IntegerToString(GetLastError()) + ")");
return true;
}
//+------------------------------------------------------------------+
refactor(comments): box headers to stdlib length The //| box blocks were excluded from 0b06f8e and 5efdb48 and were what remained: 160 of them ran to 10+ lines, the longest to 88. Compressed to their leading topic sentences - 5 lines for a function header, 8 for a file header - keeping the box format and the standard MQL5 name/author lines verbatim. Verified at the BYTE level this time, across every in-scope file: the list of non-comment lines is byte-identical to HEAD and braces balance. The first check compared a locale-decoded 'git show' against a UTF-8 read and flagged 25 files that had not changed at all - every BOM and every non-ASCII line mismatched. 47,696 -> 40,665 lines in scope; comment share 38% -> 26%. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-22 00:30:14 -04:00
//| Backward. Called by the layer BELOW this one (inverted-call |
//| convention, same as Conv/Pool/LSTM): consumes this layer's own |
//| Gradient (dL/dy, already filled by calcHiddenGradients against |
//| the layer above) and writes dL/dx into NeuronOCL's Gradient. |
feat(ai): batch normalization between dense layers The only bounded stage in the entire forward path was the sigmoid classification head - every hidden stage is PRELU. That is a network with no internal scale control, and the failure ordered exactly by depth: on SP500 H1 the shallow perceptron held ~52% balanced accuracy while the deepest topology sat on the 33.3% one-class floor, with the per-bar logit spread decaying monotonically (0.45 -> 0.38 over ~200 eras) until the evidence tilt fell under the class-prior tilt. That is the signature of internal covariate shift, which chapter 6.1 of the reference book is entirely about and which the NeuroNet_DNG engine addresses with a layer this project never had. Two mechanisms make this the right fix rather than more hyperparameter nudging: - it decouples WEIGHT_DECAY from the learned function (van Laarhoven 2017) - with a normalized layer downstream, decay can no longer grind the discriminative signal away, it only rescales the effective learning rate; - it is the precondition for ever running an unbounded logit head here. The 2026-07-27 attempt blew up (IS error 5.6e15) precisely because nothing upstream constrained scale. Implementation notes: - CNeuronBatchNormOCL computes host-side rather than as a fourth copy of a kernel across Network.cl + WarriorCPU.cpp + WarriorDML.cpp. The math is elementwise O(n); this way it behaves identically on all four compute tiers, needs no DLL rebuild, and cannot drift between backends. Same precedent as the softmax+CCE gradient and the per-sample loss weighting, both computed in MQL5 for that reason. - Statistics are exponential moving, not a stored mini-batch: training is pure online SGD, one update per sample, so there is no batch to average over. BatchNormWindow is an EMA window length. - gamma/beta are excluded from weight decay, deliberately - decaying gamma toward zero is the exact pathology being fixed. - The layer self-sizes from whatever sits below it, because a conv/pool stage's output width is derived inside the CNet constructor and is not knowable to the topology builder. - Checkpoint capture/restore/blend carry gamma/beta and the running statistics alongside the dense matrix, so the plateau ladder cannot restore a mismatched pair. - SeedOutputLayerBias accepted only an exact defNeuronBaseOCL as the weight-carrying penultimate layer; with normalization enabled that is the batch-norm layer, so the cold-start bias seed would have silently stopped being applied. - Refuses to build, loudly, if a topology asks for normalization with no compute backend at all - rather than quietly training a different architecture than the one requested. EnableBatchNorm (default on) and BatchNormWindow (1000 samples) are inputs so the effect can be A/B'd without a recompile. Both feed the weights-filename fingerprint, appended conditionally so existing non-BN configs keep their fingerprints and are not forced to retrain. Verified: analytic gradients match finite differences to 1.5e-7 relative over 200 random cases; a faithful port of the full forward/backward chain collapses to the 33.3% floor by era 4 without this layer and holds 36-43% with it. Compiles 0 errors, 0 warnings. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-29 12:34:29 -04:00
//+------------------------------------------------------------------+
bool CNeuronBatchNormOCL::calcInputGradients(CNeuronBaseOCL *NeuronOCL)
{
if(CheckPointer(NeuronOCL) == POINTER_INVALID || CheckPointer(BatchOptions) == POINTER_INVALID)
return false;
fix: live trades now use the geometry the gate certifies; perf: BN kernels Three changes, one theme: the trade placed, the trade graded, and the trade computed are now the same trade. 1) GEOMETRY WIRE (correctness, the ranked #1 open issue). The measured barrier pair reached the LABELS only - OpenParams still placed orders at the enum geometry (2*ATR/6*ATR), so the deploy gate certified "reaches 1.62*ATR before 3.33*ATR above break-even" about trades the EA never placed. Published via g_DerivedSlAtrMult/g_DerivedTpAtrMult (ConfidenceBridge, same same-tick contract as the confidence globals, because OpenParams runs on the root signal which has no pointer to the AI filter). Two writers: DeriveBarrierGeometry at era 0, and the .cfg adoption a deployed model takes. Overrides both legs and both Intelligent modes - the certificate is exact or it is nothing. TP is ATR-anchored like the label, NOT risk-relative, so a floor-widened stop cannot reshape the certified target. 2) BATCH NORM RUNS DEVICE-SIDE ON OPENCL. Four kernels in Network.cl - forward, hidden gradient, gamma/beta accumulate, gamma/beta apply - each a line-for-line transcription of the host implementation (NormalizeHost / HiddenGradHost / StepGammaBeta) including every NaN guard, clamp, and the exact moment-write ordering. The host copies remain the runtime for the DLL and pure-MQL5 tiers and the reference the kernels must match. Because this box has no OpenCL platform, the safety story is layered: - shim validation: kernels compiled as C and driven against a fp64 host transcription over NaN-poisoned stats, NaN gamma, over-clamp inputs, the frozen path, both optimizers, 3 batches - ALL PASS, worst normalized diff 0.132 vs tolerance 1.0 - in-situ self-check: each kernel is compared against its host twin ON FIRST USE on the real device (SelfCheckBn*), covering what the shim cannot - arg indices and buffer bindings. Any disagreement resyncs from the good copy, latches all BN kernels off process-wide, and training continues host-side. A transcription bug costs a warning and some speed, never a poisoned .nnw. - sync discipline: BatchOptions is now a CBufferDouble with explicit authority tracking (m_bnDeviceAuthoritative). Checkpoints/saves pull read-only; restores/loads/resets push; a mid-batch handover drains the device gamma/beta accumulator into the host arrays so no sample is lost. 3) SMALL FIXES. Apply-kernel build failure now latches the dispatch path at init (one warning instead of warning + failed Execute). Build tag bumped to win-scoring-gpu-v1 - first tag change since expectancy-stop-v1 despite five binary-changing commits. Both build variants compile 0 errors, 0 warnings. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-09 17:51:40 -04:00
int units = Neurons();
int inputsCount = NeuronOCL.Neurons();
if(units <= 0 || inputsCount != units)
return false; // batch norm is elementwise - a size mismatch means the topology is wrong
//--- DEVICE PATH: the gradient, the previous output and the statistics are all already on the
//--- device; the result lands in the previous layer's device gradient where its own backward
//--- kernels read it. Zero transfers.
if(BnDeviceEligible() && NeuronOCL.getOutputIndex() >= 0 && NeuronOCL.getGradientIndex() >= 0 &&
getGradientIndex() >= 0 && EnsureBnDeviceBuffers())
{
if(!m_bnCheckedGrad)
return SelfCheckBnHiddenGrad(NeuronOCL);
if(DispatchBnHiddenGrad(NeuronOCL))
return true;
LatchBnKernelsOff("BatchNormHiddenGrad dispatch failed (error " +
IntegerToString(GetLastError()) + ")");
}
//--- HOST PATH.
EnsureHostAuthoritative();
feat(ai): batch normalization between dense layers The only bounded stage in the entire forward path was the sigmoid classification head - every hidden stage is PRELU. That is a network with no internal scale control, and the failure ordered exactly by depth: on SP500 H1 the shallow perceptron held ~52% balanced accuracy while the deepest topology sat on the 33.3% one-class floor, with the per-bar logit spread decaying monotonically (0.45 -> 0.38 over ~200 eras) until the evidence tilt fell under the class-prior tilt. That is the signature of internal covariate shift, which chapter 6.1 of the reference book is entirely about and which the NeuroNet_DNG engine addresses with a layer this project never had. Two mechanisms make this the right fix rather than more hyperparameter nudging: - it decouples WEIGHT_DECAY from the learned function (van Laarhoven 2017) - with a normalized layer downstream, decay can no longer grind the discriminative signal away, it only rescales the effective learning rate; - it is the precondition for ever running an unbounded logit head here. The 2026-07-27 attempt blew up (IS error 5.6e15) precisely because nothing upstream constrained scale. Implementation notes: - CNeuronBatchNormOCL computes host-side rather than as a fourth copy of a kernel across Network.cl + WarriorCPU.cpp + WarriorDML.cpp. The math is elementwise O(n); this way it behaves identically on all four compute tiers, needs no DLL rebuild, and cannot drift between backends. Same precedent as the softmax+CCE gradient and the per-sample loss weighting, both computed in MQL5 for that reason. - Statistics are exponential moving, not a stored mini-batch: training is pure online SGD, one update per sample, so there is no batch to average over. BatchNormWindow is an EMA window length. - gamma/beta are excluded from weight decay, deliberately - decaying gamma toward zero is the exact pathology being fixed. - The layer self-sizes from whatever sits below it, because a conv/pool stage's output width is derived inside the CNet constructor and is not knowable to the topology builder. - Checkpoint capture/restore/blend carry gamma/beta and the running statistics alongside the dense matrix, so the plateau ladder cannot restore a mismatched pair. - SeedOutputLayerBias accepted only an exact defNeuronBaseOCL as the weight-carrying penultimate layer; with normalization enabled that is the batch-norm layer, so the cold-start bias seed would have silently stopped being applied. - Refuses to build, loudly, if a topology asks for normalization with no compute backend at all - rather than quietly training a different architecture than the one requested. EnableBatchNorm (default on) and BatchNormWindow (1000 samples) are inputs so the effect can be A/B'd without a recompile. Both feed the weights-filename fingerprint, appended conditionally so existing non-BN configs keep their fingerprints and are not forced to retrain. Verified: analytic gradients match finite differences to 1.5e-7 relative over 200 random cases; a faithful port of the full forward/backward chain collapses to the 33.3% floor by era 4 without this layer and holds 36-43% with it. Compiles 0 errors, 0 warnings. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-29 12:34:29 -04:00
double grad[];
int count = getGradient(grad);
//--- Gradient is allocated with one slot more than Neurons() (see CNeuronBaseOCL::Init); only the
//--- real units carry a value.
fix: live trades now use the geometry the gate certifies; perf: BN kernels Three changes, one theme: the trade placed, the trade graded, and the trade computed are now the same trade. 1) GEOMETRY WIRE (correctness, the ranked #1 open issue). The measured barrier pair reached the LABELS only - OpenParams still placed orders at the enum geometry (2*ATR/6*ATR), so the deploy gate certified "reaches 1.62*ATR before 3.33*ATR above break-even" about trades the EA never placed. Published via g_DerivedSlAtrMult/g_DerivedTpAtrMult (ConfidenceBridge, same same-tick contract as the confidence globals, because OpenParams runs on the root signal which has no pointer to the AI filter). Two writers: DeriveBarrierGeometry at era 0, and the .cfg adoption a deployed model takes. Overrides both legs and both Intelligent modes - the certificate is exact or it is nothing. TP is ATR-anchored like the label, NOT risk-relative, so a floor-widened stop cannot reshape the certified target. 2) BATCH NORM RUNS DEVICE-SIDE ON OPENCL. Four kernels in Network.cl - forward, hidden gradient, gamma/beta accumulate, gamma/beta apply - each a line-for-line transcription of the host implementation (NormalizeHost / HiddenGradHost / StepGammaBeta) including every NaN guard, clamp, and the exact moment-write ordering. The host copies remain the runtime for the DLL and pure-MQL5 tiers and the reference the kernels must match. Because this box has no OpenCL platform, the safety story is layered: - shim validation: kernels compiled as C and driven against a fp64 host transcription over NaN-poisoned stats, NaN gamma, over-clamp inputs, the frozen path, both optimizers, 3 batches - ALL PASS, worst normalized diff 0.132 vs tolerance 1.0 - in-situ self-check: each kernel is compared against its host twin ON FIRST USE on the real device (SelfCheckBn*), covering what the shim cannot - arg indices and buffer bindings. Any disagreement resyncs from the good copy, latches all BN kernels off process-wide, and training continues host-side. A transcription bug costs a warning and some speed, never a poisoned .nnw. - sync discipline: BatchOptions is now a CBufferDouble with explicit authority tracking (m_bnDeviceAuthoritative). Checkpoints/saves pull read-only; restores/loads/resets push; a mid-batch handover drains the device gamma/beta accumulator into the host arrays so no sample is lost. 3) SMALL FIXES. Apply-kernel build failure now latches the dispatch path at init (one warning instead of warning + failed Execute). Build tag bumped to win-scoring-gpu-v1 - first tag change since expectancy-stop-v1 despite five binary-changing commits. Both build variants compile 0 errors, 0 warnings. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-09 17:51:40 -04:00
if(count < units)
feat(ai): batch normalization between dense layers The only bounded stage in the entire forward path was the sigmoid classification head - every hidden stage is PRELU. That is a network with no internal scale control, and the failure ordered exactly by depth: on SP500 H1 the shallow perceptron held ~52% balanced accuracy while the deepest topology sat on the 33.3% one-class floor, with the per-bar logit spread decaying monotonically (0.45 -> 0.38 over ~200 eras) until the evidence tilt fell under the class-prior tilt. That is the signature of internal covariate shift, which chapter 6.1 of the reference book is entirely about and which the NeuroNet_DNG engine addresses with a layer this project never had. Two mechanisms make this the right fix rather than more hyperparameter nudging: - it decouples WEIGHT_DECAY from the learned function (van Laarhoven 2017) - with a normalized layer downstream, decay can no longer grind the discriminative signal away, it only rescales the effective learning rate; - it is the precondition for ever running an unbounded logit head here. The 2026-07-27 attempt blew up (IS error 5.6e15) precisely because nothing upstream constrained scale. Implementation notes: - CNeuronBatchNormOCL computes host-side rather than as a fourth copy of a kernel across Network.cl + WarriorCPU.cpp + WarriorDML.cpp. The math is elementwise O(n); this way it behaves identically on all four compute tiers, needs no DLL rebuild, and cannot drift between backends. Same precedent as the softmax+CCE gradient and the per-sample loss weighting, both computed in MQL5 for that reason. - Statistics are exponential moving, not a stored mini-batch: training is pure online SGD, one update per sample, so there is no batch to average over. BatchNormWindow is an EMA window length. - gamma/beta are excluded from weight decay, deliberately - decaying gamma toward zero is the exact pathology being fixed. - The layer self-sizes from whatever sits below it, because a conv/pool stage's output width is derived inside the CNet constructor and is not knowable to the topology builder. - Checkpoint capture/restore/blend carry gamma/beta and the running statistics alongside the dense matrix, so the plateau ladder cannot restore a mismatched pair. - SeedOutputLayerBias accepted only an exact defNeuronBaseOCL as the weight-carrying penultimate layer; with normalization enabled that is the batch-norm layer, so the cold-start bias seed would have silently stopped being applied. - Refuses to build, loudly, if a topology asks for normalization with no compute backend at all - rather than quietly training a different architecture than the one requested. EnableBatchNorm (default on) and BatchNormWindow (1000 samples) are inputs so the effect can be A/B'd without a recompile. Both feed the weights-filename fingerprint, appended conditionally so existing non-BN configs keep their fingerprints and are not forced to retrain. Verified: analytic gradients match finite differences to 1.5e-7 relative over 200 random cases; a faithful port of the full forward/backward chain collapses to the 33.3% floor by era 4 without this layer and holds 36-43% with it. Compiles 0 errors, 0 warnings. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-29 12:34:29 -04:00
return false;
perf: batch norm stops re-reading buffers it already has Batch norm computes host-side while its neighbours are device-resident, so every value it touches crosses the bus - and the cost is the BLOCKING SYNC per crossing, not the bytes. Per sample it did four reads, two of which were exact duplicates: feedForward -> previous layer's Output calcInputGradients -> own Gradient, and the previous layer's Output AGAIN update/accumulate -> own Gradient AGAIN Nothing writes the previous layer's Output between the forward and backward passes, and nothing writes this layer's Gradient between the gradient pass and the weight-update pass - backPropOCL runs those as two separate top-to-bottom loops and only the first writes gradients. So the repeats are removable and the result is bit-exact: the same values, read once instead of twice. Each cache is armed by its producer and disarmed by feedForward, so a consumer whose producer did not run this sample falls back to reading the buffer rather than using the previous sample's data. That is not hypothetical - a batch norm at layer 1 never gets calcInputGradients called at all, the same asymmetry backPropOCL already documents for a layer-1 LSTM, so its gradient cache is never armed and it takes the fallback every time. Both build variants compile 0 errors, 0 warnings. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-09 17:16:08 -04:00
//--- Armed for the weight-update pass, which reads this same Gradient again - see the declaration.
if(ArrayCopy(m_gradCache, grad, 0, 0, count) == count)
m_gradCached = true;
//--- The previous layer's Output, from this sample's forward pass rather than a second read of the
//--- same device buffer. Falls back to reading whenever the cache is not armed or does not match.
feat(ai): batch normalization between dense layers The only bounded stage in the entire forward path was the sigmoid classification head - every hidden stage is PRELU. That is a network with no internal scale control, and the failure ordered exactly by depth: on SP500 H1 the shallow perceptron held ~52% balanced accuracy while the deepest topology sat on the 33.3% one-class floor, with the per-bar logit spread decaying monotonically (0.45 -> 0.38 over ~200 eras) until the evidence tilt fell under the class-prior tilt. That is the signature of internal covariate shift, which chapter 6.1 of the reference book is entirely about and which the NeuroNet_DNG engine addresses with a layer this project never had. Two mechanisms make this the right fix rather than more hyperparameter nudging: - it decouples WEIGHT_DECAY from the learned function (van Laarhoven 2017) - with a normalized layer downstream, decay can no longer grind the discriminative signal away, it only rescales the effective learning rate; - it is the precondition for ever running an unbounded logit head here. The 2026-07-27 attempt blew up (IS error 5.6e15) precisely because nothing upstream constrained scale. Implementation notes: - CNeuronBatchNormOCL computes host-side rather than as a fourth copy of a kernel across Network.cl + WarriorCPU.cpp + WarriorDML.cpp. The math is elementwise O(n); this way it behaves identically on all four compute tiers, needs no DLL rebuild, and cannot drift between backends. Same precedent as the softmax+CCE gradient and the per-sample loss weighting, both computed in MQL5 for that reason. - Statistics are exponential moving, not a stored mini-batch: training is pure online SGD, one update per sample, so there is no batch to average over. BatchNormWindow is an EMA window length. - gamma/beta are excluded from weight decay, deliberately - decaying gamma toward zero is the exact pathology being fixed. - The layer self-sizes from whatever sits below it, because a conv/pool stage's output width is derived inside the CNet constructor and is not knowable to the topology builder. - Checkpoint capture/restore/blend carry gamma/beta and the running statistics alongside the dense matrix, so the plateau ladder cannot restore a mismatched pair. - SeedOutputLayerBias accepted only an exact defNeuronBaseOCL as the weight-carrying penultimate layer; with normalization enabled that is the batch-norm layer, so the cold-start bias seed would have silently stopped being applied. - Refuses to build, loudly, if a topology asks for normalization with no compute backend at all - rather than quietly training a different architecture than the one requested. EnableBatchNorm (default on) and BatchNormWindow (1000 samples) are inputs so the effect can be A/B'd without a recompile. Both feed the weights-filename fingerprint, appended conditionally so existing non-BN configs keep their fingerprints and are not forced to retrain. Verified: analytic gradients match finite differences to 1.5e-7 relative over 200 random cases; a faithful port of the full forward/backward chain collapses to the 33.3% floor by era 4 without this layer and holds 36-43% with it. Compiles 0 errors, 0 warnings. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-29 12:34:29 -04:00
double prevOut[];
perf: batch norm stops re-reading buffers it already has Batch norm computes host-side while its neighbours are device-resident, so every value it touches crosses the bus - and the cost is the BLOCKING SYNC per crossing, not the bytes. Per sample it did four reads, two of which were exact duplicates: feedForward -> previous layer's Output calcInputGradients -> own Gradient, and the previous layer's Output AGAIN update/accumulate -> own Gradient AGAIN Nothing writes the previous layer's Output between the forward and backward passes, and nothing writes this layer's Gradient between the gradient pass and the weight-update pass - backPropOCL runs those as two separate top-to-bottom loops and only the first writes gradients. So the repeats are removable and the result is bit-exact: the same values, read once instead of twice. Each cache is armed by its producer and disarmed by feedForward, so a consumer whose producer did not run this sample falls back to reading the buffer rather than using the previous sample's data. That is not hypothetical - a batch norm at layer 1 never gets calcInputGradients called at all, the same asymmetry backPropOCL already documents for a layer-1 LSTM, so its gradient cache is never armed and it takes the fallback every time. Both build variants compile 0 errors, 0 warnings. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-09 17:16:08 -04:00
if(m_fwdInputCached && ArraySize(m_fwdInputCache) >= inputsCount)
{
if(ArrayCopy(prevOut, m_fwdInputCache, 0, 0, inputsCount) != inputsCount)
return false;
}
else
if(NeuronOCL.getOutputVal(prevOut) < inputsCount)
return false;
feat(ai): batch normalization between dense layers The only bounded stage in the entire forward path was the sigmoid classification head - every hidden stage is PRELU. That is a network with no internal scale control, and the failure ordered exactly by depth: on SP500 H1 the shallow perceptron held ~52% balanced accuracy while the deepest topology sat on the 33.3% one-class floor, with the per-bar logit spread decaying monotonically (0.45 -> 0.38 over ~200 eras) until the evidence tilt fell under the class-prior tilt. That is the signature of internal covariate shift, which chapter 6.1 of the reference book is entirely about and which the NeuroNet_DNG engine addresses with a layer this project never had. Two mechanisms make this the right fix rather than more hyperparameter nudging: - it decouples WEIGHT_DECAY from the learned function (van Laarhoven 2017) - with a normalized layer downstream, decay can no longer grind the discriminative signal away, it only rescales the effective learning rate; - it is the precondition for ever running an unbounded logit head here. The 2026-07-27 attempt blew up (IS error 5.6e15) precisely because nothing upstream constrained scale. Implementation notes: - CNeuronBatchNormOCL computes host-side rather than as a fourth copy of a kernel across Network.cl + WarriorCPU.cpp + WarriorDML.cpp. The math is elementwise O(n); this way it behaves identically on all four compute tiers, needs no DLL rebuild, and cannot drift between backends. Same precedent as the softmax+CCE gradient and the per-sample loss weighting, both computed in MQL5 for that reason. - Statistics are exponential moving, not a stored mini-batch: training is pure online SGD, one update per sample, so there is no batch to average over. BatchNormWindow is an EMA window length. - gamma/beta are excluded from weight decay, deliberately - decaying gamma toward zero is the exact pathology being fixed. - The layer self-sizes from whatever sits below it, because a conv/pool stage's output width is derived inside the CNet constructor and is not knowable to the topology builder. - Checkpoint capture/restore/blend carry gamma/beta and the running statistics alongside the dense matrix, so the plateau ladder cannot restore a mismatched pair. - SeedOutputLayerBias accepted only an exact defNeuronBaseOCL as the weight-carrying penultimate layer; with normalization enabled that is the batch-norm layer, so the cold-start bias seed would have silently stopped being applied. - Refuses to build, loudly, if a topology asks for normalization with no compute backend at all - rather than quietly training a different architecture than the one requested. EnableBatchNorm (default on) and BatchNormWindow (1000 samples) are inputs so the effect can be A/B'd without a recompile. Both feed the weights-filename fingerprint, appended conditionally so existing non-BN configs keep their fingerprints and are not forced to retrain. Verified: analytic gradients match finite differences to 1.5e-7 relative over 200 random cases; a faithful port of the full forward/backward chain collapses to the 33.3% floor by era 4 without this layer and holds 36-43% with it. Compiles 0 errors, 0 warnings. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-29 12:34:29 -04:00
double ig[];
if(ArrayResize(ig, inputsCount) != inputsCount)
return false;
fix: live trades now use the geometry the gate certifies; perf: BN kernels Three changes, one theme: the trade placed, the trade graded, and the trade computed are now the same trade. 1) GEOMETRY WIRE (correctness, the ranked #1 open issue). The measured barrier pair reached the LABELS only - OpenParams still placed orders at the enum geometry (2*ATR/6*ATR), so the deploy gate certified "reaches 1.62*ATR before 3.33*ATR above break-even" about trades the EA never placed. Published via g_DerivedSlAtrMult/g_DerivedTpAtrMult (ConfidenceBridge, same same-tick contract as the confidence globals, because OpenParams runs on the root signal which has no pointer to the AI filter). Two writers: DeriveBarrierGeometry at era 0, and the .cfg adoption a deployed model takes. Overrides both legs and both Intelligent modes - the certificate is exact or it is nothing. TP is ATR-anchored like the label, NOT risk-relative, so a floor-widened stop cannot reshape the certified target. 2) BATCH NORM RUNS DEVICE-SIDE ON OPENCL. Four kernels in Network.cl - forward, hidden gradient, gamma/beta accumulate, gamma/beta apply - each a line-for-line transcription of the host implementation (NormalizeHost / HiddenGradHost / StepGammaBeta) including every NaN guard, clamp, and the exact moment-write ordering. The host copies remain the runtime for the DLL and pure-MQL5 tiers and the reference the kernels must match. Because this box has no OpenCL platform, the safety story is layered: - shim validation: kernels compiled as C and driven against a fp64 host transcription over NaN-poisoned stats, NaN gamma, over-clamp inputs, the frozen path, both optimizers, 3 batches - ALL PASS, worst normalized diff 0.132 vs tolerance 1.0 - in-situ self-check: each kernel is compared against its host twin ON FIRST USE on the real device (SelfCheckBn*), covering what the shim cannot - arg indices and buffer bindings. Any disagreement resyncs from the good copy, latches all BN kernels off process-wide, and training continues host-side. A transcription bug costs a warning and some speed, never a poisoned .nnw. - sync discipline: BatchOptions is now a CBufferDouble with explicit authority tracking (m_bnDeviceAuthoritative). Checkpoints/saves pull read-only; restores/loads/resets push; a mid-batch handover drains the device gamma/beta accumulator into the host arrays so no sample is lost. 3) SMALL FIXES. Apply-kernel build failure now latches the dispatch path at init (one warning instead of warning + failed Execute). Build tag bumped to win-scoring-gpu-v1 - first tag change since expectancy-stop-v1 despite five binary-changing commits. Both build variants compile 0 errors, 0 warnings. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-09 17:51:40 -04:00
HiddenGradHost(grad, prevOut, NeuronOCL.Activation(), ig, inputsCount);
return NeuronOCL.setGradient(ig);
}
//+------------------------------------------------------------------+
//| The elementwise backward math, factored out of calcInputGradients |
//| (2026-08-09) so the kernel self-check compares against LITERALLY |
//| the code the host path runs, not a second copy that could drift. |
//+------------------------------------------------------------------+
void CNeuronBatchNormOCL::HiddenGradHost(const double &grad[], const double &prevOut[],
ENUM_ACTIVATION act, double &ig[], int n)
{
for(int i = 0; i < n; i++)
feat(ai): batch normalization between dense layers The only bounded stage in the entire forward path was the sigmoid classification head - every hidden stage is PRELU. That is a network with no internal scale control, and the failure ordered exactly by depth: on SP500 H1 the shallow perceptron held ~52% balanced accuracy while the deepest topology sat on the 33.3% one-class floor, with the per-bar logit spread decaying monotonically (0.45 -> 0.38 over ~200 eras) until the evidence tilt fell under the class-prior tilt. That is the signature of internal covariate shift, which chapter 6.1 of the reference book is entirely about and which the NeuroNet_DNG engine addresses with a layer this project never had. Two mechanisms make this the right fix rather than more hyperparameter nudging: - it decouples WEIGHT_DECAY from the learned function (van Laarhoven 2017) - with a normalized layer downstream, decay can no longer grind the discriminative signal away, it only rescales the effective learning rate; - it is the precondition for ever running an unbounded logit head here. The 2026-07-27 attempt blew up (IS error 5.6e15) precisely because nothing upstream constrained scale. Implementation notes: - CNeuronBatchNormOCL computes host-side rather than as a fourth copy of a kernel across Network.cl + WarriorCPU.cpp + WarriorDML.cpp. The math is elementwise O(n); this way it behaves identically on all four compute tiers, needs no DLL rebuild, and cannot drift between backends. Same precedent as the softmax+CCE gradient and the per-sample loss weighting, both computed in MQL5 for that reason. - Statistics are exponential moving, not a stored mini-batch: training is pure online SGD, one update per sample, so there is no batch to average over. BatchNormWindow is an EMA window length. - gamma/beta are excluded from weight decay, deliberately - decaying gamma toward zero is the exact pathology being fixed. - The layer self-sizes from whatever sits below it, because a conv/pool stage's output width is derived inside the CNet constructor and is not knowable to the topology builder. - Checkpoint capture/restore/blend carry gamma/beta and the running statistics alongside the dense matrix, so the plateau ladder cannot restore a mismatched pair. - SeedOutputLayerBias accepted only an exact defNeuronBaseOCL as the weight-carrying penultimate layer; with normalization enabled that is the batch-norm layer, so the cold-start bias seed would have silently stopped being applied. - Refuses to build, loudly, if a topology asks for normalization with no compute backend at all - rather than quietly training a different architecture than the one requested. EnableBatchNorm (default on) and BatchNormWindow (1000 samples) are inputs so the effect can be A/B'd without a recompile. Both feed the weights-filename fingerprint, appended conditionally so existing non-BN configs keep their fingerprints and are not forced to retrain. Verified: analytic gradients match finite differences to 1.5e-7 relative over 200 random cases; a faithful port of the full forward/backward chain collapses to the 33.3% floor by era 4 without this layer and holds 36-43% with it. Compiles 0 errors, 0 warnings. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-29 12:34:29 -04:00
{
int shift = i * BN_OPT_STRIDE;
double g = grad[i];
if(iBatchSize > 1)
{
refactor(comments): stdlib comment style across the remaining in-scope files Same pass as 0b06f8e, applied file by file: comment runs of 4+ lines compressed to their leading topic sentences, capped at 4 lines, whole sentences only. Warning sentences (NEVER / MUST / trap / would-have) survive the budget. Every file was checked the same way before committing: the list of non-comment lines is byte-identical to HEAD, and braces balance. No code was touched. Panel/, Enumerations/ and the already-terse System headers needed little or nothing - PooledGate, TradeChecks, BinomialStats and Random came through with no blocks over the threshold at all. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-22 00:25:52 -04:00
//--- THE CLAMP'S OWN DERIVATIVE. If the forward pass bound this unit at +-BN_MAX_NX then
//--- the output stopped depending on the input there, so d(nx)/dx is 0 and no gradient may
//--- pass.
fix(batchnorm): bound the normalized value - a constant input feature was amplified 1e4x and pinned PAI's head to its rails BN_MIN_STD = 1e-4 caps the per-unit gain at 1/1e-4 = 1e4, and the comment above it states that as though it were a safety property. It is not. A unit whose running variance is ~0 is a CONSTANT feature carrying no information, and dividing its rounding noise by 1e-4 hands the next layer an activation of several hundred. BN's contract is "output has ~unit variance"; a unit that cannot supply that must contribute nothing, not the largest signal in the layer. MEASURED, 2026-08-17 SP500 H4, four topologies on identical separate charts: model spread Neutral CHOSE Neutral TIED rail CONV 0.386 0.68% 0.10% 0.48% LSTM 0.392 0.63% 0.00% 0.00% HYB 0.376 1.79% 0.00% 0.01% PAI 0.192 0.09% 80.63% 99.99% bn1's cached nx normed 1.38e4 over 800 units. PAI's SIGMOID head was on its rails on 99.99% of bars, with Buy and Sell landing on the SAME rail so they compared exactly equal, and ApplyClassificationSoftmax()'s strict-majority rule reported that tie as Neutral on ~80% of bars. So the long-running "PAI is heavily biased toward Neutral" was never a class-prior problem: the net CHOSE Neutral on 0.09% of bars. It was float equality on a saturated head. The 331ab29 counters answered it on their first run. PAI-only because it is the one topology whose FIRST batch norm sits on the raw 800-dim input vector - CONV/LSTM/CONVLSTM all have a conv or LSTM stage in front, so their first BN sees a learned representation with no degenerate units. That asymmetry was already on file as a suspicion; this is the mechanism. FIX, mirrored in both backends (host NeuronBatchNorm.mqh and device Network.cl): forward nx = clamp(delta/sd, -BN_MAX_NX, +BN_MAX_NX), BN_MAX_NX = 8 backward if the forward bound this unit, the output stopped depending on the input, so d(nx)/dx = 0 and NO gradient passes The backward half is not optional. g is divided by the same sd the forward multiplies by, so a degenerate unit gets its GRADIENT amplified 1e4x too - the "receives gradients divided by sqrt(var) ~ 500" pathology already noted in Network.cl's Adam kernel. Bounding only the forward would move the explosion downstream. 8 sigma is inert on anything healthy (|nx| > 8 is a ~1e-15 event under normality); it binds only on degenerate units, which is the entire point. Same clamp-to-range idiom the activation derivatives beside it already use. SelfCheckBnForward/SelfCheckBnHiddenGrad already prove host against kernel, and BN_OPT_NX was already consumed in the backward for the gamma gradient, so the new read adds no lifetime assumption. Expect PAI to change behaviour and CONV/LSTM/CONVLSTM not to (their rail rate is ~0%, so the clamp never binds). No .nnw format or fingerprint change. ALSO: print the zero-skill reference on the era line. m_oosWinLongTotal and m_oosWinShortTotal have been accumulated for a long time and NEVER printed, which is why three separate topologies all sitting at 62% read as a mysterious coincidence rather than the obvious base rate. It is not a coincidence: with the target (1.70 ATR) nearer than the stop (3.07 ATR), BOTH sides win on 24.5% of bars, so winLong+winShort covers ~124% of them and a no-edge caller collects 62.1% whichever way it calls - against a 64.3% break-even. Derived from this run's own label counts: (11329 - 2776 + 2*2776) / (2*11350) = 62.14%. Every win rate on that line must be read against this, not against 50%. NOT COMPILED - user compiles. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-17 00:21:51 -04:00
if(MathAbs(BatchOptions.At(shift + BN_OPT_NX)) >= BN_MAX_NX)
g = 0.0;
else
{
double sd = MathMax(MathSqrt(BatchOptions.At(shift + BN_OPT_VAR) + BN_EPSILON), BN_MIN_STD);
g = g * BatchOptions.At(shift + BN_OPT_GAMMA) / sd;
}
feat(ai): batch normalization between dense layers The only bounded stage in the entire forward path was the sigmoid classification head - every hidden stage is PRELU. That is a network with no internal scale control, and the failure ordered exactly by depth: on SP500 H1 the shallow perceptron held ~52% balanced accuracy while the deepest topology sat on the 33.3% one-class floor, with the per-bar logit spread decaying monotonically (0.45 -> 0.38 over ~200 eras) until the evidence tilt fell under the class-prior tilt. That is the signature of internal covariate shift, which chapter 6.1 of the reference book is entirely about and which the NeuroNet_DNG engine addresses with a layer this project never had. Two mechanisms make this the right fix rather than more hyperparameter nudging: - it decouples WEIGHT_DECAY from the learned function (van Laarhoven 2017) - with a normalized layer downstream, decay can no longer grind the discriminative signal away, it only rescales the effective learning rate; - it is the precondition for ever running an unbounded logit head here. The 2026-07-27 attempt blew up (IS error 5.6e15) precisely because nothing upstream constrained scale. Implementation notes: - CNeuronBatchNormOCL computes host-side rather than as a fourth copy of a kernel across Network.cl + WarriorCPU.cpp + WarriorDML.cpp. The math is elementwise O(n); this way it behaves identically on all four compute tiers, needs no DLL rebuild, and cannot drift between backends. Same precedent as the softmax+CCE gradient and the per-sample loss weighting, both computed in MQL5 for that reason. - Statistics are exponential moving, not a stored mini-batch: training is pure online SGD, one update per sample, so there is no batch to average over. BatchNormWindow is an EMA window length. - gamma/beta are excluded from weight decay, deliberately - decaying gamma toward zero is the exact pathology being fixed. - The layer self-sizes from whatever sits below it, because a conv/pool stage's output width is derived inside the CNet constructor and is not knowable to the topology builder. - Checkpoint capture/restore/blend carry gamma/beta and the running statistics alongside the dense matrix, so the plateau ladder cannot restore a mismatched pair. - SeedOutputLayerBias accepted only an exact defNeuronBaseOCL as the weight-carrying penultimate layer; with normalization enabled that is the batch-norm layer, so the cold-start bias seed would have silently stopped being applied. - Refuses to build, loudly, if a topology asks for normalization with no compute backend at all - rather than quietly training a different architecture than the one requested. EnableBatchNorm (default on) and BatchNormWindow (1000 samples) are inputs so the effect can be A/B'd without a recompile. Both feed the weights-filename fingerprint, appended conditionally so existing non-BN configs keep their fingerprints and are not forced to retrain. Verified: analytic gradients match finite differences to 1.5e-7 relative over 200 random cases; a faithful port of the full forward/backward chain collapses to the 33.3% floor by era 4 without this layer and holds 36-43% with it. Compiles 0 errors, 0 warnings. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-29 12:34:29 -04:00
}
//--- Then the previous layer's own activation derivative, byte-for-byte the same treatment
//--- Network.cl's CaclHiddenGradient applies - including the clamp-to-range "implied target"
//--- reformulation - so that from the previous layer's point of view a batch-norm layer is
//--- indistinguishable from any other. NONE falls through unscaled.
double out = prevOut[i];
fix: live trades now use the geometry the gate certifies; perf: BN kernels Three changes, one theme: the trade placed, the trade graded, and the trade computed are now the same trade. 1) GEOMETRY WIRE (correctness, the ranked #1 open issue). The measured barrier pair reached the LABELS only - OpenParams still placed orders at the enum geometry (2*ATR/6*ATR), so the deploy gate certified "reaches 1.62*ATR before 3.33*ATR above break-even" about trades the EA never placed. Published via g_DerivedSlAtrMult/g_DerivedTpAtrMult (ConfidenceBridge, same same-tick contract as the confidence globals, because OpenParams runs on the root signal which has no pointer to the AI filter). Two writers: DeriveBarrierGeometry at era 0, and the .cfg adoption a deployed model takes. Overrides both legs and both Intelligent modes - the certificate is exact or it is nothing. TP is ATR-anchored like the label, NOT risk-relative, so a floor-widened stop cannot reshape the certified target. 2) BATCH NORM RUNS DEVICE-SIDE ON OPENCL. Four kernels in Network.cl - forward, hidden gradient, gamma/beta accumulate, gamma/beta apply - each a line-for-line transcription of the host implementation (NormalizeHost / HiddenGradHost / StepGammaBeta) including every NaN guard, clamp, and the exact moment-write ordering. The host copies remain the runtime for the DLL and pure-MQL5 tiers and the reference the kernels must match. Because this box has no OpenCL platform, the safety story is layered: - shim validation: kernels compiled as C and driven against a fp64 host transcription over NaN-poisoned stats, NaN gamma, over-clamp inputs, the frozen path, both optimizers, 3 batches - ALL PASS, worst normalized diff 0.132 vs tolerance 1.0 - in-situ self-check: each kernel is compared against its host twin ON FIRST USE on the real device (SelfCheckBn*), covering what the shim cannot - arg indices and buffer bindings. Any disagreement resyncs from the good copy, latches all BN kernels off process-wide, and training continues host-side. A transcription bug costs a warning and some speed, never a poisoned .nnw. - sync discipline: BatchOptions is now a CBufferDouble with explicit authority tracking (m_bnDeviceAuthoritative). Checkpoints/saves pull read-only; restores/loads/resets push; a mid-batch handover drains the device gamma/beta accumulator into the host arrays so no sample is lost. 3) SMALL FIXES. Apply-kernel build failure now latches the dispatch path at init (one warning instead of warning + failed Execute). Build tag bumped to win-scoring-gpu-v1 - first tag change since expectancy-stop-v1 despite five binary-changing commits. Both build variants compile 0 errors, 0 warnings. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-09 17:51:40 -04:00
switch(act)
feat(ai): batch normalization between dense layers The only bounded stage in the entire forward path was the sigmoid classification head - every hidden stage is PRELU. That is a network with no internal scale control, and the failure ordered exactly by depth: on SP500 H1 the shallow perceptron held ~52% balanced accuracy while the deepest topology sat on the 33.3% one-class floor, with the per-bar logit spread decaying monotonically (0.45 -> 0.38 over ~200 eras) until the evidence tilt fell under the class-prior tilt. That is the signature of internal covariate shift, which chapter 6.1 of the reference book is entirely about and which the NeuroNet_DNG engine addresses with a layer this project never had. Two mechanisms make this the right fix rather than more hyperparameter nudging: - it decouples WEIGHT_DECAY from the learned function (van Laarhoven 2017) - with a normalized layer downstream, decay can no longer grind the discriminative signal away, it only rescales the effective learning rate; - it is the precondition for ever running an unbounded logit head here. The 2026-07-27 attempt blew up (IS error 5.6e15) precisely because nothing upstream constrained scale. Implementation notes: - CNeuronBatchNormOCL computes host-side rather than as a fourth copy of a kernel across Network.cl + WarriorCPU.cpp + WarriorDML.cpp. The math is elementwise O(n); this way it behaves identically on all four compute tiers, needs no DLL rebuild, and cannot drift between backends. Same precedent as the softmax+CCE gradient and the per-sample loss weighting, both computed in MQL5 for that reason. - Statistics are exponential moving, not a stored mini-batch: training is pure online SGD, one update per sample, so there is no batch to average over. BatchNormWindow is an EMA window length. - gamma/beta are excluded from weight decay, deliberately - decaying gamma toward zero is the exact pathology being fixed. - The layer self-sizes from whatever sits below it, because a conv/pool stage's output width is derived inside the CNet constructor and is not knowable to the topology builder. - Checkpoint capture/restore/blend carry gamma/beta and the running statistics alongside the dense matrix, so the plateau ladder cannot restore a mismatched pair. - SeedOutputLayerBias accepted only an exact defNeuronBaseOCL as the weight-carrying penultimate layer; with normalization enabled that is the batch-norm layer, so the cold-start bias seed would have silently stopped being applied. - Refuses to build, loudly, if a topology asks for normalization with no compute backend at all - rather than quietly training a different architecture than the one requested. EnableBatchNorm (default on) and BatchNormWindow (1000 samples) are inputs so the effect can be A/B'd without a recompile. Both feed the weights-filename fingerprint, appended conditionally so existing non-BN configs keep their fingerprints and are not forced to retrain. Verified: analytic gradients match finite differences to 1.5e-7 relative over 200 random cases; a faithful port of the full forward/backward chain collapses to the 33.3% floor by era 4 without this layer and holds 36-43% with it. Compiles 0 errors, 0 warnings. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-29 12:34:29 -04:00
{
case TANH:
g = MathMax(-1.0, MathMin(1.0, g + out)) - out;
g = g * MathMax(MIN_ACTIVATION_DERIVATIVE, 1.0 - out * out);
break;
case SIGMOID:
g = MathMax(0.0, MathMin(1.0, g + out)) - out;
g = g * MathMax(MIN_ACTIVATION_DERIVATIVE, out * (1.0 - out));
break;
case PRELU:
g = g * (out >= 0 ? 1.0 : 0.01);
break;
default:
break;
}
ig[i] = g;
}
}
//+------------------------------------------------------------------+
refactor(comments): box headers to stdlib length The //| box blocks were excluded from 0b06f8e and 5efdb48 and were what remained: 160 of them ran to 10+ lines, the longest to 88. Compressed to their leading topic sentences - 5 lines for a function header, 8 for a file header - keeping the box format and the standard MQL5 name/author lines verbatim. Verified at the BYTE level this time, across every in-scope file: the list of non-comment lines is byte-identical to HEAD and braces balance. The first check compared a locale-decoded 'git show' against a UTF-8 read and flagged 25 files that had not changed at all - every BOM and every non-ASCII line mismatched. 47,696 -> 40,665 lines in scope; comment share 38% -> 26%. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-22 00:30:14 -04:00
//| gamma/beta update. NeuronOCL is the previous layer and is |
//| deliberately unused: batch norm consumes its output elementwise, |
//| so there is no incoming weight matrix to update (the CNet |
//| constructor gives the previous layer 0 outgoing weights when |
//| this layer follows it). |
feat(ai): batch normalization between dense layers The only bounded stage in the entire forward path was the sigmoid classification head - every hidden stage is PRELU. That is a network with no internal scale control, and the failure ordered exactly by depth: on SP500 H1 the shallow perceptron held ~52% balanced accuracy while the deepest topology sat on the 33.3% one-class floor, with the per-bar logit spread decaying monotonically (0.45 -> 0.38 over ~200 eras) until the evidence tilt fell under the class-prior tilt. That is the signature of internal covariate shift, which chapter 6.1 of the reference book is entirely about and which the NeuroNet_DNG engine addresses with a layer this project never had. Two mechanisms make this the right fix rather than more hyperparameter nudging: - it decouples WEIGHT_DECAY from the learned function (van Laarhoven 2017) - with a normalized layer downstream, decay can no longer grind the discriminative signal away, it only rescales the effective learning rate; - it is the precondition for ever running an unbounded logit head here. The 2026-07-27 attempt blew up (IS error 5.6e15) precisely because nothing upstream constrained scale. Implementation notes: - CNeuronBatchNormOCL computes host-side rather than as a fourth copy of a kernel across Network.cl + WarriorCPU.cpp + WarriorDML.cpp. The math is elementwise O(n); this way it behaves identically on all four compute tiers, needs no DLL rebuild, and cannot drift between backends. Same precedent as the softmax+CCE gradient and the per-sample loss weighting, both computed in MQL5 for that reason. - Statistics are exponential moving, not a stored mini-batch: training is pure online SGD, one update per sample, so there is no batch to average over. BatchNormWindow is an EMA window length. - gamma/beta are excluded from weight decay, deliberately - decaying gamma toward zero is the exact pathology being fixed. - The layer self-sizes from whatever sits below it, because a conv/pool stage's output width is derived inside the CNet constructor and is not knowable to the topology builder. - Checkpoint capture/restore/blend carry gamma/beta and the running statistics alongside the dense matrix, so the plateau ladder cannot restore a mismatched pair. - SeedOutputLayerBias accepted only an exact defNeuronBaseOCL as the weight-carrying penultimate layer; with normalization enabled that is the batch-norm layer, so the cold-start bias seed would have silently stopped being applied. - Refuses to build, loudly, if a topology asks for normalization with no compute backend at all - rather than quietly training a different architecture than the one requested. EnableBatchNorm (default on) and BatchNormWindow (1000 samples) are inputs so the effect can be A/B'd without a recompile. Both feed the weights-filename fingerprint, appended conditionally so existing non-BN configs keep their fingerprints and are not forced to retrain. Verified: analytic gradients match finite differences to 1.5e-7 relative over 200 random cases; a faithful port of the full forward/backward chain collapses to the 33.3% floor by era 4 without this layer and holds 36-43% with it. Compiles 0 errors, 0 warnings. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-29 12:34:29 -04:00
//+------------------------------------------------------------------+
bool CNeuronBatchNormOCL::updateInputWeights(CNeuronBaseOCL *NeuronOCL)
{
if(CheckPointer(BatchOptions) == POINTER_INVALID)
return false;
if(iBatchSize <= 1)
return true; // normalization off - gamma/beta are not in the graph
int units = Neurons();
refactor(comments): stdlib comment style across the remaining in-scope files Same pass as 0b06f8e, applied file by file: comment runs of 4+ lines compressed to their leading topic sentences, capped at 4 lines, whole sentences only. Warning sentences (NEVER / MUST / trap / would-have) survive the budget. Every file was checked the same way before committing: the list of non-comment lines is byte-identical to HEAD, and braces balance. No code was touched. Panel/, Enumerations/ and the already-terse System headers needed little or nothing - PooledGate, TradeChecks, BinomialStats and Random came through with no blocks over the threshold at all. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-22 00:25:52 -04:00
//--- This per-sample path stays HOST-side even in kernel mode, deliberately: it only runs when
//--- the net's train batch is 1 (online learning on a deployed model - once per confirmed bar,
//--- not per training sample), so kernelizing it buys nothing, and the host step is the
//--- reference implementation.
fix: live trades now use the geometry the gate certifies; perf: BN kernels Three changes, one theme: the trade placed, the trade graded, and the trade computed are now the same trade. 1) GEOMETRY WIRE (correctness, the ranked #1 open issue). The measured barrier pair reached the LABELS only - OpenParams still placed orders at the enum geometry (2*ATR/6*ATR), so the deploy gate certified "reaches 1.62*ATR before 3.33*ATR above break-even" about trades the EA never placed. Published via g_DerivedSlAtrMult/g_DerivedTpAtrMult (ConfidenceBridge, same same-tick contract as the confidence globals, because OpenParams runs on the root signal which has no pointer to the AI filter). Two writers: DeriveBarrierGeometry at era 0, and the .cfg adoption a deployed model takes. Overrides both legs and both Intelligent modes - the certificate is exact or it is nothing. TP is ATR-anchored like the label, NOT risk-relative, so a floor-widened stop cannot reshape the certified target. 2) BATCH NORM RUNS DEVICE-SIDE ON OPENCL. Four kernels in Network.cl - forward, hidden gradient, gamma/beta accumulate, gamma/beta apply - each a line-for-line transcription of the host implementation (NormalizeHost / HiddenGradHost / StepGammaBeta) including every NaN guard, clamp, and the exact moment-write ordering. The host copies remain the runtime for the DLL and pure-MQL5 tiers and the reference the kernels must match. Because this box has no OpenCL platform, the safety story is layered: - shim validation: kernels compiled as C and driven against a fp64 host transcription over NaN-poisoned stats, NaN gamma, over-clamp inputs, the frozen path, both optimizers, 3 batches - ALL PASS, worst normalized diff 0.132 vs tolerance 1.0 - in-situ self-check: each kernel is compared against its host twin ON FIRST USE on the real device (SelfCheckBn*), covering what the shim cannot - arg indices and buffer bindings. Any disagreement resyncs from the good copy, latches all BN kernels off process-wide, and training continues host-side. A transcription bug costs a warning and some speed, never a poisoned .nnw. - sync discipline: BatchOptions is now a CBufferDouble with explicit authority tracking (m_bnDeviceAuthoritative). Checkpoints/saves pull read-only; restores/loads/resets push; a mid-batch handover drains the device gamma/beta accumulator into the host arrays so no sample is lost. 3) SMALL FIXES. Apply-kernel build failure now latches the dispatch path at init (one warning instead of warning + failed Execute). Build tag bumped to win-scoring-gpu-v1 - first tag change since expectancy-stop-v1 despite five binary-changing commits. Both build variants compile 0 errors, 0 warnings. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-09 17:51:40 -04:00
EnsureHostAuthoritative();
perf: batch norm stops re-reading buffers it already has Batch norm computes host-side while its neighbours are device-resident, so every value it touches crosses the bus - and the cost is the BLOCKING SYNC per crossing, not the bytes. Per sample it did four reads, two of which were exact duplicates: feedForward -> previous layer's Output calcInputGradients -> own Gradient, and the previous layer's Output AGAIN update/accumulate -> own Gradient AGAIN Nothing writes the previous layer's Output between the forward and backward passes, and nothing writes this layer's Gradient between the gradient pass and the weight-update pass - backPropOCL runs those as two separate top-to-bottom loops and only the first writes gradients. So the repeats are removable and the result is bit-exact: the same values, read once instead of twice. Each cache is armed by its producer and disarmed by feedForward, so a consumer whose producer did not run this sample falls back to reading the buffer rather than using the previous sample's data. That is not hypothetical - a batch norm at layer 1 never gets calcInputGradients called at all, the same asymmetry backPropOCL already documents for a layer-1 LSTM, so its gradient cache is never armed and it takes the fallback every time. Both build variants compile 0 errors, 0 warnings. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-09 17:16:08 -04:00
//--- Gradient from calcInputGradients' read this same sample, not a second device round trip.
double grad[];
int count = m_gradCached ? ArrayCopy(grad, m_gradCache) : getGradient(grad);
feat(ai): batch normalization between dense layers The only bounded stage in the entire forward path was the sigmoid classification head - every hidden stage is PRELU. That is a network with no internal scale control, and the failure ordered exactly by depth: on SP500 H1 the shallow perceptron held ~52% balanced accuracy while the deepest topology sat on the 33.3% one-class floor, with the per-bar logit spread decaying monotonically (0.45 -> 0.38 over ~200 eras) until the evidence tilt fell under the class-prior tilt. That is the signature of internal covariate shift, which chapter 6.1 of the reference book is entirely about and which the NeuroNet_DNG engine addresses with a layer this project never had. Two mechanisms make this the right fix rather than more hyperparameter nudging: - it decouples WEIGHT_DECAY from the learned function (van Laarhoven 2017) - with a normalized layer downstream, decay can no longer grind the discriminative signal away, it only rescales the effective learning rate; - it is the precondition for ever running an unbounded logit head here. The 2026-07-27 attempt blew up (IS error 5.6e15) precisely because nothing upstream constrained scale. Implementation notes: - CNeuronBatchNormOCL computes host-side rather than as a fourth copy of a kernel across Network.cl + WarriorCPU.cpp + WarriorDML.cpp. The math is elementwise O(n); this way it behaves identically on all four compute tiers, needs no DLL rebuild, and cannot drift between backends. Same precedent as the softmax+CCE gradient and the per-sample loss weighting, both computed in MQL5 for that reason. - Statistics are exponential moving, not a stored mini-batch: training is pure online SGD, one update per sample, so there is no batch to average over. BatchNormWindow is an EMA window length. - gamma/beta are excluded from weight decay, deliberately - decaying gamma toward zero is the exact pathology being fixed. - The layer self-sizes from whatever sits below it, because a conv/pool stage's output width is derived inside the CNet constructor and is not knowable to the topology builder. - Checkpoint capture/restore/blend carry gamma/beta and the running statistics alongside the dense matrix, so the plateau ladder cannot restore a mismatched pair. - SeedOutputLayerBias accepted only an exact defNeuronBaseOCL as the weight-carrying penultimate layer; with normalization enabled that is the batch-norm layer, so the cold-start bias seed would have silently stopped being applied. - Refuses to build, loudly, if a topology asks for normalization with no compute backend at all - rather than quietly training a different architecture than the one requested. EnableBatchNorm (default on) and BatchNormWindow (1000 samples) are inputs so the effect can be A/B'd without a recompile. Both feed the weights-filename fingerprint, appended conditionally so existing non-BN configs keep their fingerprints and are not forced to retrain. Verified: analytic gradients match finite differences to 1.5e-7 relative over 200 random cases; a faithful port of the full forward/backward chain collapses to the 33.3% floor by era 4 without this layer and holds 36-43% with it. Compiles 0 errors, 0 warnings. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-29 12:34:29 -04:00
if(count < units || units <= 0)
return false;
//--- Same bias-corrected step size the dense Adam path computes, so gamma/beta move on the same
//--- schedule as the weights around them.
fix(build): g_eta - the learning rate global no longer shadows a stdlib local MetaEditor: "declaration of 'eta' hides global variable" (Math.mqh:792 vs Network.mqh:80). The standard library's Math\Stat\Math.mqh declares a local `double eta` in its incomplete-gamma branch, and our bare global of the same name is in scope there. Same fault as the b1/b2/lr/momentum macros retired in ea2552e: a single-token global name living in a header that library code gets compiled beside. The library cannot move, so ours does - 112 references across 13 files, whole-word only. Named g_eta rather than g_learningRate to stay inside the vocabulary already around it (ETA_DECAY_FACTOR, ETA_MIN, m_etaCeiling, etaBefore), all of which are untouched and none of which shadow anything. One log line said "continuing to explore without decaying eta", where the word was prose rather than a symbol reference; that reads "the learning rate" now instead of naming a variable at the trader. Scanned for the next occurrence rather than waiting for it: the only other bare lowercase globals in the tree are eaName and tableschema, both distinctive enough not to collide with a library local. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-20 07:00:09 -04:00
double lt = (optimization == SGD) ? 0.0 : g_eta * sqrt(1 - pow(AdamBeta2, t)) / (1 - pow(AdamBeta1, t));
feat(ai): batch normalization between dense layers The only bounded stage in the entire forward path was the sigmoid classification head - every hidden stage is PRELU. That is a network with no internal scale control, and the failure ordered exactly by depth: on SP500 H1 the shallow perceptron held ~52% balanced accuracy while the deepest topology sat on the 33.3% one-class floor, with the per-bar logit spread decaying monotonically (0.45 -> 0.38 over ~200 eras) until the evidence tilt fell under the class-prior tilt. That is the signature of internal covariate shift, which chapter 6.1 of the reference book is entirely about and which the NeuroNet_DNG engine addresses with a layer this project never had. Two mechanisms make this the right fix rather than more hyperparameter nudging: - it decouples WEIGHT_DECAY from the learned function (van Laarhoven 2017) - with a normalized layer downstream, decay can no longer grind the discriminative signal away, it only rescales the effective learning rate; - it is the precondition for ever running an unbounded logit head here. The 2026-07-27 attempt blew up (IS error 5.6e15) precisely because nothing upstream constrained scale. Implementation notes: - CNeuronBatchNormOCL computes host-side rather than as a fourth copy of a kernel across Network.cl + WarriorCPU.cpp + WarriorDML.cpp. The math is elementwise O(n); this way it behaves identically on all four compute tiers, needs no DLL rebuild, and cannot drift between backends. Same precedent as the softmax+CCE gradient and the per-sample loss weighting, both computed in MQL5 for that reason. - Statistics are exponential moving, not a stored mini-batch: training is pure online SGD, one update per sample, so there is no batch to average over. BatchNormWindow is an EMA window length. - gamma/beta are excluded from weight decay, deliberately - decaying gamma toward zero is the exact pathology being fixed. - The layer self-sizes from whatever sits below it, because a conv/pool stage's output width is derived inside the CNet constructor and is not knowable to the topology builder. - Checkpoint capture/restore/blend carry gamma/beta and the running statistics alongside the dense matrix, so the plateau ladder cannot restore a mismatched pair. - SeedOutputLayerBias accepted only an exact defNeuronBaseOCL as the weight-carrying penultimate layer; with normalization enabled that is the batch-norm layer, so the cold-start bias seed would have silently stopped being applied. - Refuses to build, loudly, if a topology asks for normalization with no compute backend at all - rather than quietly training a different architecture than the one requested. EnableBatchNorm (default on) and BatchNormWindow (1000 samples) are inputs so the effect can be A/B'd without a recompile. Both feed the weights-filename fingerprint, appended conditionally so existing non-BN configs keep their fingerprints and are not forced to retrain. Verified: analytic gradients match finite differences to 1.5e-7 relative over 200 random cases; a faithful port of the full forward/backward chain collapses to the 33.3% floor by era 4 without this layer and holds 36-43% with it. Compiles 0 errors, 0 warnings. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-29 12:34:29 -04:00
for(int i = 0; i < units; i++)
{
int shift = i * BN_OPT_STRIDE;
double g = grad[i];
//--- dL/dgamma = dL/dy * nx | dL/dbeta = dL/dy
double gGamma = g * BatchOptions.At(shift + BN_OPT_NX);
double gBeta = g;
feat: mini-batch gradient accumulation (F4), front-end-aware capacity budget (F6), split Wyckoff categoricals (N1) Completes the 2026-08-09 training audit. FORCES A RETRAIN of every Wyckoff-enabled config (N1 re-keys the fingerprint), and BOTH DLLs must be redeployed alongside the .ex5 - they carry new exports. F4 - mini-batch accumulation, TRAIN_BATCH_SIZE=32. Training was pure online SGD (one weight update per bar), which is the mechanical source of the era-to-era whipsaw every downstream guard was built to cope with. The O(n^2) outer product is native - AccumulateWeightGrad / AccumulateWeightGradConv / AccumulateBufferInto in Network.cl, WarriorCPU and WarriorDML - while the optimizer step is host-side MQL5 shared by all tiers (ApplyAccumToBlock), so there is one Adam/SGD implementation instead of four that can drift. - the LSTM needs no outer-product kernel (WeightsGradient already holds the sample's full dW) but could NOT simply be left un-zeroed between samples: CPU_LSTMSeqBackward/DML_LSTMSeqBackward memset it on entry. Hence a separate accumulator plus an elementwise add. - batch-norm gamma/beta accumulate in host arrays, not new BatchOptions slots - BN_OPT_STRIDE is baked into every persisted .nnw. - scoped to pass 2; online learning keeps immediate updates. Every save / checkpoint / scoring boundary flushes, scaling by the real sample count. - degrades to per-sample updates (one log line) on a tier that cannot accumulate, so old devices and DLL-free builds are unaffected. - verified offline: DirectML/batch_accum_check.cpp drives the real exports against an independent reference; at B=1 the accumulator matches the shipped unbatched kernel's own gradient to 1.1e-16. Math only - the in-situ check remains the per-layer dW/W report on a real era. F6 - ComputeFirstLayerWidth budgeted against the RAW input width even where a conv/LSTM front end had already reduced it, so an LSTM's dense stack was charged for 1,280 inputs when it receives 64. Confirmed from the deployed .cfg files: CONV, LSTM and HYBRID were all pinned at the 16-unit floor. Now budgeted against the front-end output and capped at it (never fan out), with the derivation reordered so both stages settle first. N1 - EventCode/EventPhase/StructuralPhase are signed categoricals packing direction and Wyckoff stage into one scalar across a sign discontinuity. Split into direction + [0,1] magnitude, the same convention the base OHLC block uses. Information-preserving; 13 readings now occupy 16 inputs. Compiled clean (0 errors, 0 warnings); both DLLs rebuilt. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-09 11:48:03 -04:00
if(!StepGammaBeta(shift, gGamma, gBeta, lt))
return false;
}
if(optimization != SGD && t < INT_MAX)
t++;
fix: live trades now use the geometry the gate certifies; perf: BN kernels Three changes, one theme: the trade placed, the trade graded, and the trade computed are now the same trade. 1) GEOMETRY WIRE (correctness, the ranked #1 open issue). The measured barrier pair reached the LABELS only - OpenParams still placed orders at the enum geometry (2*ATR/6*ATR), so the deploy gate certified "reaches 1.62*ATR before 3.33*ATR above break-even" about trades the EA never placed. Published via g_DerivedSlAtrMult/g_DerivedTpAtrMult (ConfidenceBridge, same same-tick contract as the confidence globals, because OpenParams runs on the root signal which has no pointer to the AI filter). Two writers: DeriveBarrierGeometry at era 0, and the .cfg adoption a deployed model takes. Overrides both legs and both Intelligent modes - the certificate is exact or it is nothing. TP is ATR-anchored like the label, NOT risk-relative, so a floor-widened stop cannot reshape the certified target. 2) BATCH NORM RUNS DEVICE-SIDE ON OPENCL. Four kernels in Network.cl - forward, hidden gradient, gamma/beta accumulate, gamma/beta apply - each a line-for-line transcription of the host implementation (NormalizeHost / HiddenGradHost / StepGammaBeta) including every NaN guard, clamp, and the exact moment-write ordering. The host copies remain the runtime for the DLL and pure-MQL5 tiers and the reference the kernels must match. Because this box has no OpenCL platform, the safety story is layered: - shim validation: kernels compiled as C and driven against a fp64 host transcription over NaN-poisoned stats, NaN gamma, over-clamp inputs, the frozen path, both optimizers, 3 batches - ALL PASS, worst normalized diff 0.132 vs tolerance 1.0 - in-situ self-check: each kernel is compared against its host twin ON FIRST USE on the real device (SelfCheckBn*), covering what the shim cannot - arg indices and buffer bindings. Any disagreement resyncs from the good copy, latches all BN kernels off process-wide, and training continues host-side. A transcription bug costs a warning and some speed, never a poisoned .nnw. - sync discipline: BatchOptions is now a CBufferDouble with explicit authority tracking (m_bnDeviceAuthoritative). Checkpoints/saves pull read-only; restores/loads/resets push; a mid-batch handover drains the device gamma/beta accumulator into the host arrays so no sample is lost. 3) SMALL FIXES. Apply-kernel build failure now latches the dispatch path at init (one warning instead of warning + failed Execute). Build tag bumped to win-scoring-gpu-v1 - first tag change since expectancy-stop-v1 despite five binary-changing commits. Both build variants compile 0 errors, 0 warnings. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-09 17:51:40 -04:00
//--- Keep the device copy coherent so a later kernel forward reads the stepped gamma/beta.
if(BatchOptions.GetIndex() >= 0 && !BatchOptions.BufferWrite())
return false;
feat: mini-batch gradient accumulation (F4), front-end-aware capacity budget (F6), split Wyckoff categoricals (N1) Completes the 2026-08-09 training audit. FORCES A RETRAIN of every Wyckoff-enabled config (N1 re-keys the fingerprint), and BOTH DLLs must be redeployed alongside the .ex5 - they carry new exports. F4 - mini-batch accumulation, TRAIN_BATCH_SIZE=32. Training was pure online SGD (one weight update per bar), which is the mechanical source of the era-to-era whipsaw every downstream guard was built to cope with. The O(n^2) outer product is native - AccumulateWeightGrad / AccumulateWeightGradConv / AccumulateBufferInto in Network.cl, WarriorCPU and WarriorDML - while the optimizer step is host-side MQL5 shared by all tiers (ApplyAccumToBlock), so there is one Adam/SGD implementation instead of four that can drift. - the LSTM needs no outer-product kernel (WeightsGradient already holds the sample's full dW) but could NOT simply be left un-zeroed between samples: CPU_LSTMSeqBackward/DML_LSTMSeqBackward memset it on entry. Hence a separate accumulator plus an elementwise add. - batch-norm gamma/beta accumulate in host arrays, not new BatchOptions slots - BN_OPT_STRIDE is baked into every persisted .nnw. - scoped to pass 2; online learning keeps immediate updates. Every save / checkpoint / scoring boundary flushes, scaling by the real sample count. - degrades to per-sample updates (one log line) on a tier that cannot accumulate, so old devices and DLL-free builds are unaffected. - verified offline: DirectML/batch_accum_check.cpp drives the real exports against an independent reference; at B=1 the accumulator matches the shipped unbatched kernel's own gradient to 1.1e-16. Math only - the in-situ check remains the per-layer dW/W report on a real era. F6 - ComputeFirstLayerWidth budgeted against the RAW input width even where a conv/LSTM front end had already reduced it, so an LSTM's dense stack was charged for 1,280 inputs when it receives 64. Confirmed from the deployed .cfg files: CONV, LSTM and HYBRID were all pinned at the 16-unit floor. Now budgeted against the front-end output and capped at it (never fan out), with the derivation reordered so both stages settle first. N1 - EventCode/EventPhase/StructuralPhase are signed categoricals packing direction and Wyckoff stage into one scalar across a sign discontinuity. Split into direction + [0,1] magnitude, the same convention the base OHLC block uses. Information-preserving; 13 readings now occupy 16 inputs. Compiled clean (0 errors, 0 warnings); both DLLs rebuilt. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-09 11:48:03 -04:00
return true;
}
//+------------------------------------------------------------------+
//| ONE unit's gamma/beta optimizer step, factored out of |
//| updateInputWeights so the mini-batch path |
//| (ApplyAccumulatedGradients) takes the identical step on the batch |
//| mean instead of carrying a second copy of this arithmetic. |
//+------------------------------------------------------------------+
bool CNeuronBatchNormOCL::StepGammaBeta(int shift, double gGamma, double gBeta, double lt)
{
{
feat(ai): batch normalization between dense layers The only bounded stage in the entire forward path was the sigmoid classification head - every hidden stage is PRELU. That is a network with no internal scale control, and the failure ordered exactly by depth: on SP500 H1 the shallow perceptron held ~52% balanced accuracy while the deepest topology sat on the 33.3% one-class floor, with the per-bar logit spread decaying monotonically (0.45 -> 0.38 over ~200 eras) until the evidence tilt fell under the class-prior tilt. That is the signature of internal covariate shift, which chapter 6.1 of the reference book is entirely about and which the NeuroNet_DNG engine addresses with a layer this project never had. Two mechanisms make this the right fix rather than more hyperparameter nudging: - it decouples WEIGHT_DECAY from the learned function (van Laarhoven 2017) - with a normalized layer downstream, decay can no longer grind the discriminative signal away, it only rescales the effective learning rate; - it is the precondition for ever running an unbounded logit head here. The 2026-07-27 attempt blew up (IS error 5.6e15) precisely because nothing upstream constrained scale. Implementation notes: - CNeuronBatchNormOCL computes host-side rather than as a fourth copy of a kernel across Network.cl + WarriorCPU.cpp + WarriorDML.cpp. The math is elementwise O(n); this way it behaves identically on all four compute tiers, needs no DLL rebuild, and cannot drift between backends. Same precedent as the softmax+CCE gradient and the per-sample loss weighting, both computed in MQL5 for that reason. - Statistics are exponential moving, not a stored mini-batch: training is pure online SGD, one update per sample, so there is no batch to average over. BatchNormWindow is an EMA window length. - gamma/beta are excluded from weight decay, deliberately - decaying gamma toward zero is the exact pathology being fixed. - The layer self-sizes from whatever sits below it, because a conv/pool stage's output width is derived inside the CNet constructor and is not knowable to the topology builder. - Checkpoint capture/restore/blend carry gamma/beta and the running statistics alongside the dense matrix, so the plateau ladder cannot restore a mismatched pair. - SeedOutputLayerBias accepted only an exact defNeuronBaseOCL as the weight-carrying penultimate layer; with normalization enabled that is the batch-norm layer, so the cold-start bias seed would have silently stopped being applied. - Refuses to build, loudly, if a topology asks for normalization with no compute backend at all - rather than quietly training a different architecture than the one requested. EnableBatchNorm (default on) and BatchNormWindow (1000 samples) are inputs so the effect can be A/B'd without a recompile. Both feed the weights-filename fingerprint, appended conditionally so existing non-BN configs keep their fingerprints and are not forced to retrain. Verified: analytic gradients match finite differences to 1.5e-7 relative over 200 random cases; a faithful port of the full forward/backward chain collapses to the 33.3% floor by era 4 without this layer and holds 36-43% with it. Compiles 0 errors, 0 warnings. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-29 12:34:29 -04:00
double gamma = BatchOptions.At(shift + BN_OPT_GAMMA);
double beta = BatchOptions.At(shift + BN_OPT_BETA);
fix: improve error messages and suppress false sharing-violation logs - BufferDouble: replace hardcoded "DirectML/CPU-DLL" with dynamic backend name and add buffer index/element count to all error prints for easier debugging. - NetPersistence: distinguish missing file from transient lock by probing FileIsExist before logging, eliminating false "sharing violation" warnings when no saved model exists on first run.
2026-08-02 01:09:18 -04:00
//--- Matches NormalizeHost's self-heal, and this is the copy that makes it STICK: the clamped
//--- write at the end of this loop persists the repaired value, whereas the forward pass only
//--- substitutes one locally. Without this a model that loaded a NaN gamma would normalize
//--- correctly but never train that unit's scale again, since NaN + anything stays NaN.
if(!MathIsValidNumber(gamma))
gamma = 1.0;
if(!MathIsValidNumber(beta))
beta = 0.0;
feat(ai): batch normalization between dense layers The only bounded stage in the entire forward path was the sigmoid classification head - every hidden stage is PRELU. That is a network with no internal scale control, and the failure ordered exactly by depth: on SP500 H1 the shallow perceptron held ~52% balanced accuracy while the deepest topology sat on the 33.3% one-class floor, with the per-bar logit spread decaying monotonically (0.45 -> 0.38 over ~200 eras) until the evidence tilt fell under the class-prior tilt. That is the signature of internal covariate shift, which chapter 6.1 of the reference book is entirely about and which the NeuroNet_DNG engine addresses with a layer this project never had. Two mechanisms make this the right fix rather than more hyperparameter nudging: - it decouples WEIGHT_DECAY from the learned function (van Laarhoven 2017) - with a normalized layer downstream, decay can no longer grind the discriminative signal away, it only rescales the effective learning rate; - it is the precondition for ever running an unbounded logit head here. The 2026-07-27 attempt blew up (IS error 5.6e15) precisely because nothing upstream constrained scale. Implementation notes: - CNeuronBatchNormOCL computes host-side rather than as a fourth copy of a kernel across Network.cl + WarriorCPU.cpp + WarriorDML.cpp. The math is elementwise O(n); this way it behaves identically on all four compute tiers, needs no DLL rebuild, and cannot drift between backends. Same precedent as the softmax+CCE gradient and the per-sample loss weighting, both computed in MQL5 for that reason. - Statistics are exponential moving, not a stored mini-batch: training is pure online SGD, one update per sample, so there is no batch to average over. BatchNormWindow is an EMA window length. - gamma/beta are excluded from weight decay, deliberately - decaying gamma toward zero is the exact pathology being fixed. - The layer self-sizes from whatever sits below it, because a conv/pool stage's output width is derived inside the CNet constructor and is not knowable to the topology builder. - Checkpoint capture/restore/blend carry gamma/beta and the running statistics alongside the dense matrix, so the plateau ladder cannot restore a mismatched pair. - SeedOutputLayerBias accepted only an exact defNeuronBaseOCL as the weight-carrying penultimate layer; with normalization enabled that is the batch-norm layer, so the cold-start bias seed would have silently stopped being applied. - Refuses to build, loudly, if a topology asks for normalization with no compute backend at all - rather than quietly training a different architecture than the one requested. EnableBatchNorm (default on) and BatchNormWindow (1000 samples) are inputs so the effect can be A/B'd without a recompile. Both feed the weights-filename fingerprint, appended conditionally so existing non-BN configs keep their fingerprints and are not forced to retrain. Verified: analytic gradients match finite differences to 1.5e-7 relative over 200 random cases; a faithful port of the full forward/backward chain collapses to the 33.3% floor by era 4 without this layer and holds 36-43% with it. Compiles 0 errors, 0 warnings. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-29 12:34:29 -04:00
double dGamma = 0.0, dBeta = 0.0;
if(optimization == SGD)
{
fix(build): g_eta - the learning rate global no longer shadows a stdlib local MetaEditor: "declaration of 'eta' hides global variable" (Math.mqh:792 vs Network.mqh:80). The standard library's Math\Stat\Math.mqh declares a local `double eta` in its incomplete-gamma branch, and our bare global of the same name is in scope there. Same fault as the b1/b2/lr/momentum macros retired in ea2552e: a single-token global name living in a header that library code gets compiled beside. The library cannot move, so ours does - 112 references across 13 files, whole-word only. Named g_eta rather than g_learningRate to stay inside the vocabulary already around it (ETA_DECAY_FACTOR, ETA_MIN, m_etaCeiling, etaBefore), all of which are untouched and none of which shadow anything. One log line said "continuing to explore without decaying eta", where the word was prose rather than a symbol reference; that reads "the learning rate" now instead of naming a variable at the trader. Scanned for the next occurrence rather than waiting for it: the only other bare lowercase globals in the tree are eaName and tableschema, both distinctive enough not to collide with a library local. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-20 07:00:09 -04:00
dGamma = g_eta * gGamma + alpha * BatchOptions.At(shift + BN_OPT_MG);
dBeta = g_eta * gBeta + alpha * BatchOptions.At(shift + BN_OPT_MB);
feat(ai): batch normalization between dense layers The only bounded stage in the entire forward path was the sigmoid classification head - every hidden stage is PRELU. That is a network with no internal scale control, and the failure ordered exactly by depth: on SP500 H1 the shallow perceptron held ~52% balanced accuracy while the deepest topology sat on the 33.3% one-class floor, with the per-bar logit spread decaying monotonically (0.45 -> 0.38 over ~200 eras) until the evidence tilt fell under the class-prior tilt. That is the signature of internal covariate shift, which chapter 6.1 of the reference book is entirely about and which the NeuroNet_DNG engine addresses with a layer this project never had. Two mechanisms make this the right fix rather than more hyperparameter nudging: - it decouples WEIGHT_DECAY from the learned function (van Laarhoven 2017) - with a normalized layer downstream, decay can no longer grind the discriminative signal away, it only rescales the effective learning rate; - it is the precondition for ever running an unbounded logit head here. The 2026-07-27 attempt blew up (IS error 5.6e15) precisely because nothing upstream constrained scale. Implementation notes: - CNeuronBatchNormOCL computes host-side rather than as a fourth copy of a kernel across Network.cl + WarriorCPU.cpp + WarriorDML.cpp. The math is elementwise O(n); this way it behaves identically on all four compute tiers, needs no DLL rebuild, and cannot drift between backends. Same precedent as the softmax+CCE gradient and the per-sample loss weighting, both computed in MQL5 for that reason. - Statistics are exponential moving, not a stored mini-batch: training is pure online SGD, one update per sample, so there is no batch to average over. BatchNormWindow is an EMA window length. - gamma/beta are excluded from weight decay, deliberately - decaying gamma toward zero is the exact pathology being fixed. - The layer self-sizes from whatever sits below it, because a conv/pool stage's output width is derived inside the CNet constructor and is not knowable to the topology builder. - Checkpoint capture/restore/blend carry gamma/beta and the running statistics alongside the dense matrix, so the plateau ladder cannot restore a mismatched pair. - SeedOutputLayerBias accepted only an exact defNeuronBaseOCL as the weight-carrying penultimate layer; with normalization enabled that is the batch-norm layer, so the cold-start bias seed would have silently stopped being applied. - Refuses to build, loudly, if a topology asks for normalization with no compute backend at all - rather than quietly training a different architecture than the one requested. EnableBatchNorm (default on) and BatchNormWindow (1000 samples) are inputs so the effect can be A/B'd without a recompile. Both feed the weights-filename fingerprint, appended conditionally so existing non-BN configs keep their fingerprints and are not forced to retrain. Verified: analytic gradients match finite differences to 1.5e-7 relative over 200 random cases; a faithful port of the full forward/backward chain collapses to the 33.3% floor by era 4 without this layer and holds 36-43% with it. Compiles 0 errors, 0 warnings. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-29 12:34:29 -04:00
if(!BatchOptions.Update(shift + BN_OPT_MG, dGamma) ||
!BatchOptions.Update(shift + BN_OPT_MB, dBeta))
return false;
}
else
{
//--- Second momentum is stored ALREADY square-rooted so it can be used as the denominator
//--- directly, the same convention this engine's UpdateWeightsAdam kernels use
//--- (matrix_v[wi] = sqrt(...)).
refactor(stdlib): adopt Math\Stat for the deploy gate's normal tail; retire the b1/b2/lr/momentum macros The gate's NormalUpperTail was a hand-rolled Abramowitz & Stegun 26.2.17 approximation. Its own comment gave the reason - "drags a chain of headers behind it" - and that turned out to be one file: Math\Stat\Normal.mqh includes only Math.mqh, which includes nothing. Swapped for Cody's rational approximation in the library (~18 significant digits vs |error| < 7.5e-8). No past verdict changes: at the z the gate operates on, the difference is orders of magnitude below DEPLOY_FAMILY_WISE_ALPHA. Adopting it needed the four bare macros in AI\Network.mqh gone first. "#define b1 AdamBeta1" collides with an identifier in Math.mqh, so the include would have macro-expanded the library's own local and failed to compile - the same landmine that made the original author rename the approximation's coefficients to ntB1..ntB5 rather than use the reference's b1..b5. lr, b2 and momentum are the same class of hazard: single-token global macros in a 52k-line codebase. All four now resolve to the input names they always aliased, which is a pure textual identity - verified zero bare occurrences remain. Also: - SelectionSort over the buffered signals was O(n^2) with an O(n^2) count of StructToTime calls, because the comparison rebuilt both datetimes from the six int date fields every time. Now materialises the keys once and does an insertion sort; ArraySort cannot permute a struct array. IsEarlier goes with it, MakeDateTime becomes SignalTime. - Seven FileOpen sites lacked FILE_SHARE_READ|FILE_SHARE_WRITE, including AtomicWriteBegin, which stages every model save. All 43 sites now carry them - an exclusive open fails outright when another process holds the path, which here has meant a silently skipped save. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-19 19:31:36 -04:00
double mg = AdamBeta1 * BatchOptions.At(shift + BN_OPT_MG) + (1 - AdamBeta1) * gGamma;
double mb = AdamBeta1 * BatchOptions.At(shift + BN_OPT_MB) + (1 - AdamBeta1) * gBeta;
double vg = sqrt(AdamBeta2 * pow(BatchOptions.At(shift + BN_OPT_VG), 2) + (1 - AdamBeta2) * gGamma * gGamma);
double vb = sqrt(AdamBeta2 * pow(BatchOptions.At(shift + BN_OPT_VB), 2) + (1 - AdamBeta2) * gBeta * gBeta);
feat(ai): batch normalization between dense layers The only bounded stage in the entire forward path was the sigmoid classification head - every hidden stage is PRELU. That is a network with no internal scale control, and the failure ordered exactly by depth: on SP500 H1 the shallow perceptron held ~52% balanced accuracy while the deepest topology sat on the 33.3% one-class floor, with the per-bar logit spread decaying monotonically (0.45 -> 0.38 over ~200 eras) until the evidence tilt fell under the class-prior tilt. That is the signature of internal covariate shift, which chapter 6.1 of the reference book is entirely about and which the NeuroNet_DNG engine addresses with a layer this project never had. Two mechanisms make this the right fix rather than more hyperparameter nudging: - it decouples WEIGHT_DECAY from the learned function (van Laarhoven 2017) - with a normalized layer downstream, decay can no longer grind the discriminative signal away, it only rescales the effective learning rate; - it is the precondition for ever running an unbounded logit head here. The 2026-07-27 attempt blew up (IS error 5.6e15) precisely because nothing upstream constrained scale. Implementation notes: - CNeuronBatchNormOCL computes host-side rather than as a fourth copy of a kernel across Network.cl + WarriorCPU.cpp + WarriorDML.cpp. The math is elementwise O(n); this way it behaves identically on all four compute tiers, needs no DLL rebuild, and cannot drift between backends. Same precedent as the softmax+CCE gradient and the per-sample loss weighting, both computed in MQL5 for that reason. - Statistics are exponential moving, not a stored mini-batch: training is pure online SGD, one update per sample, so there is no batch to average over. BatchNormWindow is an EMA window length. - gamma/beta are excluded from weight decay, deliberately - decaying gamma toward zero is the exact pathology being fixed. - The layer self-sizes from whatever sits below it, because a conv/pool stage's output width is derived inside the CNet constructor and is not knowable to the topology builder. - Checkpoint capture/restore/blend carry gamma/beta and the running statistics alongside the dense matrix, so the plateau ladder cannot restore a mismatched pair. - SeedOutputLayerBias accepted only an exact defNeuronBaseOCL as the weight-carrying penultimate layer; with normalization enabled that is the batch-norm layer, so the cold-start bias seed would have silently stopped being applied. - Refuses to build, loudly, if a topology asks for normalization with no compute backend at all - rather than quietly training a different architecture than the one requested. EnableBatchNorm (default on) and BatchNormWindow (1000 samples) are inputs so the effect can be A/B'd without a recompile. Both feed the weights-filename fingerprint, appended conditionally so existing non-BN configs keep their fingerprints and are not forced to retrain. Verified: analytic gradients match finite differences to 1.5e-7 relative over 200 random cases; a faithful port of the full forward/backward chain collapses to the 33.3% floor by era 4 without this layer and holds 36-43% with it. Compiles 0 errors, 0 warnings. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-29 12:34:29 -04:00
dGamma = lt * mg / (vg > 0 ? vg : lt * 10);
dBeta = lt * mb / (vb > 0 ? vb : lt * 10);
if(!BatchOptions.Update(shift + BN_OPT_MG, mg) ||
!BatchOptions.Update(shift + BN_OPT_MB, mb) ||
!BatchOptions.Update(shift + BN_OPT_VG, vg) ||
!BatchOptions.Update(shift + BN_OPT_VB, vb))
return false;
}
dGamma = MathMax(-MAX_WEIGHT_DELTA, MathMin(MAX_WEIGHT_DELTA, dGamma));
dBeta = MathMax(-MAX_WEIGHT_DELTA, MathMin(MAX_WEIGHT_DELTA, dBeta));
feat: mini-batch gradient accumulation (F4), front-end-aware capacity budget (F6), split Wyckoff categoricals (N1) Completes the 2026-08-09 training audit. FORCES A RETRAIN of every Wyckoff-enabled config (N1 re-keys the fingerprint), and BOTH DLLs must be redeployed alongside the .ex5 - they carry new exports. F4 - mini-batch accumulation, TRAIN_BATCH_SIZE=32. Training was pure online SGD (one weight update per bar), which is the mechanical source of the era-to-era whipsaw every downstream guard was built to cope with. The O(n^2) outer product is native - AccumulateWeightGrad / AccumulateWeightGradConv / AccumulateBufferInto in Network.cl, WarriorCPU and WarriorDML - while the optimizer step is host-side MQL5 shared by all tiers (ApplyAccumToBlock), so there is one Adam/SGD implementation instead of four that can drift. - the LSTM needs no outer-product kernel (WeightsGradient already holds the sample's full dW) but could NOT simply be left un-zeroed between samples: CPU_LSTMSeqBackward/DML_LSTMSeqBackward memset it on entry. Hence a separate accumulator plus an elementwise add. - batch-norm gamma/beta accumulate in host arrays, not new BatchOptions slots - BN_OPT_STRIDE is baked into every persisted .nnw. - scoped to pass 2; online learning keeps immediate updates. Every save / checkpoint / scoring boundary flushes, scaling by the real sample count. - degrades to per-sample updates (one log line) on a tier that cannot accumulate, so old devices and DLL-free builds are unaffected. - verified offline: DirectML/batch_accum_check.cpp drives the real exports against an independent reference; at B=1 the accumulator matches the shipped unbatched kernel's own gradient to 1.1e-16. Math only - the in-situ check remains the per-layer dW/W report on a real era. F6 - ComputeFirstLayerWidth budgeted against the RAW input width even where a conv/LSTM front end had already reduced it, so an LSTM's dense stack was charged for 1,280 inputs when it receives 64. Confirmed from the deployed .cfg files: CONV, LSTM and HYBRID were all pinned at the 16-unit floor. Now budgeted against the front-end output and capped at it (never fan out), with the derivation reordered so both stages settle first. N1 - EventCode/EventPhase/StructuralPhase are signed categoricals packing direction and Wyckoff stage into one scalar across a sign discontinuity. Split into direction + [0,1] magnitude, the same convention the base OHLC block uses. Information-preserving; 13 readings now occupy 16 inputs. Compiled clean (0 errors, 0 warnings); both DLLs rebuilt. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-09 11:48:03 -04:00
//--- `return true` and not `continue`: this is one unit's step now that the loop lives in the
//--- caller, and a non-finite delta means SKIP this unit, never fail the layer.
feat(ai): batch normalization between dense layers The only bounded stage in the entire forward path was the sigmoid classification head - every hidden stage is PRELU. That is a network with no internal scale control, and the failure ordered exactly by depth: on SP500 H1 the shallow perceptron held ~52% balanced accuracy while the deepest topology sat on the 33.3% one-class floor, with the per-bar logit spread decaying monotonically (0.45 -> 0.38 over ~200 eras) until the evidence tilt fell under the class-prior tilt. That is the signature of internal covariate shift, which chapter 6.1 of the reference book is entirely about and which the NeuroNet_DNG engine addresses with a layer this project never had. Two mechanisms make this the right fix rather than more hyperparameter nudging: - it decouples WEIGHT_DECAY from the learned function (van Laarhoven 2017) - with a normalized layer downstream, decay can no longer grind the discriminative signal away, it only rescales the effective learning rate; - it is the precondition for ever running an unbounded logit head here. The 2026-07-27 attempt blew up (IS error 5.6e15) precisely because nothing upstream constrained scale. Implementation notes: - CNeuronBatchNormOCL computes host-side rather than as a fourth copy of a kernel across Network.cl + WarriorCPU.cpp + WarriorDML.cpp. The math is elementwise O(n); this way it behaves identically on all four compute tiers, needs no DLL rebuild, and cannot drift between backends. Same precedent as the softmax+CCE gradient and the per-sample loss weighting, both computed in MQL5 for that reason. - Statistics are exponential moving, not a stored mini-batch: training is pure online SGD, one update per sample, so there is no batch to average over. BatchNormWindow is an EMA window length. - gamma/beta are excluded from weight decay, deliberately - decaying gamma toward zero is the exact pathology being fixed. - The layer self-sizes from whatever sits below it, because a conv/pool stage's output width is derived inside the CNet constructor and is not knowable to the topology builder. - Checkpoint capture/restore/blend carry gamma/beta and the running statistics alongside the dense matrix, so the plateau ladder cannot restore a mismatched pair. - SeedOutputLayerBias accepted only an exact defNeuronBaseOCL as the weight-carrying penultimate layer; with normalization enabled that is the batch-norm layer, so the cold-start bias seed would have silently stopped being applied. - Refuses to build, loudly, if a topology asks for normalization with no compute backend at all - rather than quietly training a different architecture than the one requested. EnableBatchNorm (default on) and BatchNormWindow (1000 samples) are inputs so the effect can be A/B'd without a recompile. Both feed the weights-filename fingerprint, appended conditionally so existing non-BN configs keep their fingerprints and are not forced to retrain. Verified: analytic gradients match finite differences to 1.5e-7 relative over 200 random cases; a faithful port of the full forward/backward chain collapses to the 33.3% floor by era 4 without this layer and holds 36-43% with it. Compiles 0 errors, 0 warnings. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-29 12:34:29 -04:00
if(!MathIsValidNumber(dGamma) || !MathIsValidNumber(dBeta))
feat: mini-batch gradient accumulation (F4), front-end-aware capacity budget (F6), split Wyckoff categoricals (N1) Completes the 2026-08-09 training audit. FORCES A RETRAIN of every Wyckoff-enabled config (N1 re-keys the fingerprint), and BOTH DLLs must be redeployed alongside the .ex5 - they carry new exports. F4 - mini-batch accumulation, TRAIN_BATCH_SIZE=32. Training was pure online SGD (one weight update per bar), which is the mechanical source of the era-to-era whipsaw every downstream guard was built to cope with. The O(n^2) outer product is native - AccumulateWeightGrad / AccumulateWeightGradConv / AccumulateBufferInto in Network.cl, WarriorCPU and WarriorDML - while the optimizer step is host-side MQL5 shared by all tiers (ApplyAccumToBlock), so there is one Adam/SGD implementation instead of four that can drift. - the LSTM needs no outer-product kernel (WeightsGradient already holds the sample's full dW) but could NOT simply be left un-zeroed between samples: CPU_LSTMSeqBackward/DML_LSTMSeqBackward memset it on entry. Hence a separate accumulator plus an elementwise add. - batch-norm gamma/beta accumulate in host arrays, not new BatchOptions slots - BN_OPT_STRIDE is baked into every persisted .nnw. - scoped to pass 2; online learning keeps immediate updates. Every save / checkpoint / scoring boundary flushes, scaling by the real sample count. - degrades to per-sample updates (one log line) on a tier that cannot accumulate, so old devices and DLL-free builds are unaffected. - verified offline: DirectML/batch_accum_check.cpp drives the real exports against an independent reference; at B=1 the accumulator matches the shipped unbatched kernel's own gradient to 1.1e-16. Math only - the in-situ check remains the per-layer dW/W report on a real era. F6 - ComputeFirstLayerWidth budgeted against the RAW input width even where a conv/LSTM front end had already reduced it, so an LSTM's dense stack was charged for 1,280 inputs when it receives 64. Confirmed from the deployed .cfg files: CONV, LSTM and HYBRID were all pinned at the 16-unit floor. Now budgeted against the front-end output and capped at it (never fan out), with the derivation reordered so both stages settle first. N1 - EventCode/EventPhase/StructuralPhase are signed categoricals packing direction and Wyckoff stage into one scalar across a sign discontinuity. Split into direction + [0,1] magnitude, the same convention the base OHLC block uses. Information-preserving; 13 readings now occupy 16 inputs. Compiled clean (0 errors, 0 warnings); both DLLs rebuilt. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-09 11:48:03 -04:00
return true;
feat(ai): batch normalization between dense layers The only bounded stage in the entire forward path was the sigmoid classification head - every hidden stage is PRELU. That is a network with no internal scale control, and the failure ordered exactly by depth: on SP500 H1 the shallow perceptron held ~52% balanced accuracy while the deepest topology sat on the 33.3% one-class floor, with the per-bar logit spread decaying monotonically (0.45 -> 0.38 over ~200 eras) until the evidence tilt fell under the class-prior tilt. That is the signature of internal covariate shift, which chapter 6.1 of the reference book is entirely about and which the NeuroNet_DNG engine addresses with a layer this project never had. Two mechanisms make this the right fix rather than more hyperparameter nudging: - it decouples WEIGHT_DECAY from the learned function (van Laarhoven 2017) - with a normalized layer downstream, decay can no longer grind the discriminative signal away, it only rescales the effective learning rate; - it is the precondition for ever running an unbounded logit head here. The 2026-07-27 attempt blew up (IS error 5.6e15) precisely because nothing upstream constrained scale. Implementation notes: - CNeuronBatchNormOCL computes host-side rather than as a fourth copy of a kernel across Network.cl + WarriorCPU.cpp + WarriorDML.cpp. The math is elementwise O(n); this way it behaves identically on all four compute tiers, needs no DLL rebuild, and cannot drift between backends. Same precedent as the softmax+CCE gradient and the per-sample loss weighting, both computed in MQL5 for that reason. - Statistics are exponential moving, not a stored mini-batch: training is pure online SGD, one update per sample, so there is no batch to average over. BatchNormWindow is an EMA window length. - gamma/beta are excluded from weight decay, deliberately - decaying gamma toward zero is the exact pathology being fixed. - The layer self-sizes from whatever sits below it, because a conv/pool stage's output width is derived inside the CNet constructor and is not knowable to the topology builder. - Checkpoint capture/restore/blend carry gamma/beta and the running statistics alongside the dense matrix, so the plateau ladder cannot restore a mismatched pair. - SeedOutputLayerBias accepted only an exact defNeuronBaseOCL as the weight-carrying penultimate layer; with normalization enabled that is the batch-norm layer, so the cold-start bias seed would have silently stopped being applied. - Refuses to build, loudly, if a topology asks for normalization with no compute backend at all - rather than quietly training a different architecture than the one requested. EnableBatchNorm (default on) and BatchNormWindow (1000 samples) are inputs so the effect can be A/B'd without a recompile. Both feed the weights-filename fingerprint, appended conditionally so existing non-BN configs keep their fingerprints and are not forced to retrain. Verified: analytic gradients match finite differences to 1.5e-7 relative over 200 random cases; a faithful port of the full forward/backward chain collapses to the 33.3% floor by era 4 without this layer and holds 36-43% with it. Compiles 0 errors, 0 warnings. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-29 12:34:29 -04:00
if(!BatchOptions.Update(shift + BN_OPT_GAMMA, MathMax(-MAX_WEIGHT, MathMin(MAX_WEIGHT, gamma + dGamma))) ||
!BatchOptions.Update(shift + BN_OPT_BETA, MathMax(-MAX_WEIGHT, MathMin(MAX_WEIGHT, beta + dBeta))))
return false;
}
feat: mini-batch gradient accumulation (F4), front-end-aware capacity budget (F6), split Wyckoff categoricals (N1) Completes the 2026-08-09 training audit. FORCES A RETRAIN of every Wyckoff-enabled config (N1 re-keys the fingerprint), and BOTH DLLs must be redeployed alongside the .ex5 - they carry new exports. F4 - mini-batch accumulation, TRAIN_BATCH_SIZE=32. Training was pure online SGD (one weight update per bar), which is the mechanical source of the era-to-era whipsaw every downstream guard was built to cope with. The O(n^2) outer product is native - AccumulateWeightGrad / AccumulateWeightGradConv / AccumulateBufferInto in Network.cl, WarriorCPU and WarriorDML - while the optimizer step is host-side MQL5 shared by all tiers (ApplyAccumToBlock), so there is one Adam/SGD implementation instead of four that can drift. - the LSTM needs no outer-product kernel (WeightsGradient already holds the sample's full dW) but could NOT simply be left un-zeroed between samples: CPU_LSTMSeqBackward/DML_LSTMSeqBackward memset it on entry. Hence a separate accumulator plus an elementwise add. - batch-norm gamma/beta accumulate in host arrays, not new BatchOptions slots - BN_OPT_STRIDE is baked into every persisted .nnw. - scoped to pass 2; online learning keeps immediate updates. Every save / checkpoint / scoring boundary flushes, scaling by the real sample count. - degrades to per-sample updates (one log line) on a tier that cannot accumulate, so old devices and DLL-free builds are unaffected. - verified offline: DirectML/batch_accum_check.cpp drives the real exports against an independent reference; at B=1 the accumulator matches the shipped unbatched kernel's own gradient to 1.1e-16. Math only - the in-situ check remains the per-layer dW/W report on a real era. F6 - ComputeFirstLayerWidth budgeted against the RAW input width even where a conv/LSTM front end had already reduced it, so an LSTM's dense stack was charged for 1,280 inputs when it receives 64. Confirmed from the deployed .cfg files: CONV, LSTM and HYBRID were all pinned at the 16-unit floor. Now budgeted against the front-end output and capped at it (never fan out), with the derivation reordered so both stages settle first. N1 - EventCode/EventPhase/StructuralPhase are signed categoricals packing direction and Wyckoff stage into one scalar across a sign discontinuity. Split into direction + [0,1] magnitude, the same convention the base OHLC block uses. Information-preserving; 13 readings now occupy 16 inputs. Compiled clean (0 errors, 0 warnings); both DLLs rebuilt. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-09 11:48:03 -04:00
return true;
}
//+------------------------------------------------------------------+
refactor(comments): box headers to stdlib length The //| box blocks were excluded from 0b06f8e and 5efdb48 and were what remained: 160 of them ran to 10+ lines, the longest to 88. Compressed to their leading topic sentences - 5 lines for a function header, 8 for a file header - keeping the box format and the standard MQL5 name/author lines verbatim. Verified at the BYTE level this time, across every in-scope file: the list of non-comment lines is byte-identical to HEAD and braces balance. The first check compared a locale-decoded 'git show' against a UTF-8 read and flagged 25 files that had not changed at all - every BOM and every non-ASCII line mismatched. 47,696 -> 40,665 lines in scope; comment share 38% -> 26%. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-22 00:30:14 -04:00
//| MINI-BATCH (2026-08-09 audit, F4). gamma/beta are host-side |
//| parameters, so their accumulation is a plain host sum - no |
//| kernel and no extra device buffer. |
feat: mini-batch gradient accumulation (F4), front-end-aware capacity budget (F6), split Wyckoff categoricals (N1) Completes the 2026-08-09 training audit. FORCES A RETRAIN of every Wyckoff-enabled config (N1 re-keys the fingerprint), and BOTH DLLs must be redeployed alongside the .ex5 - they carry new exports. F4 - mini-batch accumulation, TRAIN_BATCH_SIZE=32. Training was pure online SGD (one weight update per bar), which is the mechanical source of the era-to-era whipsaw every downstream guard was built to cope with. The O(n^2) outer product is native - AccumulateWeightGrad / AccumulateWeightGradConv / AccumulateBufferInto in Network.cl, WarriorCPU and WarriorDML - while the optimizer step is host-side MQL5 shared by all tiers (ApplyAccumToBlock), so there is one Adam/SGD implementation instead of four that can drift. - the LSTM needs no outer-product kernel (WeightsGradient already holds the sample's full dW) but could NOT simply be left un-zeroed between samples: CPU_LSTMSeqBackward/DML_LSTMSeqBackward memset it on entry. Hence a separate accumulator plus an elementwise add. - batch-norm gamma/beta accumulate in host arrays, not new BatchOptions slots - BN_OPT_STRIDE is baked into every persisted .nnw. - scoped to pass 2; online learning keeps immediate updates. Every save / checkpoint / scoring boundary flushes, scaling by the real sample count. - degrades to per-sample updates (one log line) on a tier that cannot accumulate, so old devices and DLL-free builds are unaffected. - verified offline: DirectML/batch_accum_check.cpp drives the real exports against an independent reference; at B=1 the accumulator matches the shipped unbatched kernel's own gradient to 1.1e-16. Math only - the in-situ check remains the per-layer dW/W report on a real era. F6 - ComputeFirstLayerWidth budgeted against the RAW input width even where a conv/LSTM front end had already reduced it, so an LSTM's dense stack was charged for 1,280 inputs when it receives 64. Confirmed from the deployed .cfg files: CONV, LSTM and HYBRID were all pinned at the 16-unit floor. Now budgeted against the front-end output and capped at it (never fan out), with the derivation reordered so both stages settle first. N1 - EventCode/EventPhase/StructuralPhase are signed categoricals packing direction and Wyckoff stage into one scalar across a sign discontinuity. Split into direction + [0,1] magnitude, the same convention the base OHLC block uses. Information-preserving; 13 readings now occupy 16 inputs. Compiled clean (0 errors, 0 warnings); both DLLs rebuilt. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-09 11:48:03 -04:00
//+------------------------------------------------------------------+
bool CNeuronBatchNormOCL::accumulateInputWeightGrads(CNeuronBaseOCL *NeuronOCL)
{
//--- The outgoing dense matrix is accumulated by the layer above, exactly as for a plain dense
//--- neuron; this adds only the normalization parameters' own gradients.
if(CheckPointer(BatchOptions) == POINTER_INVALID || iBatchSize <= 1)
return true; // normalization off - gamma/beta are not in the graph
int units = Neurons();
fix: live trades now use the geometry the gate certifies; perf: BN kernels Three changes, one theme: the trade placed, the trade graded, and the trade computed are now the same trade. 1) GEOMETRY WIRE (correctness, the ranked #1 open issue). The measured barrier pair reached the LABELS only - OpenParams still placed orders at the enum geometry (2*ATR/6*ATR), so the deploy gate certified "reaches 1.62*ATR before 3.33*ATR above break-even" about trades the EA never placed. Published via g_DerivedSlAtrMult/g_DerivedTpAtrMult (ConfidenceBridge, same same-tick contract as the confidence globals, because OpenParams runs on the root signal which has no pointer to the AI filter). Two writers: DeriveBarrierGeometry at era 0, and the .cfg adoption a deployed model takes. Overrides both legs and both Intelligent modes - the certificate is exact or it is nothing. TP is ATR-anchored like the label, NOT risk-relative, so a floor-widened stop cannot reshape the certified target. 2) BATCH NORM RUNS DEVICE-SIDE ON OPENCL. Four kernels in Network.cl - forward, hidden gradient, gamma/beta accumulate, gamma/beta apply - each a line-for-line transcription of the host implementation (NormalizeHost / HiddenGradHost / StepGammaBeta) including every NaN guard, clamp, and the exact moment-write ordering. The host copies remain the runtime for the DLL and pure-MQL5 tiers and the reference the kernels must match. Because this box has no OpenCL platform, the safety story is layered: - shim validation: kernels compiled as C and driven against a fp64 host transcription over NaN-poisoned stats, NaN gamma, over-clamp inputs, the frozen path, both optimizers, 3 batches - ALL PASS, worst normalized diff 0.132 vs tolerance 1.0 - in-situ self-check: each kernel is compared against its host twin ON FIRST USE on the real device (SelfCheckBn*), covering what the shim cannot - arg indices and buffer bindings. Any disagreement resyncs from the good copy, latches all BN kernels off process-wide, and training continues host-side. A transcription bug costs a warning and some speed, never a poisoned .nnw. - sync discipline: BatchOptions is now a CBufferDouble with explicit authority tracking (m_bnDeviceAuthoritative). Checkpoints/saves pull read-only; restores/loads/resets push; a mid-batch handover drains the device gamma/beta accumulator into the host arrays so no sample is lost. 3) SMALL FIXES. Apply-kernel build failure now latches the dispatch path at init (one warning instead of warning + failed Execute). Build tag bumped to win-scoring-gpu-v1 - first tag change since expectancy-stop-v1 despite five binary-changing commits. Both build variants compile 0 errors, 0 warnings. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-09 17:51:40 -04:00
//--- DEVICE PATH: the gradient and NX are both device-fresh, so the accumulation happens where
//--- they already live. One dispatch, no transfers.
if(BnDeviceEligible() && getGradientIndex() >= 0 && EnsureBnDeviceBuffers())
{
if(!m_bnCheckedAccum)
return SelfCheckBnAccum();
if(DispatchBnAccum())
return true;
LatchBnKernelsOff("BatchNormAccumGammaBeta dispatch failed (error " +
IntegerToString(GetLastError()) + ")");
//--- The latch drained whatever the kernels had accumulated this batch into m_accGamma/
//--- m_accBeta; the host code below adds THIS sample on top, so nothing is lost or doubled.
}
EnsureHostAuthoritative();
perf: batch norm stops re-reading buffers it already has Batch norm computes host-side while its neighbours are device-resident, so every value it touches crosses the bus - and the cost is the BLOCKING SYNC per crossing, not the bytes. Per sample it did four reads, two of which were exact duplicates: feedForward -> previous layer's Output calcInputGradients -> own Gradient, and the previous layer's Output AGAIN update/accumulate -> own Gradient AGAIN Nothing writes the previous layer's Output between the forward and backward passes, and nothing writes this layer's Gradient between the gradient pass and the weight-update pass - backPropOCL runs those as two separate top-to-bottom loops and only the first writes gradients. So the repeats are removable and the result is bit-exact: the same values, read once instead of twice. Each cache is armed by its producer and disarmed by feedForward, so a consumer whose producer did not run this sample falls back to reading the buffer rather than using the previous sample's data. That is not hypothetical - a batch norm at layer 1 never gets calcInputGradients called at all, the same asymmetry backPropOCL already documents for a layer-1 LSTM, so its gradient cache is never armed and it takes the fallback every time. Both build variants compile 0 errors, 0 warnings. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-09 17:16:08 -04:00
//--- Gradient from calcInputGradients' read this same sample - see the declaration comment. This is
//--- the batched twin of updateInputWeights above and takes the value from the same place.
double grad[];
int count = m_gradCached ? ArrayCopy(grad, m_gradCache) : getGradient(grad);
feat: mini-batch gradient accumulation (F4), front-end-aware capacity budget (F6), split Wyckoff categoricals (N1) Completes the 2026-08-09 training audit. FORCES A RETRAIN of every Wyckoff-enabled config (N1 re-keys the fingerprint), and BOTH DLLs must be redeployed alongside the .ex5 - they carry new exports. F4 - mini-batch accumulation, TRAIN_BATCH_SIZE=32. Training was pure online SGD (one weight update per bar), which is the mechanical source of the era-to-era whipsaw every downstream guard was built to cope with. The O(n^2) outer product is native - AccumulateWeightGrad / AccumulateWeightGradConv / AccumulateBufferInto in Network.cl, WarriorCPU and WarriorDML - while the optimizer step is host-side MQL5 shared by all tiers (ApplyAccumToBlock), so there is one Adam/SGD implementation instead of four that can drift. - the LSTM needs no outer-product kernel (WeightsGradient already holds the sample's full dW) but could NOT simply be left un-zeroed between samples: CPU_LSTMSeqBackward/DML_LSTMSeqBackward memset it on entry. Hence a separate accumulator plus an elementwise add. - batch-norm gamma/beta accumulate in host arrays, not new BatchOptions slots - BN_OPT_STRIDE is baked into every persisted .nnw. - scoped to pass 2; online learning keeps immediate updates. Every save / checkpoint / scoring boundary flushes, scaling by the real sample count. - degrades to per-sample updates (one log line) on a tier that cannot accumulate, so old devices and DLL-free builds are unaffected. - verified offline: DirectML/batch_accum_check.cpp drives the real exports against an independent reference; at B=1 the accumulator matches the shipped unbatched kernel's own gradient to 1.1e-16. Math only - the in-situ check remains the per-layer dW/W report on a real era. F6 - ComputeFirstLayerWidth budgeted against the RAW input width even where a conv/LSTM front end had already reduced it, so an LSTM's dense stack was charged for 1,280 inputs when it receives 64. Confirmed from the deployed .cfg files: CONV, LSTM and HYBRID were all pinned at the 16-unit floor. Now budgeted against the front-end output and capped at it (never fan out), with the derivation reordered so both stages settle first. N1 - EventCode/EventPhase/StructuralPhase are signed categoricals packing direction and Wyckoff stage into one scalar across a sign discontinuity. Split into direction + [0,1] magnitude, the same convention the base OHLC block uses. Information-preserving; 13 readings now occupy 16 inputs. Compiled clean (0 errors, 0 warnings); both DLLs rebuilt. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-09 11:48:03 -04:00
if(count < units || units <= 0)
return false;
if(ArraySize(m_accGamma) != units || ArraySize(m_accBeta) != units)
{
ArrayResize(m_accGamma, units);
ArrayResize(m_accBeta, units);
ArrayInitialize(m_accGamma, 0.0);
ArrayInitialize(m_accBeta, 0.0);
}
for(int i = 0; i < units; i++)
{
double g = grad[i];
m_accGamma[i] += g * BatchOptions.At(i * BN_OPT_STRIDE + BN_OPT_NX);
m_accBeta[i] += g;
}
return true;
}
//+------------------------------------------------------------------+
bool CNeuronBatchNormOCL::BeginGradAccum(void)
{
bool ok = CNeuronBaseOCL::BeginGradAccum();
ArrayInitialize(m_accGamma, 0.0);
ArrayInitialize(m_accBeta, 0.0);
fix: live trades now use the geometry the gate certifies; perf: BN kernels Three changes, one theme: the trade placed, the trade graded, and the trade computed are now the same trade. 1) GEOMETRY WIRE (correctness, the ranked #1 open issue). The measured barrier pair reached the LABELS only - OpenParams still placed orders at the enum geometry (2*ATR/6*ATR), so the deploy gate certified "reaches 1.62*ATR before 3.33*ATR above break-even" about trades the EA never placed. Published via g_DerivedSlAtrMult/g_DerivedTpAtrMult (ConfidenceBridge, same same-tick contract as the confidence globals, because OpenParams runs on the root signal which has no pointer to the AI filter). Two writers: DeriveBarrierGeometry at era 0, and the .cfg adoption a deployed model takes. Overrides both legs and both Intelligent modes - the certificate is exact or it is nothing. TP is ATR-anchored like the label, NOT risk-relative, so a floor-widened stop cannot reshape the certified target. 2) BATCH NORM RUNS DEVICE-SIDE ON OPENCL. Four kernels in Network.cl - forward, hidden gradient, gamma/beta accumulate, gamma/beta apply - each a line-for-line transcription of the host implementation (NormalizeHost / HiddenGradHost / StepGammaBeta) including every NaN guard, clamp, and the exact moment-write ordering. The host copies remain the runtime for the DLL and pure-MQL5 tiers and the reference the kernels must match. Because this box has no OpenCL platform, the safety story is layered: - shim validation: kernels compiled as C and driven against a fp64 host transcription over NaN-poisoned stats, NaN gamma, over-clamp inputs, the frozen path, both optimizers, 3 batches - ALL PASS, worst normalized diff 0.132 vs tolerance 1.0 - in-situ self-check: each kernel is compared against its host twin ON FIRST USE on the real device (SelfCheckBn*), covering what the shim cannot - arg indices and buffer bindings. Any disagreement resyncs from the good copy, latches all BN kernels off process-wide, and training continues host-side. A transcription bug costs a warning and some speed, never a poisoned .nnw. - sync discipline: BatchOptions is now a CBufferDouble with explicit authority tracking (m_bnDeviceAuthoritative). Checkpoints/saves pull read-only; restores/loads/resets push; a mid-batch handover drains the device gamma/beta accumulator into the host arrays so no sample is lost. 3) SMALL FIXES. Apply-kernel build failure now latches the dispatch path at init (one warning instead of warning + failed Execute). Build tag bumped to win-scoring-gpu-v1 - first tag change since expectancy-stop-v1 despite five binary-changing commits. Both build variants compile 0 errors, 0 warnings. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-09 17:51:40 -04:00
//--- Belt and braces for the device accumulator: the apply kernel zeroes it itself, but a batch
//--- abandoned mid-way (era boundary, restore) must not leak its partial sums into the next one.
if(CheckPointer(m_bnAcc) != POINTER_INVALID && m_bnAcc.GetIndex() >= 0)
ok = ZeroOptimizerBuffer(m_bnAcc) && ok;
feat: mini-batch gradient accumulation (F4), front-end-aware capacity budget (F6), split Wyckoff categoricals (N1) Completes the 2026-08-09 training audit. FORCES A RETRAIN of every Wyckoff-enabled config (N1 re-keys the fingerprint), and BOTH DLLs must be redeployed alongside the .ex5 - they carry new exports. F4 - mini-batch accumulation, TRAIN_BATCH_SIZE=32. Training was pure online SGD (one weight update per bar), which is the mechanical source of the era-to-era whipsaw every downstream guard was built to cope with. The O(n^2) outer product is native - AccumulateWeightGrad / AccumulateWeightGradConv / AccumulateBufferInto in Network.cl, WarriorCPU and WarriorDML - while the optimizer step is host-side MQL5 shared by all tiers (ApplyAccumToBlock), so there is one Adam/SGD implementation instead of four that can drift. - the LSTM needs no outer-product kernel (WeightsGradient already holds the sample's full dW) but could NOT simply be left un-zeroed between samples: CPU_LSTMSeqBackward/DML_LSTMSeqBackward memset it on entry. Hence a separate accumulator plus an elementwise add. - batch-norm gamma/beta accumulate in host arrays, not new BatchOptions slots - BN_OPT_STRIDE is baked into every persisted .nnw. - scoped to pass 2; online learning keeps immediate updates. Every save / checkpoint / scoring boundary flushes, scaling by the real sample count. - degrades to per-sample updates (one log line) on a tier that cannot accumulate, so old devices and DLL-free builds are unaffected. - verified offline: DirectML/batch_accum_check.cpp drives the real exports against an independent reference; at B=1 the accumulator matches the shipped unbatched kernel's own gradient to 1.1e-16. Math only - the in-situ check remains the per-layer dW/W report on a real era. F6 - ComputeFirstLayerWidth budgeted against the RAW input width even where a conv/LSTM front end had already reduced it, so an LSTM's dense stack was charged for 1,280 inputs when it receives 64. Confirmed from the deployed .cfg files: CONV, LSTM and HYBRID were all pinned at the 16-unit floor. Now budgeted against the front-end output and capped at it (never fan out), with the derivation reordered so both stages settle first. N1 - EventCode/EventPhase/StructuralPhase are signed categoricals packing direction and Wyckoff stage into one scalar across a sign discontinuity. Split into direction + [0,1] magnitude, the same convention the base OHLC block uses. Information-preserving; 13 readings now occupy 16 inputs. Compiled clean (0 errors, 0 warnings); both DLLs rebuilt. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-09 11:48:03 -04:00
return ok;
}
//+------------------------------------------------------------------+
bool CNeuronBatchNormOCL::ApplyAccumulatedGradients(double scale)
{
//--- Outgoing dense matrix first, through the shared block optimizer.
bool ok = ApplyAccumToBlock(Weights, GradAccum, FirstMomentum, SecondMomentum, DeltaWeights, scale);
if(CheckPointer(BatchOptions) != POINTER_INVALID && iBatchSize > 1)
{
fix(build): g_eta - the learning rate global no longer shadows a stdlib local MetaEditor: "declaration of 'eta' hides global variable" (Math.mqh:792 vs Network.mqh:80). The standard library's Math\Stat\Math.mqh declares a local `double eta` in its incomplete-gamma branch, and our bare global of the same name is in scope there. Same fault as the b1/b2/lr/momentum macros retired in ea2552e: a single-token global name living in a header that library code gets compiled beside. The library cannot move, so ours does - 112 references across 13 files, whole-word only. Named g_eta rather than g_learningRate to stay inside the vocabulary already around it (ETA_DECAY_FACTOR, ETA_MIN, m_etaCeiling, etaBefore), all of which are untouched and none of which shadow anything. One log line said "continuing to explore without decaying eta", where the word was prose rather than a symbol reference; that reads "the learning rate" now instead of naming a variable at the trader. Scanned for the next occurrence rather than waiting for it: the only other bare lowercase globals in the tree are eaName and tableschema, both distinctive enough not to collide with a library local. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-20 07:00:09 -04:00
double lt = (optimization == SGD) ? 0.0 : g_eta * sqrt(1 - pow(AdamBeta2, t)) / (1 - pow(AdamBeta1, t));
fix: live trades now use the geometry the gate certifies; perf: BN kernels Three changes, one theme: the trade placed, the trade graded, and the trade computed are now the same trade. 1) GEOMETRY WIRE (correctness, the ranked #1 open issue). The measured barrier pair reached the LABELS only - OpenParams still placed orders at the enum geometry (2*ATR/6*ATR), so the deploy gate certified "reaches 1.62*ATR before 3.33*ATR above break-even" about trades the EA never placed. Published via g_DerivedSlAtrMult/g_DerivedTpAtrMult (ConfidenceBridge, same same-tick contract as the confidence globals, because OpenParams runs on the root signal which has no pointer to the AI filter). Two writers: DeriveBarrierGeometry at era 0, and the .cfg adoption a deployed model takes. Overrides both legs and both Intelligent modes - the certificate is exact or it is nothing. TP is ATR-anchored like the label, NOT risk-relative, so a floor-widened stop cannot reshape the certified target. 2) BATCH NORM RUNS DEVICE-SIDE ON OPENCL. Four kernels in Network.cl - forward, hidden gradient, gamma/beta accumulate, gamma/beta apply - each a line-for-line transcription of the host implementation (NormalizeHost / HiddenGradHost / StepGammaBeta) including every NaN guard, clamp, and the exact moment-write ordering. The host copies remain the runtime for the DLL and pure-MQL5 tiers and the reference the kernels must match. Because this box has no OpenCL platform, the safety story is layered: - shim validation: kernels compiled as C and driven against a fp64 host transcription over NaN-poisoned stats, NaN gamma, over-clamp inputs, the frozen path, both optimizers, 3 batches - ALL PASS, worst normalized diff 0.132 vs tolerance 1.0 - in-situ self-check: each kernel is compared against its host twin ON FIRST USE on the real device (SelfCheckBn*), covering what the shim cannot - arg indices and buffer bindings. Any disagreement resyncs from the good copy, latches all BN kernels off process-wide, and training continues host-side. A transcription bug costs a warning and some speed, never a poisoned .nnw. - sync discipline: BatchOptions is now a CBufferDouble with explicit authority tracking (m_bnDeviceAuthoritative). Checkpoints/saves pull read-only; restores/loads/resets push; a mid-batch handover drains the device gamma/beta accumulator into the host arrays so no sample is lost. 3) SMALL FIXES. Apply-kernel build failure now latches the dispatch path at init (one warning instead of warning + failed Execute). Build tag bumped to win-scoring-gpu-v1 - first tag change since expectancy-stop-v1 despite five binary-changing commits. Both build variants compile 0 errors, 0 warnings. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-09 17:51:40 -04:00
//--- DEVICE PATH: the batch was accumulated by the kernels, so it is stepped by the kernel too.
//--- A dispatch failure latches (which drains the device sums into m_accGamma/m_accBeta) and
//--- drops to the host loop below, so the batch is stepped exactly once either way.
bool stepped = false;
if(BnDeviceEligible() && CheckPointer(m_bnAcc) != POINTER_INVALID && m_bnAcc.GetIndex() >= 0)
{
if(!m_bnCheckedApply)
stepped = SelfCheckBnApply(scale, lt);
else
if(DispatchBnApply(scale, lt))
stepped = true;
else
LatchBnKernelsOff("BatchNormApplyGammaBeta dispatch failed (error " +
IntegerToString(GetLastError()) + ")");
}
if(!stepped)
{
EnsureHostAuthoritative();
int units = MathMin(Neurons(), MathMin(ArraySize(m_accGamma), ArraySize(m_accBeta)));
for(int i = 0; i < units; i++)
if(!StepGammaBeta(i * BN_OPT_STRIDE, m_accGamma[i] * scale, m_accBeta[i] * scale, lt))
{
ok = false;
break;
}
ArrayInitialize(m_accGamma, 0.0);
ArrayInitialize(m_accBeta, 0.0);
//--- Keep a created-but-idle device copy coherent with the host step.
if(BatchOptions.GetIndex() >= 0)
ok = BatchOptions.BufferWrite() && ok;
}
feat: mini-batch gradient accumulation (F4), front-end-aware capacity budget (F6), split Wyckoff categoricals (N1) Completes the 2026-08-09 training audit. FORCES A RETRAIN of every Wyckoff-enabled config (N1 re-keys the fingerprint), and BOTH DLLs must be redeployed alongside the .ex5 - they carry new exports. F4 - mini-batch accumulation, TRAIN_BATCH_SIZE=32. Training was pure online SGD (one weight update per bar), which is the mechanical source of the era-to-era whipsaw every downstream guard was built to cope with. The O(n^2) outer product is native - AccumulateWeightGrad / AccumulateWeightGradConv / AccumulateBufferInto in Network.cl, WarriorCPU and WarriorDML - while the optimizer step is host-side MQL5 shared by all tiers (ApplyAccumToBlock), so there is one Adam/SGD implementation instead of four that can drift. - the LSTM needs no outer-product kernel (WeightsGradient already holds the sample's full dW) but could NOT simply be left un-zeroed between samples: CPU_LSTMSeqBackward/DML_LSTMSeqBackward memset it on entry. Hence a separate accumulator plus an elementwise add. - batch-norm gamma/beta accumulate in host arrays, not new BatchOptions slots - BN_OPT_STRIDE is baked into every persisted .nnw. - scoped to pass 2; online learning keeps immediate updates. Every save / checkpoint / scoring boundary flushes, scaling by the real sample count. - degrades to per-sample updates (one log line) on a tier that cannot accumulate, so old devices and DLL-free builds are unaffected. - verified offline: DirectML/batch_accum_check.cpp drives the real exports against an independent reference; at B=1 the accumulator matches the shipped unbatched kernel's own gradient to 1.1e-16. Math only - the in-situ check remains the per-layer dW/W report on a real era. F6 - ComputeFirstLayerWidth budgeted against the RAW input width even where a conv/LSTM front end had already reduced it, so an LSTM's dense stack was charged for 1,280 inputs when it receives 64. Confirmed from the deployed .cfg files: CONV, LSTM and HYBRID were all pinned at the 16-unit floor. Now budgeted against the front-end output and capped at it (never fan out), with the derivation reordered so both stages settle first. N1 - EventCode/EventPhase/StructuralPhase are signed categoricals packing direction and Wyckoff stage into one scalar across a sign discontinuity. Split into direction + [0,1] magnitude, the same convention the base OHLC block uses. Information-preserving; 13 readings now occupy 16 inputs. Compiled clean (0 errors, 0 warnings); both DLLs rebuilt. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-09 11:48:03 -04:00
}
//--- One step, so t advances once - matching every other layer's batched apply.
feat(ai): batch normalization between dense layers The only bounded stage in the entire forward path was the sigmoid classification head - every hidden stage is PRELU. That is a network with no internal scale control, and the failure ordered exactly by depth: on SP500 H1 the shallow perceptron held ~52% balanced accuracy while the deepest topology sat on the 33.3% one-class floor, with the per-bar logit spread decaying monotonically (0.45 -> 0.38 over ~200 eras) until the evidence tilt fell under the class-prior tilt. That is the signature of internal covariate shift, which chapter 6.1 of the reference book is entirely about and which the NeuroNet_DNG engine addresses with a layer this project never had. Two mechanisms make this the right fix rather than more hyperparameter nudging: - it decouples WEIGHT_DECAY from the learned function (van Laarhoven 2017) - with a normalized layer downstream, decay can no longer grind the discriminative signal away, it only rescales the effective learning rate; - it is the precondition for ever running an unbounded logit head here. The 2026-07-27 attempt blew up (IS error 5.6e15) precisely because nothing upstream constrained scale. Implementation notes: - CNeuronBatchNormOCL computes host-side rather than as a fourth copy of a kernel across Network.cl + WarriorCPU.cpp + WarriorDML.cpp. The math is elementwise O(n); this way it behaves identically on all four compute tiers, needs no DLL rebuild, and cannot drift between backends. Same precedent as the softmax+CCE gradient and the per-sample loss weighting, both computed in MQL5 for that reason. - Statistics are exponential moving, not a stored mini-batch: training is pure online SGD, one update per sample, so there is no batch to average over. BatchNormWindow is an EMA window length. - gamma/beta are excluded from weight decay, deliberately - decaying gamma toward zero is the exact pathology being fixed. - The layer self-sizes from whatever sits below it, because a conv/pool stage's output width is derived inside the CNet constructor and is not knowable to the topology builder. - Checkpoint capture/restore/blend carry gamma/beta and the running statistics alongside the dense matrix, so the plateau ladder cannot restore a mismatched pair. - SeedOutputLayerBias accepted only an exact defNeuronBaseOCL as the weight-carrying penultimate layer; with normalization enabled that is the batch-norm layer, so the cold-start bias seed would have silently stopped being applied. - Refuses to build, loudly, if a topology asks for normalization with no compute backend at all - rather than quietly training a different architecture than the one requested. EnableBatchNorm (default on) and BatchNormWindow (1000 samples) are inputs so the effect can be A/B'd without a recompile. Both feed the weights-filename fingerprint, appended conditionally so existing non-BN configs keep their fingerprints and are not forced to retrain. Verified: analytic gradients match finite differences to 1.5e-7 relative over 200 random cases; a faithful port of the full forward/backward chain collapses to the 33.3% floor by era 4 without this layer and holds 36-43% with it. Compiles 0 errors, 0 warnings. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-29 12:34:29 -04:00
if(optimization != SGD && t < INT_MAX)
t++;
feat: mini-batch gradient accumulation (F4), front-end-aware capacity budget (F6), split Wyckoff categoricals (N1) Completes the 2026-08-09 training audit. FORCES A RETRAIN of every Wyckoff-enabled config (N1 re-keys the fingerprint), and BOTH DLLs must be redeployed alongside the .ex5 - they carry new exports. F4 - mini-batch accumulation, TRAIN_BATCH_SIZE=32. Training was pure online SGD (one weight update per bar), which is the mechanical source of the era-to-era whipsaw every downstream guard was built to cope with. The O(n^2) outer product is native - AccumulateWeightGrad / AccumulateWeightGradConv / AccumulateBufferInto in Network.cl, WarriorCPU and WarriorDML - while the optimizer step is host-side MQL5 shared by all tiers (ApplyAccumToBlock), so there is one Adam/SGD implementation instead of four that can drift. - the LSTM needs no outer-product kernel (WeightsGradient already holds the sample's full dW) but could NOT simply be left un-zeroed between samples: CPU_LSTMSeqBackward/DML_LSTMSeqBackward memset it on entry. Hence a separate accumulator plus an elementwise add. - batch-norm gamma/beta accumulate in host arrays, not new BatchOptions slots - BN_OPT_STRIDE is baked into every persisted .nnw. - scoped to pass 2; online learning keeps immediate updates. Every save / checkpoint / scoring boundary flushes, scaling by the real sample count. - degrades to per-sample updates (one log line) on a tier that cannot accumulate, so old devices and DLL-free builds are unaffected. - verified offline: DirectML/batch_accum_check.cpp drives the real exports against an independent reference; at B=1 the accumulator matches the shipped unbatched kernel's own gradient to 1.1e-16. Math only - the in-situ check remains the per-layer dW/W report on a real era. F6 - ComputeFirstLayerWidth budgeted against the RAW input width even where a conv/LSTM front end had already reduced it, so an LSTM's dense stack was charged for 1,280 inputs when it receives 64. Confirmed from the deployed .cfg files: CONV, LSTM and HYBRID were all pinned at the 16-unit floor. Now budgeted against the front-end output and capped at it (never fan out), with the derivation reordered so both stages settle first. N1 - EventCode/EventPhase/StructuralPhase are signed categoricals packing direction and Wyckoff stage into one scalar across a sign discontinuity. Split into direction + [0,1] magnitude, the same convention the base OHLC block uses. Information-preserving; 13 readings now occupy 16 inputs. Compiled clean (0 errors, 0 warnings); both DLLs rebuilt. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-09 11:48:03 -04:00
return ok;
feat(ai): batch normalization between dense layers The only bounded stage in the entire forward path was the sigmoid classification head - every hidden stage is PRELU. That is a network with no internal scale control, and the failure ordered exactly by depth: on SP500 H1 the shallow perceptron held ~52% balanced accuracy while the deepest topology sat on the 33.3% one-class floor, with the per-bar logit spread decaying monotonically (0.45 -> 0.38 over ~200 eras) until the evidence tilt fell under the class-prior tilt. That is the signature of internal covariate shift, which chapter 6.1 of the reference book is entirely about and which the NeuroNet_DNG engine addresses with a layer this project never had. Two mechanisms make this the right fix rather than more hyperparameter nudging: - it decouples WEIGHT_DECAY from the learned function (van Laarhoven 2017) - with a normalized layer downstream, decay can no longer grind the discriminative signal away, it only rescales the effective learning rate; - it is the precondition for ever running an unbounded logit head here. The 2026-07-27 attempt blew up (IS error 5.6e15) precisely because nothing upstream constrained scale. Implementation notes: - CNeuronBatchNormOCL computes host-side rather than as a fourth copy of a kernel across Network.cl + WarriorCPU.cpp + WarriorDML.cpp. The math is elementwise O(n); this way it behaves identically on all four compute tiers, needs no DLL rebuild, and cannot drift between backends. Same precedent as the softmax+CCE gradient and the per-sample loss weighting, both computed in MQL5 for that reason. - Statistics are exponential moving, not a stored mini-batch: training is pure online SGD, one update per sample, so there is no batch to average over. BatchNormWindow is an EMA window length. - gamma/beta are excluded from weight decay, deliberately - decaying gamma toward zero is the exact pathology being fixed. - The layer self-sizes from whatever sits below it, because a conv/pool stage's output width is derived inside the CNet constructor and is not knowable to the topology builder. - Checkpoint capture/restore/blend carry gamma/beta and the running statistics alongside the dense matrix, so the plateau ladder cannot restore a mismatched pair. - SeedOutputLayerBias accepted only an exact defNeuronBaseOCL as the weight-carrying penultimate layer; with normalization enabled that is the batch-norm layer, so the cold-start bias seed would have silently stopped being applied. - Refuses to build, loudly, if a topology asks for normalization with no compute backend at all - rather than quietly training a different architecture than the one requested. EnableBatchNorm (default on) and BatchNormWindow (1000 samples) are inputs so the effect can be A/B'd without a recompile. Both feed the weights-filename fingerprint, appended conditionally so existing non-BN configs keep their fingerprints and are not forced to retrain. Verified: analytic gradients match finite differences to 1.5e-7 relative over 200 random cases; a faithful port of the full forward/backward chain collapses to the 33.3% floor by era 4 without this layer and holds 36-43% with it. Compiles 0 errors, 0 warnings. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-29 12:34:29 -04:00
}
//+------------------------------------------------------------------+
//| Outgoing dense weight matrix followed by the whole BatchOptions |
//| block, as one flat array - see the declaration comment. |
//+------------------------------------------------------------------+
int CNeuronBatchNormOCL::getWeightsBN(double &values[])
{
fix: live trades now use the geometry the gate certifies; perf: BN kernels Three changes, one theme: the trade placed, the trade graded, and the trade computed are now the same trade. 1) GEOMETRY WIRE (correctness, the ranked #1 open issue). The measured barrier pair reached the LABELS only - OpenParams still placed orders at the enum geometry (2*ATR/6*ATR), so the deploy gate certified "reaches 1.62*ATR before 3.33*ATR above break-even" about trades the EA never placed. Published via g_DerivedSlAtrMult/g_DerivedTpAtrMult (ConfidenceBridge, same same-tick contract as the confidence globals, because OpenParams runs on the root signal which has no pointer to the AI filter). Two writers: DeriveBarrierGeometry at era 0, and the .cfg adoption a deployed model takes. Overrides both legs and both Intelligent modes - the certificate is exact or it is nothing. TP is ATR-anchored like the label, NOT risk-relative, so a floor-widened stop cannot reshape the certified target. 2) BATCH NORM RUNS DEVICE-SIDE ON OPENCL. Four kernels in Network.cl - forward, hidden gradient, gamma/beta accumulate, gamma/beta apply - each a line-for-line transcription of the host implementation (NormalizeHost / HiddenGradHost / StepGammaBeta) including every NaN guard, clamp, and the exact moment-write ordering. The host copies remain the runtime for the DLL and pure-MQL5 tiers and the reference the kernels must match. Because this box has no OpenCL platform, the safety story is layered: - shim validation: kernels compiled as C and driven against a fp64 host transcription over NaN-poisoned stats, NaN gamma, over-clamp inputs, the frozen path, both optimizers, 3 batches - ALL PASS, worst normalized diff 0.132 vs tolerance 1.0 - in-situ self-check: each kernel is compared against its host twin ON FIRST USE on the real device (SelfCheckBn*), covering what the shim cannot - arg indices and buffer bindings. Any disagreement resyncs from the good copy, latches all BN kernels off process-wide, and training continues host-side. A transcription bug costs a warning and some speed, never a poisoned .nnw. - sync discipline: BatchOptions is now a CBufferDouble with explicit authority tracking (m_bnDeviceAuthoritative). Checkpoints/saves pull read-only; restores/loads/resets push; a mid-batch handover drains the device gamma/beta accumulator into the host arrays so no sample is lost. 3) SMALL FIXES. Apply-kernel build failure now latches the dispatch path at init (one warning instead of warning + failed Execute). Build tag bumped to win-scoring-gpu-v1 - first tag change since expectancy-stop-v1 despite five binary-changing commits. Both build variants compile 0 errors, 0 warnings. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-09 17:51:40 -04:00
//--- Checkpoints and the health report call this mid-training; in kernel mode the statistics live
//--- on the device, so pull them first. Read-only - the device stays authoritative.
SyncOptionsToHost();
feat(ai): batch normalization between dense layers The only bounded stage in the entire forward path was the sigmoid classification head - every hidden stage is PRELU. That is a network with no internal scale control, and the failure ordered exactly by depth: on SP500 H1 the shallow perceptron held ~52% balanced accuracy while the deepest topology sat on the 33.3% one-class floor, with the per-bar logit spread decaying monotonically (0.45 -> 0.38 over ~200 eras) until the evidence tilt fell under the class-prior tilt. That is the signature of internal covariate shift, which chapter 6.1 of the reference book is entirely about and which the NeuroNet_DNG engine addresses with a layer this project never had. Two mechanisms make this the right fix rather than more hyperparameter nudging: - it decouples WEIGHT_DECAY from the learned function (van Laarhoven 2017) - with a normalized layer downstream, decay can no longer grind the discriminative signal away, it only rescales the effective learning rate; - it is the precondition for ever running an unbounded logit head here. The 2026-07-27 attempt blew up (IS error 5.6e15) precisely because nothing upstream constrained scale. Implementation notes: - CNeuronBatchNormOCL computes host-side rather than as a fourth copy of a kernel across Network.cl + WarriorCPU.cpp + WarriorDML.cpp. The math is elementwise O(n); this way it behaves identically on all four compute tiers, needs no DLL rebuild, and cannot drift between backends. Same precedent as the softmax+CCE gradient and the per-sample loss weighting, both computed in MQL5 for that reason. - Statistics are exponential moving, not a stored mini-batch: training is pure online SGD, one update per sample, so there is no batch to average over. BatchNormWindow is an EMA window length. - gamma/beta are excluded from weight decay, deliberately - decaying gamma toward zero is the exact pathology being fixed. - The layer self-sizes from whatever sits below it, because a conv/pool stage's output width is derived inside the CNet constructor and is not knowable to the topology builder. - Checkpoint capture/restore/blend carry gamma/beta and the running statistics alongside the dense matrix, so the plateau ladder cannot restore a mismatched pair. - SeedOutputLayerBias accepted only an exact defNeuronBaseOCL as the weight-carrying penultimate layer; with normalization enabled that is the batch-norm layer, so the cold-start bias seed would have silently stopped being applied. - Refuses to build, loudly, if a topology asks for normalization with no compute backend at all - rather than quietly training a different architecture than the one requested. EnableBatchNorm (default on) and BatchNormWindow (1000 samples) are inputs so the effect can be A/B'd without a recompile. Both feed the weights-filename fingerprint, appended conditionally so existing non-BN configs keep their fingerprints and are not forced to retrain. Verified: analytic gradients match finite differences to 1.5e-7 relative over 200 random cases; a faithful port of the full forward/backward chain collapses to the 33.3% floor by era 4 without this layer and holds 36-43% with it. Compiles 0 errors, 0 warnings. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-29 12:34:29 -04:00
double w[];
int wCount = getWeights(w);
if(wCount < 0)
wCount = 0;
int oCount = (CheckPointer(BatchOptions) == POINTER_INVALID) ? 0 : BatchOptions.Total();
if(ArrayResize(values, wCount + oCount) != wCount + oCount)
return 0;
for(int i = 0; i < wCount; i++)
values[i] = w[i];
for(int i = 0; i < oCount; i++)
values[wCount + i] = BatchOptions.At(i);
return wCount + oCount;
}
//+------------------------------------------------------------------+
//| |
//+------------------------------------------------------------------+
bool CNeuronBatchNormOCL::setWeightsBN(double &values[])
{
int total = ArraySize(values);
int oCount = (CheckPointer(BatchOptions) == POINTER_INVALID) ? 0 : BatchOptions.Total();
int wCount = total - oCount;
if(wCount < 0)
return false; // snapshot predates this layer's parameters - refuse rather than half-restore
if(wCount > 0)
{
double w[];
if(ArrayResize(w, wCount) != wCount)
return false;
for(int i = 0; i < wCount; i++)
w[i] = values[i];
if(!setWeights(w))
return false;
}
for(int i = 0; i < oCount; i++)
if(!BatchOptions.Update(i, values[wCount + i]))
return false;
fix: live trades now use the geometry the gate certifies; perf: BN kernels Three changes, one theme: the trade placed, the trade graded, and the trade computed are now the same trade. 1) GEOMETRY WIRE (correctness, the ranked #1 open issue). The measured barrier pair reached the LABELS only - OpenParams still placed orders at the enum geometry (2*ATR/6*ATR), so the deploy gate certified "reaches 1.62*ATR before 3.33*ATR above break-even" about trades the EA never placed. Published via g_DerivedSlAtrMult/g_DerivedTpAtrMult (ConfidenceBridge, same same-tick contract as the confidence globals, because OpenParams runs on the root signal which has no pointer to the AI filter). Two writers: DeriveBarrierGeometry at era 0, and the .cfg adoption a deployed model takes. Overrides both legs and both Intelligent modes - the certificate is exact or it is nothing. TP is ATR-anchored like the label, NOT risk-relative, so a floor-widened stop cannot reshape the certified target. 2) BATCH NORM RUNS DEVICE-SIDE ON OPENCL. Four kernels in Network.cl - forward, hidden gradient, gamma/beta accumulate, gamma/beta apply - each a line-for-line transcription of the host implementation (NormalizeHost / HiddenGradHost / StepGammaBeta) including every NaN guard, clamp, and the exact moment-write ordering. The host copies remain the runtime for the DLL and pure-MQL5 tiers and the reference the kernels must match. Because this box has no OpenCL platform, the safety story is layered: - shim validation: kernels compiled as C and driven against a fp64 host transcription over NaN-poisoned stats, NaN gamma, over-clamp inputs, the frozen path, both optimizers, 3 batches - ALL PASS, worst normalized diff 0.132 vs tolerance 1.0 - in-situ self-check: each kernel is compared against its host twin ON FIRST USE on the real device (SelfCheckBn*), covering what the shim cannot - arg indices and buffer bindings. Any disagreement resyncs from the good copy, latches all BN kernels off process-wide, and training continues host-side. A transcription bug costs a warning and some speed, never a poisoned .nnw. - sync discipline: BatchOptions is now a CBufferDouble with explicit authority tracking (m_bnDeviceAuthoritative). Checkpoints/saves pull read-only; restores/loads/resets push; a mid-batch handover drains the device gamma/beta accumulator into the host arrays so no sample is lost. 3) SMALL FIXES. Apply-kernel build failure now latches the dispatch path at init (one warning instead of warning + failed Execute). Build tag bumped to win-scoring-gpu-v1 - first tag change since expectancy-stop-v1 despite five binary-changing commits. Both build variants compile 0 errors, 0 warnings. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-09 17:51:40 -04:00
//--- Full host overwrite of every slot: push it so a kernel-mode net keeps computing on the
//--- RESTORED statistics rather than the diverged ones the device still holds. After this the two
//--- copies are identical, so whichever side was authoritative remains consistent.
if(BatchOptions.GetIndex() >= 0 && !BatchOptions.BufferWrite())
return false;
feat(ai): batch normalization between dense layers The only bounded stage in the entire forward path was the sigmoid classification head - every hidden stage is PRELU. That is a network with no internal scale control, and the failure ordered exactly by depth: on SP500 H1 the shallow perceptron held ~52% balanced accuracy while the deepest topology sat on the 33.3% one-class floor, with the per-bar logit spread decaying monotonically (0.45 -> 0.38 over ~200 eras) until the evidence tilt fell under the class-prior tilt. That is the signature of internal covariate shift, which chapter 6.1 of the reference book is entirely about and which the NeuroNet_DNG engine addresses with a layer this project never had. Two mechanisms make this the right fix rather than more hyperparameter nudging: - it decouples WEIGHT_DECAY from the learned function (van Laarhoven 2017) - with a normalized layer downstream, decay can no longer grind the discriminative signal away, it only rescales the effective learning rate; - it is the precondition for ever running an unbounded logit head here. The 2026-07-27 attempt blew up (IS error 5.6e15) precisely because nothing upstream constrained scale. Implementation notes: - CNeuronBatchNormOCL computes host-side rather than as a fourth copy of a kernel across Network.cl + WarriorCPU.cpp + WarriorDML.cpp. The math is elementwise O(n); this way it behaves identically on all four compute tiers, needs no DLL rebuild, and cannot drift between backends. Same precedent as the softmax+CCE gradient and the per-sample loss weighting, both computed in MQL5 for that reason. - Statistics are exponential moving, not a stored mini-batch: training is pure online SGD, one update per sample, so there is no batch to average over. BatchNormWindow is an EMA window length. - gamma/beta are excluded from weight decay, deliberately - decaying gamma toward zero is the exact pathology being fixed. - The layer self-sizes from whatever sits below it, because a conv/pool stage's output width is derived inside the CNet constructor and is not knowable to the topology builder. - Checkpoint capture/restore/blend carry gamma/beta and the running statistics alongside the dense matrix, so the plateau ladder cannot restore a mismatched pair. - SeedOutputLayerBias accepted only an exact defNeuronBaseOCL as the weight-carrying penultimate layer; with normalization enabled that is the batch-norm layer, so the cold-start bias seed would have silently stopped being applied. - Refuses to build, loudly, if a topology asks for normalization with no compute backend at all - rather than quietly training a different architecture than the one requested. EnableBatchNorm (default on) and BatchNormWindow (1000 samples) are inputs so the effect can be A/B'd without a recompile. Both feed the weights-filename fingerprint, appended conditionally so existing non-BN configs keep their fingerprints and are not forced to retrain. Verified: analytic gradients match finite differences to 1.5e-7 relative over 200 random cases; a faithful port of the full forward/backward chain collapses to the 33.3% floor by era 4 without this layer and holds 36-43% with it. Compiles 0 errors, 0 warnings. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-29 12:34:29 -04:00
return true;
}
//+------------------------------------------------------------------+
//| |
//+------------------------------------------------------------------+
bool CNeuronBatchNormOCL::Save(const int file_handle)
{
fix: live trades now use the geometry the gate certifies; perf: BN kernels Three changes, one theme: the trade placed, the trade graded, and the trade computed are now the same trade. 1) GEOMETRY WIRE (correctness, the ranked #1 open issue). The measured barrier pair reached the LABELS only - OpenParams still placed orders at the enum geometry (2*ATR/6*ATR), so the deploy gate certified "reaches 1.62*ATR before 3.33*ATR above break-even" about trades the EA never placed. Published via g_DerivedSlAtrMult/g_DerivedTpAtrMult (ConfidenceBridge, same same-tick contract as the confidence globals, because OpenParams runs on the root signal which has no pointer to the AI filter). Two writers: DeriveBarrierGeometry at era 0, and the .cfg adoption a deployed model takes. Overrides both legs and both Intelligent modes - the certificate is exact or it is nothing. TP is ATR-anchored like the label, NOT risk-relative, so a floor-widened stop cannot reshape the certified target. 2) BATCH NORM RUNS DEVICE-SIDE ON OPENCL. Four kernels in Network.cl - forward, hidden gradient, gamma/beta accumulate, gamma/beta apply - each a line-for-line transcription of the host implementation (NormalizeHost / HiddenGradHost / StepGammaBeta) including every NaN guard, clamp, and the exact moment-write ordering. The host copies remain the runtime for the DLL and pure-MQL5 tiers and the reference the kernels must match. Because this box has no OpenCL platform, the safety story is layered: - shim validation: kernels compiled as C and driven against a fp64 host transcription over NaN-poisoned stats, NaN gamma, over-clamp inputs, the frozen path, both optimizers, 3 batches - ALL PASS, worst normalized diff 0.132 vs tolerance 1.0 - in-situ self-check: each kernel is compared against its host twin ON FIRST USE on the real device (SelfCheckBn*), covering what the shim cannot - arg indices and buffer bindings. Any disagreement resyncs from the good copy, latches all BN kernels off process-wide, and training continues host-side. A transcription bug costs a warning and some speed, never a poisoned .nnw. - sync discipline: BatchOptions is now a CBufferDouble with explicit authority tracking (m_bnDeviceAuthoritative). Checkpoints/saves pull read-only; restores/loads/resets push; a mid-batch handover drains the device gamma/beta accumulator into the host arrays so no sample is lost. 3) SMALL FIXES. Apply-kernel build failure now latches the dispatch path at init (one warning instead of warning + failed Execute). Build tag bumped to win-scoring-gpu-v1 - first tag change since expectancy-stop-v1 despite five binary-changing commits. Both build variants compile 0 errors, 0 warnings. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-09 17:51:40 -04:00
//--- The statistics ARE the layer's inference behaviour; in kernel mode their current values live
//--- on the device, and a .nnw written from the stale host mirror would be a different model.
SyncOptionsToHost();
feat(ai): batch normalization between dense layers The only bounded stage in the entire forward path was the sigmoid classification head - every hidden stage is PRELU. That is a network with no internal scale control, and the failure ordered exactly by depth: on SP500 H1 the shallow perceptron held ~52% balanced accuracy while the deepest topology sat on the 33.3% one-class floor, with the per-bar logit spread decaying monotonically (0.45 -> 0.38 over ~200 eras) until the evidence tilt fell under the class-prior tilt. That is the signature of internal covariate shift, which chapter 6.1 of the reference book is entirely about and which the NeuroNet_DNG engine addresses with a layer this project never had. Two mechanisms make this the right fix rather than more hyperparameter nudging: - it decouples WEIGHT_DECAY from the learned function (van Laarhoven 2017) - with a normalized layer downstream, decay can no longer grind the discriminative signal away, it only rescales the effective learning rate; - it is the precondition for ever running an unbounded logit head here. The 2026-07-27 attempt blew up (IS error 5.6e15) precisely because nothing upstream constrained scale. Implementation notes: - CNeuronBatchNormOCL computes host-side rather than as a fourth copy of a kernel across Network.cl + WarriorCPU.cpp + WarriorDML.cpp. The math is elementwise O(n); this way it behaves identically on all four compute tiers, needs no DLL rebuild, and cannot drift between backends. Same precedent as the softmax+CCE gradient and the per-sample loss weighting, both computed in MQL5 for that reason. - Statistics are exponential moving, not a stored mini-batch: training is pure online SGD, one update per sample, so there is no batch to average over. BatchNormWindow is an EMA window length. - gamma/beta are excluded from weight decay, deliberately - decaying gamma toward zero is the exact pathology being fixed. - The layer self-sizes from whatever sits below it, because a conv/pool stage's output width is derived inside the CNet constructor and is not knowable to the topology builder. - Checkpoint capture/restore/blend carry gamma/beta and the running statistics alongside the dense matrix, so the plateau ladder cannot restore a mismatched pair. - SeedOutputLayerBias accepted only an exact defNeuronBaseOCL as the weight-carrying penultimate layer; with normalization enabled that is the batch-norm layer, so the cold-start bias seed would have silently stopped being applied. - Refuses to build, loudly, if a topology asks for normalization with no compute backend at all - rather than quietly training a different architecture than the one requested. EnableBatchNorm (default on) and BatchNormWindow (1000 samples) are inputs so the effect can be A/B'd without a recompile. Both feed the weights-filename fingerprint, appended conditionally so existing non-BN configs keep their fingerprints and are not forced to retrain. Verified: analytic gradients match finite differences to 1.5e-7 relative over 200 random cases; a faithful port of the full forward/backward chain collapses to the 33.3% floor by era 4 without this layer and holds 36-43% with it. Compiles 0 errors, 0 warnings. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-29 12:34:29 -04:00
if(!CNeuronBaseOCL::Save(file_handle))
return false;
if(FileWriteInteger(file_handle, iBatchSize, INT_VALUE) < INT_VALUE)
return false;
if(FileWriteInteger(file_handle, iSamplesSeen, INT_VALUE) < INT_VALUE)
return false;
if(CheckPointer(BatchOptions) == POINTER_INVALID)
return false;
return BatchOptions.Save(file_handle);
}
//+------------------------------------------------------------------+
//| |
//+------------------------------------------------------------------+
bool CNeuronBatchNormOCL::Load(const int file_handle)
{
if(!CNeuronBaseOCL::Load(file_handle))
return false;
iBatchSize = FileReadInteger(file_handle, INT_VALUE);
iSamplesSeen = FileReadInteger(file_handle, INT_VALUE);
if(CheckPointer(BatchOptions) == POINTER_INVALID)
{
fix: live trades now use the geometry the gate certifies; perf: BN kernels Three changes, one theme: the trade placed, the trade graded, and the trade computed are now the same trade. 1) GEOMETRY WIRE (correctness, the ranked #1 open issue). The measured barrier pair reached the LABELS only - OpenParams still placed orders at the enum geometry (2*ATR/6*ATR), so the deploy gate certified "reaches 1.62*ATR before 3.33*ATR above break-even" about trades the EA never placed. Published via g_DerivedSlAtrMult/g_DerivedTpAtrMult (ConfidenceBridge, same same-tick contract as the confidence globals, because OpenParams runs on the root signal which has no pointer to the AI filter). Two writers: DeriveBarrierGeometry at era 0, and the .cfg adoption a deployed model takes. Overrides both legs and both Intelligent modes - the certificate is exact or it is nothing. TP is ATR-anchored like the label, NOT risk-relative, so a floor-widened stop cannot reshape the certified target. 2) BATCH NORM RUNS DEVICE-SIDE ON OPENCL. Four kernels in Network.cl - forward, hidden gradient, gamma/beta accumulate, gamma/beta apply - each a line-for-line transcription of the host implementation (NormalizeHost / HiddenGradHost / StepGammaBeta) including every NaN guard, clamp, and the exact moment-write ordering. The host copies remain the runtime for the DLL and pure-MQL5 tiers and the reference the kernels must match. Because this box has no OpenCL platform, the safety story is layered: - shim validation: kernels compiled as C and driven against a fp64 host transcription over NaN-poisoned stats, NaN gamma, over-clamp inputs, the frozen path, both optimizers, 3 batches - ALL PASS, worst normalized diff 0.132 vs tolerance 1.0 - in-situ self-check: each kernel is compared against its host twin ON FIRST USE on the real device (SelfCheckBn*), covering what the shim cannot - arg indices and buffer bindings. Any disagreement resyncs from the good copy, latches all BN kernels off process-wide, and training continues host-side. A transcription bug costs a warning and some speed, never a poisoned .nnw. - sync discipline: BatchOptions is now a CBufferDouble with explicit authority tracking (m_bnDeviceAuthoritative). Checkpoints/saves pull read-only; restores/loads/resets push; a mid-batch handover drains the device gamma/beta accumulator into the host arrays so no sample is lost. 3) SMALL FIXES. Apply-kernel build failure now latches the dispatch path at init (one warning instead of warning + failed Execute). Build tag bumped to win-scoring-gpu-v1 - first tag change since expectancy-stop-v1 despite five binary-changing commits. Both build variants compile 0 errors, 0 warnings. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-09 17:51:40 -04:00
BatchOptions = new CBufferDouble();
feat(ai): batch normalization between dense layers The only bounded stage in the entire forward path was the sigmoid classification head - every hidden stage is PRELU. That is a network with no internal scale control, and the failure ordered exactly by depth: on SP500 H1 the shallow perceptron held ~52% balanced accuracy while the deepest topology sat on the 33.3% one-class floor, with the per-bar logit spread decaying monotonically (0.45 -> 0.38 over ~200 eras) until the evidence tilt fell under the class-prior tilt. That is the signature of internal covariate shift, which chapter 6.1 of the reference book is entirely about and which the NeuroNet_DNG engine addresses with a layer this project never had. Two mechanisms make this the right fix rather than more hyperparameter nudging: - it decouples WEIGHT_DECAY from the learned function (van Laarhoven 2017) - with a normalized layer downstream, decay can no longer grind the discriminative signal away, it only rescales the effective learning rate; - it is the precondition for ever running an unbounded logit head here. The 2026-07-27 attempt blew up (IS error 5.6e15) precisely because nothing upstream constrained scale. Implementation notes: - CNeuronBatchNormOCL computes host-side rather than as a fourth copy of a kernel across Network.cl + WarriorCPU.cpp + WarriorDML.cpp. The math is elementwise O(n); this way it behaves identically on all four compute tiers, needs no DLL rebuild, and cannot drift between backends. Same precedent as the softmax+CCE gradient and the per-sample loss weighting, both computed in MQL5 for that reason. - Statistics are exponential moving, not a stored mini-batch: training is pure online SGD, one update per sample, so there is no batch to average over. BatchNormWindow is an EMA window length. - gamma/beta are excluded from weight decay, deliberately - decaying gamma toward zero is the exact pathology being fixed. - The layer self-sizes from whatever sits below it, because a conv/pool stage's output width is derived inside the CNet constructor and is not knowable to the topology builder. - Checkpoint capture/restore/blend carry gamma/beta and the running statistics alongside the dense matrix, so the plateau ladder cannot restore a mismatched pair. - SeedOutputLayerBias accepted only an exact defNeuronBaseOCL as the weight-carrying penultimate layer; with normalization enabled that is the batch-norm layer, so the cold-start bias seed would have silently stopped being applied. - Refuses to build, loudly, if a topology asks for normalization with no compute backend at all - rather than quietly training a different architecture than the one requested. EnableBatchNorm (default on) and BatchNormWindow (1000 samples) are inputs so the effect can be A/B'd without a recompile. Both feed the weights-filename fingerprint, appended conditionally so existing non-BN configs keep their fingerprints and are not forced to retrain. Verified: analytic gradients match finite differences to 1.5e-7 relative over 200 random cases; a faithful port of the full forward/backward chain collapses to the 33.3% floor by era 4 without this layer and holds 36-43% with it. Compiles 0 errors, 0 warnings. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-29 12:34:29 -04:00
if(CheckPointer(BatchOptions) == POINTER_INVALID)
return false;
}
if(!BatchOptions.Load(file_handle))
return false;
fix: live trades now use the geometry the gate certifies; perf: BN kernels Three changes, one theme: the trade placed, the trade graded, and the trade computed are now the same trade. 1) GEOMETRY WIRE (correctness, the ranked #1 open issue). The measured barrier pair reached the LABELS only - OpenParams still placed orders at the enum geometry (2*ATR/6*ATR), so the deploy gate certified "reaches 1.62*ATR before 3.33*ATR above break-even" about trades the EA never placed. Published via g_DerivedSlAtrMult/g_DerivedTpAtrMult (ConfidenceBridge, same same-tick contract as the confidence globals, because OpenParams runs on the root signal which has no pointer to the AI filter). Two writers: DeriveBarrierGeometry at era 0, and the .cfg adoption a deployed model takes. Overrides both legs and both Intelligent modes - the certificate is exact or it is nothing. TP is ATR-anchored like the label, NOT risk-relative, so a floor-widened stop cannot reshape the certified target. 2) BATCH NORM RUNS DEVICE-SIDE ON OPENCL. Four kernels in Network.cl - forward, hidden gradient, gamma/beta accumulate, gamma/beta apply - each a line-for-line transcription of the host implementation (NormalizeHost / HiddenGradHost / StepGammaBeta) including every NaN guard, clamp, and the exact moment-write ordering. The host copies remain the runtime for the DLL and pure-MQL5 tiers and the reference the kernels must match. Because this box has no OpenCL platform, the safety story is layered: - shim validation: kernels compiled as C and driven against a fp64 host transcription over NaN-poisoned stats, NaN gamma, over-clamp inputs, the frozen path, both optimizers, 3 batches - ALL PASS, worst normalized diff 0.132 vs tolerance 1.0 - in-situ self-check: each kernel is compared against its host twin ON FIRST USE on the real device (SelfCheckBn*), covering what the shim cannot - arg indices and buffer bindings. Any disagreement resyncs from the good copy, latches all BN kernels off process-wide, and training continues host-side. A transcription bug costs a warning and some speed, never a poisoned .nnw. - sync discipline: BatchOptions is now a CBufferDouble with explicit authority tracking (m_bnDeviceAuthoritative). Checkpoints/saves pull read-only; restores/loads/resets push; a mid-batch handover drains the device gamma/beta accumulator into the host arrays so no sample is lost. 3) SMALL FIXES. Apply-kernel build failure now latches the dispatch path at init (one warning instead of warning + failed Execute). Build tag bumped to win-scoring-gpu-v1 - first tag change since expectancy-stop-v1 despite five binary-changing commits. Both build variants compile 0 errors, 0 warnings. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-09 17:51:40 -04:00
//--- Loaded state is a full host overwrite - push it if a device copy already exists (a reload into
//--- a live net), and the host is authoritative until the first kernel forward.
if(BatchOptions.GetIndex() >= 0 && !BatchOptions.BufferWrite())
return false;
m_bnDeviceAuthoritative = false;
feat(ai): batch normalization between dense layers The only bounded stage in the entire forward path was the sigmoid classification head - every hidden stage is PRELU. That is a network with no internal scale control, and the failure ordered exactly by depth: on SP500 H1 the shallow perceptron held ~52% balanced accuracy while the deepest topology sat on the 33.3% one-class floor, with the per-bar logit spread decaying monotonically (0.45 -> 0.38 over ~200 eras) until the evidence tilt fell under the class-prior tilt. That is the signature of internal covariate shift, which chapter 6.1 of the reference book is entirely about and which the NeuroNet_DNG engine addresses with a layer this project never had. Two mechanisms make this the right fix rather than more hyperparameter nudging: - it decouples WEIGHT_DECAY from the learned function (van Laarhoven 2017) - with a normalized layer downstream, decay can no longer grind the discriminative signal away, it only rescales the effective learning rate; - it is the precondition for ever running an unbounded logit head here. The 2026-07-27 attempt blew up (IS error 5.6e15) precisely because nothing upstream constrained scale. Implementation notes: - CNeuronBatchNormOCL computes host-side rather than as a fourth copy of a kernel across Network.cl + WarriorCPU.cpp + WarriorDML.cpp. The math is elementwise O(n); this way it behaves identically on all four compute tiers, needs no DLL rebuild, and cannot drift between backends. Same precedent as the softmax+CCE gradient and the per-sample loss weighting, both computed in MQL5 for that reason. - Statistics are exponential moving, not a stored mini-batch: training is pure online SGD, one update per sample, so there is no batch to average over. BatchNormWindow is an EMA window length. - gamma/beta are excluded from weight decay, deliberately - decaying gamma toward zero is the exact pathology being fixed. - The layer self-sizes from whatever sits below it, because a conv/pool stage's output width is derived inside the CNet constructor and is not knowable to the topology builder. - Checkpoint capture/restore/blend carry gamma/beta and the running statistics alongside the dense matrix, so the plateau ladder cannot restore a mismatched pair. - SeedOutputLayerBias accepted only an exact defNeuronBaseOCL as the weight-carrying penultimate layer; with normalization enabled that is the batch-norm layer, so the cold-start bias seed would have silently stopped being applied. - Refuses to build, loudly, if a topology asks for normalization with no compute backend at all - rather than quietly training a different architecture than the one requested. EnableBatchNorm (default on) and BatchNormWindow (1000 samples) are inputs so the effect can be A/B'd without a recompile. Both feed the weights-filename fingerprint, appended conditionally so existing non-BN configs keep their fingerprints and are not forced to retrain. Verified: analytic gradients match finite differences to 1.5e-7 relative over 200 random cases; a faithful port of the full forward/backward chain collapses to the 33.3% floor by era 4 without this layer and holds 36-43% with it. Compiles 0 errors, 0 warnings. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-29 12:34:29 -04:00
//--- A model whose statistics block does not match its own width is not usable for inference -
//--- fail loudly here rather than index past the end on the first forward pass.
if(BatchOptions.Total() != Neurons() * BN_OPT_STRIDE)
{
Print(__FUNCTION__ + ": batch-norm parameter block is " + IntegerToString(BatchOptions.Total()) +
" values but this layer has " + IntegerToString(Neurons()) + " units (expected " +
IntegerToString(Neurons() * BN_OPT_STRIDE) + ") - file does not match the topology");
return false;
}
return true;
}
#endif // WARRIOR_NEURON_BATCHNORM_MQH
//+------------------------------------------------------------------+
Copy permalink