Rixa2020/Warrior_EA
Watch
1
0
Fork
You've already forked Warrior_EA
 0
forked from MrBaro75/Warrior_EA
Code Pull requests Activity
Warrior_EA/DirectML/WarriorCPU.h

155 lines
9.9 KiB
C++
RawPermalinkNormal ViewHistory

feat: add max-pooling and convolution OpenCL kernels, clean up barrier and signal code - Define MAX_WEIGHT constant (1.0e6) for weight limits in clusters - Remove redundant barrier from FeedForward kernel (prevents sync issues) - Port FeedForwardProof and CalcInputGradientProof kernels for max-pooling (no weights, sliding max) - Port FeedForwardConv kernel for convolution layers (shared weights, multiple output channels) - Remove unused code and refactor signal condition logic (CSignalPAI)
2026-07-13 03:23:39 -04:00
//+------------------------------------------------------------------+
//| Warrior_EA |
//| Multithreaded CPU compute fallback - used when neither |
//| OpenCL nor the D3D12/DirectML tier are available (e.g. |
//| a VM with no GPU passthrough). Same buffer-handle model |
//| as WarriorDML.dll but full double precision throughout |
//| (no GPU float roundtrip) and work is spread across a |
//| configurable pool of worker threads instead of a device. |
//+------------------------------------------------------------------+
// Flat C ABI so MQL5 can #import this DLL directly. Function names and
// argument order/semantics mirror WarriorDML.h / AI\Network.cl 1:1 so the
// two backends are interchangeable behind CDirectMLMy in AI\Network.mqh.
#pragma once
#define WARRIORCPU_API extern "C" __declspec(dllexport)
fix: correct OpenCL gradient and weight update for classification layers - In CaclOutputGradient kernel: - Case 1 (sigmoid classification): removed erroneous multiplication by out*(1-out) which dampened gradients – the binary cross-entropy loss already cancels the sigmoid derivative, so direct (target-out) is correct. - Added default case for NONE activation (softmax classification) to compute plain error (target-out); previously unhandled, resulting in zero gradients that froze the entire network when OpenCL was active. - In UpdateWeightsMomentum and UpdateWeightsAdam kernels: added clamp to MAX_WEIGHT when updating weights to prevent gradient spikes from producing ±Infinity and subsequent NaN propagation through dense layers (e.g., classification output head).
2026-07-15 21:47:09 -04:00
// Every exported function (besides CPU_Init/CPU_GetHardwareConcurrency) takes a
// CpuHandle as its first argument - an opaque pointer to a heap-allocated,
// self-contained context (its own thread pool, buffer table and mutex) that
// CPU_Init() allocates and the caller (CDirectMLMy on the MQL5 side) is
// responsible for remembering and passing back on every subsequent call, then
// releasing via CPU_Shutdown(). No state is shared between contexts and this
// DLL keeps no global/static mutable state of its own, so it can be loaded any
// number of times and driven by any number of instances/threads in parallel -
// a fault or a wedged call against one context can never poison another
// context's calls, unlike a process-wide singleton would.
// A plain `long long` (not C++ `long`, which is only 32 bits on Windows) so it
// round-trips exactly through MQL5's 64-bit `long` on the #import side.
typedef long long CpuHandle;
// Lifecycle. CPU_Init(threads) always succeeds (returns a non-zero handle)
// since it needs no hardware - threads<=0 means "use
// std::thread::hardware_concurrency()". Returns 0 on failure.
WARRIORCPU_API CpuHandle __stdcall CPU_Init(int threads);
WARRIORCPU_API void __stdcall CPU_Shutdown(CpuHandle ctx);
WARRIORCPU_API int __stdcall CPU_GetLastError(CpuHandle ctx);
WARRIORCPU_API int __stdcall CPU_GetThreadCount(CpuHandle ctx);
// Stateless: true std::thread::hardware_concurrency(), independent of any
// context's pool size. Use this (not a CPU_Init(0)/GetThreadCount()/Shutdown()
// probe) to size a CPU_Init() request.
feat: add percentage-based CPU load input for fallback tier Replace the absolute thread count input (CpuDllThreads) with a percentage-based CPU_LOAD_PRESET enum (TargetCPULoad). This allows users to specify a percentage of detected cores to use when the CPU DLL fallback is active, improving flexibility and preventing issues when multiple instances share the same CPU DLL pool. Also adds CPU_GetHardwareConcurrency() for accurate core detection.
2026-07-14 18:04:48 -04:00
WARRIORCPU_API int __stdcall CPU_GetHardwareConcurrency();
feat: add max-pooling and convolution OpenCL kernels, clean up barrier and signal code - Define MAX_WEIGHT constant (1.0e6) for weight limits in clusters - Remove redundant barrier from FeedForward kernel (prevents sync issues) - Port FeedForwardProof and CalcInputGradientProof kernels for max-pooling (no weights, sliding max) - Port FeedForwardConv kernel for convolution layers (shared weights, multiple output channels) - Remove unused code and refactor signal condition logic (CSignalPAI)
2026-07-13 03:23:39 -04:00
fix: correct OpenCL gradient and weight update for classification layers - In CaclOutputGradient kernel: - Case 1 (sigmoid classification): removed erroneous multiplication by out*(1-out) which dampened gradients – the binary cross-entropy loss already cancels the sigmoid derivative, so direct (target-out) is correct. - Added default case for NONE activation (softmax classification) to compute plain error (target-out); previously unhandled, resulting in zero gradients that froze the entire network when OpenCL was active. - In UpdateWeightsMomentum and UpdateWeightsAdam kernels: added clamp to MAX_WEIGHT when updating weights to prevent gradient spikes from producing ±Infinity and subsequent NaN propagation through dense layers (e.g., classification output head).
2026-07-15 21:47:09 -04:00
// Buffer management. Handles are small non-negative integers scoped to ctx;
// -1 means failure. Buffers are plain double vectors owned by ctx; Write/Read
feat: add max-pooling and convolution OpenCL kernels, clean up barrier and signal code - Define MAX_WEIGHT constant (1.0e6) for weight limits in clusters - Remove redundant barrier from FeedForward kernel (prevents sync issues) - Port FeedForwardProof and CalcInputGradientProof kernels for max-pooling (no weights, sliding max) - Port FeedForwardConv kernel for convolution layers (shared weights, multiple output channels) - Remove unused code and refactor signal condition logic (CSignalPAI)
2026-07-13 03:23:39 -04:00
// just memcpy in/out, there is no upload/download step on CPU.
fix: correct OpenCL gradient and weight update for classification layers - In CaclOutputGradient kernel: - Case 1 (sigmoid classification): removed erroneous multiplication by out*(1-out) which dampened gradients – the binary cross-entropy loss already cancels the sigmoid derivative, so direct (target-out) is correct. - Added default case for NONE activation (softmax classification) to compute plain error (target-out); previously unhandled, resulting in zero gradients that froze the entire network when OpenCL was active. - In UpdateWeightsMomentum and UpdateWeightsAdam kernels: added clamp to MAX_WEIGHT when updating weights to prevent gradient spikes from producing ±Infinity and subsequent NaN propagation through dense layers (e.g., classification output head).
2026-07-15 21:47:09 -04:00
WARRIORCPU_API int __stdcall CPU_BufferCreate(CpuHandle ctx, int elementCount);
WARRIORCPU_API int __stdcall CPU_BufferWrite(CpuHandle ctx, int handle, const double *data, int count);
WARRIORCPU_API int __stdcall CPU_BufferRead(CpuHandle ctx, int handle, double *data, int count);
WARRIORCPU_API void __stdcall CPU_BufferFree(CpuHandle ctx, int handle);
feat: add max-pooling and convolution OpenCL kernels, clean up barrier and signal code - Define MAX_WEIGHT constant (1.0e6) for weight limits in clusters - Remove redundant barrier from FeedForward kernel (prevents sync issues) - Port FeedForwardProof and CalcInputGradientProof kernels for max-pooling (no weights, sliding max) - Port FeedForwardConv kernel for convolution layers (shared weights, multiple output channels) - Remove unused code and refactor signal condition logic (CSignalPAI)
2026-07-13 03:23:39 -04:00
// Compute kernels - argument order/semantics mirror AI\Network.cl and
// WarriorDML.h 1:1. activation: 0 = TANH, 1 = SIGMOID, 2 = PReLU (conv only).
fix: correct OpenCL gradient and weight update for classification layers - In CaclOutputGradient kernel: - Case 1 (sigmoid classification): removed erroneous multiplication by out*(1-out) which dampened gradients – the binary cross-entropy loss already cancels the sigmoid derivative, so direct (target-out) is correct. - Added default case for NONE activation (softmax classification) to compute plain error (target-out); previously unhandled, resulting in zero gradients that froze the entire network when OpenCL was active. - In UpdateWeightsMomentum and UpdateWeightsAdam kernels: added clamp to MAX_WEIGHT when updating weights to prevent gradient spikes from producing ±Infinity and subsequent NaN propagation through dense layers (e.g., classification output head).
2026-07-15 21:47:09 -04:00
WARRIORCPU_API int __stdcall CPU_FeedForward(CpuHandle ctx, int wHandle, int iHandle, int oHandle,
feat: add max-pooling and convolution OpenCL kernels, clean up barrier and signal code - Define MAX_WEIGHT constant (1.0e6) for weight limits in clusters - Remove redundant barrier from FeedForward kernel (prevents sync issues) - Port FeedForwardProof and CalcInputGradientProof kernels for max-pooling (no weights, sliding max) - Port FeedForwardConv kernel for convolution layers (shared weights, multiple output channels) - Remove unused code and refactor signal condition logic (CSignalPAI)
2026-07-13 03:23:39 -04:00
 int inputs, int activation);
fix: correct OpenCL gradient and weight update for classification layers - In CaclOutputGradient kernel: - Case 1 (sigmoid classification): removed erroneous multiplication by out*(1-out) which dampened gradients – the binary cross-entropy loss already cancels the sigmoid derivative, so direct (target-out) is correct. - Added default case for NONE activation (softmax classification) to compute plain error (target-out); previously unhandled, resulting in zero gradients that froze the entire network when OpenCL was active. - In UpdateWeightsMomentum and UpdateWeightsAdam kernels: added clamp to MAX_WEIGHT when updating weights to prevent gradient spikes from producing ±Infinity and subsequent NaN propagation through dense layers (e.g., classification output head).
2026-07-15 21:47:09 -04:00
WARRIORCPU_API int __stdcall CPU_CalcOutputGradient(CpuHandle ctx, int tHandle, int oHandle, int igHandle,
feat: add max-pooling and convolution OpenCL kernels, clean up barrier and signal code - Define MAX_WEIGHT constant (1.0e6) for weight limits in clusters - Remove redundant barrier from FeedForward kernel (prevents sync issues) - Port FeedForwardProof and CalcInputGradientProof kernels for max-pooling (no weights, sliding max) - Port FeedForwardConv kernel for convolution layers (shared weights, multiple output channels) - Remove unused code and refactor signal condition logic (CSignalPAI)
2026-07-13 03:23:39 -04:00
 int activation, int count);
fix: correct OpenCL gradient and weight update for classification layers - In CaclOutputGradient kernel: - Case 1 (sigmoid classification): removed erroneous multiplication by out*(1-out) which dampened gradients – the binary cross-entropy loss already cancels the sigmoid derivative, so direct (target-out) is correct. - Added default case for NONE activation (softmax classification) to compute plain error (target-out); previously unhandled, resulting in zero gradients that froze the entire network when OpenCL was active. - In UpdateWeightsMomentum and UpdateWeightsAdam kernels: added clamp to MAX_WEIGHT when updating weights to prevent gradient spikes from producing ±Infinity and subsequent NaN propagation through dense layers (e.g., classification output head).
2026-07-15 21:47:09 -04:00
WARRIORCPU_API int __stdcall CPU_CalcHiddenGradient(CpuHandle ctx, int wHandle, int gHandle, int oHandle, int igHandle,
feat: add max-pooling and convolution OpenCL kernels, clean up barrier and signal code - Define MAX_WEIGHT constant (1.0e6) for weight limits in clusters - Remove redundant barrier from FeedForward kernel (prevents sync issues) - Port FeedForwardProof and CalcInputGradientProof kernels for max-pooling (no weights, sliding max) - Port FeedForwardConv kernel for convolution layers (shared weights, multiple output channels) - Remove unused code and refactor signal condition logic (CSignalPAI)
2026-07-13 03:23:39 -04:00
 int outputs, int activation, int count);
fix: correct OpenCL gradient and weight update for classification layers - In CaclOutputGradient kernel: - Case 1 (sigmoid classification): removed erroneous multiplication by out*(1-out) which dampened gradients – the binary cross-entropy loss already cancels the sigmoid derivative, so direct (target-out) is correct. - Added default case for NONE activation (softmax classification) to compute plain error (target-out); previously unhandled, resulting in zero gradients that froze the entire network when OpenCL was active. - In UpdateWeightsMomentum and UpdateWeightsAdam kernels: added clamp to MAX_WEIGHT when updating weights to prevent gradient spikes from producing ±Infinity and subsequent NaN propagation through dense layers (e.g., classification output head).
2026-07-15 21:47:09 -04:00
WARRIORCPU_API int __stdcall CPU_UpdateWeightsMomentum(CpuHandle ctx, int wHandle, int gHandle, int iHandle, int dwHandle,
feat(opencl): add feedback alignment support to weight update kernels Introduce an `optimizer` parameter to UpdateWeightsMomentum, UpdateWeightsConvMomentum, and UpdateWeightsAdam kernels. When set to a non-zero value, the gradient used for weight updates is multiplied by ±1 based on the parity of the weight index, implementing a basic feedback alignment signal for experimentation. When zero, the standard gradient is used unchanged. This allows A/B testing of alternative learning signals without modifying the rest of the training pipeline.
2026-07-28 15:01:40 -04:00
 int inputs, double learningRate, double momentumRate, int neurons, int optimizer);
feat: add max-pooling and convolution OpenCL kernels, clean up barrier and signal code - Define MAX_WEIGHT constant (1.0e6) for weight limits in clusters - Remove redundant barrier from FeedForward kernel (prevents sync issues) - Port FeedForwardProof and CalcInputGradientProof kernels for max-pooling (no weights, sliding max) - Port FeedForwardConv kernel for convolution layers (shared weights, multiple output channels) - Remove unused code and refactor signal condition logic (CSignalPAI)
2026-07-13 03:23:39 -04:00
fix: correct OpenCL gradient and weight update for classification layers - In CaclOutputGradient kernel: - Case 1 (sigmoid classification): removed erroneous multiplication by out*(1-out) which dampened gradients – the binary cross-entropy loss already cancels the sigmoid derivative, so direct (target-out) is correct. - Added default case for NONE activation (softmax classification) to compute plain error (target-out); previously unhandled, resulting in zero gradients that froze the entire network when OpenCL was active. - In UpdateWeightsMomentum and UpdateWeightsAdam kernels: added clamp to MAX_WEIGHT when updating weights to prevent gradient spikes from producing ±Infinity and subsequent NaN propagation through dense layers (e.g., classification output head).
2026-07-15 21:47:09 -04:00
WARRIORCPU_API int __stdcall CPU_UpdateWeightsAdam(CpuHandle ctx, int wHandle, int gHandle, int iHandle,
feat: add max-pooling and convolution OpenCL kernels, clean up barrier and signal code - Define MAX_WEIGHT constant (1.0e6) for weight limits in clusters - Remove redundant barrier from FeedForward kernel (prevents sync issues) - Port FeedForwardProof and CalcInputGradientProof kernels for max-pooling (no weights, sliding max) - Port FeedForwardConv kernel for convolution layers (shared weights, multiple output channels) - Remove unused code and refactor signal condition logic (CSignalPAI)
2026-07-13 03:23:39 -04:00
 int mHandle, int vHandle,
int inputs, double lt, double b1, double b2, int neurons);
feat: mini-batch gradient accumulation (F4), front-end-aware capacity budget (F6), split Wyckoff categoricals (N1) Completes the 2026-08-09 training audit. FORCES A RETRAIN of every Wyckoff-enabled config (N1 re-keys the fingerprint), and BOTH DLLs must be redeployed alongside the .ex5 - they carry new exports. F4 - mini-batch accumulation, TRAIN_BATCH_SIZE=32. Training was pure online SGD (one weight update per bar), which is the mechanical source of the era-to-era whipsaw every downstream guard was built to cope with. The O(n^2) outer product is native - AccumulateWeightGrad / AccumulateWeightGradConv / AccumulateBufferInto in Network.cl, WarriorCPU and WarriorDML - while the optimizer step is host-side MQL5 shared by all tiers (ApplyAccumToBlock), so there is one Adam/SGD implementation instead of four that can drift. - the LSTM needs no outer-product kernel (WeightsGradient already holds the sample's full dW) but could NOT simply be left un-zeroed between samples: CPU_LSTMSeqBackward/DML_LSTMSeqBackward memset it on entry. Hence a separate accumulator plus an elementwise add. - batch-norm gamma/beta accumulate in host arrays, not new BatchOptions slots - BN_OPT_STRIDE is baked into every persisted .nnw. - scoped to pass 2; online learning keeps immediate updates. Every save / checkpoint / scoring boundary flushes, scaling by the real sample count. - degrades to per-sample updates (one log line) on a tier that cannot accumulate, so old devices and DLL-free builds are unaffected. - verified offline: DirectML/batch_accum_check.cpp drives the real exports against an independent reference; at B=1 the accumulator matches the shipped unbatched kernel's own gradient to 1.1e-16. Math only - the in-situ check remains the per-layer dW/W report on a real era. F6 - ComputeFirstLayerWidth budgeted against the RAW input width even where a conv/LSTM front end had already reduced it, so an LSTM's dense stack was charged for 1,280 inputs when it receives 64. Confirmed from the deployed .cfg files: CONV, LSTM and HYBRID were all pinned at the 16-unit floor. Now budgeted against the front-end output and capped at it (never fan out), with the derivation reordered so both stages settle first. N1 - EventCode/EventPhase/StructuralPhase are signed categoricals packing direction and Wyckoff stage into one scalar across a sign discontinuity. Split into direction + [0,1] magnitude, the same convention the base OHLC block uses. Information-preserving; 13 readings now occupy 16 inputs. Compiled clean (0 errors, 0 warnings); both DLLs rebuilt. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-09 11:48:03 -04:00
// Mini-batch gradient accumulation (2026-08-09 audit, F4) - mirrors AI\Network.cl's
// AccumulateWeightGrad / AccumulateWeightGradConv. These ADD one sample's per-weight gradient into an
perf(train): DLL-side mini-batch apply + 300ms slice - the era bottleneck "Hundreds of times slower than a regular EA" decomposed into two multiplied factors, both measured: 1. THE OPTIMIZER STEP RAN IN INTERPRETED MQL5. The CPU tier shipped the F4 accumulate exports with deliberately no matching apply (WarriorCPU.h said so), so on the DLL backend - this box - every TRAIN_BATCH_SIZE=8 batch fell to the host loop in ApplyAccumToBlock: a per-weight MQL5 pass through CBufferDouble.At()/Update() plus four full weight-matrix BufferRead/Write round trips. The 2026-07-26 profile had already shown the per-sample Adam step at 81% of ALL runtime (feedForward: 8%; feature building: 0.35%) - sqrt+divide per weight vs one multiply-add; moving it into MQL5 made it worse. New CPU_ApplyAccumAdam / CPU_ApplyAccumMomentum: one element-wise ParallelFor takes the batch-mean step and zeroes the accumulator DLL-side, generic over any flat block (dense/conv/LSTM/batch-norm - all apply paths funnel through ApplyAccumToBlock, which now tries the DLL first, with the same one-warning failure latch as the OpenCL fast path). Math is the shipped step to the last clamp: sqrt-stored v, ClampDelta, AdamW decay, ClampWeight. batch_accum_check extended (check 6) and ALL PASS: apply == host reference at B=8/B=4, accumulator zeroed, and B=1 accumulate+apply == the unbatched Adam kernel BIT-EXACTLY (kernel-vs-kernel, no transcription). DLL rebuilt with the shipped /fp:fast recipe. 2. A 24% DUTY CYCLE. Train sliced 120ms per 500ms timer period (30ms/member x4), leaving the chart thread idle 76% of the time. Now 300ms total (75ms/member): ~60% duty, ~2.5x, click latency bounded at ~300ms while training runs - between the fully-reactive 120 and the documented "sticky drag" 480. DEPLOYMENT COUPLING: the new .ex5 #imports the new exports, so it will NOT LOAD against the old WarriorCPU.dll ("cannot find function"). Copy DirectML\WarriorCPU.dll into MQL5\Libraries (terminal closed) in the same step as deploying the new .ex5. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-25 19:32:15 -04:00
// accumulator buffer; the optimizer step on the batch mean is CPU_ApplyAccumAdam /
// CPU_ApplyAccumMomentum below (2026-08-25 - the original design applied it host-side in MQL5, which
// profiled at ~10x the whole forward pass and dominated every training era on the CPU tier).
feat: mini-batch gradient accumulation (F4), front-end-aware capacity budget (F6), split Wyckoff categoricals (N1) Completes the 2026-08-09 training audit. FORCES A RETRAIN of every Wyckoff-enabled config (N1 re-keys the fingerprint), and BOTH DLLs must be redeployed alongside the .ex5 - they carry new exports. F4 - mini-batch accumulation, TRAIN_BATCH_SIZE=32. Training was pure online SGD (one weight update per bar), which is the mechanical source of the era-to-era whipsaw every downstream guard was built to cope with. The O(n^2) outer product is native - AccumulateWeightGrad / AccumulateWeightGradConv / AccumulateBufferInto in Network.cl, WarriorCPU and WarriorDML - while the optimizer step is host-side MQL5 shared by all tiers (ApplyAccumToBlock), so there is one Adam/SGD implementation instead of four that can drift. - the LSTM needs no outer-product kernel (WeightsGradient already holds the sample's full dW) but could NOT simply be left un-zeroed between samples: CPU_LSTMSeqBackward/DML_LSTMSeqBackward memset it on entry. Hence a separate accumulator plus an elementwise add. - batch-norm gamma/beta accumulate in host arrays, not new BatchOptions slots - BN_OPT_STRIDE is baked into every persisted .nnw. - scoped to pass 2; online learning keeps immediate updates. Every save / checkpoint / scoring boundary flushes, scaling by the real sample count. - degrades to per-sample updates (one log line) on a tier that cannot accumulate, so old devices and DLL-free builds are unaffected. - verified offline: DirectML/batch_accum_check.cpp drives the real exports against an independent reference; at B=1 the accumulator matches the shipped unbatched kernel's own gradient to 1.1e-16. Math only - the in-situ check remains the per-layer dW/W report on a real era. F6 - ComputeFirstLayerWidth budgeted against the RAW input width even where a conv/LSTM front end had already reduced it, so an LSTM's dense stack was charged for 1,280 inputs when it receives 64. Confirmed from the deployed .cfg files: CONV, LSTM and HYBRID were all pinned at the 16-unit floor. Now budgeted against the front-end output and capped at it (never fan out), with the derivation reordered so both stages settle first. N1 - EventCode/EventPhase/StructuralPhase are signed categoricals packing direction and Wyckoff stage into one scalar across a sign discontinuity. Split into direction + [0,1] magnitude, the same convention the base OHLC block uses. Information-preserving; 13 readings now occupy 16 inputs. Compiled clean (0 errors, 0 warnings); both DLLs rebuilt. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-09 11:48:03 -04:00
WARRIORCPU_API int __stdcall CPU_AccumulateWeightGrad(CpuHandle ctx, int accHandle, int gHandle, int iHandle,
int inputs, int neurons);
WARRIORCPU_API int __stdcall CPU_AccumulateWeightGradConv(CpuHandle ctx, int accHandle, int gHandle, int iHandle,
int inputs, int windowIn, int windowOut, int step);
WARRIORCPU_API int __stdcall CPU_AccumulateBufferInto(CpuHandle ctx, int dstHandle, int srcHandle, int count);
perf(train): DLL-side mini-batch apply + 300ms slice - the era bottleneck "Hundreds of times slower than a regular EA" decomposed into two multiplied factors, both measured: 1. THE OPTIMIZER STEP RAN IN INTERPRETED MQL5. The CPU tier shipped the F4 accumulate exports with deliberately no matching apply (WarriorCPU.h said so), so on the DLL backend - this box - every TRAIN_BATCH_SIZE=8 batch fell to the host loop in ApplyAccumToBlock: a per-weight MQL5 pass through CBufferDouble.At()/Update() plus four full weight-matrix BufferRead/Write round trips. The 2026-07-26 profile had already shown the per-sample Adam step at 81% of ALL runtime (feedForward: 8%; feature building: 0.35%) - sqrt+divide per weight vs one multiply-add; moving it into MQL5 made it worse. New CPU_ApplyAccumAdam / CPU_ApplyAccumMomentum: one element-wise ParallelFor takes the batch-mean step and zeroes the accumulator DLL-side, generic over any flat block (dense/conv/LSTM/batch-norm - all apply paths funnel through ApplyAccumToBlock, which now tries the DLL first, with the same one-warning failure latch as the OpenCL fast path). Math is the shipped step to the last clamp: sqrt-stored v, ClampDelta, AdamW decay, ClampWeight. batch_accum_check extended (check 6) and ALL PASS: apply == host reference at B=8/B=4, accumulator zeroed, and B=1 accumulate+apply == the unbatched Adam kernel BIT-EXACTLY (kernel-vs-kernel, no transcription). DLL rebuilt with the shipped /fp:fast recipe. 2. A 24% DUTY CYCLE. Train sliced 120ms per 500ms timer period (30ms/member x4), leaving the chart thread idle 76% of the time. Now 300ms total (75ms/member): ~60% duty, ~2.5x, click latency bounded at ~300ms while training runs - between the fully-reactive 120 and the documented "sticky drag" 480. DEPLOYMENT COUPLING: the new .ex5 #imports the new exports, so it will NOT LOAD against the old WarriorCPU.dll ("cannot find function"). Copy DirectML\WarriorCPU.dll into MQL5\Libraries (terminal closed) in the same step as deploying the new .ex5. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-25 19:32:15 -04:00
// Mini-batch APPLY (2026-08-25): one optimizer step on the batch mean, element-wise over `total`
// weights, then zero the accumulator. Same math as CPU_UpdateWeightsAdam / the momentum kernel, so
// batch size 1 reproduces the unbatched path exactly. Generic over any flat weight block (dense,
// conv, LSTM, batch norm) - the caller passes the block's buffers, no shape needed.
WARRIORCPU_API int __stdcall CPU_ApplyAccumAdam(CpuHandle ctx, int wHandle, int accHandle, int mHandle, int vHandle,
int total, double scale, double lt, double b1, double b2);
WARRIORCPU_API int __stdcall CPU_ApplyAccumMomentum(CpuHandle ctx, int wHandle, int accHandle, int dwHandle,
int total, double scale, double learningRate, double momentum);
perf(tester,bn): no sub-second timer in the tester + BN kernels on the DLL tier THE OPTIMIZER ("0.1% an hour per agent", 0 of 39 passes in 78 min, 12 agents): the tester fires OnTimer on SIMULATED time, so the live chart's 500ms EventSetMillisecondTimer over a 2016-2026 pass is ~600 MILLION OnTimer calls - each walking 4x PollTraining, the vote readout's string build, the overlay advance and the deployed census. None of it serves an inference-only pass: training never runs, per-bar inference is driven by OnTickHandler off the tick stream, the risk budget re-checks in OnTick, and there is no chart to keep fresh. StepSetTimer now arms EventSetTimer(3600) in tester/optimizer/forward (~2,600 calls per pass) and keeps the 500ms timer for live charts. Plus a TESTER PASS SELF-PROFILE: per-tick buckets (pre / Expert.OnTick / journal) and the timer total, printed once at the pass's OnDeinit - so if a pass is still slow it names its own consumer instead of being diagnosed from outside. OFFLOAD (operator: "as much calculation as possible to DLL/OpenCL"): batch norm was the ONE stage still host-side on the DLL tier - the device path was OpenCL-only, so every sample crossed the bus twice per BN layer and normalized in interpreted MQL5 (and every model runs batchnorm ON). Four new exports mirror AI\Network.cl's BatchNorm* kernels 1:1 in DOUBLE precision (closer to the host reference than the float OpenCL kernels): forward with running stats + frozen flag, hidden gradient with the clamp derivative, gamma/beta accumulate, and the batch-mean apply (no weight decay, moments-before-skip ordering, sqrt-stored v). BnDeviceEligible/EnsureBnDeviceBuffers/all four Dispatch* now route by backend; the EXISTING in-situ self-checks (host-vs-device on the first real sample, latch-off + host fallback on mismatch) verify the DLL kernels exactly as they verified OpenCL ones. batch_accum_check regression: ALL CHECKS PASSED on the rebuilt DLL. Same deployment coupling as bd46374: the .ex5 imports the new exports - copy DirectML\WarriorCPU.dll into MQL5\Libraries (terminal closed) together with the new .ex5, and re-copy it to the tester agents (or just run DirectML\build_cpu.bat once with everything closed - it deploys to every discovered Libraries folder). Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-25 20:44:18 -04:00
// Batch norm (2026-08-25) - mirrors AI\Network.cl's BatchNorm* kernels in double precision. Until
// these existed the BN layers were the one stage that ran host-side on the DLL tier, with two bus
// crossings per layer per sample. The options layout is AI\Network.mqh's BN_OPT_* stride-9 record.
WARRIORCPU_API int __stdcall CPU_BatchNormForward(CpuHandle ctx, int iHandle, int oHandle, int optHandle,
double w, int frozen, int count);
WARRIORCPU_API int __stdcall CPU_BatchNormHiddenGrad(CpuHandle ctx, int gHandle, int prevOHandle, int prevGHandle,
int optHandle, int activation, int count);
WARRIORCPU_API int __stdcall CPU_BatchNormAccumGammaBeta(CpuHandle ctx, int gHandle, int optHandle, int accHandle, int count);
WARRIORCPU_API int __stdcall CPU_BatchNormApplyGammaBeta(CpuHandle ctx, int optHandle, int accHandle,
double scale, double lt, double b1, double b2, double lr, double momentum, int optimizer, int count);
fix: correct OpenCL gradient and weight update for classification layers - In CaclOutputGradient kernel: - Case 1 (sigmoid classification): removed erroneous multiplication by out*(1-out) which dampened gradients – the binary cross-entropy loss already cancels the sigmoid derivative, so direct (target-out) is correct. - Added default case for NONE activation (softmax classification) to compute plain error (target-out); previously unhandled, resulting in zero gradients that froze the entire network when OpenCL was active. - In UpdateWeightsMomentum and UpdateWeightsAdam kernels: added clamp to MAX_WEIGHT when updating weights to prevent gradient spikes from producing ±Infinity and subsequent NaN propagation through dense layers (e.g., classification output head).
2026-07-15 21:47:09 -04:00
WARRIORCPU_API int __stdcall CPU_FeedForwardConv(CpuHandle ctx, int wHandle, int iHandle, int oHandle,
feat: add max-pooling and convolution OpenCL kernels, clean up barrier and signal code - Define MAX_WEIGHT constant (1.0e6) for weight limits in clusters - Remove redundant barrier from FeedForward kernel (prevents sync issues) - Port FeedForwardProof and CalcInputGradientProof kernels for max-pooling (no weights, sliding max) - Port FeedForwardConv kernel for convolution layers (shared weights, multiple output channels) - Remove unused code and refactor signal condition logic (CSignalPAI)
2026-07-13 03:23:39 -04:00
 int inputs, int step, int windowIn, int windowOut, int activation, int positions);
fix: correct OpenCL gradient and weight update for classification layers - In CaclOutputGradient kernel: - Case 1 (sigmoid classification): removed erroneous multiplication by out*(1-out) which dampened gradients – the binary cross-entropy loss already cancels the sigmoid derivative, so direct (target-out) is correct. - Added default case for NONE activation (softmax classification) to compute plain error (target-out); previously unhandled, resulting in zero gradients that froze the entire network when OpenCL was active. - In UpdateWeightsMomentum and UpdateWeightsAdam kernels: added clamp to MAX_WEIGHT when updating weights to prevent gradient spikes from producing ±Infinity and subsequent NaN propagation through dense layers (e.g., classification output head).
2026-07-15 21:47:09 -04:00
WARRIORCPU_API int __stdcall CPU_CalcHiddenGradientConv(CpuHandle ctx, int wHandle, int gHandle, int oHandle, int igHandle,
feat: add max-pooling and convolution OpenCL kernels, clean up barrier and signal code - Define MAX_WEIGHT constant (1.0e6) for weight limits in clusters - Remove redundant barrier from FeedForward kernel (prevents sync issues) - Port FeedForwardProof and CalcInputGradientProof kernels for max-pooling (no weights, sliding max) - Port FeedForwardConv kernel for convolution layers (shared weights, multiple output channels) - Remove unused code and refactor signal condition logic (CSignalPAI)
2026-07-13 03:23:39 -04:00
 int outputs, int step, int windowIn, int windowOut, int activation, int inputCount);
fix: correct OpenCL gradient and weight update for classification layers - In CaclOutputGradient kernel: - Case 1 (sigmoid classification): removed erroneous multiplication by out*(1-out) which dampened gradients – the binary cross-entropy loss already cancels the sigmoid derivative, so direct (target-out) is correct. - Added default case for NONE activation (softmax classification) to compute plain error (target-out); previously unhandled, resulting in zero gradients that froze the entire network when OpenCL was active. - In UpdateWeightsMomentum and UpdateWeightsAdam kernels: added clamp to MAX_WEIGHT when updating weights to prevent gradient spikes from producing ±Infinity and subsequent NaN propagation through dense layers (e.g., classification output head).
2026-07-15 21:47:09 -04:00
WARRIORCPU_API int __stdcall CPU_UpdateWeightsConvMomentum(CpuHandle ctx, int wHandle, int gHandle, int iHandle, int dwHandle,
feat(opencl): add feedback alignment support to weight update kernels Introduce an `optimizer` parameter to UpdateWeightsMomentum, UpdateWeightsConvMomentum, and UpdateWeightsAdam kernels. When set to a non-zero value, the gradient used for weight updates is multiplied by ±1 based on the parity of the weight index, implementing a basic feedback alignment signal for experimentation. When zero, the standard gradient is used unchanged. This allows A/B testing of alternative learning signals without modifying the rest of the training pipeline.
2026-07-28 15:01:40 -04:00
 int inputs, double learningRate, double momentumRate, int windowIn, int windowOut, int step, int optimizer);
feat: add max-pooling and convolution OpenCL kernels, clean up barrier and signal code - Define MAX_WEIGHT constant (1.0e6) for weight limits in clusters - Remove redundant barrier from FeedForward kernel (prevents sync issues) - Port FeedForwardProof and CalcInputGradientProof kernels for max-pooling (no weights, sliding max) - Port FeedForwardConv kernel for convolution layers (shared weights, multiple output channels) - Remove unused code and refactor signal condition logic (CSignalPAI)
2026-07-13 03:23:39 -04:00
fix: correct OpenCL gradient and weight update for classification layers - In CaclOutputGradient kernel: - Case 1 (sigmoid classification): removed erroneous multiplication by out*(1-out) which dampened gradients – the binary cross-entropy loss already cancels the sigmoid derivative, so direct (target-out) is correct. - Added default case for NONE activation (softmax classification) to compute plain error (target-out); previously unhandled, resulting in zero gradients that froze the entire network when OpenCL was active. - In UpdateWeightsMomentum and UpdateWeightsAdam kernels: added clamp to MAX_WEIGHT when updating weights to prevent gradient spikes from producing ±Infinity and subsequent NaN propagation through dense layers (e.g., classification output head).
2026-07-15 21:47:09 -04:00
WARRIORCPU_API int __stdcall CPU_UpdateWeightsConvAdam(CpuHandle ctx, int wHandle, int gHandle, int iHandle,
feat: add max-pooling and convolution OpenCL kernels, clean up barrier and signal code - Define MAX_WEIGHT constant (1.0e6) for weight limits in clusters - Remove redundant barrier from FeedForward kernel (prevents sync issues) - Port FeedForwardProof and CalcInputGradientProof kernels for max-pooling (no weights, sliding max) - Port FeedForwardConv kernel for convolution layers (shared weights, multiple output channels) - Remove unused code and refactor signal condition logic (CSignalPAI)
2026-07-13 03:23:39 -04:00
 int mHandle, int vHandle,
int inputs, double lt, double b1, double b2, int windowIn, int windowOut, int step);
fix: correct OpenCL gradient and weight update for classification layers - In CaclOutputGradient kernel: - Case 1 (sigmoid classification): removed erroneous multiplication by out*(1-out) which dampened gradients – the binary cross-entropy loss already cancels the sigmoid derivative, so direct (target-out) is correct. - Added default case for NONE activation (softmax classification) to compute plain error (target-out); previously unhandled, resulting in zero gradients that froze the entire network when OpenCL was active. - In UpdateWeightsMomentum and UpdateWeightsAdam kernels: added clamp to MAX_WEIGHT when updating weights to prevent gradient spikes from producing ±Infinity and subsequent NaN propagation through dense layers (e.g., classification output head).
2026-07-15 21:47:09 -04:00
WARRIORCPU_API int __stdcall CPU_LSTMGates(CpuHandle ctx, int wHandle, int hiddenPrevHandle, int inputsHandle,
feat: add max-pooling and convolution OpenCL kernels, clean up barrier and signal code - Define MAX_WEIGHT constant (1.0e6) for weight limits in clusters - Remove redundant barrier from FeedForward kernel (prevents sync issues) - Port FeedForwardProof and CalcInputGradientProof kernels for max-pooling (no weights, sliding max) - Port FeedForwardConv kernel for convolution layers (shared weights, multiple output channels) - Remove unused code and refactor signal condition logic (CSignalPAI)
2026-07-13 03:23:39 -04:00
 int concatenatedHandle, int hiddenSize, int inputSize);
fix: correct OpenCL gradient and weight update for classification layers - In CaclOutputGradient kernel: - Case 1 (sigmoid classification): removed erroneous multiplication by out*(1-out) which dampened gradients – the binary cross-entropy loss already cancels the sigmoid derivative, so direct (target-out) is correct. - Added default case for NONE activation (softmax classification) to compute plain error (target-out); previously unhandled, resulting in zero gradients that froze the entire network when OpenCL was active. - In UpdateWeightsMomentum and UpdateWeightsAdam kernels: added clamp to MAX_WEIGHT when updating weights to prevent gradient spikes from producing ±Infinity and subsequent NaN propagation through dense layers (e.g., classification output head).
2026-07-15 21:47:09 -04:00
WARRIORCPU_API int __stdcall CPU_LSTMState(CpuHandle ctx, int concatenatedHandle, int memoryHandle, int hiddenPrevHandle,
feat: add max-pooling and convolution OpenCL kernels, clean up barrier and signal code - Define MAX_WEIGHT constant (1.0e6) for weight limits in clusters - Remove redundant barrier from FeedForward kernel (prevents sync issues) - Port FeedForwardProof and CalcInputGradientProof kernels for max-pooling (no weights, sliding max) - Port FeedForwardConv kernel for convolution layers (shared weights, multiple output channels) - Remove unused code and refactor signal condition logic (CSignalPAI)
2026-07-13 03:23:39 -04:00
 int hiddenCacheHandle, int outputHandle, int hiddenSize);
fix: correct OpenCL gradient and weight update for classification layers - In CaclOutputGradient kernel: - Case 1 (sigmoid classification): removed erroneous multiplication by out*(1-out) which dampened gradients – the binary cross-entropy loss already cancels the sigmoid derivative, so direct (target-out) is correct. - Added default case for NONE activation (softmax classification) to compute plain error (target-out); previously unhandled, resulting in zero gradients that froze the entire network when OpenCL was active. - In UpdateWeightsMomentum and UpdateWeightsAdam kernels: added clamp to MAX_WEIGHT when updating weights to prevent gradient spikes from producing ±Infinity and subsequent NaN propagation through dense layers (e.g., classification output head).
2026-07-15 21:47:09 -04:00
WARRIORCPU_API int __stdcall CPU_LSTMGateGradient(CpuHandle ctx, int gradientHandle, int memoryHandle, int concatenatedHandle,
feat: add max-pooling and convolution OpenCL kernels, clean up barrier and signal code - Define MAX_WEIGHT constant (1.0e6) for weight limits in clusters - Remove redundant barrier from FeedForward kernel (prevents sync issues) - Port FeedForwardProof and CalcInputGradientProof kernels for max-pooling (no weights, sliding max) - Port FeedForwardConv kernel for convolution layers (shared weights, multiple output channels) - Remove unused code and refactor signal condition logic (CSignalPAI)
2026-07-13 03:23:39 -04:00
 int concatenatedGradientHandle, int hiddenSize);
fix: correct OpenCL gradient and weight update for classification layers - In CaclOutputGradient kernel: - Case 1 (sigmoid classification): removed erroneous multiplication by out*(1-out) which dampened gradients – the binary cross-entropy loss already cancels the sigmoid derivative, so direct (target-out) is correct. - Added default case for NONE activation (softmax classification) to compute plain error (target-out); previously unhandled, resulting in zero gradients that froze the entire network when OpenCL was active. - In UpdateWeightsMomentum and UpdateWeightsAdam kernels: added clamp to MAX_WEIGHT when updating weights to prevent gradient spikes from producing ±Infinity and subsequent NaN propagation through dense layers (e.g., classification output head).
2026-07-15 21:47:09 -04:00
WARRIORCPU_API int __stdcall CPU_LSTMWeightsGradient(CpuHandle ctx, int concatenatedGradientHandle, int hiddenCacheHandle,
feat: add max-pooling and convolution OpenCL kernels, clean up barrier and signal code - Define MAX_WEIGHT constant (1.0e6) for weight limits in clusters - Remove redundant barrier from FeedForward kernel (prevents sync issues) - Port FeedForwardProof and CalcInputGradientProof kernels for max-pooling (no weights, sliding max) - Port FeedForwardConv kernel for convolution layers (shared weights, multiple output channels) - Remove unused code and refactor signal condition logic (CSignalPAI)
2026-07-13 03:23:39 -04:00
 int inputsHandle, int weightsGradientHandle, int hiddenSize, int inputSize);
fix: correct OpenCL gradient and weight update for classification layers - In CaclOutputGradient kernel: - Case 1 (sigmoid classification): removed erroneous multiplication by out*(1-out) which dampened gradients – the binary cross-entropy loss already cancels the sigmoid derivative, so direct (target-out) is correct. - Added default case for NONE activation (softmax classification) to compute plain error (target-out); previously unhandled, resulting in zero gradients that froze the entire network when OpenCL was active. - In UpdateWeightsMomentum and UpdateWeightsAdam kernels: added clamp to MAX_WEIGHT when updating weights to prevent gradient spikes from producing ±Infinity and subsequent NaN propagation through dense layers (e.g., classification output head).
2026-07-15 21:47:09 -04:00
WARRIORCPU_API int __stdcall CPU_LSTMInputsGradient(CpuHandle ctx, int concatenatedGradientHandle, int wHandle,
feat: add max-pooling and convolution OpenCL kernels, clean up barrier and signal code - Define MAX_WEIGHT constant (1.0e6) for weight limits in clusters - Remove redundant barrier from FeedForward kernel (prevents sync issues) - Port FeedForwardProof and CalcInputGradientProof kernels for max-pooling (no weights, sliding max) - Port FeedForwardConv kernel for convolution layers (shared weights, multiple output channels) - Remove unused code and refactor signal condition logic (CSignalPAI)
2026-07-13 03:23:39 -04:00
 int inputsGradientHandle, int hiddenSize, int inputSize);
feat(dll): fused sequence-LSTM kernels with real backpropagation-through-time The per-step entry points cannot express a sequence model. CPU_LSTMGates takes the ENTIRE flattened input as one timestep, and CPU_LSTMGateGradient has no parameter for dc arriving from the following step - so the recurrent gradient path does not exist and cannot be assembled from these primitives at any call pattern. The layer built on them is a gated dense layer that the class comment already described honestly: "single-timestep-truncated BPTT". Adds CPU_LSTMSeqForward / CPU_LSTMSeqBackward: the whole unrolled sequence in one call each, weights shared across timesteps, dW accumulated over all of them (the per-step CPU_LSTMWeightsGradient assigns rather than accumulates, so it could not have been reused even with the dc term). Fused rather than dispatched per step because the recurrence is sequential - T round trips would serialise T lock/dispatch pairs for a few thousand FLOPs each. h_{-1} and c_{-1} are zero per sample. The old layer carried its cell state across forward passes, so under shuffled training every sample inherited the state of an unrelated one. DirectML gets the same math host-side (readback, compute in double, upload) rather than HLSL: the recurrence needs a barrier per timestep, the GPU buffers are float and BPTT accumulation is where that hurts most, and no D3D12 device exists on this machine to test a shader against. Documented at the definition. Verified with lstm_seq_gradcheck.cpp - central-difference check of dW and dX against an asymmetric loss over the final hidden state. Max relative error 2.3e-10 on both, with a non-trivial gradient magnitude asserted so the check cannot pass on an all-zero result. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-30 18:13:50 -04:00
// Fused unrolled sequence LSTM - see the block comment above the definitions in WarriorCPU.cpp.
// These replace the per-step entry points above for sequence models: the per-step ones treat the whole
// input as ONE timestep, and CPU_LSTMGateGradient cannot accept dc from the following step, so real
// backpropagation-through-time cannot be assembled from them.
WARRIORCPU_API int __stdcall CPU_LSTMSeqForward(CpuHandle ctx, int wHandle, int inputsHandle,
int cacheGatesHandle, int cacheCellHandle, int cacheHiddenHandle, int outputHandle,
int hiddenSize, int stepInputs, int steps);
WARRIORCPU_API int __stdcall CPU_LSTMSeqBackward(CpuHandle ctx, int wHandle, int inputsHandle,
int cacheGatesHandle, int cacheCellHandle, int cacheHiddenHandle, int outGradientHandle,
int weightsGradientHandle, int inputsGradientHandle, int hiddenSize, int stepInputs, int steps);
fix: correct OpenCL gradient and weight update for classification layers - In CaclOutputGradient kernel: - Case 1 (sigmoid classification): removed erroneous multiplication by out*(1-out) which dampened gradients – the binary cross-entropy loss already cancels the sigmoid derivative, so direct (target-out) is correct. - Added default case for NONE activation (softmax classification) to compute plain error (target-out); previously unhandled, resulting in zero gradients that froze the entire network when OpenCL was active. - In UpdateWeightsMomentum and UpdateWeightsAdam kernels: added clamp to MAX_WEIGHT when updating weights to prevent gradient spikes from producing ±Infinity and subsequent NaN propagation through dense layers (e.g., classification output head).
2026-07-15 21:47:09 -04:00
WARRIORCPU_API int __stdcall CPU_LSTMUpdateWeightsAdam(CpuHandle ctx, int wHandle, int weightsGradientHandle,
feat: add max-pooling and convolution OpenCL kernels, clean up barrier and signal code - Define MAX_WEIGHT constant (1.0e6) for weight limits in clusters - Remove redundant barrier from FeedForward kernel (prevents sync issues) - Port FeedForwardProof and CalcInputGradientProof kernels for max-pooling (no weights, sliding max) - Port FeedForwardConv kernel for convolution layers (shared weights, multiple output channels) - Remove unused code and refactor signal condition logic (CSignalPAI)
2026-07-13 03:23:39 -04:00
 int mHandle, int vHandle, double l, double b1, double b2, int total);
feat: add SGD+momentum optimizer and input-driven hyperparameters Replace hardcoded lr and momentum with new input variables for Adam and SGD+momentum. Add OpenCL kernel LSTM_UpdateWeightsMomentum alongside the existing Adam kernel. Update comments and revert beta1 to book default 0.9.
2026-07-18 14:56:41 -04:00
WARRIORCPU_API int __stdcall CPU_LSTMUpdateWeightsMomentum(CpuHandle ctx, int wHandle, int weightsGradientHandle,
feat(opencl): add feedback alignment support to weight update kernels Introduce an `optimizer` parameter to UpdateWeightsMomentum, UpdateWeightsConvMomentum, and UpdateWeightsAdam kernels. When set to a non-zero value, the gradient used for weight updates is multiplied by ±1 based on the parity of the weight index, implementing a basic feedback alignment signal for experimentation. When zero, the standard gradient is used unchanged. This allows A/B testing of alternative learning signals without modifying the rest of the training pipeline.
2026-07-28 15:01:40 -04:00
 int dwHandle, double learningRate, double momentumRate, int total, int optimizer);
feat: add SGD+momentum optimizer and input-driven hyperparameters Replace hardcoded lr and momentum with new input variables for Adam and SGD+momentum. Add OpenCL kernel LSTM_UpdateWeightsMomentum alongside the existing Adam kernel. Update comments and revert beta1 to book default 0.9.
2026-07-18 14:56:41 -04:00
fix: correct OpenCL gradient and weight update for classification layers - In CaclOutputGradient kernel: - Case 1 (sigmoid classification): removed erroneous multiplication by out*(1-out) which dampened gradients – the binary cross-entropy loss already cancels the sigmoid derivative, so direct (target-out) is correct. - Added default case for NONE activation (softmax classification) to compute plain error (target-out); previously unhandled, resulting in zero gradients that froze the entire network when OpenCL was active. - In UpdateWeightsMomentum and UpdateWeightsAdam kernels: added clamp to MAX_WEIGHT when updating weights to prevent gradient spikes from producing ±Infinity and subsequent NaN propagation through dense layers (e.g., classification output head).
2026-07-15 21:47:09 -04:00
WARRIORCPU_API int __stdcall CPU_FeedForwardProof(CpuHandle ctx, int iHandle, int oHandle, int inputs, int window, int step, int outputs);
feat: add max-pooling and convolution OpenCL kernels, clean up barrier and signal code - Define MAX_WEIGHT constant (1.0e6) for weight limits in clusters - Remove redundant barrier from FeedForward kernel (prevents sync issues) - Port FeedForwardProof and CalcInputGradientProof kernels for max-pooling (no weights, sliding max) - Port FeedForwardConv kernel for convolution layers (shared weights, multiple output channels) - Remove unused code and refactor signal condition logic (CSignalPAI)
2026-07-13 03:23:39 -04:00
fix: correct OpenCL gradient and weight update for classification layers - In CaclOutputGradient kernel: - Case 1 (sigmoid classification): removed erroneous multiplication by out*(1-out) which dampened gradients – the binary cross-entropy loss already cancels the sigmoid derivative, so direct (target-out) is correct. - Added default case for NONE activation (softmax classification) to compute plain error (target-out); previously unhandled, resulting in zero gradients that froze the entire network when OpenCL was active. - In UpdateWeightsMomentum and UpdateWeightsAdam kernels: added clamp to MAX_WEIGHT when updating weights to prevent gradient spikes from producing ±Infinity and subsequent NaN propagation through dense layers (e.g., classification output head).
2026-07-15 21:47:09 -04:00
WARRIORCPU_API int __stdcall CPU_CalcInputGradientProof(CpuHandle ctx, int iHandle, int gHandle, int oHandle, int igHandle,
feat: add max-pooling and convolution OpenCL kernels, clean up barrier and signal code - Define MAX_WEIGHT constant (1.0e6) for weight limits in clusters - Remove redundant barrier from FeedForward kernel (prevents sync issues) - Port FeedForwardProof and CalcInputGradientProof kernels for max-pooling (no weights, sliding max) - Port FeedForwardConv kernel for convolution layers (shared weights, multiple output channels) - Remove unused code and refactor signal condition logic (CSignalPAI)
2026-07-13 03:23:39 -04:00
 int outputs, int window, int step, int inputs);
Copy permalink