koros0111/Warrior_EA
1
0
Fork 0
forked from animatedread/Warrior_EA
Code Pull requests Activity
Warrior_EA/DirectML/WarriorCPU.h

106 lines
6.5 KiB
C
Raw Permalink Normal View History

feat: add max-pooling and convolution OpenCL kernels, clean up barrier and signal code - Define MAX_WEIGHT constant (1.0e6) for weight limits in clusters - Remove redundant barrier from FeedForward kernel (prevents sync issues) - Port FeedForwardProof and CalcInputGradientProof kernels for max-pooling (no weights, sliding max) - Port FeedForwardConv kernel for convolution layers (shared weights, multiple output channels) - Remove unused code and refactor signal condition logic (CSignalPAI)
2026-07-13 03:23:39 -04:00
//+------------------------------------------------------------------+
//| Warrior_EA |
//| Multithreaded CPU compute fallback - used when neither |
//| OpenCL nor the D3D12/DirectML tier are available (e.g. |
//| a VM with no GPU passthrough). Same buffer-handle model |
//| as WarriorDML.dll but full double precision throughout |
//| (no GPU float roundtrip) and work is spread across a |
//| configurable pool of worker threads instead of a device. |
//+------------------------------------------------------------------+
// Flat C ABI so MQL5 can #import this DLL directly. Function names and
// argument order/semantics mirror WarriorDML.h / AI\Network.cl 1:1 so the
// two backends are interchangeable behind CDirectMLMy in AI\Network.mqh.
#pragma once
#define WARRIORCPU_API extern "C" __declspec(dllexport)
fix: correct OpenCL gradient and weight update for classification layers - In CaclOutputGradient kernel: - Case 1 (sigmoid classification): removed erroneous multiplication by out*(1-out) which dampened gradients – the binary cross-entropy loss already cancels the sigmoid derivative, so direct (target-out) is correct. - Added default case for NONE activation (softmax classification) to compute plain error (target-out); previously unhandled, resulting in zero gradients that froze the entire network when OpenCL was active. - In UpdateWeightsMomentum and UpdateWeightsAdam kernels: added clamp to MAX_WEIGHT when updating weights to prevent gradient spikes from producing ±Infinity and subsequent NaN propagation through dense layers (e.g., classification output head).
2026-07-15 21:47:09 -04:00
// Every exported function (besides CPU_Init/CPU_GetHardwareConcurrency) takes a
// CpuHandle as its first argument - an opaque pointer to a heap-allocated,
// self-contained context (its own thread pool, buffer table and mutex) that
// CPU_Init() allocates and the caller (CDirectMLMy on the MQL5 side) is
// responsible for remembering and passing back on every subsequent call, then
// releasing via CPU_Shutdown(). No state is shared between contexts and this
// DLL keeps no global/static mutable state of its own, so it can be loaded any
// number of times and driven by any number of instances/threads in parallel -
// a fault or a wedged call against one context can never poison another
// context's calls, unlike a process-wide singleton would.
// A plain `long long` (not C++ `long`, which is only 32 bits on Windows) so it
// round-trips exactly through MQL5's 64-bit `long` on the #import side.
typedef long long CpuHandle;
// Lifecycle. CPU_Init(threads) always succeeds (returns a non-zero handle)
// since it needs no hardware - threads<=0 means "use
// std::thread::hardware_concurrency()". Returns 0 on failure.
WARRIORCPU_API CpuHandle __stdcall CPU_Init(int threads);
WARRIORCPU_API void __stdcall CPU_Shutdown(CpuHandle ctx);
WARRIORCPU_API int __stdcall CPU_GetLastError(CpuHandle ctx);
WARRIORCPU_API int __stdcall CPU_GetThreadCount(CpuHandle ctx);
// Stateless: true std::thread::hardware_concurrency(), independent of any
// context's pool size. Use this (not a CPU_Init(0)/GetThreadCount()/Shutdown()
// probe) to size a CPU_Init() request.
feat: add percentage-based CPU load input for fallback tier Replace the absolute thread count input (CpuDllThreads) with a percentage-based CPU_LOAD_PRESET enum (TargetCPULoad). This allows users to specify a percentage of detected cores to use when the CPU DLL fallback is active, improving flexibility and preventing issues when multiple instances share the same CPU DLL pool. Also adds CPU_GetHardwareConcurrency() for accurate core detection.
2026-07-14 18:04:48 -04:00
WARRIORCPU_API int __stdcall CPU_GetHardwareConcurrency();
feat: add max-pooling and convolution OpenCL kernels, clean up barrier and signal code - Define MAX_WEIGHT constant (1.0e6) for weight limits in clusters - Remove redundant barrier from FeedForward kernel (prevents sync issues) - Port FeedForwardProof and CalcInputGradientProof kernels for max-pooling (no weights, sliding max) - Port FeedForwardConv kernel for convolution layers (shared weights, multiple output channels) - Remove unused code and refactor signal condition logic (CSignalPAI)
2026-07-13 03:23:39 -04:00
fix: correct OpenCL gradient and weight update for classification layers - In CaclOutputGradient kernel: - Case 1 (sigmoid classification): removed erroneous multiplication by out*(1-out) which dampened gradients – the binary cross-entropy loss already cancels the sigmoid derivative, so direct (target-out) is correct. - Added default case for NONE activation (softmax classification) to compute plain error (target-out); previously unhandled, resulting in zero gradients that froze the entire network when OpenCL was active. - In UpdateWeightsMomentum and UpdateWeightsAdam kernels: added clamp to MAX_WEIGHT when updating weights to prevent gradient spikes from producing ±Infinity and subsequent NaN propagation through dense layers (e.g., classification output head).
2026-07-15 21:47:09 -04:00
// Buffer management. Handles are small non-negative integers scoped to ctx;
// -1 means failure. Buffers are plain double vectors owned by ctx; Write/Read
feat: add max-pooling and convolution OpenCL kernels, clean up barrier and signal code - Define MAX_WEIGHT constant (1.0e6) for weight limits in clusters - Remove redundant barrier from FeedForward kernel (prevents sync issues) - Port FeedForwardProof and CalcInputGradientProof kernels for max-pooling (no weights, sliding max) - Port FeedForwardConv kernel for convolution layers (shared weights, multiple output channels) - Remove unused code and refactor signal condition logic (CSignalPAI)
2026-07-13 03:23:39 -04:00
// just memcpy in/out, there is no upload/download step on CPU.
fix: correct OpenCL gradient and weight update for classification layers - In CaclOutputGradient kernel: - Case 1 (sigmoid classification): removed erroneous multiplication by out*(1-out) which dampened gradients – the binary cross-entropy loss already cancels the sigmoid derivative, so direct (target-out) is correct. - Added default case for NONE activation (softmax classification) to compute plain error (target-out); previously unhandled, resulting in zero gradients that froze the entire network when OpenCL was active. - In UpdateWeightsMomentum and UpdateWeightsAdam kernels: added clamp to MAX_WEIGHT when updating weights to prevent gradient spikes from producing ±Infinity and subsequent NaN propagation through dense layers (e.g., classification output head).
2026-07-15 21:47:09 -04:00
WARRIORCPU_API int __stdcall CPU_BufferCreate(CpuHandle ctx, int elementCount);
WARRIORCPU_API int __stdcall CPU_BufferWrite(CpuHandle ctx, int handle, const double *data, int count);
WARRIORCPU_API int __stdcall CPU_BufferRead(CpuHandle ctx, int handle, double *data, int count);
WARRIORCPU_API void __stdcall CPU_BufferFree(CpuHandle ctx, int handle);
feat: add max-pooling and convolution OpenCL kernels, clean up barrier and signal code - Define MAX_WEIGHT constant (1.0e6) for weight limits in clusters - Remove redundant barrier from FeedForward kernel (prevents sync issues) - Port FeedForwardProof and CalcInputGradientProof kernels for max-pooling (no weights, sliding max) - Port FeedForwardConv kernel for convolution layers (shared weights, multiple output channels) - Remove unused code and refactor signal condition logic (CSignalPAI)
2026-07-13 03:23:39 -04:00
// Compute kernels - argument order/semantics mirror AI\Network.cl and
// WarriorDML.h 1:1. activation: 0 = TANH, 1 = SIGMOID, 2 = PReLU (conv only).
fix: correct OpenCL gradient and weight update for classification layers - In CaclOutputGradient kernel: - Case 1 (sigmoid classification): removed erroneous multiplication by out*(1-out) which dampened gradients – the binary cross-entropy loss already cancels the sigmoid derivative, so direct (target-out) is correct. - Added default case for NONE activation (softmax classification) to compute plain error (target-out); previously unhandled, resulting in zero gradients that froze the entire network when OpenCL was active. - In UpdateWeightsMomentum and UpdateWeightsAdam kernels: added clamp to MAX_WEIGHT when updating weights to prevent gradient spikes from producing ±Infinity and subsequent NaN propagation through dense layers (e.g., classification output head).
2026-07-15 21:47:09 -04:00
WARRIORCPU_API int __stdcall CPU_FeedForward(CpuHandle ctx, int wHandle, int iHandle, int oHandle,
feat: add max-pooling and convolution OpenCL kernels, clean up barrier and signal code - Define MAX_WEIGHT constant (1.0e6) for weight limits in clusters - Remove redundant barrier from FeedForward kernel (prevents sync issues) - Port FeedForwardProof and CalcInputGradientProof kernels for max-pooling (no weights, sliding max) - Port FeedForwardConv kernel for convolution layers (shared weights, multiple output channels) - Remove unused code and refactor signal condition logic (CSignalPAI)
2026-07-13 03:23:39 -04:00
int inputs, int activation);
fix: correct OpenCL gradient and weight update for classification layers - In CaclOutputGradient kernel: - Case 1 (sigmoid classification): removed erroneous multiplication by out*(1-out) which dampened gradients – the binary cross-entropy loss already cancels the sigmoid derivative, so direct (target-out) is correct. - Added default case for NONE activation (softmax classification) to compute plain error (target-out); previously unhandled, resulting in zero gradients that froze the entire network when OpenCL was active. - In UpdateWeightsMomentum and UpdateWeightsAdam kernels: added clamp to MAX_WEIGHT when updating weights to prevent gradient spikes from producing ±Infinity and subsequent NaN propagation through dense layers (e.g., classification output head).
2026-07-15 21:47:09 -04:00
WARRIORCPU_API int __stdcall CPU_CalcOutputGradient(CpuHandle ctx, int tHandle, int oHandle, int igHandle,
feat: add max-pooling and convolution OpenCL kernels, clean up barrier and signal code - Define MAX_WEIGHT constant (1.0e6) for weight limits in clusters - Remove redundant barrier from FeedForward kernel (prevents sync issues) - Port FeedForwardProof and CalcInputGradientProof kernels for max-pooling (no weights, sliding max) - Port FeedForwardConv kernel for convolution layers (shared weights, multiple output channels) - Remove unused code and refactor signal condition logic (CSignalPAI)
2026-07-13 03:23:39 -04:00
int activation, int count);
fix: correct OpenCL gradient and weight update for classification layers - In CaclOutputGradient kernel: - Case 1 (sigmoid classification): removed erroneous multiplication by out*(1-out) which dampened gradients – the binary cross-entropy loss already cancels the sigmoid derivative, so direct (target-out) is correct. - Added default case for NONE activation (softmax classification) to compute plain error (target-out); previously unhandled, resulting in zero gradients that froze the entire network when OpenCL was active. - In UpdateWeightsMomentum and UpdateWeightsAdam kernels: added clamp to MAX_WEIGHT when updating weights to prevent gradient spikes from producing ±Infinity and subsequent NaN propagation through dense layers (e.g., classification output head).
2026-07-15 21:47:09 -04:00
WARRIORCPU_API int __stdcall CPU_CalcHiddenGradient(CpuHandle ctx, int wHandle, int gHandle, int oHandle, int igHandle,
feat: add max-pooling and convolution OpenCL kernels, clean up barrier and signal code - Define MAX_WEIGHT constant (1.0e6) for weight limits in clusters - Remove redundant barrier from FeedForward kernel (prevents sync issues) - Port FeedForwardProof and CalcInputGradientProof kernels for max-pooling (no weights, sliding max) - Port FeedForwardConv kernel for convolution layers (shared weights, multiple output channels) - Remove unused code and refactor signal condition logic (CSignalPAI)
2026-07-13 03:23:39 -04:00
int outputs, int activation, int count);
fix: correct OpenCL gradient and weight update for classification layers - In CaclOutputGradient kernel: - Case 1 (sigmoid classification): removed erroneous multiplication by out*(1-out) which dampened gradients – the binary cross-entropy loss already cancels the sigmoid derivative, so direct (target-out) is correct. - Added default case for NONE activation (softmax classification) to compute plain error (target-out); previously unhandled, resulting in zero gradients that froze the entire network when OpenCL was active. - In UpdateWeightsMomentum and UpdateWeightsAdam kernels: added clamp to MAX_WEIGHT when updating weights to prevent gradient spikes from producing ±Infinity and subsequent NaN propagation through dense layers (e.g., classification output head).
2026-07-15 21:47:09 -04:00
WARRIORCPU_API int __stdcall CPU_UpdateWeightsMomentum(CpuHandle ctx, int wHandle, int gHandle, int iHandle, int dwHandle,
feat: add max-pooling and convolution OpenCL kernels, clean up barrier and signal code - Define MAX_WEIGHT constant (1.0e6) for weight limits in clusters - Remove redundant barrier from FeedForward kernel (prevents sync issues) - Port FeedForwardProof and CalcInputGradientProof kernels for max-pooling (no weights, sliding max) - Port FeedForwardConv kernel for convolution layers (shared weights, multiple output channels) - Remove unused code and refactor signal condition logic (CSignalPAI)
2026-07-13 03:23:39 -04:00
int inputs, double learningRate, double momentum, int neurons);
fix: correct OpenCL gradient and weight update for classification layers - In CaclOutputGradient kernel: - Case 1 (sigmoid classification): removed erroneous multiplication by out*(1-out) which dampened gradients – the binary cross-entropy loss already cancels the sigmoid derivative, so direct (target-out) is correct. - Added default case for NONE activation (softmax classification) to compute plain error (target-out); previously unhandled, resulting in zero gradients that froze the entire network when OpenCL was active. - In UpdateWeightsMomentum and UpdateWeightsAdam kernels: added clamp to MAX_WEIGHT when updating weights to prevent gradient spikes from producing ±Infinity and subsequent NaN propagation through dense layers (e.g., classification output head).
2026-07-15 21:47:09 -04:00
WARRIORCPU_API int __stdcall CPU_UpdateWeightsAdam(CpuHandle ctx, int wHandle, int gHandle, int iHandle,
feat: add max-pooling and convolution OpenCL kernels, clean up barrier and signal code - Define MAX_WEIGHT constant (1.0e6) for weight limits in clusters - Remove redundant barrier from FeedForward kernel (prevents sync issues) - Port FeedForwardProof and CalcInputGradientProof kernels for max-pooling (no weights, sliding max) - Port FeedForwardConv kernel for convolution layers (shared weights, multiple output channels) - Remove unused code and refactor signal condition logic (CSignalPAI)
2026-07-13 03:23:39 -04:00
int mHandle, int vHandle,
int inputs, double lt, double b1, double b2, int neurons);
fix: correct OpenCL gradient and weight update for classification layers - In CaclOutputGradient kernel: - Case 1 (sigmoid classification): removed erroneous multiplication by out*(1-out) which dampened gradients – the binary cross-entropy loss already cancels the sigmoid derivative, so direct (target-out) is correct. - Added default case for NONE activation (softmax classification) to compute plain error (target-out); previously unhandled, resulting in zero gradients that froze the entire network when OpenCL was active. - In UpdateWeightsMomentum and UpdateWeightsAdam kernels: added clamp to MAX_WEIGHT when updating weights to prevent gradient spikes from producing ±Infinity and subsequent NaN propagation through dense layers (e.g., classification output head).
2026-07-15 21:47:09 -04:00
WARRIORCPU_API int __stdcall CPU_FeedForwardConv(CpuHandle ctx, int wHandle, int iHandle, int oHandle,
feat: add max-pooling and convolution OpenCL kernels, clean up barrier and signal code - Define MAX_WEIGHT constant (1.0e6) for weight limits in clusters - Remove redundant barrier from FeedForward kernel (prevents sync issues) - Port FeedForwardProof and CalcInputGradientProof kernels for max-pooling (no weights, sliding max) - Port FeedForwardConv kernel for convolution layers (shared weights, multiple output channels) - Remove unused code and refactor signal condition logic (CSignalPAI)
2026-07-13 03:23:39 -04:00
int inputs, int step, int windowIn, int windowOut, int activation, int positions);
fix: correct OpenCL gradient and weight update for classification layers - In CaclOutputGradient kernel: - Case 1 (sigmoid classification): removed erroneous multiplication by out*(1-out) which dampened gradients – the binary cross-entropy loss already cancels the sigmoid derivative, so direct (target-out) is correct. - Added default case for NONE activation (softmax classification) to compute plain error (target-out); previously unhandled, resulting in zero gradients that froze the entire network when OpenCL was active. - In UpdateWeightsMomentum and UpdateWeightsAdam kernels: added clamp to MAX_WEIGHT when updating weights to prevent gradient spikes from producing ±Infinity and subsequent NaN propagation through dense layers (e.g., classification output head).
2026-07-15 21:47:09 -04:00
WARRIORCPU_API int __stdcall CPU_CalcHiddenGradientConv(CpuHandle ctx, int wHandle, int gHandle, int oHandle, int igHandle,
feat: add max-pooling and convolution OpenCL kernels, clean up barrier and signal code - Define MAX_WEIGHT constant (1.0e6) for weight limits in clusters - Remove redundant barrier from FeedForward kernel (prevents sync issues) - Port FeedForwardProof and CalcInputGradientProof kernels for max-pooling (no weights, sliding max) - Port FeedForwardConv kernel for convolution layers (shared weights, multiple output channels) - Remove unused code and refactor signal condition logic (CSignalPAI)
2026-07-13 03:23:39 -04:00
int outputs, int step, int windowIn, int windowOut, int activation, int inputCount);
fix: correct OpenCL gradient and weight update for classification layers - In CaclOutputGradient kernel: - Case 1 (sigmoid classification): removed erroneous multiplication by out*(1-out) which dampened gradients – the binary cross-entropy loss already cancels the sigmoid derivative, so direct (target-out) is correct. - Added default case for NONE activation (softmax classification) to compute plain error (target-out); previously unhandled, resulting in zero gradients that froze the entire network when OpenCL was active. - In UpdateWeightsMomentum and UpdateWeightsAdam kernels: added clamp to MAX_WEIGHT when updating weights to prevent gradient spikes from producing ±Infinity and subsequent NaN propagation through dense layers (e.g., classification output head).
2026-07-15 21:47:09 -04:00
WARRIORCPU_API int __stdcall CPU_UpdateWeightsConvMomentum(CpuHandle ctx, int wHandle, int gHandle, int iHandle, int dwHandle,
feat: add max-pooling and convolution OpenCL kernels, clean up barrier and signal code - Define MAX_WEIGHT constant (1.0e6) for weight limits in clusters - Remove redundant barrier from FeedForward kernel (prevents sync issues) - Port FeedForwardProof and CalcInputGradientProof kernels for max-pooling (no weights, sliding max) - Port FeedForwardConv kernel for convolution layers (shared weights, multiple output channels) - Remove unused code and refactor signal condition logic (CSignalPAI)
2026-07-13 03:23:39 -04:00
int inputs, double learningRate, double momentum, int windowIn, int windowOut, int step);
fix: correct OpenCL gradient and weight update for classification layers - In CaclOutputGradient kernel: - Case 1 (sigmoid classification): removed erroneous multiplication by out*(1-out) which dampened gradients – the binary cross-entropy loss already cancels the sigmoid derivative, so direct (target-out) is correct. - Added default case for NONE activation (softmax classification) to compute plain error (target-out); previously unhandled, resulting in zero gradients that froze the entire network when OpenCL was active. - In UpdateWeightsMomentum and UpdateWeightsAdam kernels: added clamp to MAX_WEIGHT when updating weights to prevent gradient spikes from producing ±Infinity and subsequent NaN propagation through dense layers (e.g., classification output head).
2026-07-15 21:47:09 -04:00
WARRIORCPU_API int __stdcall CPU_UpdateWeightsConvAdam(CpuHandle ctx, int wHandle, int gHandle, int iHandle,
feat: add max-pooling and convolution OpenCL kernels, clean up barrier and signal code - Define MAX_WEIGHT constant (1.0e6) for weight limits in clusters - Remove redundant barrier from FeedForward kernel (prevents sync issues) - Port FeedForwardProof and CalcInputGradientProof kernels for max-pooling (no weights, sliding max) - Port FeedForwardConv kernel for convolution layers (shared weights, multiple output channels) - Remove unused code and refactor signal condition logic (CSignalPAI)
2026-07-13 03:23:39 -04:00
int mHandle, int vHandle,
int inputs, double lt, double b1, double b2, int windowIn, int windowOut, int step);
fix: correct OpenCL gradient and weight update for classification layers - In CaclOutputGradient kernel: - Case 1 (sigmoid classification): removed erroneous multiplication by out*(1-out) which dampened gradients – the binary cross-entropy loss already cancels the sigmoid derivative, so direct (target-out) is correct. - Added default case for NONE activation (softmax classification) to compute plain error (target-out); previously unhandled, resulting in zero gradients that froze the entire network when OpenCL was active. - In UpdateWeightsMomentum and UpdateWeightsAdam kernels: added clamp to MAX_WEIGHT when updating weights to prevent gradient spikes from producing ±Infinity and subsequent NaN propagation through dense layers (e.g., classification output head).
2026-07-15 21:47:09 -04:00
WARRIORCPU_API int __stdcall CPU_LSTMGates(CpuHandle ctx, int wHandle, int hiddenPrevHandle, int inputsHandle,
feat: add max-pooling and convolution OpenCL kernels, clean up barrier and signal code - Define MAX_WEIGHT constant (1.0e6) for weight limits in clusters - Remove redundant barrier from FeedForward kernel (prevents sync issues) - Port FeedForwardProof and CalcInputGradientProof kernels for max-pooling (no weights, sliding max) - Port FeedForwardConv kernel for convolution layers (shared weights, multiple output channels) - Remove unused code and refactor signal condition logic (CSignalPAI)
2026-07-13 03:23:39 -04:00
int concatenatedHandle, int hiddenSize, int inputSize);
fix: correct OpenCL gradient and weight update for classification layers - In CaclOutputGradient kernel: - Case 1 (sigmoid classification): removed erroneous multiplication by out*(1-out) which dampened gradients – the binary cross-entropy loss already cancels the sigmoid derivative, so direct (target-out) is correct. - Added default case for NONE activation (softmax classification) to compute plain error (target-out); previously unhandled, resulting in zero gradients that froze the entire network when OpenCL was active. - In UpdateWeightsMomentum and UpdateWeightsAdam kernels: added clamp to MAX_WEIGHT when updating weights to prevent gradient spikes from producing ±Infinity and subsequent NaN propagation through dense layers (e.g., classification output head).
2026-07-15 21:47:09 -04:00
WARRIORCPU_API int __stdcall CPU_LSTMState(CpuHandle ctx, int concatenatedHandle, int memoryHandle, int hiddenPrevHandle,
feat: add max-pooling and convolution OpenCL kernels, clean up barrier and signal code - Define MAX_WEIGHT constant (1.0e6) for weight limits in clusters - Remove redundant barrier from FeedForward kernel (prevents sync issues) - Port FeedForwardProof and CalcInputGradientProof kernels for max-pooling (no weights, sliding max) - Port FeedForwardConv kernel for convolution layers (shared weights, multiple output channels) - Remove unused code and refactor signal condition logic (CSignalPAI)
2026-07-13 03:23:39 -04:00
int hiddenCacheHandle, int outputHandle, int hiddenSize);
fix: correct OpenCL gradient and weight update for classification layers - In CaclOutputGradient kernel: - Case 1 (sigmoid classification): removed erroneous multiplication by out*(1-out) which dampened gradients – the binary cross-entropy loss already cancels the sigmoid derivative, so direct (target-out) is correct. - Added default case for NONE activation (softmax classification) to compute plain error (target-out); previously unhandled, resulting in zero gradients that froze the entire network when OpenCL was active. - In UpdateWeightsMomentum and UpdateWeightsAdam kernels: added clamp to MAX_WEIGHT when updating weights to prevent gradient spikes from producing ±Infinity and subsequent NaN propagation through dense layers (e.g., classification output head).
2026-07-15 21:47:09 -04:00
WARRIORCPU_API int __stdcall CPU_LSTMGateGradient(CpuHandle ctx, int gradientHandle, int memoryHandle, int concatenatedHandle,
feat: add max-pooling and convolution OpenCL kernels, clean up barrier and signal code - Define MAX_WEIGHT constant (1.0e6) for weight limits in clusters - Remove redundant barrier from FeedForward kernel (prevents sync issues) - Port FeedForwardProof and CalcInputGradientProof kernels for max-pooling (no weights, sliding max) - Port FeedForwardConv kernel for convolution layers (shared weights, multiple output channels) - Remove unused code and refactor signal condition logic (CSignalPAI)
2026-07-13 03:23:39 -04:00
int concatenatedGradientHandle, int hiddenSize);
fix: correct OpenCL gradient and weight update for classification layers - In CaclOutputGradient kernel: - Case 1 (sigmoid classification): removed erroneous multiplication by out*(1-out) which dampened gradients – the binary cross-entropy loss already cancels the sigmoid derivative, so direct (target-out) is correct. - Added default case for NONE activation (softmax classification) to compute plain error (target-out); previously unhandled, resulting in zero gradients that froze the entire network when OpenCL was active. - In UpdateWeightsMomentum and UpdateWeightsAdam kernels: added clamp to MAX_WEIGHT when updating weights to prevent gradient spikes from producing ±Infinity and subsequent NaN propagation through dense layers (e.g., classification output head).
2026-07-15 21:47:09 -04:00
WARRIORCPU_API int __stdcall CPU_LSTMWeightsGradient(CpuHandle ctx, int concatenatedGradientHandle, int hiddenCacheHandle,
feat: add max-pooling and convolution OpenCL kernels, clean up barrier and signal code - Define MAX_WEIGHT constant (1.0e6) for weight limits in clusters - Remove redundant barrier from FeedForward kernel (prevents sync issues) - Port FeedForwardProof and CalcInputGradientProof kernels for max-pooling (no weights, sliding max) - Port FeedForwardConv kernel for convolution layers (shared weights, multiple output channels) - Remove unused code and refactor signal condition logic (CSignalPAI)
2026-07-13 03:23:39 -04:00
int inputsHandle, int weightsGradientHandle, int hiddenSize, int inputSize);
fix: correct OpenCL gradient and weight update for classification layers - In CaclOutputGradient kernel: - Case 1 (sigmoid classification): removed erroneous multiplication by out*(1-out) which dampened gradients – the binary cross-entropy loss already cancels the sigmoid derivative, so direct (target-out) is correct. - Added default case for NONE activation (softmax classification) to compute plain error (target-out); previously unhandled, resulting in zero gradients that froze the entire network when OpenCL was active. - In UpdateWeightsMomentum and UpdateWeightsAdam kernels: added clamp to MAX_WEIGHT when updating weights to prevent gradient spikes from producing ±Infinity and subsequent NaN propagation through dense layers (e.g., classification output head).
2026-07-15 21:47:09 -04:00
WARRIORCPU_API int __stdcall CPU_LSTMInputsGradient(CpuHandle ctx, int concatenatedGradientHandle, int wHandle,
feat: add max-pooling and convolution OpenCL kernels, clean up barrier and signal code - Define MAX_WEIGHT constant (1.0e6) for weight limits in clusters - Remove redundant barrier from FeedForward kernel (prevents sync issues) - Port FeedForwardProof and CalcInputGradientProof kernels for max-pooling (no weights, sliding max) - Port FeedForwardConv kernel for convolution layers (shared weights, multiple output channels) - Remove unused code and refactor signal condition logic (CSignalPAI)
2026-07-13 03:23:39 -04:00
int inputsGradientHandle, int hiddenSize, int inputSize);
fix: correct OpenCL gradient and weight update for classification layers - In CaclOutputGradient kernel: - Case 1 (sigmoid classification): removed erroneous multiplication by out*(1-out) which dampened gradients – the binary cross-entropy loss already cancels the sigmoid derivative, so direct (target-out) is correct. - Added default case for NONE activation (softmax classification) to compute plain error (target-out); previously unhandled, resulting in zero gradients that froze the entire network when OpenCL was active. - In UpdateWeightsMomentum and UpdateWeightsAdam kernels: added clamp to MAX_WEIGHT when updating weights to prevent gradient spikes from producing ±Infinity and subsequent NaN propagation through dense layers (e.g., classification output head).
2026-07-15 21:47:09 -04:00
WARRIORCPU_API int __stdcall CPU_LSTMUpdateWeightsAdam(CpuHandle ctx, int wHandle, int weightsGradientHandle,
feat: add max-pooling and convolution OpenCL kernels, clean up barrier and signal code - Define MAX_WEIGHT constant (1.0e6) for weight limits in clusters - Remove redundant barrier from FeedForward kernel (prevents sync issues) - Port FeedForwardProof and CalcInputGradientProof kernels for max-pooling (no weights, sliding max) - Port FeedForwardConv kernel for convolution layers (shared weights, multiple output channels) - Remove unused code and refactor signal condition logic (CSignalPAI)
2026-07-13 03:23:39 -04:00
int mHandle, int vHandle, double l, double b1, double b2, int total);
feat: add SGD+momentum optimizer and input-driven hyperparameters Replace hardcoded lr and momentum with new input variables for Adam and SGD+momentum. Add OpenCL kernel LSTM_UpdateWeightsMomentum alongside the existing Adam kernel. Update comments and revert beta1 to book default 0.9.
2026-07-18 14:56:41 -04:00
WARRIORCPU_API int __stdcall CPU_LSTMUpdateWeightsMomentum(CpuHandle ctx, int wHandle, int weightsGradientHandle,
int dwHandle, double learningRate, double momentum, int total);
fix: correct OpenCL gradient and weight update for classification layers - In CaclOutputGradient kernel: - Case 1 (sigmoid classification): removed erroneous multiplication by out*(1-out) which dampened gradients – the binary cross-entropy loss already cancels the sigmoid derivative, so direct (target-out) is correct. - Added default case for NONE activation (softmax classification) to compute plain error (target-out); previously unhandled, resulting in zero gradients that froze the entire network when OpenCL was active. - In UpdateWeightsMomentum and UpdateWeightsAdam kernels: added clamp to MAX_WEIGHT when updating weights to prevent gradient spikes from producing ±Infinity and subsequent NaN propagation through dense layers (e.g., classification output head).
2026-07-15 21:47:09 -04:00
WARRIORCPU_API int __stdcall CPU_FeedForwardProof(CpuHandle ctx, int iHandle, int oHandle, int inputs, int window, int step, int outputs);
feat: add max-pooling and convolution OpenCL kernels, clean up barrier and signal code - Define MAX_WEIGHT constant (1.0e6) for weight limits in clusters - Remove redundant barrier from FeedForward kernel (prevents sync issues) - Port FeedForwardProof and CalcInputGradientProof kernels for max-pooling (no weights, sliding max) - Port FeedForwardConv kernel for convolution layers (shared weights, multiple output channels) - Remove unused code and refactor signal condition logic (CSignalPAI)
2026-07-13 03:23:39 -04:00
fix: correct OpenCL gradient and weight update for classification layers - In CaclOutputGradient kernel: - Case 1 (sigmoid classification): removed erroneous multiplication by out*(1-out) which dampened gradients – the binary cross-entropy loss already cancels the sigmoid derivative, so direct (target-out) is correct. - Added default case for NONE activation (softmax classification) to compute plain error (target-out); previously unhandled, resulting in zero gradients that froze the entire network when OpenCL was active. - In UpdateWeightsMomentum and UpdateWeightsAdam kernels: added clamp to MAX_WEIGHT when updating weights to prevent gradient spikes from producing ±Infinity and subsequent NaN propagation through dense layers (e.g., classification output head).
2026-07-15 21:47:09 -04:00
WARRIORCPU_API int __stdcall CPU_CalcInputGradientProof(CpuHandle ctx, int iHandle, int gHandle, int oHandle, int igHandle,
feat: add max-pooling and convolution OpenCL kernels, clean up barrier and signal code - Define MAX_WEIGHT constant (1.0e6) for weight limits in clusters - Remove redundant barrier from FeedForward kernel (prevents sync issues) - Port FeedForwardProof and CalcInputGradientProof kernels for max-pooling (no weights, sliding max) - Port FeedForwardConv kernel for convolution layers (shared weights, multiple output channels) - Remove unused code and refactor signal condition logic (CSignalPAI)
2026-07-13 03:23:39 -04:00
int outputs, int window, int step, int inputs);
Copy permalink