animatedread/Warrior_EA
Watch
4
6
Fork
You've already forked Warrior_EA
 7
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions
Warrior_EA/DirectML/WarriorDML.h

137 lines
8.3 KiB
C
Raw Permalink Normal View History

feat: add max-pooling and convolution OpenCL kernels, clean up barrier and signal code - Define MAX_WEIGHT constant (1.0e6) for weight limits in clusters - Remove redundant barrier from FeedForward kernel (prevents sync issues) - Port FeedForwardProof and CalcInputGradientProof kernels for max-pooling (no weights, sliding max) - Port FeedForwardConv kernel for convolution layers (shared weights, multiple output channels) - Remove unused code and refactor signal condition logic (CSignalPAI)
2026-07-13 03:23:39 -04:00
//+------------------------------------------------------------------+
//| Warrior_EA |
//| D3D12 compute fallback for GPUs|
//| without OpenCL (DirectX12/D3D)|
//+------------------------------------------------------------------+
// Flat C ABI so MQL5 can #import this DLL directly. Every buffer is a
// plain array of 64-bit doubles on the MQL5 side; internally the GPU
// works in 32-bit floats (see WarriorDML.cpp for why) and this layer
// does the conversion, so callers never see the difference.
#pragma once
#define WARRIORDML_API extern "C" __declspec(dllexport)
fix: correct OpenCL gradient and weight update for classification layers - In CaclOutputGradient kernel: - Case 1 (sigmoid classification): removed erroneous multiplication by out*(1-out) which dampened gradients – the binary cross-entropy loss already cancels the sigmoid derivative, so direct (target-out) is correct. - Added default case for NONE activation (softmax classification) to compute plain error (target-out); previously unhandled, resulting in zero gradients that froze the entire network when OpenCL was active. - In UpdateWeightsMomentum and UpdateWeightsAdam kernels: added clamp to MAX_WEIGHT when updating weights to prevent gradient spikes from producing ±Infinity and subsequent NaN propagation through dense layers (e.g., classification output head).
2026-07-15 21:47:09 -04:00
// Every exported function (besides DML_Init) takes a DmlHandle as its first
// argument - an opaque pointer to a heap-allocated, self-contained context
// (its own D3D12 device/queue/command list/fence, compiled kernels and
// buffer table) that DML_Init() allocates and the caller (CDirectMLMy on the
// MQL5 side) is responsible for remembering and passing back on every
// subsequent call, then releasing via DML_Shutdown(). No state is shared
// between contexts and this DLL keeps no global/static mutable state of its
// own, so it can be loaded any number of times and driven by any number of
// instances/threads in parallel - a driver fault, a device-removal, or a
// wedged call against one context can never poison another context's calls,
// unlike a process-wide singleton device would.
// A plain `long long` (not C++ `long`, which is only 32 bits on Windows) so
// it round-trips exactly through MQL5's 64-bit `long` on the #import side.
typedef long long DmlHandle;
// Lifecycle. DML_Init() returns a non-zero handle on success (a usable D3D12
// device was created), 0 otherwise - callers should fall back to CPU on 0. A
// failed attempt allocates no context to hand back (there is nothing to pass
// to DML_Shutdown() or DML_GetLastError()), so the failure stage is instead
// written directly to *outError (may be NULL if the caller doesn't care) so
// the MQL5 log can show *why* (e.g. "no DX12 adapter" vs "kernel compile
// failed") without this DLL needing to keep any state alive past the failure.
// See DML_GetLastError() below for the error codes.
WARRIORDML_API DmlHandle __stdcall DML_Init(int *outError);
WARRIORDML_API void __stdcall DML_Shutdown(DmlHandle ctx);
feat: add max-pooling and convolution OpenCL kernels, clean up barrier and signal code - Define MAX_WEIGHT constant (1.0e6) for weight limits in clusters - Remove redundant barrier from FeedForward kernel (prevents sync issues) - Port FeedForwardProof and CalcInputGradientProof kernels for max-pooling (no weights, sliding max) - Port FeedForwardConv kernel for convolution layers (shared weights, multiple output channels) - Remove unused code and refactor signal condition logic (CSignalPAI)
2026-07-13 03:23:39 -04:00
// 0 = no error / not yet run
// 1 = CreateDXGIFactory1 failed
// 2 = no hardware D3D_FEATURE_LEVEL_11_0 adapter found (only software/none enumerated)
// 3 = command queue creation failed
// 4 = command allocator creation failed
// 5 = command list creation failed
// 6 = fence creation failed
// 7 = fence event creation failed
// 8 = one or more compute kernels failed to compile/create (HLSL/PSO)
fix: correct OpenCL gradient and weight update for classification layers - In CaclOutputGradient kernel: - Case 1 (sigmoid classification): removed erroneous multiplication by out*(1-out) which dampened gradients – the binary cross-entropy loss already cancels the sigmoid derivative, so direct (target-out) is correct. - Added default case for NONE activation (softmax classification) to compute plain error (target-out); previously unhandled, resulting in zero gradients that froze the entire network when OpenCL was active. - In UpdateWeightsMomentum and UpdateWeightsAdam kernels: added clamp to MAX_WEIGHT when updating weights to prevent gradient spikes from producing ±Infinity and subsequent NaN propagation through dense layers (e.g., classification output head).
2026-07-15 21:47:09 -04:00
// 9 = device lost (TDR/removal/wedge) after a successful Init, or a fault
// during a kernel dispatch
// Reads the error off a live context (e.g. after a DML_* call returns 0) -
// see DML_Init()'s outError parameter for reading an Init() failure itself.
WARRIORDML_API int __stdcall DML_GetLastError(DmlHandle ctx);
// Buffer management. Handles are small non-negative integers scoped to ctx;
// -1 means failure. Buffers are GPU-resident; Write uploads, Read reads back.
WARRIORDML_API int __stdcall DML_BufferCreate(DmlHandle ctx, int elementCount);
WARRIORDML_API int __stdcall DML_BufferWrite(DmlHandle ctx, int handle, const double *data, int count);
WARRIORDML_API int __stdcall DML_BufferRead(DmlHandle ctx, int handle, double *data, int count);
WARRIORDML_API void __stdcall DML_BufferFree(DmlHandle ctx, int handle);
feat: add max-pooling and convolution OpenCL kernels, clean up barrier and signal code - Define MAX_WEIGHT constant (1.0e6) for weight limits in clusters - Remove redundant barrier from FeedForward kernel (prevents sync issues) - Port FeedForwardProof and CalcInputGradientProof kernels for max-pooling (no weights, sliding max) - Port FeedForwardConv kernel for convolution layers (shared weights, multiple output channels) - Remove unused code and refactor signal condition logic (CSignalPAI)
2026-07-13 03:23:39 -04:00
// Compute kernels - argument order/semantics mirror AI\Network.cl 1:1.
// activation: 0 = TANH, 1 = SIGMOID.
fix: correct OpenCL gradient and weight update for classification layers - In CaclOutputGradient kernel: - Case 1 (sigmoid classification): removed erroneous multiplication by out*(1-out) which dampened gradients – the binary cross-entropy loss already cancels the sigmoid derivative, so direct (target-out) is correct. - Added default case for NONE activation (softmax classification) to compute plain error (target-out); previously unhandled, resulting in zero gradients that froze the entire network when OpenCL was active. - In UpdateWeightsMomentum and UpdateWeightsAdam kernels: added clamp to MAX_WEIGHT when updating weights to prevent gradient spikes from producing ±Infinity and subsequent NaN propagation through dense layers (e.g., classification output head).
2026-07-15 21:47:09 -04:00
WARRIORDML_API int __stdcall DML_FeedForward(DmlHandle ctx, int wHandle, int iHandle, int oHandle,
feat: add max-pooling and convolution OpenCL kernels, clean up barrier and signal code - Define MAX_WEIGHT constant (1.0e6) for weight limits in clusters - Remove redundant barrier from FeedForward kernel (prevents sync issues) - Port FeedForwardProof and CalcInputGradientProof kernels for max-pooling (no weights, sliding max) - Port FeedForwardConv kernel for convolution layers (shared weights, multiple output channels) - Remove unused code and refactor signal condition logic (CSignalPAI)
2026-07-13 03:23:39 -04:00
int inputs, int activation);
fix: correct OpenCL gradient and weight update for classification layers - In CaclOutputGradient kernel: - Case 1 (sigmoid classification): removed erroneous multiplication by out*(1-out) which dampened gradients – the binary cross-entropy loss already cancels the sigmoid derivative, so direct (target-out) is correct. - Added default case for NONE activation (softmax classification) to compute plain error (target-out); previously unhandled, resulting in zero gradients that froze the entire network when OpenCL was active. - In UpdateWeightsMomentum and UpdateWeightsAdam kernels: added clamp to MAX_WEIGHT when updating weights to prevent gradient spikes from producing ±Infinity and subsequent NaN propagation through dense layers (e.g., classification output head).
2026-07-15 21:47:09 -04:00
WARRIORDML_API int __stdcall DML_CalcOutputGradient(DmlHandle ctx, int tHandle, int oHandle, int igHandle,
feat: add max-pooling and convolution OpenCL kernels, clean up barrier and signal code - Define MAX_WEIGHT constant (1.0e6) for weight limits in clusters - Remove redundant barrier from FeedForward kernel (prevents sync issues) - Port FeedForwardProof and CalcInputGradientProof kernels for max-pooling (no weights, sliding max) - Port FeedForwardConv kernel for convolution layers (shared weights, multiple output channels) - Remove unused code and refactor signal condition logic (CSignalPAI)
2026-07-13 03:23:39 -04:00
int activation, int count);
fix: correct OpenCL gradient and weight update for classification layers - In CaclOutputGradient kernel: - Case 1 (sigmoid classification): removed erroneous multiplication by out*(1-out) which dampened gradients – the binary cross-entropy loss already cancels the sigmoid derivative, so direct (target-out) is correct. - Added default case for NONE activation (softmax classification) to compute plain error (target-out); previously unhandled, resulting in zero gradients that froze the entire network when OpenCL was active. - In UpdateWeightsMomentum and UpdateWeightsAdam kernels: added clamp to MAX_WEIGHT when updating weights to prevent gradient spikes from producing ±Infinity and subsequent NaN propagation through dense layers (e.g., classification output head).
2026-07-15 21:47:09 -04:00
WARRIORDML_API int __stdcall DML_CalcHiddenGradient(DmlHandle ctx, int wHandle, int gHandle, int oHandle, int igHandle,
feat: add max-pooling and convolution OpenCL kernels, clean up barrier and signal code - Define MAX_WEIGHT constant (1.0e6) for weight limits in clusters - Remove redundant barrier from FeedForward kernel (prevents sync issues) - Port FeedForwardProof and CalcInputGradientProof kernels for max-pooling (no weights, sliding max) - Port FeedForwardConv kernel for convolution layers (shared weights, multiple output channels) - Remove unused code and refactor signal condition logic (CSignalPAI)
2026-07-13 03:23:39 -04:00
int outputs, int activation, int count);
fix: correct OpenCL gradient and weight update for classification layers - In CaclOutputGradient kernel: - Case 1 (sigmoid classification): removed erroneous multiplication by out*(1-out) which dampened gradients – the binary cross-entropy loss already cancels the sigmoid derivative, so direct (target-out) is correct. - Added default case for NONE activation (softmax classification) to compute plain error (target-out); previously unhandled, resulting in zero gradients that froze the entire network when OpenCL was active. - In UpdateWeightsMomentum and UpdateWeightsAdam kernels: added clamp to MAX_WEIGHT when updating weights to prevent gradient spikes from producing ±Infinity and subsequent NaN propagation through dense layers (e.g., classification output head).
2026-07-15 21:47:09 -04:00
WARRIORDML_API int __stdcall DML_UpdateWeightsMomentum(DmlHandle ctx, int wHandle, int gHandle, int iHandle, int dwHandle,
feat(opencl): add feedback alignment support to weight update kernels Introduce an `optimizer` parameter to UpdateWeightsMomentum, UpdateWeightsConvMomentum, and UpdateWeightsAdam kernels. When set to a non-zero value, the gradient used for weight updates is multiplied by ±1 based on the parity of the weight index, implementing a basic feedback alignment signal for experimentation. When zero, the standard gradient is used unchanged. This allows A/B testing of alternative learning signals without modifying the rest of the training pipeline.
2026-07-28 15:01:40 -04:00
int inputs, double learningRate, double momentumRate, int neurons, int optimizer);
feat: add max-pooling and convolution OpenCL kernels, clean up barrier and signal code - Define MAX_WEIGHT constant (1.0e6) for weight limits in clusters - Remove redundant barrier from FeedForward kernel (prevents sync issues) - Port FeedForwardProof and CalcInputGradientProof kernels for max-pooling (no weights, sliding max) - Port FeedForwardConv kernel for convolution layers (shared weights, multiple output channels) - Remove unused code and refactor signal condition logic (CSignalPAI)
2026-07-13 03:23:39 -04:00
fix: correct OpenCL gradient and weight update for classification layers - In CaclOutputGradient kernel: - Case 1 (sigmoid classification): removed erroneous multiplication by out*(1-out) which dampened gradients – the binary cross-entropy loss already cancels the sigmoid derivative, so direct (target-out) is correct. - Added default case for NONE activation (softmax classification) to compute plain error (target-out); previously unhandled, resulting in zero gradients that froze the entire network when OpenCL was active. - In UpdateWeightsMomentum and UpdateWeightsAdam kernels: added clamp to MAX_WEIGHT when updating weights to prevent gradient spikes from producing ±Infinity and subsequent NaN propagation through dense layers (e.g., classification output head).
2026-07-15 21:47:09 -04:00
WARRIORDML_API int __stdcall DML_UpdateWeightsAdam(DmlHandle ctx, int wHandle, int gHandle, int iHandle,
feat: add max-pooling and convolution OpenCL kernels, clean up barrier and signal code - Define MAX_WEIGHT constant (1.0e6) for weight limits in clusters - Remove redundant barrier from FeedForward kernel (prevents sync issues) - Port FeedForwardProof and CalcInputGradientProof kernels for max-pooling (no weights, sliding max) - Port FeedForwardConv kernel for convolution layers (shared weights, multiple output channels) - Remove unused code and refactor signal condition logic (CSignalPAI)
2026-07-13 03:23:39 -04:00
int mHandle, int vHandle,
int inputs, double lt, double b1, double b2, int neurons);
// Convolution layer kernels (CNeuronConvOCL) - mirror AI\Network.cl's
// FeedForwardConv/CalcHiddenGradientConv/UpdateWeightsConv* 1:1. A single
// (windowIn+1)*windowOut weight block is shared across every sliding position.
// activation: 0 = TANH, 1 = SIGMOID, 2 = PReLU (fixed param 0.01).
fix: correct OpenCL gradient and weight update for classification layers - In CaclOutputGradient kernel: - Case 1 (sigmoid classification): removed erroneous multiplication by out*(1-out) which dampened gradients – the binary cross-entropy loss already cancels the sigmoid derivative, so direct (target-out) is correct. - Added default case for NONE activation (softmax classification) to compute plain error (target-out); previously unhandled, resulting in zero gradients that froze the entire network when OpenCL was active. - In UpdateWeightsMomentum and UpdateWeightsAdam kernels: added clamp to MAX_WEIGHT when updating weights to prevent gradient spikes from producing ±Infinity and subsequent NaN propagation through dense layers (e.g., classification output head).
2026-07-15 21:47:09 -04:00
WARRIORDML_API int __stdcall DML_FeedForwardConv(DmlHandle ctx, int wHandle, int iHandle, int oHandle,
feat: add max-pooling and convolution OpenCL kernels, clean up barrier and signal code - Define MAX_WEIGHT constant (1.0e6) for weight limits in clusters - Remove redundant barrier from FeedForward kernel (prevents sync issues) - Port FeedForwardProof and CalcInputGradientProof kernels for max-pooling (no weights, sliding max) - Port FeedForwardConv kernel for convolution layers (shared weights, multiple output channels) - Remove unused code and refactor signal condition logic (CSignalPAI)
2026-07-13 03:23:39 -04:00
int inputs, int step, int windowIn, int windowOut, int activation, int positions);
fix: correct OpenCL gradient and weight update for classification layers - In CaclOutputGradient kernel: - Case 1 (sigmoid classification): removed erroneous multiplication by out*(1-out) which dampened gradients – the binary cross-entropy loss already cancels the sigmoid derivative, so direct (target-out) is correct. - Added default case for NONE activation (softmax classification) to compute plain error (target-out); previously unhandled, resulting in zero gradients that froze the entire network when OpenCL was active. - In UpdateWeightsMomentum and UpdateWeightsAdam kernels: added clamp to MAX_WEIGHT when updating weights to prevent gradient spikes from producing ±Infinity and subsequent NaN propagation through dense layers (e.g., classification output head).
2026-07-15 21:47:09 -04:00
WARRIORDML_API int __stdcall DML_CalcHiddenGradientConv(DmlHandle ctx, int wHandle, int gHandle, int oHandle, int igHandle,
feat: add max-pooling and convolution OpenCL kernels, clean up barrier and signal code - Define MAX_WEIGHT constant (1.0e6) for weight limits in clusters - Remove redundant barrier from FeedForward kernel (prevents sync issues) - Port FeedForwardProof and CalcInputGradientProof kernels for max-pooling (no weights, sliding max) - Port FeedForwardConv kernel for convolution layers (shared weights, multiple output channels) - Remove unused code and refactor signal condition logic (CSignalPAI)
2026-07-13 03:23:39 -04:00
int outputs, int step, int windowIn, int windowOut, int activation, int inputCount);
fix: correct OpenCL gradient and weight update for classification layers - In CaclOutputGradient kernel: - Case 1 (sigmoid classification): removed erroneous multiplication by out*(1-out) which dampened gradients – the binary cross-entropy loss already cancels the sigmoid derivative, so direct (target-out) is correct. - Added default case for NONE activation (softmax classification) to compute plain error (target-out); previously unhandled, resulting in zero gradients that froze the entire network when OpenCL was active. - In UpdateWeightsMomentum and UpdateWeightsAdam kernels: added clamp to MAX_WEIGHT when updating weights to prevent gradient spikes from producing ±Infinity and subsequent NaN propagation through dense layers (e.g., classification output head).
2026-07-15 21:47:09 -04:00
WARRIORDML_API int __stdcall DML_UpdateWeightsConvMomentum(DmlHandle ctx, int wHandle, int gHandle, int iHandle, int dwHandle,
feat(opencl): add feedback alignment support to weight update kernels Introduce an `optimizer` parameter to UpdateWeightsMomentum, UpdateWeightsConvMomentum, and UpdateWeightsAdam kernels. When set to a non-zero value, the gradient used for weight updates is multiplied by ±1 based on the parity of the weight index, implementing a basic feedback alignment signal for experimentation. When zero, the standard gradient is used unchanged. This allows A/B testing of alternative learning signals without modifying the rest of the training pipeline.
2026-07-28 15:01:40 -04:00
int inputs, double learningRate, double momentumRate, int windowIn, int windowOut, int step, int optimizer);
feat: add max-pooling and convolution OpenCL kernels, clean up barrier and signal code - Define MAX_WEIGHT constant (1.0e6) for weight limits in clusters - Remove redundant barrier from FeedForward kernel (prevents sync issues) - Port FeedForwardProof and CalcInputGradientProof kernels for max-pooling (no weights, sliding max) - Port FeedForwardConv kernel for convolution layers (shared weights, multiple output channels) - Remove unused code and refactor signal condition logic (CSignalPAI)
2026-07-13 03:23:39 -04:00
fix: correct OpenCL gradient and weight update for classification layers - In CaclOutputGradient kernel: - Case 1 (sigmoid classification): removed erroneous multiplication by out*(1-out) which dampened gradients – the binary cross-entropy loss already cancels the sigmoid derivative, so direct (target-out) is correct. - Added default case for NONE activation (softmax classification) to compute plain error (target-out); previously unhandled, resulting in zero gradients that froze the entire network when OpenCL was active. - In UpdateWeightsMomentum and UpdateWeightsAdam kernels: added clamp to MAX_WEIGHT when updating weights to prevent gradient spikes from producing ±Infinity and subsequent NaN propagation through dense layers (e.g., classification output head).
2026-07-15 21:47:09 -04:00
WARRIORDML_API int __stdcall DML_UpdateWeightsConvAdam(DmlHandle ctx, int wHandle, int gHandle, int iHandle,
feat: add max-pooling and convolution OpenCL kernels, clean up barrier and signal code - Define MAX_WEIGHT constant (1.0e6) for weight limits in clusters - Remove redundant barrier from FeedForward kernel (prevents sync issues) - Port FeedForwardProof and CalcInputGradientProof kernels for max-pooling (no weights, sliding max) - Port FeedForwardConv kernel for convolution layers (shared weights, multiple output channels) - Remove unused code and refactor signal condition logic (CSignalPAI)
2026-07-13 03:23:39 -04:00
int mHandle, int vHandle,
int inputs, double lt, double b1, double b2, int windowIn, int windowOut, int step);
// LSTM layer kernels (CNeuronLSTMOCL) - single-timestep-truncated BPTT,
// Adam-only. Derived from the standard LSTM equations, NOT ported from any
// reference (see AI\Network.cl for why). Gate order: forget, input, output,
// candidate; weight layout per gate is a H*(H+I+1) block (H weights on
// h_prev, I weights on the current input, 1 bias), 4 such blocks total.
fix: correct OpenCL gradient and weight update for classification layers - In CaclOutputGradient kernel: - Case 1 (sigmoid classification): removed erroneous multiplication by out*(1-out) which dampened gradients – the binary cross-entropy loss already cancels the sigmoid derivative, so direct (target-out) is correct. - Added default case for NONE activation (softmax classification) to compute plain error (target-out); previously unhandled, resulting in zero gradients that froze the entire network when OpenCL was active. - In UpdateWeightsMomentum and UpdateWeightsAdam kernels: added clamp to MAX_WEIGHT when updating weights to prevent gradient spikes from producing ±Infinity and subsequent NaN propagation through dense layers (e.g., classification output head).
2026-07-15 21:47:09 -04:00
WARRIORDML_API int __stdcall DML_LSTMGates(DmlHandle ctx, int wHandle, int hiddenPrevHandle, int inputsHandle,
feat: add max-pooling and convolution OpenCL kernels, clean up barrier and signal code - Define MAX_WEIGHT constant (1.0e6) for weight limits in clusters - Remove redundant barrier from FeedForward kernel (prevents sync issues) - Port FeedForwardProof and CalcInputGradientProof kernels for max-pooling (no weights, sliding max) - Port FeedForwardConv kernel for convolution layers (shared weights, multiple output channels) - Remove unused code and refactor signal condition logic (CSignalPAI)
2026-07-13 03:23:39 -04:00
int concatenatedHandle, int hiddenSize, int inputSize);
fix: correct OpenCL gradient and weight update for classification layers - In CaclOutputGradient kernel: - Case 1 (sigmoid classification): removed erroneous multiplication by out*(1-out) which dampened gradients – the binary cross-entropy loss already cancels the sigmoid derivative, so direct (target-out) is correct. - Added default case for NONE activation (softmax classification) to compute plain error (target-out); previously unhandled, resulting in zero gradients that froze the entire network when OpenCL was active. - In UpdateWeightsMomentum and UpdateWeightsAdam kernels: added clamp to MAX_WEIGHT when updating weights to prevent gradient spikes from producing ±Infinity and subsequent NaN propagation through dense layers (e.g., classification output head).
2026-07-15 21:47:09 -04:00
WARRIORDML_API int __stdcall DML_LSTMState(DmlHandle ctx, int concatenatedHandle, int memoryHandle, int hiddenPrevHandle,
feat: add max-pooling and convolution OpenCL kernels, clean up barrier and signal code - Define MAX_WEIGHT constant (1.0e6) for weight limits in clusters - Remove redundant barrier from FeedForward kernel (prevents sync issues) - Port FeedForwardProof and CalcInputGradientProof kernels for max-pooling (no weights, sliding max) - Port FeedForwardConv kernel for convolution layers (shared weights, multiple output channels) - Remove unused code and refactor signal condition logic (CSignalPAI)
2026-07-13 03:23:39 -04:00
int hiddenCacheHandle, int outputHandle, int hiddenSize);
fix: correct OpenCL gradient and weight update for classification layers - In CaclOutputGradient kernel: - Case 1 (sigmoid classification): removed erroneous multiplication by out*(1-out) which dampened gradients – the binary cross-entropy loss already cancels the sigmoid derivative, so direct (target-out) is correct. - Added default case for NONE activation (softmax classification) to compute plain error (target-out); previously unhandled, resulting in zero gradients that froze the entire network when OpenCL was active. - In UpdateWeightsMomentum and UpdateWeightsAdam kernels: added clamp to MAX_WEIGHT when updating weights to prevent gradient spikes from producing ±Infinity and subsequent NaN propagation through dense layers (e.g., classification output head).
2026-07-15 21:47:09 -04:00
WARRIORDML_API int __stdcall DML_LSTMGateGradient(DmlHandle ctx, int gradientHandle, int memoryHandle, int concatenatedHandle,
feat: add max-pooling and convolution OpenCL kernels, clean up barrier and signal code - Define MAX_WEIGHT constant (1.0e6) for weight limits in clusters - Remove redundant barrier from FeedForward kernel (prevents sync issues) - Port FeedForwardProof and CalcInputGradientProof kernels for max-pooling (no weights, sliding max) - Port FeedForwardConv kernel for convolution layers (shared weights, multiple output channels) - Remove unused code and refactor signal condition logic (CSignalPAI)
2026-07-13 03:23:39 -04:00
int concatenatedGradientHandle, int hiddenSize);
fix: correct OpenCL gradient and weight update for classification layers - In CaclOutputGradient kernel: - Case 1 (sigmoid classification): removed erroneous multiplication by out*(1-out) which dampened gradients – the binary cross-entropy loss already cancels the sigmoid derivative, so direct (target-out) is correct. - Added default case for NONE activation (softmax classification) to compute plain error (target-out); previously unhandled, resulting in zero gradients that froze the entire network when OpenCL was active. - In UpdateWeightsMomentum and UpdateWeightsAdam kernels: added clamp to MAX_WEIGHT when updating weights to prevent gradient spikes from producing ±Infinity and subsequent NaN propagation through dense layers (e.g., classification output head).
2026-07-15 21:47:09 -04:00
WARRIORDML_API int __stdcall DML_LSTMWeightsGradient(DmlHandle ctx, int concatenatedGradientHandle, int hiddenCacheHandle,
feat: add max-pooling and convolution OpenCL kernels, clean up barrier and signal code - Define MAX_WEIGHT constant (1.0e6) for weight limits in clusters - Remove redundant barrier from FeedForward kernel (prevents sync issues) - Port FeedForwardProof and CalcInputGradientProof kernels for max-pooling (no weights, sliding max) - Port FeedForwardConv kernel for convolution layers (shared weights, multiple output channels) - Remove unused code and refactor signal condition logic (CSignalPAI)
2026-07-13 03:23:39 -04:00
int inputsHandle, int weightsGradientHandle, int hiddenSize, int inputSize);
fix: correct OpenCL gradient and weight update for classification layers - In CaclOutputGradient kernel: - Case 1 (sigmoid classification): removed erroneous multiplication by out*(1-out) which dampened gradients – the binary cross-entropy loss already cancels the sigmoid derivative, so direct (target-out) is correct. - Added default case for NONE activation (softmax classification) to compute plain error (target-out); previously unhandled, resulting in zero gradients that froze the entire network when OpenCL was active. - In UpdateWeightsMomentum and UpdateWeightsAdam kernels: added clamp to MAX_WEIGHT when updating weights to prevent gradient spikes from producing ±Infinity and subsequent NaN propagation through dense layers (e.g., classification output head).
2026-07-15 21:47:09 -04:00
WARRIORDML_API int __stdcall DML_LSTMInputsGradient(DmlHandle ctx, int concatenatedGradientHandle, int wHandle,
feat: add max-pooling and convolution OpenCL kernels, clean up barrier and signal code - Define MAX_WEIGHT constant (1.0e6) for weight limits in clusters - Remove redundant barrier from FeedForward kernel (prevents sync issues) - Port FeedForwardProof and CalcInputGradientProof kernels for max-pooling (no weights, sliding max) - Port FeedForwardConv kernel for convolution layers (shared weights, multiple output channels) - Remove unused code and refactor signal condition logic (CSignalPAI)
2026-07-13 03:23:39 -04:00
int inputsGradientHandle, int hiddenSize, int inputSize);
feat(dll): fused sequence-LSTM kernels with real backpropagation-through-time The per-step entry points cannot express a sequence model. CPU_LSTMGates takes the ENTIRE flattened input as one timestep, and CPU_LSTMGateGradient has no parameter for dc arriving from the following step - so the recurrent gradient path does not exist and cannot be assembled from these primitives at any call pattern. The layer built on them is a gated dense layer that the class comment already described honestly: "single-timestep-truncated BPTT". Adds CPU_LSTMSeqForward / CPU_LSTMSeqBackward: the whole unrolled sequence in one call each, weights shared across timesteps, dW accumulated over all of them (the per-step CPU_LSTMWeightsGradient assigns rather than accumulates, so it could not have been reused even with the dc term). Fused rather than dispatched per step because the recurrence is sequential - T round trips would serialise T lock/dispatch pairs for a few thousand FLOPs each. h_{-1} and c_{-1} are zero per sample. The old layer carried its cell state across forward passes, so under shuffled training every sample inherited the state of an unrelated one. DirectML gets the same math host-side (readback, compute in double, upload) rather than HLSL: the recurrence needs a barrier per timestep, the GPU buffers are float and BPTT accumulation is where that hurts most, and no D3D12 device exists on this machine to test a shader against. Documented at the definition. Verified with lstm_seq_gradcheck.cpp - central-difference check of dW and dX against an asymmetric loss over the final hidden state. Max relative error 2.3e-10 on both, with a non-trivial gradient magnitude asserted so the check cannot pass on an all-zero result. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-30 18:13:50 -04:00
// Fused unrolled sequence LSTM - see the block comments in WarriorDML.cpp (host-side by design) and
// WarriorCPU.cpp (the authoritative contract).
WARRIORDML_API int __stdcall DML_LSTMSeqForward(DmlHandle ctx, int wHandle, int inputsHandle,
int cacheGatesHandle, int cacheCellHandle, int cacheHiddenHandle, int outputHandle,
int hiddenSize, int stepInputs, int steps);
WARRIORDML_API int __stdcall DML_LSTMSeqBackward(DmlHandle ctx, int wHandle, int inputsHandle,
int cacheGatesHandle, int cacheCellHandle, int cacheHiddenHandle, int outGradientHandle,
int weightsGradientHandle, int inputsGradientHandle, int hiddenSize, int stepInputs, int steps);
fix: correct OpenCL gradient and weight update for classification layers - In CaclOutputGradient kernel: - Case 1 (sigmoid classification): removed erroneous multiplication by out*(1-out) which dampened gradients – the binary cross-entropy loss already cancels the sigmoid derivative, so direct (target-out) is correct. - Added default case for NONE activation (softmax classification) to compute plain error (target-out); previously unhandled, resulting in zero gradients that froze the entire network when OpenCL was active. - In UpdateWeightsMomentum and UpdateWeightsAdam kernels: added clamp to MAX_WEIGHT when updating weights to prevent gradient spikes from producing ±Infinity and subsequent NaN propagation through dense layers (e.g., classification output head).
2026-07-15 21:47:09 -04:00
WARRIORDML_API int __stdcall DML_LSTMUpdateWeightsAdam(DmlHandle ctx, int wHandle, int weightsGradientHandle,
feat: add max-pooling and convolution OpenCL kernels, clean up barrier and signal code - Define MAX_WEIGHT constant (1.0e6) for weight limits in clusters - Remove redundant barrier from FeedForward kernel (prevents sync issues) - Port FeedForwardProof and CalcInputGradientProof kernels for max-pooling (no weights, sliding max) - Port FeedForwardConv kernel for convolution layers (shared weights, multiple output channels) - Remove unused code and refactor signal condition logic (CSignalPAI)
2026-07-13 03:23:39 -04:00
int mHandle, int vHandle, double l, double b1, double b2, int total);
feat: add SGD+momentum optimizer and input-driven hyperparameters Replace hardcoded lr and momentum with new input variables for Adam and SGD+momentum. Add OpenCL kernel LSTM_UpdateWeightsMomentum alongside the existing Adam kernel. Update comments and revert beta1 to book default 0.9.
2026-07-18 14:56:41 -04:00
WARRIORDML_API int __stdcall DML_LSTMUpdateWeightsMomentum(DmlHandle ctx, int wHandle, int weightsGradientHandle,
feat(opencl): add feedback alignment support to weight update kernels Introduce an `optimizer` parameter to UpdateWeightsMomentum, UpdateWeightsConvMomentum, and UpdateWeightsAdam kernels. When set to a non-zero value, the gradient used for weight updates is multiplied by ±1 based on the parity of the weight index, implementing a basic feedback alignment signal for experimentation. When zero, the standard gradient is used unchanged. This allows A/B testing of alternative learning signals without modifying the rest of the training pipeline.
2026-07-28 15:01:40 -04:00
int dwHandle, double learningRate, double momentumRate, int total, int optimizer);
feat: add SGD+momentum optimizer and input-driven hyperparameters Replace hardcoded lr and momentum with new input variables for Adam and SGD+momentum. Add OpenCL kernel LSTM_UpdateWeightsMomentum alongside the existing Adam kernel. Update comments and revert beta1 to book default 0.9.
2026-07-18 14:56:41 -04:00
feat: add max-pooling and convolution OpenCL kernels, clean up barrier and signal code - Define MAX_WEIGHT constant (1.0e6) for weight limits in clusters - Remove redundant barrier from FeedForward kernel (prevents sync issues) - Port FeedForwardProof and CalcInputGradientProof kernels for max-pooling (no weights, sliding max) - Port FeedForwardConv kernel for convolution layers (shared weights, multiple output channels) - Remove unused code and refactor signal condition logic (CSignalPAI)
2026-07-13 03:23:39 -04:00
// Max-pooling layer kernels (CNeuronPoolOCL) - no weights.
fix: correct OpenCL gradient and weight update for classification layers - In CaclOutputGradient kernel: - Case 1 (sigmoid classification): removed erroneous multiplication by out*(1-out) which dampened gradients – the binary cross-entropy loss already cancels the sigmoid derivative, so direct (target-out) is correct. - Added default case for NONE activation (softmax classification) to compute plain error (target-out); previously unhandled, resulting in zero gradients that froze the entire network when OpenCL was active. - In UpdateWeightsMomentum and UpdateWeightsAdam kernels: added clamp to MAX_WEIGHT when updating weights to prevent gradient spikes from producing ±Infinity and subsequent NaN propagation through dense layers (e.g., classification output head).
2026-07-15 21:47:09 -04:00
WARRIORDML_API int __stdcall DML_FeedForwardProof(DmlHandle ctx, int iHandle, int oHandle, int inputs, int window, int step, int outputs);
feat: add max-pooling and convolution OpenCL kernels, clean up barrier and signal code - Define MAX_WEIGHT constant (1.0e6) for weight limits in clusters - Remove redundant barrier from FeedForward kernel (prevents sync issues) - Port FeedForwardProof and CalcInputGradientProof kernels for max-pooling (no weights, sliding max) - Port FeedForwardConv kernel for convolution layers (shared weights, multiple output channels) - Remove unused code and refactor signal condition logic (CSignalPAI)
2026-07-13 03:23:39 -04:00
fix: correct OpenCL gradient and weight update for classification layers - In CaclOutputGradient kernel: - Case 1 (sigmoid classification): removed erroneous multiplication by out*(1-out) which dampened gradients – the binary cross-entropy loss already cancels the sigmoid derivative, so direct (target-out) is correct. - Added default case for NONE activation (softmax classification) to compute plain error (target-out); previously unhandled, resulting in zero gradients that froze the entire network when OpenCL was active. - In UpdateWeightsMomentum and UpdateWeightsAdam kernels: added clamp to MAX_WEIGHT when updating weights to prevent gradient spikes from producing ±Infinity and subsequent NaN propagation through dense layers (e.g., classification output head).
2026-07-15 21:47:09 -04:00
WARRIORDML_API int __stdcall DML_CalcInputGradientProof(DmlHandle ctx, int iHandle, int gHandle, int oHandle, int igHandle,
feat: add max-pooling and convolution OpenCL kernels, clean up barrier and signal code - Define MAX_WEIGHT constant (1.0e6) for weight limits in clusters - Remove redundant barrier from FeedForward kernel (prevents sync issues) - Port FeedForwardProof and CalcInputGradientProof kernels for max-pooling (no weights, sliding max) - Port FeedForwardConv kernel for convolution layers (shared weights, multiple output channels) - Remove unused code and refactor signal condition logic (CSignalPAI)
2026-07-13 03:23:39 -04:00
int outputs, int window, int step, int inputs);
Reference in a new issue Copy permalink