Commit graph

58 commits

Author SHA1 Message Date
AnimateDread
e2df5cfdc5 fix: improve OOS accuracy regression logging to include balanced accuracy and blended values 2026-07-21 14:23:13 -04:00
AnimateDread
3d5475ebcb refactor: defer arrow drawing to era-end sweep when NMS enabled
When `m_signalClusterWindow > 0`, training passes (1,2,3) now only record signal scores into `m_arrowSignalCache[]` and skip drawing arrows. The end-of-era `PruneDirectionalClusters()` sweep handles all rendering, preventing raw mid-era clusters from appearing and decoupling NMS from pass 2’s shuffled draw order.
2026-07-21 13:18:35 -04:00
AnimateDread
3f83bf192c fix: improve NMS clustering and cross-direction resolution in signal AI base
Change the non-maximum suppression algorithm so that same-direction
contiguous runs collapse to a single arrow by tracking the last SEEN
bar (instead of the last KEPT bar), preventing re-emission every
window+1 bars inside a long run.  Add confidence-based resolution
between opposite signals within the cluster window: only the higher-
confidence side is kept, reducing flicker around genuine turn zones.

Introduce new state variables (`m_nmsLiveBuyAccept`,
`m_nmsLiveSellAccept`, `m_nmsLiveKeptTime`, `m_nmsLiveKeptDir`,
`m_nmsLiveKeptConf`) to support idempotent re-evaluation of the same
live bar and cross-direction filtering.  Update `NmsLiveAccept` and
`PruneDirectionalClusters` to apply the same logic consistently for
both historical and live paths.
2026-07-21 12:30:29 -04:00
AnimateDread
c024e8dede feat: add non-max suppression and balanced accuracy for signal clustering and checkpoint ranking
Implement non-max suppression (NMS) to declutter signal visualizations by keeping only the first bar of each same-direction run, controlled by m_signalClusterWindow. Also replace blended accuracy with balanced accuracy (macro-recall) for checkpoint selection to avoid neutral bias, tracked via m_bestBalancedOos. This improves signal clarity and model deployment quality.
2026-07-21 00:03:45 -04:00
AnimateDread
7aec5c4916 feat(ExpertSignalAIBase): extend swing context with recent price-action features
Increase the swing context neuron count from 5 to 9 by adding four fresh, non-repainting features: Donchian range position at 20 and 50 bars, 20-bar return, and 20-bar SMA extension. These are computed from closed bars only, eliminating lookahead. They provide immediate trend/position context that the stale (~100-bar-old) confirmed pivot anchor cannot, enabling the network to differentiate genuine reversals at range extremes from mid-trend bars that resemble pivot shapes.
2026-07-20 19:17:23 -04:00
AnimateDread
e13ee77c94 feat(signal): add oversampling parity fraction and min signal confidence threshold
Introduce OVERSAMPLE_PARITY_FRACTION (0.7) to control minority class oversampling scaling, reducing low-quality directional calls by not fully replicating to parity. Add m_minSignalConfidence (0.5) to require a minimum softmax probability for live Buy/Sell orders, preventing firing on bare plurality (~0.34) and improving signal quality.
2026-07-20 15:55:21 -04:00
AnimateDread
9f6e2b3b34 refactor: set MAX_OVERSAMPLE to 100 and revert to exact labels
- Increased MAX_OVERSAMPLE_REPLICAS from 5 to 100 to serve only as an absolute sanity ceiling against degenerate tallies, not a binding limit – oversampling rate is now measurement-driven per era.
- Reverted LABEL_WINDOW_BARS to 0 (exact reversal candle only), as the earlier low recall motivating widening was observed on a flawed training stack that has since been fixed; exact labels get a clean re-test.
- Removed ArrayResize calls in Train that preallocated queues using MAX_OVERSAMPLE_REPLICAS, consistent with the shift to dynamic, measurement-based allocation.
2026-07-19 22:40:56 -04:00
AnimateDread
3ceb4c6431 fix: prevent silent mislabeling and false convergence after label cache wipe
When a new candle closes mid-run (e.g., after weekend market reopen), the label cache is
wiped and the fallback per-bar computation labels everything Neutral. Without this fix,
the rest of the training run silently trains and scores against an all-Neutral world,
causing false convergence (e.g., 84.9% accuracy with 100% Neutral truth). Now the
prebuild is re-armed after cache invalidation to defer the era until relabeling completes,
and convergence finalization requires directional recall to be actually measured (not n/a),
preventing a Neutral-only model from being certified.

Also improves logging to distinguish era 0 seeding from mid-run rebuilds.
2026-07-19 18:22:46 -04:00
AnimateDread
afee4cc4c1 fix: lower WEIGHT_DECAY to 0.001 to prevent argmax degeneration to Neutral
Reduce weight decay from 0.01 to 0.001 across all backends (Network.cl, Network.mqh, WarriorCPU.cpp) to fix a training collapse issue. The original 0.01 AdamW default caused discriminative weights to decay below the calibration-capped class-prior offsets, resulting in a monotonically shrinking per-bar logit spread and eventual constant Neutral predictions (argmax degenerated once evidence tilt dropped under the prior tilt). The new value 0.001 lifts the evidence ceiling 10× while still bounding long-run weight growth, restoring effective discrimination. Note: this change must remain in sync across all four backends.
2026-07-19 17:05:58 -04:00
AnimateDread
96a9b414b4 feat: remove sign-agreement gate from Adam weight updates in OpenCL kernels
The sign-agreement gate in `UpdateWeightsAdam`, `UpdateWeightsConvAdam`, and `LSTM_UpdateWeightsAdam` caused a gradient ratchet effect under one-hot softmax with categorical cross-entropy, leading to an all-Neutral collapse. Removing this gate aligns all four backends (CPU, GPU, OpenCL, MQL) and restores correct gradient flow.
2026-07-19 14:50:52 -04:00
AnimateDread
e8452913c0 feat: add swing-context feature with confirmed zigzag pivot
Introduce m_useSwingContext flag and FindConfirmedZigZagPivot method to compute normalized swing direction/magnitude/age features from the existing ADZigZag indicator. Only pivots that are at least m_swingConfirmationBars old are trusted, preventing lookahead bias. The SWING_SCAN_CAP_BARS macro limits backward scan depth. Default is off.
2026-07-19 11:04:38 -04:00
AnimateDread
3fbe16127e docs: simplify comments and enum descriptions across AI and Enums
Remove overly verbose explanations in Network.mqh comments and InputEnums.mqh enum values. Shorten descriptions to improve readability without losing essential information.
2026-07-18 23:59:40 -04:00
AnimateDread
6f95a402f4 refactor: replace class-balance loss weighting with data-level oversampling
Class-balance correction is now performed entirely via data-level oversampling (duplicating minority-class bars) instead of using a per-occurrence loss-weight multiplier combined with oversampling. The loss-weight mechanism proved ineffective due to Adam's near-invariance to constant gradient rescaling (Kingma & Ba 2015). The CLASS_SAMPLE_WEIGHT_PRESET enum is retained for a possible future supplemental loss weight, but m_maxClassSampleWeight currently has no effect. The MAX_OVERSAMPLE_REPLICAS constant is now the sole bound on correction strength.

Updated comments in InputEnums.mqh and ExpertSignalAIBase.mqh to reflect this change.
2026-07-18 23:25:54 -04:00
AnimateDread
39d785b0de fix: prevent oversampling weight compounding and fix era display lag
- Clarify MAX_OVERSAMPLE_REPLICAS comment: it is an outer safety cap; actual repCount is derived from m_maxClassSampleWeight.
- Introduce m_isTrainQueueWeightScale array to scale per-occurrence class-balance weight by repCount, avoiding independent compounding of frequency and magnitude.
- Add forceRefresh flag to UpdateTrainingStatusLabel to force panel update at era end, fixing one-era-behind display of era number.
2026-07-18 23:03:51 -04:00
AnimateDread
20f32306f3 feat(ml): add data-level oversampling for minority classes in training queue
Introduce MAX_OVERSAMPLE_REPLICAS=3 to duplicate minority-class (Buy/Sell) bars in the training queue, complementing the existing loss-level reweighting. Adam's update rule largely cancels gradient scaling, so pure loss weighting was insufficient to overcome class imbalance. Duplicating bars ensures Adam's moment estimates see minority gradients more frequently, preventing the Neutral-only collapse observed in practice.
2026-07-18 22:27:34 -04:00
AnimateDread
d75bd6d6a8 feat: add configurable news event proximity/impact as an NN input feature
Price, time, volume, and volatility were already trained-model input
features; the real economic calendar (already used for the live
NewsFilter veto) is now an optional one too, reusing
System/NewsRelevance.mqh's symbol-relevance logic from the prior fix.

New EnableNews/NewsFeatureWindowMinutes inputs gate two features per
bar: minutes-since and minutes-until the nearest symbol-relevant
calendar event, impact-weighted. Deliberately limited to proximity +
impact, not actual-vs-forecast deviation - release schedules are
public knowledge ahead of time (not lookahead bias to use for a
historical training bar), but a release's actual outcome is not.

Wired identically to the existing EnableVolume/EnableTime/EnableATR
toggles: InitIndicators() accounts for the +2 neuron count,
BufferTempDataCompute() appends the two feature values, PAI/CONV/LSTM
all wired in Warrior_EA.mq5. Compiled clean (MetaEditor, 0 errors/0
warnings).

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-07-18 17:26:04 -04:00
AnimateDread
4c0afd682b refactor(ExpertSignalAIBase): extract AutoTune param state into CADIndicatorTuner
CExpertSignalAIBase (4,326 lines, one class) carried 5 struct
definitions, 10 member fields, and 3 methods (Flatten/Unflatten/
PerturbRandom) purely for the AutoTuneIndicators search-space state -
entirely self-contained (never touches Net, Train()'s resumable state
machine, or anything else in the class). Extracted into a new
Expert/ADIndicatorTuner.mqh (CADIndicatorTuner), held as a single
m_indicatorTuner member.

TuneIndicatorsAndTrain() itself - the outer loop that actually
orchestrates Train()/Net/checkpointing around this tuner - turned out
to be exactly as tightly coupled to Train()'s resumable state machine
as Train() itself, so per the same caution already applied to Train()
in this refactor pass, it stays in CExpertSignalAIBase rather than
being pulled into the collaborator; it now calls the tuner's public
Flatten()/Unflatten()/PerturbRandom()/SaveAsBest()/RestoreBest()
instead of manipulating the structs inline.

All internal field-access renames (m_adCumDeltaParams.lookback ->
m_indicatorTuner.adCumDelta.lookback, etc., ~40 sites across the 5
InitAD*() indicator-setup methods) verified against a full grep sweep
- no leftover references to the old field/method names. Compiled
clean (MetaEditor, 0 errors/0 warnings).

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-07-18 16:22:09 -04:00
AnimateDread
e6ff2506a8 refactor(Money): extract CMoneyRiskBase to remove FixedRisk/Intelligent duplication
CMoneyFixedRisk and CMoneyIntelligent both derived independently from
CExpertMoneyCustom and carried near-identical CalculatePotentialLoss()/
CheckOpenLong()/CheckOpenShort()/CalculateLotSize() bodies, maintained
in two places. New Money/MoneyRiskBase.mqh (CMoneyRiskBase) houses the
shared core, with two virtual hooks - AdjustRiskAmount()/AdjustLotSize()
- at the exact two points CMoneyIntelligent's confidence-scaling and
trade-history Optimize() step used to diverge. CMoneyFixedRisk now has
an empty body (the base's defaults are exactly its old behavior);
CMoneyIntelligent overrides only the two hooks. CMoneyFixedLot is
untouched - it doesn't do risk-based sizing.

Pure reorganization, no behavior change; compiled clean (MetaEditor,
0 errors/0 warnings).

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-07-18 15:53:04 -04:00
AnimateDread
97cf7be556 chore: cleanup pass - dead code, stale wizard scaffolding, one edge-case guard
- Variables/Inputs.mqh: drop the dead commented-out HybridSignals input
  line; add a one-line note above the section-header input strings
  clarifying they're intentional MetaTrader GUI dividers (consumed by
  the terminal, not any MQL5 statement) so a future audit doesn't
  re-flag them as unwired.
- Signals/SignalSessionFilter.mqh: replace the never-filled-in MQL5
  Wizard template header (ProjectName/CompanyName placeholders) with
  this codebase's real header, matching every other Signals/*.mqh file.
- Signals/SignalNewsFilter.mqh: remove the stale NEWS_IMPACT template
  macro - it was only ever used as the constructor default, immediately
  overridden by the real NF_MinImpact input at wiring time.
- Expert/ExpertSignalAIBase.mqh: guard the SERIES_LASTBAR_DATE read in
  ScheduleTrainingIfNeeded() so a failed lookup (0) can't silently be
  read as "no new bar pending" and stall training/signal refresh.

Compiled clean (MetaEditor, 0 errors/0 warnings) after each change.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-07-18 15:49:59 -04:00
AnimateDread
7e27ad8729 fix(ExpertSignalAIBase): stop pass-3 resume from replaying pass 2 forever
m_isPass2Active going false was ambiguous between "pass 2 hasn't
started yet" and "pass 2 already finished" - both look identical to
a plain if(!m_isPass2Active) check. Once pass 3 (OOS scoring) needed
more than one chunk to finish, every resume into it fell through that
ambiguity on both the pass-1 and pass-2 gates and re-shuffled/replayed
the entire IS training queue from scratch, on top of never-reset
predicted-class counters - observed as era 0 looping the shuffle phase
forever with predicted Buy/Sell collapsing to 0 and predicted Neutral
climbing past the real bar count.

Adds a distinct m_isPass2Done flag so a resume can tell "not started"
apart from "already done" and correctly skip straight into pass 3.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-07-18 15:44:48 -04:00
AnimateDread
6a687cda41 feat: add SGD+momentum optimizer and input-driven hyperparameters
Replace hardcoded lr and momentum with new input variables for Adam and
SGD+momentum. Add OpenCL kernel LSTM_UpdateWeightsMomentum alongside the
existing Adam kernel. Update comments and revert beta1 to book default 0.9.
2026-07-18 14:56:41 -04:00
AnimateDread
86cb37595d feat: introduce Pass 3 for OOS scoring and extract status label
Add a new post-training pass (m_isPass3Active) that scores the OOS region chronologically after pass 2 has trained the IS data. This fixes the previous stale scoring (which used pre-era weights) and ensures OOS metrics reflect the trained model.

New member variables track the OOS cursor and start index; UpdateTrainingStatusLabel extracts the inline formatting into a dedicated method to keep the panel updating during all three passes, eliminating the "stuck" appearance during pass 2/3.

Reset m_isPass3Active at era start and remove redundant inline formatting code from Train().
2026-07-18 12:10:37 -04:00
AnimateDread
379db0827c fix(ExpertSignalAIBase): add shuffled backprop pass 2 to prevent training instability
Adam's momentum (b=0.9) has an effective memory of ~1/(1-b)=10 steps, which closely matches the typical 5-bar contiguous same-class label runs around ZigZag pivots. Replaying these runs in the same chronological order every era allowed momentum to lock onto whichever run it was currently passing through, causing the network's end-of-era state to disproportionately reflect the most recent run (recency bias). This led to IS error climbing era-over-era (0.16→0.55+ over 25 eras) and OOS Buy/Sell recall whipsawing between near-0% and 70-100% despite near-equal label counts.

The fix implements a second pass within the resumable-chunk training loop: after pass 1 (sequential feedforward/scoring) queues all IS-eligible bars into an array, pass 2 trains on those bars in a freshly shuffled order – the standard SGD solution to break momentum’s lock on repeating patterns. New member variables `m_isTrainQueue`, `m_isTrainCursor`, and `m_isPass2Active` manage the queue and state transitions between passes, preserving the existing `TRAIN_TIME_BUDGET_MS` chunk-yield mechanism.
2026-07-18 11:33:21 -04:00
AnimateDread
87ca08f21b refactor: tune optimizer momentum and integrate focal loss to reduce multi-era bias
- Lowered Adam momentum parameter b1 from 0.9 to 0.8, shortening the effective memory window (~5 steps) to reduce multi-era bias streaks observed in training runs.
- Added FOCAL_GAMMA_PRESET enum and corresponding member variable m_focalGamma to apply focal loss modulation (Lin et al. 2017) on top of class-balance weighting, targeting the same streak issue by downweighting already-confident examples.
- Updated comments to document the rationale and experimental nature of both changes.
2026-07-18 10:03:21 -04:00
AnimateDread
1a804fce28 feat: make class-balance oversampling multiplier configurable via input enum
Replace the hardcoded `MAX_CLASS_SAMPLE_WEIGHT` (3.0) with a tunable member variable `m_maxClassSampleWeight`, controlled by the new `CLASS_SAMPLE_WEIGHT_PRESET` input enum. This allows adjusting the ceiling on rare-class sample weights to prevent the previously observed anti-correlated Buy/Sell recall whipsaw (where a high multiplier caused one era's gradients to overwrite another class's separability). Default is CSW_15 (1.5x), which is gentler than the old 3.0x; lower values down to 1.0x disable rebalancing to train on raw label distribution, while higher values (up to 3.0x) converge faster but risk instability. This tuning can now be done without recompiling.
2026-07-18 02:01:06 -04:00
AnimateDread
457ae4acac refactor: Remove test holdout slice, add live signal alternation gate
- Removed TEST_HOLDOUT_PRESET enum and its associated member variables (m_testHoldoutPct, dTestForecast, m_testSamples) from ExpertSignalAIBase to simplify code and eliminate unused test slice logic.
- Added m_lastNonNeutralSignal member to track the last non-neutral signal during live inference, preventing LongCondition()/ShortCondition() from opening a second consecutive same-direction trade (which would be a false fire based on training patterns where valid reversals always alternate direction).
- This live-only mechanism does not affect historical training or backpropagation.
2026-07-17 23:55:10 -04:00
AnimateDread
e62c710d6f fix: correct array orientation and PReLU gradient backprop in hidden layers
- Ensure `tick_volume` array is set as series in ADShorteningOfThrust.mq5 to prevent future-data leak in volume calculations.
- Ensure `open` array is set as series in ADWyckoffFailedStructure.mq5 to prevent future-data leak in structure detection.
- Add missing PReLU gradient scaling (multiply by 0.01 for negative outputs) in CPU_CalcHiddenGradient and DirectML shader to match expected derivative behavior across all backends.
2026-07-17 23:21:12 -04:00
AnimateDread
e50f7d6c3b refactor: improve status label formatting in training schedule
Replace the raw debug dump (event call, prevSignal, studied time) in the status label with a clean, user-friendly summary showing era count, training state, and forecast. This provides clearer on-screen feedback when training stops, pauses, or completes.
2026-07-17 22:05:52 -04:00
AnimateDread
58049461c0 feat: show in-sample training percentage in progress log
The training progress display now includes the in-sample split percentage (100 - oosSplitPct) next to the accuracy, replacing the previously generic "IS Acc" label. This makes it clearer what proportion of data is used for in-sample training.
2026-07-17 21:56:25 -04:00
AnimateDread
ef95e19d1d fix: correct metric label in training output and improve status panel comments 2026-07-17 21:53:09 -04:00
AnimateDread
fe4c93cf71 refactor: simplify on-chart status text and remove verbose metrics
The training status panel was too tall and wordy, displaying detailed recall, precision, and continual-learning OOS simulation metrics that are not needed for at-a-glance monitoring. These metrics are still tracked internally for convergence gating, but removed from the on-chart display to keep the panel compact. Also shortened label names (e.g., "[IS] Accuracy" -> "IS Acc") to save space without reducing clarity.
2026-07-17 21:49:30 -04:00
AnimateDread
2c0cdf5e2b refactor(ai): replace Comment() with StatusLabel for training status display
Migrate all per-era classification counts, OOS confusion metrics, and training progress output from the legacy Comment() function to a dedicated StatusLabel object. This provides cleaner UI integration and avoids blocking the chart's normal info line during prolonged training cycles. A new include for StatusLabel.mqh has been added, and all related documentation comments updated accordingly.
2026-07-17 21:28:59 -04:00
AnimateDread
f5fa5b3fcc fix: always prebuild label cache to prevent silent neutral labels on checkpoint restart
Previously, on a successful checkpoint load (`netLoaded == true`), `m_labelCachePrebuilt` was set to `true`, skipping the eager prebuild scan. This caused every bar to fall back to `ComputeLabelForBar()`, a dead stub returning `buy=false/sell=false`, effectively labeling the entire era Neutral. Now we unconditionally set `m_labelCachePrebuilt = false`, ensuring the real ZigZag-based labeling logic runs via `AdvanceZigZagLabelState()` during the prebuild, regardless of checkpoint state.
2026-07-17 21:18:25 -04:00
AnimateDread
55ff1c40c4 perf: improve small layer dispatch and UI responsiveness
- Add inline threshold (512) in WarriorCPU to avoid thread-pool overhead for small dispatches; run small workloads inline on the calling thread.
- Reduce training time budget from 500ms to 120ms in ExpertSignalAIBase to keep the UI reactive while training.
2026-07-17 19:30:10 -04:00
AnimateDread
80a53d5b04 feat: cap class sample weight and increase training time budget
- Added MAX_CLASS_SAMPLE_WEIGHT (3.0) to cap the sample weight multiplier, preventing anti-correlated Buy/Sell recall whipsaw observed when per-era ratio overwrites previous separability.
- Added MIN_OOS_CLASS_SAMPLES_FOR_GATE (10) to require a minimum number of true OOS samples before trusting class recall, avoiding degenerate false-convergence cases.
- Increased TRAIN_TIME_BUDGET_MS from 80 to 500 to improve throughput under MT5's custom event dispatch model (which determines actual pacing); trades UI responsiveness (~500ms lag) for roughly 6× more bars processed per event.
2026-07-17 09:11:42 -04:00
AnimateDread
ed2285000b feat: widen label cache to include neighborhood around pivot bars
The exact-bar-only label scheme caused OOS Buy/Sell recall stuck at 0-3% because the feature window cannot distinguish a pivot bar from its near-identical neighbor. By extending each confirmed pivot into a +-LABEL_WINDOW_BARS neighborhood (default 2), the network is trained to recognize bars near a real reversal rather than only the exact tick. The spread uses a frozen snapshot of the original scan to prevent cascading propagation.
2026-07-16 22:29:49 -04:00
AnimateDread
17abf513ca feat: bias output layer toward the real class prior to fix era-0 cold-start noise
A freshly-initialized network's weights are plain random init with no
informed prior, so its argmax is close to uniform noise across the 3
classes - on this project's typically heavily Neutral-skewed label
distribution, that means era 0 fires far more spurious Buy/Sell calls
on the very first (oldest) bars than the true base rate warrants,
until enough backProp steps correct it.

The label-cache prebuild already computes the real upfront Buy/Sell/
Neutral tally before era 0 starts. Added CNet::SeedOutputLayerBias()
(AI/Network.mqh) to overwrite just the output layer's bias term (the
per-input weights stay randomly initialized and still carry the real
learning signal) with a fixed +-3.0 push toward whichever class
actually dominates - reaches one layer back from the output layer,
since that's where the weight block producing it is stored. Wired in
at the end of AdvanceLabelCachePrebuild(), fresh-network-only.
2026-07-16 20:37:36 -04:00
AnimateDread
721f2970f2 feat: normalize CumulativeDelta feature, add explicit bullish/bearish flag
ADCumulativeDelta's CumulativeDelta buffer now outputs
cumulativeDelta/sumVolume clamped +/-2 instead of a raw, unbounded,
tick-volume-scale running sum - also fixes the indicator's own
declared -2..2 min/max property, which the raw sum silently violated.
Re-added to the feature set now that it's properly scaled, as a
distinct order-flow-imbalance signal from Pressure (same term plus
Initiative/Absorption adjustments).

Added an explicit +1/-1/0 bullish/bearish/doji flag alongside the
ATR-normalized price features, so the network gets candle direction
as a clean standalone signal instead of having to disentangle it from
(close-open)/atr's combined direction+magnitude encoding.
2026-07-16 19:55:51 -04:00
AnimateDread
fbe4821df2 fix: replace class-balance oversample replay with loss weighting
The training log showed the model had genuinely collapsed to always-predict-
Neutral: 100+ consecutive eras with Buy/Sell OOS recall flat at 0% and IS
error frozen exactly at 0.14, not a "still early" transient. Root cause is
the 33:1 Buy/Sell-vs-Neutral label imbalance combined with the oversample
replay cap - capped at 3x specifically because more identical back-to-back
backProp() calls fed Adam highly-correlated gradients and caused runaway
momentum (a past incident: OOS accuracy diving from 90%+ to single digits
within ~20 eras). That cap meant minority classes never got enough gradient
influence to matter once the model settled into all-Neutral.

Replaced the N-times replay with a single backProp() call per example, with
its output-layer gradient scaled by inverse class frequency (maxCount/
trueCount from the previous era's true label distribution, uncapped - the
existing MAX_WEIGHT_DELTA per-step clip already bounds how far any single
step can move a weight regardless of gradient magnitude, so there's no
repeated-gradient momentum risk left to cap against).

AI/Network.mqh: CNet::backProp()/backPropOCL() take a new optional
sampleWeight parameter (default 1.0, so every other caller is unaffected).
For the OCL/DirectML path, since WarriorCPU.dll/WarriorDML.dll/Network.cl
have no notion of per-sample weighting, the raw gradient computed by the
native CalcOutputGradient call is read back into MQL5, scaled, and written
back via a new CNeuronBaseOCL::setGradient() before the hidden layers read
it - no changes needed to any of the 3 compute backends themselves.

Also fixed: a run that "converged" at era 63 only because that specific
era's small OOS sample happened to contain zero true Buy/Sell examples
(recall shows n/a and auto-passes the gate when a class is absent from an
era's sample) - the model had already fully collapsed several eras earlier;
this was a lucky/unlucky sampling fluke, not real convergence. Not fixed in
this commit (separate, narrower issue - the gate's n/a auto-pass exists to
avoid deadlocking on a genuinely rare class, and distinguishing that from a
collapsed model needs its own follow-up).

Needs a fresh retrain like the prior structural fixes.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-07-16 19:04:56 -04:00
AnimateDread
4f0108366f fix: remove undocumented input noise and unbounded/duplicate features
AI/Network.mqh: CNet::feedForward() was adding sin(i)/cos(i) (alternating by
index parity) directly onto every single input value, on both the CPU and
GPU/DirectML paths - an undocumented, uncontrolled +/-1 offset baked into
already-normalized features that mostly live in a much smaller range. No
comment anywhere explained it, unlike the rest of this codebase; removed.

Expert/ExpertSignalAIBase.mqh, data normalization cleanup:
- Volume relative-change feature is now clamped to +/-5 - unlike the
  ATR-normalized price features, it had no ceiling, so a thin previous bar
  (e.g. 1 tick) followed by a normal one could feed the network an outlier
  input an order of magnitude past every other feature's range.
- Dropped ADCumulativeDelta's raw CumulativeDelta buffer (1) from the feature
  set - it's an unbounded, tick-volume-scale running sum, unlike every sibling
  buffer in that same indicator (all explicitly clamped +/-2). Pressure
  (buffer 0) is this same signal already normalized, so nothing is lost.
- Dropped ADWyckoffEventStream's EventPhase buffer (1) - confirmed via source
  it's a byte-for-byte duplicate of EventCode (same underlying variable), not
  a distinct phase reading; StructuralPhase (buffer 5) is the real phase
  signal and stays.

m_neuronsCount updated accordingly (topology changes, needs a fresh retrain
same as the earlier window-offset fix).

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-07-16 18:09:06 -04:00
AnimateDread
10f52084a8 fix: training progress Comment() showed the true label, not the prediction
Moving the per-bar window/feedForward step before the label-check/backprop
step (previous commit) meant the label-check block now runs LAST each
iteration, clearing TempData and refilling it with the smoothed target label
(0.1/0.9) for backProp - so by the time the next iteration's Comment() call
read TempData[0..2] to display "Buy/Sell/Neutral Neuron", it was showing the
PREVIOUS bar's true label, not the network's actual prediction. Since ~94% of
bars are Neutral-labeled, this mostly looked like "Neutral stuck near 0.9-1,
Buy/Sell stuck near 0.1" regardless of what the model was really outputting.

Snapshot the softmax probabilities into local vars right after they're
computed (before the label-check block can clobber TempData), and moved the
whole Comment() build to run right after that snapshot instead of at the top
of the loop, so it now shows the CURRENT bar's own genuine prediction.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-07-16 17:46:27 -04:00
AnimateDread
819066819d fix: feature window never included the most recent bars near the prediction target
RefreshLatestSignal(), AdvanceOosSimulationChunk(), and Train()'s per-bar window
build all constructed the historyBars-wide feature window starting historyBars
bars past the target bar (r = i + historyBars) instead of ending at it (r = i) -
so the model was always blind to the most recent historyBars bars of price
action right before the bar it was being asked to call, both in training and
live inference. This likely explains why Buy calls fire but land poorly timed
and Sell almost never clears its recall floor.

Also reordered Train()'s main loop so each bar's own freshly-built window/
prediction is what gets checked against that bar's label immediately (matching
AdvanceOosSimulationChunk's existing predict-then-learn pattern), instead of
training against the previous iteration's stale prediction.

Also added a Print() when training genuinely converges (m_objectiveMet &&
m_oosStable) - this was previously silent, only visible via the persisted
TrainingComplete flag.

Existing trained weights encode the old, shifted task and need a full retrain.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-07-16 15:23:14 -04:00
AnimateDread
e39cfa56c1 fix: stop halving the WarriorCPU.dll thread pool for phantom concurrency
TargetCPULoad=max was only ever getting ~half the cores (e.g. 6 of 12)
even outside HYBRID mode, because every CNet (including the live net's
own EMA shadow net, which every signal has) counted as a "peer" that
divided the requested CPU budget.

That division was solving a problem that can't actually happen: MQL5
gives one chart's EA a single execution thread, and every WarriorCPU.dll
entry point blocks that thread until its own ParallelFor() completes,
so only one CNet's worker pool is ever actively computing regardless of
how many are alive (live, shadow, or - under HYBRID - PAI/CONV/LSTM's
own pairs). The idle ones sit blocked on a condvar at ~0% CPU. Dividing
the budget across "peers" that never contend just capped whichever pool
happened to be running at a fraction of the cores actually available.

Removes the now-pointless g_netPeerCount/EffectiveCpuLoadPercent()/
PeerNetworkCount() machinery entirely; SetCpuLoadPercent() now just
takes TargetCPULoad directly.
2026-07-16 13:51:28 -04:00
AnimateDread
feb82a61aa feat: log per-class OOS recall in the periodic training progress line
Recall (directionalRecallOK) was only ever printed on a regression
event, so once OOS accuracy alone cleared the target there was no way
to tell from the Journal whether recall was the thing still blocking
m_objectiveMet. Now shown on every ~5s progress line.
2026-07-16 12:25:05 -04:00
AnimateDread
78ef4c14b9 fix: classification training was structurally unable to converge
Two issues compounded to keep the network oscillating for 350+ eras
without ever reaching m_objectiveMet, despite good OOS accuracy:

- CNet::backPropOCL() zeroed the diagnostic error contribution from
  any output neuron whose target was exactly 0, unlike backProp()'s
  CPU-fallback path. For the one-hot 3-class classification head this
  meant dError only ever reflected the true class's own calibration,
  never whether the other two classes were correctly suppressed -
  purely a reporting bug, the real gradient (CPU_CalcOutputGradient
  in WarriorCPU.cpp) was already correct and unaffected.
- Train()'s convergence gate required dError < 0.1 unconditionally.
  That RMS-error floor is reachable for the single-neuron regression
  head, but for 3 one-hot outputs it demands near-perfect confident
  calibration on every bar - unreachable under normal market label
  noise, and redundant with the classification-native gates already
  in place (dOosForecast >= target, per-class directionalRecallOK).
  Now skipped when m_outputNeuronsCount == 3.
2026-07-16 08:55:54 -04:00
AnimateDread
eee45526a5 fix: remove redundant Expert.Deinit() in OnInit(); correct ADZigZag buffers
- Remove manual Expert.Deinit() calls from all failure branches in OnInit() because MQL5 automatically calls OnDeinit(REASON_INITFAILED) when returning non-INIT_SUCCEEDED, preventing double deinitialization and invalid pointer access in OnDeinit().
- Change ADZigZag buffer count from 1 to 3 to match the indicator's #property indicator_buffers (3 buffers), accounting for internal INDICATOR_CALCULATIONS buffers, consistent with other AD Wyckoff indicators.
2026-07-16 01:12:37 -04:00
AnimateDread
0bfc441139 feat: real-ZigZag training labels, HYBRID CPU-oversubscription fix, input cleanup
- Replace the hand-rolled fractal/deviation-%/ATR-trend-context ZigZag
  approximation with the actual MQL5 ZigZag indicator (rebranded as
  CustomIndicators/ADZigZag.mq5, logic untouched) as the training label
  source; bump the settle/confirmation window from 20 to 100 bars so a
  proper leg can form before being trusted, and drop the now-dead
  ZigZagDeviationPct/MinTrendATRMultiple/TrendContextBars inputs.
- Rebrand the stock Volumes indicator the same way (ADVolume.mq5).
- Fix HYBRID mode (AIType=HYBRID) oversubscribing the CPU fallback tier:
  every concurrent CNet instance was independently sizing its worker
  pool off the same global TargetCPULoad input. g_netPeerCount/
  PeerNetworkCount() now split it across however many CNet instances
  (live+shadow x active signals) are actually sharing the CPU.
- Fix NEURONS_REDUCTION_FACTOR's confusing retention-vs-reduction
  semantics so RF_70 means an actual 70% reduction; default to 4 hidden
  layers / RF_70.
- Migrate remaining free-form training/indicator inputs to enums for UI
  consistency; default news filter lookback to 1h, disable every-tick.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-07-16 00:56:33 -04:00
AnimateDread
8c66a0fb89 fix(cl): stabilize network training with tighter limits, weight decay, and sign-agreement gate
- Tighten MAX_WEIGHT from 1.0e6 to 100.0 to prevent unbounded weight growth.
- Add MIN_ACTIVATION_DERIVATIVE (1e-4) to avoid zero gradients for saturated units.
- Introduce WEIGHT_DECAY (0.01) to decouple decay from Adam updates.
- Add MAX_WEIGHT_DELTA (0.1) to clamp per-step Adam updates and prevent overshoot.
- Apply sign-agreement gate on weight updates in Adam kernel to only apply steps aligned with current gradient direction.
- Fix tanh/sigmoid derivative calculations to use the new floor instead of hard-coded edge-case values.
2026-07-15 21:47:37 -04:00
AnimateDread
c05d9d1a1f fix: correct OpenCL gradient and weight update for classification layers
- In CaclOutputGradient kernel:
  - Case 1 (sigmoid classification): removed erroneous multiplication by out*(1-out) which dampened gradients – the binary cross-entropy loss already cancels the sigmoid derivative, so direct (target-out) is correct.
  - Added default case for NONE activation (softmax classification) to compute plain error (target-out); previously unhandled, resulting in zero gradients that froze the entire network when OpenCL was active.
- In UpdateWeightsMomentum and UpdateWeightsAdam kernels: added clamp to MAX_WEIGHT when updating weights to prevent gradient spikes from producing ±Infinity and subsequent NaN propagation through dense layers (e.g., classification output head).
2026-07-15 21:47:09 -04:00
AnimateDread
ac88fc1ac5 fix(Expert/ExpertSignalAIBase): add feature cache and fix oversampling imbalance
- Introduce per-bar feature cache to avoid redundant recomputation of input vectors during training.
- Rename EnsureLabelCacheCapacity to EnsureBarCachesCapacity to reflect management of both label and feature caches.
- Fix oversampling logic to maintain balanced representation among minority classes, replacing independent 5x caps that caused relative bias.
2026-07-14 22:49:14 -04:00