User request: attaching a chart must need zero Inputs-tab edits. Private
(non-Market) build now defaults to AIType=META, all four classic families
ON (they are the sweep's candidate sources), Meta_ExportDataset=true.
Market-build defaults unchanged (AI_NONE, MA/RSI only, no export);
UseDatabaseRanking=true applies to both per the earlier request.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The user should not need a tester corpus run per symbol. Every pattern
condition in Signals\Signal{MA,RSI,MACD,Ichimoku}.mqh anchors its reads on
`int idx = StartIndex()` with zero hardcoded indices (verified), so a
name-hiding StartIndex override + EvalShift(i) on CExpertSignalCustom makes
the EXACT live ladder code answer "what would you have fired at bar i" -
the silent-divergence trap that justified the DB corpus does not exist on
this path, and neither do the GMT-offset ambiguity, the DB row caps, or
the wipe procedure.
- CExpertSignalCustom: m_evalShift + StartIndex()/EvalShift() +
SweepPrepare(bars) (deep-resizes the shared price series); the four
classic signal classes override SweepPrepare to deep-resize their own
indicator buffers.
- CSignalMETA::BuildCorpusBySweep: per bar x per source filter, run
Direction() shifted, harvest the per-side pattern slots + netVote into
the same corpus arrays the DB loader fills; entry=bar open so
MetaPrepareEra's resolution matches at offset +0 with zero price error.
DB corpus remains the fallback when classic filters are disabled.
- Warrior_EA.mq5: META gets the enabled classic filters as candidate
sources (family ids match the descriptor one-hot).
- UseDatabaseRanking default false -> true (user request): a META chart
journals + ranks out of the box.
Workflow per symbol is now: attach ONE chart with AIType=META (optionally
Meta_ExportDataset=true for the offline pool) - candidates, labels,
training and export all happen in place, ~10 seconds of sweep instead of a
tester run.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Meta_ExportDataset input: with AIType=META the chart writes its complete
training set once per attach - every resolved+labeled candidate as
[barTime|family|pattern|side|won|NetInputWidth floats] using the SAME
window builder, descriptor and label caches pass 2 trains on, so offline
examples are byte-equivalent to the EA's own. Sidecar .meta.csv carries
layout + the geometry/BE the labels were computed at. Files land in
Common\Files\Warrior_EA\MetaExport\<sym>_<period>.f32.
This is the pooling architecture decision: multi-symbol training INSIDE the
per-chart God-class would be the riskiest surgery this codebase has seen;
instead each chart exports, the pooled head trains offline (small dense+BN
net, minutes on this box), is validated per-symbol under the same
chronological splits and coverage x (p - BE) gate, and only a WINNER gets
written back into a .nnw for the EA to load natively (format fully mapped).
Also turns every future meta experiment from a 20-minute tester cycle into
minutes of offline iteration.
Cost-model note for the record (user challenge, verified): spread is 0.099
ATR = ~2% of the 4.74 ATR trade width - tiny per bar, but expressed in
win-rate points it is 0.099/4.74 = 2.1pp, which is the measured base-vs-BE
gap and the size of the entire observed skill lift. Zero-spread relabeling
would put base == BE by construction. Multi-day holds additionally pay swap,
which the label does NOT charge - the true bar is higher, not lower.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The NN now has a target that is not per-bar direction (closed, best-of-999
p=1.0000): P(win | this journaled candidate, at the EA's own SL/TP, net of
cost). One net for all 52 pattern-sides, AIType=AI_META.
- NetForward.mqh: the host-side softmax+CE gradient generalized total==3 ->
2||3 on both backprop paths; a 2-class softmax IS a logistic head, and no
compute backend changes.
- SignalMETA.mqh (new): corpus loaded read-only from the LARGEST signal DB on
disk (decoupled from the config fingerprint that burned four S1 runs); the
GMT->server offset is measured PER ROW against entryPrice vs bar open
(DST-immune, histogram logged); a window-span regime filter drops the
pre-2017 daily-backfill rows; 31-feature setup descriptor appended at the
input (26 one-hot + side + tanh netVote + SL/TP ATR + spread/ATR).
- Training.mqh: candidate-queued pass 1, binary-target pass 2, per-candidate
calibration (2.5) and OOS (3) walks. Counter mapping win->Buy / loss->Sell
lets checkpoint selection, the edge floor, the plateau ladder and the
family-wise deploy gate run UNCHANGED: precision reads as win rate among
traded candidates, chance as the base win rate, recalls as sensitivity/
specificity. Era-end META line: coverage x (p - break-even) vs the null.
- Labels are the side-conditional triple-barrier win caches - never the DB's
stop-and-reverse outcome. Logit adjustment deliberately skipped (~40% base
rate). Live inference + online learning guarded off until S3.
- Fingerprint: conditional |TGT:META1; State\META\ folder + 2-output filename
slot keep meta models fully separate from direction models.
Compiles clean (0 errors, 0 warnings). S2 run = attach a chart with
AIType=AI_META; S3 wires the votes via the per-side hooks.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Implements stage S1 of Meta_Labeling_Design.md, superseding the
original "training-time ladder sweep": the per-side journaling from
652bf81/195be20 already produces the exact candidate stream a sweep
would compute - every pattern instance the live ladders fire, both
sides, uncensored, with netVote and touchable entry price - so the
corpus is READ from the DB instead of re-implementing 26 ladder
conditions in training code. That eliminates the silent-divergence
trap outright: the corpus is by construction identical to live
behaviour. Accepted costs are documented in the module and the doc:
coverage equals the populating backtest, and sampling is one
candidate per fire-stretch (the right dedup for training anyway).
- Expert\AIBase\MetaCorpus.mqh: CMetaCorpus reader (52 tables ->
SMetaCandidate rows) + VerboseMode OnInit report: volume/closed/
S&R-win-rate per family, span, and the GMT->server bar-offset
match table (offsets +0..+3h) that S2''s label plumbing pins to -
measured, not assumed.
- DB_MaxRowsPerTable input (default 1000 = old MAX_TABLE_ROWS): a
corpus build raises it (e.g. 20000) so a 15-20 year backtest
isn''t pruned; wired through CExpertSignalCustom::MaxTableRows().
- Report-only stage: nothing downstream consumes the corpus yet.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The DB system logs objectively; the decision layer reads it to compute
win rates and adjust weights. The journaling path still had one
decision-layer tendril: rows were only written when the root''s
OpenLongParams()/OpenShortParams() succeeded. Those calls validate
ORDER PLACEMENT (broker stops-level, ATR warm-up, entry-mode
rejection) and their failures cluster in volatility/spread conditions,
so the gate non-randomly censored exactly those bars out of every
pattern''s win-rate sample - the same censoring class 652bf81/c8ef478
removed, one layer down. The ledger never needed placement to be
possible: entries are marked at the touchable side of the spread and
exits are same-pattern reversals, not broker fills.
Also documents netVote for what it is: a record of the decision
layer''s state at log time (per-pattern weights inside it drift as
ranking updates land), not an objective measure - the objective part
of a row is pattern/direction/price/result.
SIGNAL_DB_SEMANTICS_VERSION 4 -> 5: row populations gain the
previously censored bars, so the database re-keys.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Verification of 652bf81 on a fresh 7-month backtest DB surfaced the
last one-sided mechanism: ProcessSignal absorbed a reversing signal as
the exit of the opposite trade and skipped registering it. For pure
EVENT patterns that strictly alternate (MACD model 3, the zero-line
cross), every reversal was consumed and the whole ledger landed on
whichever side fired first - 60 Buy rows, 0 Sell rows - so the silent
side never earned a win rate and UpdateSignalsWeights() weighted the
pattern from one side only. State patterns escaped by re-firing one
bar later.
The reversal now closes the opposite trade AND registers its own row;
the existing duplicate/outdated/open-trade checks still bound the
table at one open trade per pattern+side. Row populations change
meaning, so SIGNAL_DB_SEMANTICS_VERSION 3 -> 4 re-keys the database
(the v3 file is orphaned, not wiped - schema is unchanged).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The labelMatchesVote gate compared a single last-writer-wins label
(LongCondition then ShortCondition) against the net vote sign, which
structurally censored the pattern tables: a long event co-occurring
with any short-side state model lost its label to the later writer and
was dropped, while the mirrored short event journaled fine. Ichimoku
models 0/3 and MA model 1 could not produce a row at all by
construction (MA model 1 was "revived" in 8710240 yet still could
never journal - its weight-10 vote is exactly cancelled by the
opposing Pattern_0 state), and every pattern's win rate was measured
on a with-trend-only subset - the exact statistic
UpdateSignalsWeights() feeds back into the weights, self-sealing:
no rows -> no win rate -> default weight -> still censored.
- Direction() now evaluates the two ladders separately and snapshots
each ladder's matched pattern into its own side slot; each side that
matched journals its own row. The flat-vote poisoning the old gate
fixed stays fixed: a label can no longer contradict its side.
- The filter's net vote (raw pattern-weight units) is stored as a new
netVote column - data, never a drop filter. Snapshot is keyed on the
ladder setting a label, not on its weight, so a 0%-win-rate pattern
keeps journaling and can recover.
- SIGNAL_DB_SEMANTICS_VERSION is folded unconditionally into the DB
filename fingerprint: pattern-definition changes (b2069bc, 8710240)
re-key the database instead of blending incompatible Pattern_N
populations under one key, which the input-hash fingerprint cannot
see. 7 months of mixed-semantics rows shared one file because of it.
- dbVersion 2.0 -> 3.0: schema changed, and inserts carry the new
column, so the version-mismatch folder wipe is the migration.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Direction is closed - normalised asymmetry fails on three instruments
with a working positive control, and the classifier's own best-of-999
era-cap test agrees (+0.9pp = 1.48 sigma, family-wise p=1.0000). SIZE is
a different question and RANGE clears at ~4x its null.
Checked the denomination before building on that, since the source memo
warns to: m_excUpCache holds (maxHigh - fill)/ATR, so "RANGE is
predictable" is a claim about travel RELATIVE to current ATR, not a
restatement of "ATR is autocorrelated". It is exactly the part a fixed
multiple (stop 3.31*ATR, target 1.64*ATR) discards.
A second small CNet, 760 -> 24 -> 32 sigmoid outputs = P(price reaches
ladder rung k) upward and downward. Survival parameterisation rather than
regressing the multiple, because it needs nothing new from CNet: sigmoid
outputs and the per-neuron delta the `total != 3` branch already applies
(a quantile head would need a linear activation and a pinball gradient in
Network.mqh, Network.cl and the DirectML path, on a class four topologies
share). Targets are free - m_ladderUpAt already records first-touch age
per rung with 0 meaning never reached.
Separate net, not extra outputs on the classifier: more outputs would
change m_outputNeuronsCount, the .nnw shape and the fingerprint, and push
the count off 3 - the exact condition backProp uses to select the joint
softmax gradient the 3-class head depends on. The classifier is
bit-for-bit unaffected and this is removable without trace.
STAGE 1 PLACES NO ORDERS. It reports a Brier skill score against the
constant per-rung base rate - the baseline a fixed ATR multiple already
assumes - with both predictors fitted IS and evaluated OOS, so neither
gets a look at the test set. Positive skill justifies Stage 2 (drive
SL/TP and sizing off ExcursionQuantile, which is defined and deliberately
uncalled). Zero or negative means ATR already carries everything and
Stage 2 must not be built.
Trains only on primary occurrences: the replay queue oversamples for
CLASS balance, and a direction-balanced sample is a biased SIZE sample.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
SignalClusterWindow 3 -> 10 for all topologies. On H1 a 3-bar window
collapsed only the tightest runs and left visible clusters at every
turn; 10 bars is closer to the spacing of genuinely distinct setups.
ALTERNATION. Rule 1 only collapses a same-direction run INSIDE the
window; past it a second Buy is emitted with no Sell between, giving
Buy/Buy/Buy/Sell. With both directions tradeable that sequence is the
model re-entering a move it is already in rather than finding a new
one. The kept sequence must now alternate: the first signal passes,
and after that a direction passes only if the last KEPT signal was the
opposite one.
Added to ALL THREE consumers, with identical logic, because they must
agree:
- NmsLiveAccept -> the live trade
- pass 3's OOS replay -> the tally the deploy gate grades
- PruneDirectionalClusters -> the drawn history
A rule applied to only some of these certifies one strategy and trades
another - the same defect class as the geometry the gate certified
while OpenParams placed something else (9a7c37f) - and would draw the
user arrows the EA would never have taken.
Deliberately NOT applied to the LABEL. The barrier target has no "must
flip" invariant: consecutive Buy labels are routinely correct, and an
earlier alternation gate was removed with the triple-barrier relabel
for exactly that reason. This filters what is ACTED ON, which is what
"applies to training" can honestly mean here - pass 3's declustered
tally is the training-side number that decides deployment.
BothDirectionsTradeable() is the stated precondition (with one side
disabled there is no opposite to wait for, so alternation would
suppress everything after the first call). This build has no
long-only/short-only input, so it is constant true - kept as a named
predicate so a future direction restriction has one place to change
rather than three call sites silently assuming both sides.
Build tag -> nms-alternate-v4.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Three changes, one theme: the trade placed, the trade graded, and the trade
computed are now the same trade.
1) GEOMETRY WIRE (correctness, the ranked #1 open issue). The measured barrier
pair reached the LABELS only - OpenParams still placed orders at the enum
geometry (2*ATR/6*ATR), so the deploy gate certified "reaches 1.62*ATR before
3.33*ATR above break-even" about trades the EA never placed. Published via
g_DerivedSlAtrMult/g_DerivedTpAtrMult (ConfidenceBridge, same same-tick
contract as the confidence globals, because OpenParams runs on the root signal
which has no pointer to the AI filter). Two writers: DeriveBarrierGeometry at
era 0, and the .cfg adoption a deployed model takes. Overrides both legs and
both Intelligent modes - the certificate is exact or it is nothing. TP is
ATR-anchored like the label, NOT risk-relative, so a floor-widened stop cannot
reshape the certified target.
2) BATCH NORM RUNS DEVICE-SIDE ON OPENCL. Four kernels in Network.cl -
forward, hidden gradient, gamma/beta accumulate, gamma/beta apply - each a
line-for-line transcription of the host implementation (NormalizeHost /
HiddenGradHost / StepGammaBeta) including every NaN guard, clamp, and the
exact moment-write ordering. The host copies remain the runtime for the DLL
and pure-MQL5 tiers and the reference the kernels must match.
Because this box has no OpenCL platform, the safety story is layered:
- shim validation: kernels compiled as C and driven against a fp64 host
transcription over NaN-poisoned stats, NaN gamma, over-clamp inputs, the
frozen path, both optimizers, 3 batches - ALL PASS, worst normalized diff
0.132 vs tolerance 1.0
- in-situ self-check: each kernel is compared against its host twin ON FIRST
USE on the real device (SelfCheckBn*), covering what the shim cannot - arg
indices and buffer bindings. Any disagreement resyncs from the good copy,
latches all BN kernels off process-wide, and training continues host-side.
A transcription bug costs a warning and some speed, never a poisoned .nnw.
- sync discipline: BatchOptions is now a CBufferDouble with explicit
authority tracking (m_bnDeviceAuthoritative). Checkpoints/saves pull
read-only; restores/loads/resets push; a mid-batch handover drains the
device gamma/beta accumulator into the host arrays so no sample is lost.
3) SMALL FIXES. Apply-kernel build failure now latches the dispatch path at
init (one warning instead of warning + failed Execute). Build tag bumped to
win-scoring-gpu-v1 - first tag change since expectancy-stop-v1 despite five
binary-changing commits.
Both build variants compile 0 errors, 0 warnings.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The barrier geometry is derived from the instrument's own excursion
distribution (stop at q75 of adverse travel, target at q50 of favourable),
and then a 1:2 floor was applied on top, raising the target to twice whatever
the stop happened to be. On SP500 H1 that pushed the target to 6.66*ATR,
reached on 3.3% of bars inside the horizon - so the label became "almost
never a win" and every topology was trained to predict an event that
essentially does not occur. A measured target has to stay measured.
The ratio never bought what it was believed to buy. A reward:risk floor does
not create expectancy; it trades hit rate against payoff at a break-even the
geometry already fixes - which this project has separately MEASURED (payoff
0.92 -> 5.72 with expectancy flat). What it did buy was two outages: four
consecutive Market validation rejections for "no trading operations" when it
rejected 100% of setups, and the label corruption above.
Removed:
- the input and the RISK_REWARD_RATIO enum (deleted, not left dangling - a
live enum with no input behind it is the shape of the stale-.set incident
that trained ~250 eras on the wrong target)
- the forced target raise in the label geometry
- the rrOK eligibility gate in the barrier-geometry scan, so every unclamped
pairing now competes on the measurement alone. Clamping stays disqualifying
for its own unrelated reason.
- the reward < minRR*risk veto in OpenParams
Kept: g_TradeRewardRiskRatio still computed and still bridged to Kelly sizing
in MoneyIntelligent - the ratio as a SIZING input was always the sound use.
Risk stays bounded where it actually is - account risk % and CRiskBudget.
The low-reachability warning survives but is re-aimed: with nothing inflating
the target, a target the market rarely reaches can only mean the horizon is
truncating the excursions the geometry is derived from.
Both build variants compile 0 errors / 0 warnings.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
AutoTuneIndicators now defaults to FALSE, and the 33 AD/Wyckoff parameters
it used to search are now inputs.
WHY THE DEFAULT FLIPPED - not because the search is broken. It is correct,
and its own Sidak gate is what proves it: 324 candidates per model on
SP500 H1, "no improvement" on all four topologies (0.00236 -> 0.00236 on
the AD configs, 0.00370 -> 0.00370 on PAI), winner rejected at p=1.0000.
It cannot do better here by construction - it ranks candidates by MARGINAL
MI, and the headline MI is 0.00370 nats against a shuffled null of
0.00379 +/- 0.00061 (p=0.4975), so every candidate is a noise draw and the
maximum over N of them is noise too. The cost is 45-56 min per model in
one synchronous call with no yield, and it was the amplifier for the
handle leak fixed in 33f106d. The EA's own report says it plainest: "no
per-feature indicator retuning will help."
THE INPUT STAYS. TuneIndicatorsByFilter is one function of twelve in
AIBase/AutoTune.mqh; the other eleven are the MI/lag/excursion/geometry
diagnostics that produced every verdict this project relies on, and they
run regardless of this flag. Removing the input invites removing the file.
WHY THE INPUTS WERE NEEDED. All 33 were literals in CADIndicatorTuner's
constructor with no input of any kind, while MA/RSI/MACD/Ichimoku have had
their periods exposed from the start. On the AD configs those indicators
contribute 28 of 64 features per bar. Survivable while the tuner searched
them; indefensible with it off, where they would freeze at values nobody
chose.
CONSOLIDATED 33 -> 18. volClimax/volHigh/rangeClimax/rangeSignificant/
stVolRatio/atr were duplicated verbatim across CumulativeDelta, Wyckoff
Events, Failed Structure and Bar Inversion - the same constants restated
3-4 times. One concept, one input. They are SEEDS: each fans out to the
indicator's own struct field, so with the tuner on it retains full
per-indicator freedom to move them apart. Same contract as PeriodMA.
NO RETRAIN. Every default is byte-identical to the literal it replaces,
and the fingerprint's new ADP token is appended ONLY on deviation
(MACD/Ichimoku/BN/XA convention), gated on the AD features being enabled.
At defaults the token is absent, so every model on disk keeps its filename
and stays loadable. Without that guard, merely EXPOSING these parameters
would have re-keyed every config and forced a from-scratch retrain of all
four topologies for a change that alters no number anywhere.
All-or-nothing rather than per-input, so the token can never encode a
partial picture of what the features were built from.
Compiles clean: 0 errors, 0 warnings.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The daily (4%) and total (8%) rules bound how FAST an account can lose. Nothing
noticed WHETHER it was losing. A negative-expectancy signal traded at 1% inside
that envelope breaches no rule and still arrives at zero - it just takes longer,
with every limit green the whole way down. That is the realistic way this EA
destroys an account, and no existing guard could see it.
THE ARITHMETIC THIS ENFORCES. Expected value per trade is p*TP - (1-p)*SL - cost.
With no directional edge p equals SL/(SL+TP), which is also the break-even rate,
so the payoff terms cancel exactly and EV = -cost. Expected P&L is -(trades) x
cost: strictly negative, proportional to activity. Measured here: directional
precision 23-24% against a 25% break-even, flat across every confidence tier,
with 58 points of spread on SP500. Sizing, stop placement and trailing move
variance around that mean; none of them changes its sign.
So every closed position now reports its result in R (net profit over money
actually at risk) and the running mean is tested against zero. Above the
configured minimum sample, if mean + sigma*SE < 0, new entries stop.
- SIGNIFICANTLY below, not merely below. A run of losers is ordinary variance
even for a profitable system; halting on the raw mean would be the same
act-on-noise error the MI gates exist to prevent. Using the standard error
means a wide spread simply demands more trades before the rule can fire.
- NET of swap and commission (ResolveClose already sums all three). Deliberate
and load-bearing: when the edge is zero, cost IS the expectancy, so a gross
version would measure a strategy nobody can trade.
- Reported in R so symbols, lot sizes and balances share one scale and one
mean. Trades without a stop are not scored rather than assigned a guessed R.
- LATCHED across restarts, like the daily halt and for the same reason: a
latch a reattach clears is not a latch. Clearing it means deleting the risk
state file, deliberately, after looking at why.
State is appended to the risk file length-guarded, so files written before this
still load and start their sample at zero rather than misreading.
Defaults 40 trades / 2 sigma; ExpectancyMinTrades = 0 disables it.
This does not make the strategy profitable and is not meant to. It stops paying
tuition on one the results say is losing, and does it on measurement rather than
on a drawdown limit finally being reached.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Three enums left the Inputs tab. They were three things a user had to pick and,
in the tester, three more axes for a genetic optimization to overfit.
Entry_Multiplier is pinned to MARKET. Its pending modes place the entry at a
LEVEL while the rest of the pipeline measures from the bar open - the exact
mismatch that manufactured the +0.097 R "retail fade" result later retracted as
a fill artifact. This codebase's fill model cannot honestly simulate a pending
entry, so it is no longer offered.
SL_Mode/TP_Mode become a STARTING pair. ReportBarrierGeometryScan now ADOPTS its
winner instead of printing "set SL_Mode/TP_Mode to X and retrain":
- only when it clears the family-wise gate from 04ee2e1 (beat the null of the
MAXIMUM, not merely the incumbent). This is why that gate had to land first:
without it, removing the inputs would hand a noise-picked geometry direct
control over the training target with no human in the loop - strictly worse
than the input it replaced. On SP500 H1 today it does NOT clear (p=0.1463),
so 2:6 is what you get - now chosen by measurement rather than assumed.
- only at m_eraCount == 0. Relabelling a partly-trained net moves the target
out from under weights already fitted to the old one.
THE GEOMETRY LEFT THE WEIGHTS-FILENAME HASH, because it is now measured. Same
rule that moved the horizon and the derived topology values out: a filename
keyed on a measured quantity changes the moment the measurement does - a few
more bars shift which pairing wins - and the EA then looks for a file that does
not exist, starts from era 0 and orphans a trained model silently. It is PINNED
IN THE .cfg instead: appended at the end (the only backward-safe change),
length-guarded like the 2026-07-30 derived pair, and ADOPTED on load rather than
compared, so a trained model keeps the barriers it actually learned and never
re-measures.
Two traps closed while wiring it, neither of which announces itself:
- m_barrierHorizonResolved latches the horizon ONCE PER PROCESS. Adopting 2:8
(wants ~192 bars) after it settled for 2:6 (128) would label the new target
against the old ceiling - the truncation fixed in 168422f, where every model
learned "target within 128 bars" while the EA holds to SL/TP. It lands in
Neutral, not in the timeout counter watching for it. Unlatched on adoption,
along with the label cache the old barriers filled.
- the .cfg adopt runs at init, before the horizon latches and before any label
is computed, so a resumed model has its pinned pair in place first. Verified,
not assumed.
FORCES A FULL RETRAIN: the fingerprint change orphans every existing .nnw.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
- Implemented sqx_audit.py to audit StrategyQuant X trade lists, focusing on performance metrics and cost analysis.
- Created sqx_portfolio.py to evaluate portfolio performance based on uncorrelated components and their impact on risk and return.
- Developed swing.py to analyze cost ratios across different holding periods and assess swing trading structures.
- Introduced test_management.py to investigate the effectiveness of exit rules on random entries and their impact on expectancy.
- BufferDouble: replace hardcoded "DirectML/CPU-DLL" with dynamic backend name
and add buffer index/element count to all error prints for easier debugging.
- NetPersistence: distinguish missing file from transient lock by probing
FileIsExist before logging, eliminating false "sharing violation" warnings
when no saved model exists on first run.
Adds spread/ATR and the spread change ratio as network inputs (EnableSpreadFeature,
default on). Spread is the one microstructure channel that is both FX-available and
genuinely historical in the Strategy Tester - "during testing, the spread is not modeled
but is taken from historical data" - so unlike swap, signed tick flow or depth of market it
is something a backtest can honestly validate.
What it encodes, stated precisely because the raw measurement overstates it.
research/test_spread.py found spr/atr the strongest single feature in this codebase, on 5
of 8 instrument/geometry cells at 2-4x any volume feature. But the barrier LABEL charges
the spread inside its own barriers, so a wide-spread bar is mechanically likelier to
resolve as a loss and the feature would partly be predicting its own cost model. Relabelling
at zero cost and re-measuring the identical feature showed 20-40% of it WAS that tautology
and the majority was not (XAUUSD retained 97%). What survives is a volatility-regime
reading: spread is near-fixed while ATR is not, so the ratio runs high exactly when
realised volatility is below its own ATR estimate, which genuinely predicts whether
ATR-scaled barriers get reached. It is UNSIGNED - Neutral-vs-directional only, never a side.
Also fixes a stale-index bug I introduced with the cross-asset panel and had just repeated
in the spread series. Both cached on length alone:
if(m_crossAsset.Bars() >= bars) return true;
MQL5 series indices are relative to NOW, so one new closed candle shifts every index by
one. Keyed only on length, the panel keeps serving its index 0 as a bar that is no longer
the newest, and every cross-asset value is read one bar out of step with the price features
sitting beside it in the same vector - silently, with no error and no shape change. This is
the same class of defect as the dtStudied watermark behind the zero-direction backtests.
Both now carry a datetime anchor on m_Time.GetData(0), the same invalidation key the
label/feature bar caches already use.
And a performance fix that fell out of it: with correct invalidation the panel rebuilds on
every new bar, and RefreshConvergedSignal runs per bar - which in the tester would mean one
full multi-symbol resample per simulated bar at training depth. Inference only reads bars
0..m_historyBars-1 plus the panel's own slow window, so it now requests exactly that. The
cache check is >=, so a deeper panel left from training still satisfies it.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Two defects surfaced by research/test_classic.py, both verified fixed by re-running the
transcription against 178k bars of EURUSD H1.
CSignalMA model 1 could never fire. For any recursive average - and MA_TYPE_EMA is the
shipped default - MA(i) = a*Close(i) + (1-a)*MA(i+1), so
DiffMA(i) = a * (Close(i) - MA(i+1))
DiffCloseMA(i) = (1-a) * (Close(i) - MA(i+1))
are positive multiples of one quantity and always share a sign. Model 1 asks for a close
BELOW a RISING average, which is precisely the combination that identity forbids: 0.000%
of bars, either direction, any symbol. The MQL5 standard library this was ported from
defaults to MODE_SMA, where the two are merely correlated - the bug arrived with the EMA
default, not with the port. Reading the slope one bar back (DiffMAPrev) breaks the tie for
every MA type while keeping the model's stated meaning. Now fires on 7.92% of bars.
CSignalIchimoku model 11 fired on 27% of bars at weight 100. Sanyaku is three standing
STATES conjoined with no transition term, so it held across long stretches - and being
last in the if-chain at the top weight, the module's highest-conviction reading was also
its most common one, overwriting all eight event models below it on a quarter of all bars.
The old comment rejected an event form because "demanding all three flip on the same bar
would fire almost never" - true, but that is not the alternative. Kouten is the TURN: the
ALIGNMENT transitions, and only one role need change for it to. Testing !Sanyaku(idx+1)
fires once per aligned stretch. Now 2.17%, in line with Kumo breakout (2.4%) and the
strong TK cross (1.1%). DataReady() extended one bar deeper to cover the lookback.
Neither pattern showed edge before or after; this is about the models meaning what they
say and the vote not being dominated by a constant.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
MEASURED COST OF THE GA, which is what retired it. Per generation:
rung 0: 8 cand x 3 seeds x 3 eras = 72 eras
rung 1: 4 cand x 3 seeds x 8 eras = 96
rung 2: 2 cand x 3 seeds x 20 eras = 120
= 288 eras/generation x 4 generations = 1152 eras BEFORE the winner's
real training began. Against the observed era times on SP500 H1:
PAI 29.1 s/era -> 9.3 h (matches the observed 00:37 -> 09:22)
CONV 41.3 s/era -> 13.2 h
LSTM 150.4 s/era -> 48.1 h
HYBRID 154.6 s/era -> 49.5 h
Two days to tune is not a first-run experience, and it is the phase in
which the panel goes quiet, which is what made it look like a hang.
It also bought nothing. The space is 90 points (10 MA periods x 9 MA
types), so 1152 evaluations revisited each point ~13 times; and rungs of
3 and 8 eras cannot separate two MA periods at all. The 2026-08-01 run
proves it: every finalist scored 25.0-25.9% balanced accuracy - below the
33.3% one-class floor, i.e. indistinguishable noise - and the search then
"deployed the winner" of that.
THE ERROR WAS THE SCORING FUNCTION, not its constants. Using a full
training run to choose a feature's period is a wrapper method paying
wrapper prices for a decision that does not need one. The reference book
does not do this: ch. 3.3 selects inputs by measuring each candidate
indicator's CORRELATION with the target and dropping the ones with none,
with no network involved.
So: rank candidates by the MUTUAL INFORMATION between the resulting
feature vector and the triple-barrier label. MI rather than correlation
because the label is 3-class categorical and the features are not
monotonically related to it. Equal-FREQUENCY binning (rank-based),
because these features are ATR-normalised and heavy-tailed - fixed-width
bins put nearly everything in one bucket and report ~0 information for a
genuinely useful feature.
Scoring is arithmetic over the feature cache, so it costs seconds and its
cost is independent of topology: LSTM now tunes as fast as the MLP.
Coordinate sweep, not product sweep - cost is the SUM of per-parameter
candidate counts, so enabling every indicator stays affordable - with a
second pass that breaks early once nothing moves.
Sampling is IS-ONLY. Letting the OOS window influence which indicator
settings ship would mean the holdout had been used for selection and had
stopped being a holdout.
HONEST LIMIT, recorded because it is the price: MI is marginal, so a
parameter that only pays off in combination with another can be missed
(Guyon & Elisseeff 2003, filter vs wrapper). Given the wrapper it
replaces was ranking pure noise at 48 h a run, this is strictly better.
Deleted with it: GaRungEras/GaExtract/GaStore/GaMutate/GaRandomCandidate/
GaBlockCrossover/GaSortAliveByScoreDesc/GaBreedNextGeneration, 14 m_ga*
members, the GA_*/TUNE_POP_* constants, and ComputeTuneTrialBudget.
AND m_evalMode/m_evalEraBudget, because nothing set them any more - 28
read sites all permanently inert. That is not a tidy-up: the `if
(!m_evalMode)` guard on UpdateClassPriors is exactly what silently
disabled the imbalance correction for entire runs two commits ago. Dead
machinery that still reads like live machinery is this codebase's most
expensive recurring bug, and leaving 28 more instances of it would have
been indefensible.
The panel's tuning-progress state goes too - tuning no longer takes long
enough to need one.
Both builds compile 0 errors / 0 warnings.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The AutoTuneIndicators input is now automatically derived via `ComputeTuneTrialBudget()`, so the default is set to false to prevent manual interference.
Adjust default stop-loss mode from SL_ATR_x1 to SL_ATR_x2 and default take-profit mode from TP_ATR_x3 to TP_ATR_x6. This improves the risk-reward alignment in line with the recommended minimum ratio and ensures setups are not rejected under the EA's target reward parameters.
Every removal below is FINGERPRINT-NEUTRAL by construction: each retired
input is pinned to the exact value it already shipped with, so running
models keep their filenames and resume rather than restarting at era 0.
Verified field by field against BuildConfigFingerprint.
Removed as inputs, kept as pinned constants (the value was never a
preference the user had a basis to change):
- OutputNeuronsCount. The regression head predicts a continuous quantity
the triple-barrier label does not contain; the target is an EVENT, so
the right output is its probability. The regression code paths stay
implemented and dormant - they cost nothing and removing them would
touch every scoring path at once.
- MinRecall. A safety floor, not a preference, and the only direction a
user can move it is the harmful one: raising it past what the config
reaches yields NO model, not a better one (observed repeatedly at 60).
- SwingConfirmationBars. Stopped gating the labels with the relabel, but
is STILL load-bearing for the swing-context input features - it is the
ZigZag repainting embargo, and without it those 9 features read a leg
the live bar could not have had yet. Pinned, not deleted.
- MaxErasPerRun (runaway backstop, never reached in a healthy run),
FreezePriorCalibration (unanswerable by a user; near-balanced labels
make the priors stable anyway), VerboseMode (developer view, joins
DebuggingMode), MACD/Ichimoku periods x6 (both indicators ship
disabled, and as optimizer dimensions they are pure overfitting
surface - the AI auto-tuner is the supported way to move them).
- SignalClusterWindow -> 3, no longer an input. Barrier labels make
consecutive setups real, which argued for 0; it is not 0 because on D1+
a 6-bar window spans over a week and two arrows a day apart on a
weekly-scale move are one event. 3 splits it correctly by timeframe.
- EnableOnlineLearning -> ON. Adapting to a changing market is what keeps
a months-attached model from going stale, and the rolling-accuracy
freeze is what makes it safe. See the caveat noted in the handoff: it
had not been forward-tested on a live feed when this became default.
Removed entirely:
- Intraday Time Filter (5 inputs + Signals/SignalITF.mqh). Two of its
five inputs were raw BITMASKS, which is an implementation detail
exposed as a control. The job is covered three times over by things
that are declarative or that learn: the session filter, the
time-of-day/day-of-week input features (the network discovers which
hours are good rather than being told), and the journal's time buckets.
- Market Depth Filter (5 inputs + Signals/SignalMarketDepth.mqh, plus
its OnInit probe and OnDeinit release). It needs real level-2 data
that this broker - and most retail MT5 brokers - do not provide, so
the module has never once executed against real data. Shipping four
tuning dropdowns for an untested path is worse than shipping nothing:
the only users who could enable it would be its first-ever testers,
live. If DOM returns it should be a FEATURE fed to the network, not a
rule-based veto with hand-tuned thresholds - imbalance is data.
- IndicatorTuneTrials, replaced by ComputeTuneTrialBudget(). The useful
budget depends on how many parameters are actually being searched,
which depends on which features are enabled - so one number meant
wildly different things run to run. The shipped 32 was ~10 candidates
per dimension against one enabled indicator (wasteful: each costs
GA_SEEDS full training runs) and under one per dimension against all
nine (blind). Now population ~ 4 x active dimensions, clamped [8,64],
with CADIndicatorTuner::ActiveDimensions() defined immediately above
PerturbRandom() so the two cannot drift apart.
- Six orphaned enums (TUNE_TRIALS_PRESET, DOM_*, ENTRY_HOUR_OF_DAY,
TIME_FILTER_DAY_OF_WEEK), 81 lines.
Other UX:
- SL_ATR_x1 / TP_ATR_x3 now carry the "(classic)" default marker every
other preset enum in the file already used. Nothing in the SL/TP
dropdowns previously told a user which pair was the shipped default -
which matters far more since the relabel, because those two define the
labels and changing either forces a retrain.
- Neural Network section moved directly ABOVE AI Input Features: choose
the architecture, then choose what it sees. NN Optimizer / Performance
stays last - the Adam/Sgd inputs are declared in AI/Network.mqh and
render immediately after that divider.
- News feature + window moved to the end of the AI feature list, below
Wyckoff Bar Inversion.
- Dropped "(0-100)" from Min vote to open - it is an enum, not a number.
Both builds compile 0 errors / 0 warnings. No retrain forced.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The 31:1 class imbalance was self-inflicted by the TARGET, not a property
of the market. Labelling only the exact bar where a ZigZag pivot confirms
gave Buy 1164 / Sell 1164 / Neutral 35841, and every correction mechanism
this codebase accumulated sits downstream of that one choice: the
logit-adjusted loss and its range cap, the prior EMA, the +-3.0 output-bias
seed, balanced-accuracy-then-precision selection with its coverage floor,
the recall floor and its catch-22, the alternation gate, NMS, and the four
oversampling designs that collapsed before them.
The reference this engine is built on (references/neuronetworksbook.pdf
ch. 3.1/3.3) also uses ZigZag, but targets the DIRECTION TO THE NEXT
EXTREMUM on every bar - ~50/50 by construction, with no imbalance to
correct at all. It never had this problem because it never asked "is this
the pivot bar".
Labels are now the triple barrier (Lopez de Prado ch. 3), using the EA's
OWN SL_Mode/TP_Mode: does a trade opened at this bar's close reach its
target before its stop, within a horizon. Buy = long resolves, Sell =
short resolves, Neutral = neither. Consequences:
- dir-precision in the era line stops being a proxy and becomes the win
rate of the strategy under its own exit rules.
- Expected balance ~25/25/50 at the shipped 1:3 (gambler's ruin), i.e.
~2:1 instead of 31:1. Measured and logged at the end of the prebuild.
- Spread is charged on both legs, so it is a NET win rate.
- Intrabar ambiguity resolves to the STOP. OHLC cannot order two touches
inside one bar and the optimistic reading is how a backtested edge
becomes a live loss.
ZigZag stays as input features (EnableSwingContext) and now also supplies
the vertical barrier: the horizon is the median confirmed leg length,
snapped to a coarse ladder. Derived, not configured, and deliberately kept
out of the filename fingerprint - a filename keyed on a measured quantity
orphans a trained model the moment the measurement moves.
Removed, because the premise died with the old target:
- the alternation gate. Correct for pivot labels (a ZigZag cannot emit two
same-type pivots in a row, so a repeat was provably a false fire), and
wrong for barrier labels, which answer each bar independently. It also
took its worst consequence with it: a one-sided model previously got ONE
trade per backtest, a hard blocker on marketplace validation.
- SignalClusterWindow now defaults off - it de-duplicated repeats that are
now real trades. Kept as an opt-in display control.
- LABEL_WINDOW_BARS, the pivot-widening pass, ConfirmedZigZagLabel.
- the era-0 output-bias seed now needs a genuinely dominant class (0.70)
rather than 0.40; at ~50% Neutral a +-3.0 seed is a distortion, not a
correction.
Also fixed, both found while wiring the above:
1. RefreshConvergedSignal sized its buffers from a date delta
(Bars(sym, period, dtStudied, TimeCurrent())). dtStudied is a training
watermark; in the tester it is loaded from a live-chart save AHEAD of
the simulated date, so the interval inverted, Bars() returned ~0, and
the buffer came out at exactly m_historyBars - deep enough for the OHLC
window and far too shallow for the Donchian-50 / 20-bar-return / SMA
extension behind it. Inference silently computed DIFFERENT features
from the ones training learned on, live as well as in the tester. Now
sized from what the feature builder actually needs.
2. The barrier horizon is resolved on the deployed path too. A deployed
model never enters Train(), so it never reached the prebuild, and
OnlineLearnStep reads the horizon as its confirmation delay - left at
the fallback it would have backpropped bars whose barriers had not
resolved. Silent lookahead in the one place that writes to a live model.
SL_Mode/TP_Mode join the weights fingerprint: they define the labels now,
so a model trained at 1:3 must never be silently reused at 1:1. This
re-keys every pre-existing model by design - none were trained on this task.
Inference census extended with the vote gate. LongCondition/ShortCondition
open with a readiness check the refresh counters never see; in the tester it
reduces to "the seeded _optcache.nnw must have LOADED", and if it did not,
every vote is hard-zeroed while the model still answers Buy. The old three
counters would have read that as "the model says Neutral" - false, and a
completely different fix. This is the leading candidate for the
zero-direction backtest and the census can now name it in one run.
Both builds compile 0 errors / 0 warnings. Forces a full retrain.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The imbalance section offered nine controls for one job. Audited against the
code, five of them did not do what their names said at the shipped defaults:
AILogitPriorStrength DEAD - Inference.mqh's post-hoc prior early-returns
whenever the adjusted loss is on, which is default.
OversampleParity DEAD in training - Training.mqh gated the replay loop
on !useLogitAdjustedLoss (correctly, citing Buda et
al. 2018). Live only in the online-learning path.
EnableMinorityReplay DEAD as replay. It survived ONLY as a focal-gamma
damper - "replay minority bars through pass-2
oversampling" was a focal-loss switch.
ConstrainReplay DEAD as a cap; it only chose damper 0.125 vs 0.25.
UseStaticPrior An exact duplicate of FreezePriorCalibration - the two
were OR'd together in the single place either is read.
So they were not five mechanisms fighting; they were one mechanism plus eight
knobs that mostly described machinery that no longer ran. That is worse than
a real conflict, because the log agreed with the names: the label-cache line
printed "reps up to 28x (90% parity) (seeding era 0's class-balance
oversampling)" on every run, describing an oversampling pass that had been
switched off. It is fixed here too - it cost this session a wrong diagnosis.
The one genuine redundancy was focal loss, running at gamma*0.125 alongside
the adjusted loss: two corrections on the same axis, the exact stacking
failure this file already cited Buda et al. for in two other places, damped
by a replay flag whose replay path was itself dead. Removed rather than
re-tuned. The plateau ladder is unaffected - its escape is the learning-rate
warm restart; the gamma anneal beside it only ever stepped toward zero.
WHAT REMAINS is logit-adjusted loss (Menon et al. 2021) plus a prior freeze:
LogitAdjustTau 0 = off; replaces the separate EnableLogitAdjusted-
Loss boolean, since a strength dial where 0 already
means off does not need an on/off switch beside it.
FreezePriorCalibration unchanged.
It is the only one of the six corrections with a consistency guarantee, and
it is consistent for exactly the balanced-error metric checkpoint selection
already ranks on - so the loss and the deploy decision optimize one thing.
The online continual-learning path keeps its own alpha-balanced focal weight,
now as constants pinned to the removed inputs' shipped defaults, so its
behaviour is unchanged. It legitimately needs its own correction:
ApplyLogitAdjustment() only runs inside a training run, so a deployed model
that was reloaded carries no logit offsets and would otherwise stream 31:1
data into itself uncorrected.
The weights-filename fingerprint is BYTE-IDENTICAL. The focal slot was a
double fed to a %d conversion and had always emitted a literal 0; the |MR:
segment is written as the constant its shipped defaults produced. Dropping
either would have re-keyed every model and forced a from-scratch retrain of
the one topology currently converged and trading.
Also removed as orphans: FOCAL_GAMMA_PRESET, MAX_OVERSAMPLE_REPLICAS,
OVERSAMPLE_PARITY_FRACTION, PLATEAU_GAMMA_STEP, and the now-unreachable
"neutralized by prior correction" diagnostic.
Both builds compile 0 errors, 0 warnings. No retrain forced.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Stops keyed to the recent swing extreme make a trade's risk a function of
how far the last swing happens to sit rather than of current volatility. On
a shallow pullback the swing sits close to the fill, so the stop is tight
enough to be taken out by noise on setups that then run to target - which is
what the Perceptron's signals were showing.
SL: lowest_low/highest_high -/+ mult*ATR -> entry -/+ mult*ATR
TP: TP_PREV_SWING (opposite swing) -> removed; ATR-from-entry
SL_PREV_SWING, TP_PREV_SWING -> removed from the enums
The SL anchors to `price` (the resolved entry), not to base_price: with a
pending entry those differ by the whole entry offset, and the risk Money
sizes against is entry-to-stop.
MIN_SL_ATR_MULTIPLIER 2.0 -> 0.5. That floor existed because a swing-
anchored stop could land arbitrarily close to the entry and needed a bound
unrelated to the chosen multiple. An entry-anchored stop is exactly
mult*ATR by construction and cannot collapse, so leaving it at 2.0 would
have silently overridden SL_ATR_x1 to 2*ATR - making the input a lie AND
forcing TP >= 4*ATR just to clear the default 1:2 rejection filter. The
broker's own stop level is enforced separately and precisely by
TCAdjustStops(), so this is now a pure sanity net.
Default TP_Mode TP_PREV_SWING -> TP_ATR_x3, so SL_ATR_x1 + TP_ATR_x3 gives a
realised 3:1 against the 1:2 filter. TP_ATR_x2 would sit EXACTLY on the 2.0
boundary where price-normalization rounding alone can reject the setup; the
default leaves a deliberate gap. This is the same interaction that once
rejected 100% of setups on every symbol (see TP_INTELLIGENT_BASE_RR).
Swing validity guards now reject only when the configuration actually uses a
swing - i.e. ENTRY_PREV_SWING. Previously an unsynced or thin history
rejected EVERY trade, including configurations whose levels no longer
reference a swing at all. The guards are kept, not deleted: a bad swing must
still never reach an entry price, and iLow/iHigh are no longer called with a
possibly-negative index.
TP_INTELLIGENT stays risk-relative. Now that risk is exactly mult*ATR the
risk- and ATR-relative forms coincide, but risk-relative keeps its
reward:risk guarantee exact after the floor or TCAdjustStops widens a stop.
Both builds compile 0 errors, 0 warnings.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Completes the derived-topology work. Three inputs removed.
AIType loses its depth suffix - AI_MLP/AI_CONV/AI_LSTM/AI_HYBRID, five
entries instead of eight. Depth is now derived from the two endpoints
the taper already has to connect (derived first-layer width, output-tied
final width) at a 2x per-layer compression target, clamped [2..5].
Asking a user to pick a layer count while the code derives the widths
those layers taper between was asking for half a decision: at 64 units
tapering to 12, four layers compress by 1.4x per step and five by 1.3x,
so the extra depth bought no abstraction. On the shipping H1/10y default
the derivation lands on 3 layers - the depth that actually won Run 2.
StudyPeriods removed. There is no case for training on less data than
the broker provides at a ~6% directional base rate; the honest
generalization read comes from the OOS holdout, not from withholding
history. Training now starts at the earliest available bar, floored by
MinTrainYear, which answers a different question (excluding dubious
pre-history) and stays.
That required closing the hazard the old code documented: the capacity
budget now MEASURES the symbol's real bar count, and a topology derived
from a measurement would widen as history downloads. Both ends are now
pinned. Every derived value left the weights-filename fingerprint -
keying a filename on a measured quantity means the EA looks for a file
that does not exist, starts from era 0 and orphans a trained model,
silently, because a missing cache is the normal first-run state. The
shape lives in the .cfg instead, where LoadAndCompare now ADOPTS the
four derived fields rather than diffing them; a mismatch there would
discard a fully-trained model over nothing the user did. Two fields
appended to the .cfg for the conv/LSTM stages, length-guarded on read
because FileReadInteger past EOF returns 0 with no error.
ForceHiddenLayers, a compile-time constant like DebuggingMode, pins
depth for diagnostic comparisons. It joins the fingerprint only when
non-zero, so forced depths get their own files - sequential comparisons
only, not simultaneous from one .ex5.
Derived shape, H1/10y defaults (21 features x 20 bars): first layer 64,
3 dense, 8 conv filters, 16 LSTM units. The LSTM block halves from
~58k to ~28k weights.
Both builds compile 0 errors, 0 warnings. Re-keys existing models.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Same defect the first-layer width had before 2026-07-29: both were
inputs whose defaults were fixed constants picked with no reference to
the input they sit on, which is the only thing that decides whether
either number is sane.
The conv layer is a per-bar projection - AddConvStage sets
window = step = one bar's features - so its filter count should be read
against the per-bar feature count. Sixteen filters COMPRESSED a
50-feature configuration 3x but EXPANDED a minimal 4-feature one 4x, and
the expanding case adds parameters below every learnable layer without
adding information. Now derived as half the per-bar feature count,
snapped down a power-of-two ladder.
The LSTM stage was the bigger miss. Its weight count is exactly
4*H*(H+inputs+1) (CNeuronLSTMOCL::SetInputs) and AddLstmStage feeds it
the whole flattened vector, so the shipped 32 units against a 540-wide
input is ~73k weights - more than DOUBLE the entire derived dense taper
it feeds. It was the one stage the capacity budget never covered, which
is why deriving the dense stack alone did not stop LSTM and HYBRID from
being over-parameterized. Now solved from the same
one-weight-per-in-sample-bar budget the first layer spends.
Factored EstimatedInSampleBars() out of ComputeFirstLayerWidth so all
three decisions spend one budget rather than each guessing at the
training-set size separately. Both new values are assigned alongside the
first-layer width, before the fingerprint that hashes them, and are
functions of inputs already in that hash - so they need no entry of
their own, and the same reasoning removes them from the DB config key.
Both builds compile 0 errors, 0 warnings. Re-keys existing models.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Three separate reports from one deploy.
1. CONV, LSTM and HYBRID all came back tagged [4109]. The weights
fingerprint omits the topology type on purpose - the file path already
separates it (State\CONV\ vs State\LSTM\ vs State\HYB\) and hashing a
value that is constant within a folder buys nothing while re-keying
every trained model into a forced retrain. So the files were never at
risk, but the tag could not do its one job. Prefixing the short id
makes it unique on the display side only; the hex half still greps
straight to the .nnw inside the folder the prefix names.
2. The default panel read like a training console. Six lines down to
three, each answering a question an owner actually has. The deploy
internals (best score, eras-since-best, ladder stage) were developer
diagnostics describing a recall floor that no longer decides anything,
and were already in the era-end journal line. In-sample accuracy left
the panel too: it grades the model on bars it trained on, so it always
flatters, and showing it beside the honest number invites reading the
wrong one. New compile-time DebuggingMode constant - deliberately not
an input - carries the IS/OOS pair and the resolved model path into
the journal instead. No extra Inputs row, no extra Market description
line, no user-reachable firehose.
3. Panel drag and buttons stuttered under training load, exactly as the
2026-07-26 note raising the chunk budget to 200ms warned they might.
Backed off to the documented 120ms - worst-case click latency is that
budget - and the derived topology (~292k weights to ~29k) makes the
throughput this costs far cheaper than when that note was written.
Also halved the panel redraw rate to 2.5 Hz: ChartRedraw repaints the
whole chart, so its cost scales with accumulated arrows, and 5 Hz was
the larger half of the stutter. Era-end still force-refreshes.
Both builds compile 0 errors, 0 warnings.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
EnableBatchNorm and BatchNormWindow demoted from inputs to constants. Batch
norm is required, not optional: measured on identical MLP_3L topologies it
was worth +11.3 points of balanced accuracy (57.0% with, 45.7% without),
stable across 150+ and 200+ eras, and the no-BN control converged to ~5% IS
and OOS accuracy with no chart signals at all. A user cannot make a good
decision here and can easily make a ruinous one, so the choice is not
offered. BatchNormWindow goes with it - a running-statistics window in
samples has no meaningful setting a trader could reason about, and its only
other reachable state (<=1) silently disables the layer.
Kept as named constants rather than deleted: the topology builder, the
weights fingerprint and the .cfg guard all read them, and a constant keeps
those paths - and the ability to flip one for a diagnostic rebuild - intact.
Fewer knobs also means a shorter Market description and less room for a
buyer to misconfigure.
EXPERIMENTS.md records runs 2 and 3, since the MT5 logs are wiped between
runs and these measurements are what the design decisions rest on. Run 3
(12h, uncapped tau=1.0) is a write-off: zero eras out of 1,993 across the
five batch-norm charts ever called a direction on fewer than half of all
bars, at a median precision equal to the ~6.1% base rate. The damage was
present at era 1 and never recovered over 292-766 eras.
Both builds compile 0 errors, 0 warnings.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
FeedForwardConv emits POSITION-MAJOR output, matrix_o[out + window_out * i],
so one bar's window_out filter responses are contiguous and consecutive bars
sit window_out apart. Both pooling implementations (FeedForwardProof and
CPU_FeedForwardProof) slide FLAT over that buffer - pos = i * step, reducing
`window` CONSECUTIVE elements. On a position-major layout those neighbours
are different FILTERS of the same bar, never one filter across time.
At the shipped 3/2 the pool computed max(bar0_f0, bar0_f1, bar0_f2), then
max(bar0_f2, bar0_f3, bar0_f4), with every 8th window straddling a bar
boundary. So it collapsed unrelated feature detectors into whichever fired
hardest, passed gradient to that winner only, and halved the feature map
while doing it - all below every learnable layer, where nothing above can
recover it. The removed inputs' own labels ("3 Bars") show time-axis pooling
was the intent throughout.
Measured cost: CONV sat pinned at ~40% balanced accuracy for 510 eras with
Sell recall 0%, while plain MLPs on the same data reached 57-61%. HYBRID,
which also carried this stage, came second-worst of the batch-norm group.
Not fixable in the topology: pooling one filter across time needs a stride
of window_out BETWEEN samples within a window, which a consecutive-window
kernel cannot express at any window/step. That needs a stride-aware kernel
in Network.cl + WarriorCPU.cpp + WarriorDML.cpp and a DLL rebuild, and is
only worth doing if a conv front-end earns its place without downsampling
first - with 20 sliding positions there is little to gain by halving them.
ConvPoolWindow/ConvPoolStep and their enums are removed with it, along with
the |CP: fingerprint term added earlier today.
Both builds compile 0 errors, 0 warnings.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Menon et al. 2021 (ICLR), "Long-tail learning via logit adjustment": add
tau*log(prior_c) to each class logit inside the training gradient. Softmax
CE on adjusted logits is consistent for BALANCED error - the metric
checkpoint selection already ranks on - so the loss and the deploy decision
finally optimize the same thing.
The engine already computed a true softmax + categorical-CE gradient and
wrote it over the per-neuron sigmoid delta, so this is an offset added to
three logits in the two places that gradient is built (backProp scalar path
and backPropOCL). No backend, kernel or DLL change; the forward pass and
every inference path are untouched, which is the point - the network learns
to absorb the offset, so its raw argmax becomes the balanced-optimal
decision with nothing applied at inference.
Replaces rather than stacks. Minority replay is disabled while this is on,
and the post-hoc inference prior is forced off. Stacking is not a
theoretical worry: simulated on the measured 1118/1119/34298 distribution
in the weak-signal regime, plain CE collapses to Neutral (33.4% balanced,
Buy 0%), replay reaches 48.1%, logit adjustment 50.9% with better balance -
and BOTH together score 45.4% with Neutral recall at 0%, worse than either
alone. Buda et al. 2018 predicts exactly that.
Motivation from the six-chart run: every topology took one direction to
~50% recall and abandoned the other, the direction chosen arbitrarily (the
batch-norm control went Buy 1% / Sell 42%, the inverse of the other five).
One era in 1,301 cleared the per-class recall floor.
Fingerprinted conditionally, so the converged 60.7% models on disk keep
their filenames and stay loadable as the fallback.
Both builds compile 0 errors, 0 warnings.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Dividing a machine budget by the live chart count was wrong twice over.
The count is a snapshot taken when each net's pool is built, and charts
attach one at a time: five charts measured 10/6/5/4/4% of the same budget,
because the first only ever saw itself and the last saw all five. So the
earliest chart got several times the threads of the latest - skewing any
cross-topology comparison run on those charts, which is the exact thing
the setting existed to make fair. Nothing rebalanced afterwards either,
and rebalancing would mean tearing down a DLL context under a live trainer.
Both problems disappear once the answer stops depending on how many charts
are running. Each net now asks for a fixed 2 worker threads, converted to
the percentage the DLL wants from the detected core count.
Two is not a compromise: since the topology became data-derived the widest
dense layer is 64 units, so each ParallelFor has almost nothing to split
and per-dispatch overhead dominates. An MLP era cost ~66s at a wildly
oversubscribed 12 threads and ~80s at 1 thread - a 20% spread across a 12x
difference in thread count. Two per net also lands six concurrent charts
exactly on a 12-core box.
Removing the input costs nothing on the product side: a Market build has no
DLL tier at all, so it was already compiled out to a constant there and no
buyer could reach it.
Both builds compile 0 errors, 0 warnings.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Deriving the first layer's width left NeuronsReduction and MinNeuronsCount
behind as inputs calibrated for something that no longer exists. Against a
hand-picked 500-wide first layer "keep 30%, floor at 20" produced a genuine
funnel - 500 -> 150 -> 45. Against the derived 64 it degenerates to
64 -> 20 -> 20: the reduction factor stops mattering after one step, and
"minimum neurons per layer" silently becomes the width of every layer but
the first. Two knobs whose labels no longer describe what they do.
The taper now runs geometrically from the derived first-layer width down to
a final hidden layer sized off the output count, spread evenly over however
many layers the chosen AIType implies:
MLP_3L 64 -> 28 -> 12 -> 3 29,151 dense weights
MLP_4L 64 -> 37 -> 21 -> 12 -> 3 30,450
CONV/LSTM/HYBRID_2L 64 -> 12 -> 3 27,763
and it stays a funnel at the floor, where the old rule could not:
D1 (first layer floored to 16) 16 -> 14 -> 12 -> 3
Both inputs are removed. With the width derived there is no freedom left in
the taper, so keeping either would only let the user contradict the
derivation. The layer COUNT stays selectable, because it is bundled into
AIType alongside the conv/LSTM front-end - depth is an architecture choice,
not a data-derived quantity, and pairing them means the two cannot
contradict each other.
m_minNeuronsCount / m_neuronsReduction survive as frozen members: nothing
reads them to build a topology any more, but they hold positional slots in
the .cfg sidecar and the weights fingerprint, and changing either value
would re-key every model on disk for no behavioural reason.
The DB config fingerprint drops both terms.
Compiles 0 errors, 0 warnings.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
InitialNeurons was an input whose only defensible value depends on two
things the user cannot see when picking from a dropdown: how wide the input
vector ended up after feature selection, and how much in-sample data the
study period actually yields. Left to a hand-picked constant it was badly
wrong - 500 units against a 420-wide input is 210,500 weights, 72% of a
292,583-weight model, against ~36,500 training bars of which only ~2,236
are directional. That is 6.6 weights per training bar, and it EXPANDS a set
of highly correlated inputs rather than compressing them.
The symptom was already in the logs and had been read as a depth problem:
the shallowest topology consistently beat the deepest (perceptron 52.7%
balanced, hybrid 41.3%). Over-parameterization predicts that ordering just
as well as covariate shift does, and only one of the two had been addressed.
ComputeFirstLayerWidth() budgets roughly one first-layer weight per
in-sample bar. Measured across the configurations in use:
M15 10y -> 256 units, 129,071 weights, 0.73 per bar
H1 10y -> 64 units, 28,727 weights, 0.65 per bar
H4 10y -> 16 units, 7,559 weights, 0.68 per bar
Two design points that matter:
- It estimates in-sample bars from the STUDY PERIOD and timeframe, not
from Bars(). What is downloaded grows over a terminal's lifetime, and a
topology that widened as history filled in would re-key its own weights
file and discard a trained model.
- The result is snapped down to a coarse power-of-two ladder, so the
estimate would have to be wrong by ~2x to change the answer.
Every field it reads is already part of the weights-filename fingerprint,
so the derived value needs no fingerprint entry of its own. The public
setter is removed - it could only have been called after construction, and
would either be ignored or silently re-key the model mid-run.
Where the data cannot support even the floor (D1 over 10 years is under
2,000 bars) it now says so and names the fixes, rather than quietly
training a model with more weights than examples.
The DB config fingerprint drops the term too, which re-keys existing
pattern databases once - correct, since a model an order of magnitude
smaller should not inherit the old one's win-rate history.
Compiles 0 errors, 0 warnings.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The only bounded stage in the entire forward path was the sigmoid
classification head - every hidden stage is PRELU. That is a network with
no internal scale control, and the failure ordered exactly by depth: on
SP500 H1 the shallow perceptron held ~52% balanced accuracy while the
deepest topology sat on the 33.3% one-class floor, with the per-bar logit
spread decaying monotonically (0.45 -> 0.38 over ~200 eras) until the
evidence tilt fell under the class-prior tilt. That is the signature of
internal covariate shift, which chapter 6.1 of the reference book is
entirely about and which the NeuroNet_DNG engine addresses with a layer
this project never had.
Two mechanisms make this the right fix rather than more hyperparameter
nudging:
- it decouples WEIGHT_DECAY from the learned function (van Laarhoven
2017) - with a normalized layer downstream, decay can no longer grind
the discriminative signal away, it only rescales the effective
learning rate;
- it is the precondition for ever running an unbounded logit head here.
The 2026-07-27 attempt blew up (IS error 5.6e15) precisely because
nothing upstream constrained scale.
Implementation notes:
- CNeuronBatchNormOCL computes host-side rather than as a fourth copy of
a kernel across Network.cl + WarriorCPU.cpp + WarriorDML.cpp. The math
is elementwise O(n); this way it behaves identically on all four
compute tiers, needs no DLL rebuild, and cannot drift between
backends. Same precedent as the softmax+CCE gradient and the
per-sample loss weighting, both computed in MQL5 for that reason.
- Statistics are exponential moving, not a stored mini-batch: training
is pure online SGD, one update per sample, so there is no batch to
average over. BatchNormWindow is an EMA window length.
- gamma/beta are excluded from weight decay, deliberately - decaying
gamma toward zero is the exact pathology being fixed.
- The layer self-sizes from whatever sits below it, because a conv/pool
stage's output width is derived inside the CNet constructor and is not
knowable to the topology builder.
- Checkpoint capture/restore/blend carry gamma/beta and the running
statistics alongside the dense matrix, so the plateau ladder cannot
restore a mismatched pair.
- SeedOutputLayerBias accepted only an exact defNeuronBaseOCL as the
weight-carrying penultimate layer; with normalization enabled that is
the batch-norm layer, so the cold-start bias seed would have silently
stopped being applied.
- Refuses to build, loudly, if a topology asks for normalization with no
compute backend at all - rather than quietly training a different
architecture than the one requested.
EnableBatchNorm (default on) and BatchNormWindow (1000 samples) are
inputs so the effect can be A/B'd without a recompile. Both feed the
weights-filename fingerprint, appended conditionally so existing non-BN
configs keep their fingerprints and are not forced to retrain.
Verified: analytic gradients match finite differences to 1.5e-7 relative
over 200 random cases; a faithful port of the full forward/backward chain
collapses to the 33.3% floor by era 4 without this layer and holds
36-43% with it. Compiles 0 errors, 0 warnings.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Evidence (MQL5\Logs, SP500 H1, 2026-07-29):
Perceptron era 61 Buy 32% Sell 27% Neut 94% bal 51%
LSTM era 160 Buy 16% Sell 11% Neut 98% bal 42% (peaked 49% @ era 44)
Hybrid era 179 Buy 5% Sell 2% Neut 99% bal 35% (peaked 41%)
CONV era 228 Buy 2% Sell 4% Neut 99% bal 35% (peaked 40% @ era 122)
Every model peaks early then decays monotonically toward Neutral, and nothing
stops it: the restore-best-weights + decay-eta handler is gated on
m_bestPassedRecall, which stays false forever when no checkpoint ever clears the
per-class floor. CONV ran 228 eras with eta pinned at its 0.000300 start. The
plateau ladder cannot end such a run either (stage 3 refuses to deploy without a
recall pass, so it resets ~27 times), making it a 1000-era one-way trip.
The gate's own justification had expired. It was written when the pre-pass
tiebreak was blended-accuracy-only, where "best" really did mean "called Neutral
most confidently". The balanced-selection change replaced that with
`balancedOosEra > m_bestBalancedOos` plus an isFullyCollapsedEra exclusion, so a
Neutral-only era now scores ~33% - the FLOOR of the balanced metric - and cannot
anchor the checkpoint at all. Pre-pass "best" now means "most class-balanced so
far", which is worth defending; and isWorseEra is itself a balanced-accuracy
regression, so it cannot fire merely for trading Neutral calls for Buy/Sell.
The original concern still holds while the best-so-far IS near-collapse, so the
escape is margin-guarded: defend the checkpoint only once balanced accuracy sits
more than BALANCED_WORTH_DEFENDING_MARGIN_PCT (5pp) above the one-class floor of
100/3. Against the run above that engages for all three stuck topologies
(42.3/41.3/50.0 vs a 38.3 threshold) while a genuinely collapsed run still
explores freely.
Two inputs restored to the regime that actually produced a deploy:
- MinRecall 60 -> 40. The one successful auto-deploy in the logs (Hybrid, 28th
00:50, best balanced 66.0%) ran against a 40% floor. 60 has never been shown
reachable here - a floor above what the config can reach is the same "target
set too high" failure the surrounding comment already warns about.
- OversampleParity 60 -> 90. 60 overcorrected. Runs now START Neutral-dominant
(Buy 0-11% recall at era 1) and call Buy/Sell on 0-4% of bars against a ~6%
true base rate - under-calling, with no headroom to converge down from. The
deploying run began at Buy 90% / Sell 36%, 24% of bars called, and settled into
the floor from above. Raw over-calling is the intended starting condition; live
calls are base-rate-calibrated by AILogitPriorStrength, which is why the input's
own note says to judge over-calling by live-fired precision, not raw counts.
Compiles 0 errors, 0 warnings.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The DFA (Direct Feedback Alignment) option was never a correct implementation:
it deterministically flipped the sign of half of all gradients based on
connection index parity, causing permanent gradient ascent for those weights
and guaranteed divergence. The backward pass was also incompatible with the
OpenCL/DirectML neuron model (layer.Total() == 1). This change removes all DFA
logic, including the enum value and `DfaFeedbackSignal` method, and replaces it
with plain gradient descent in all momentum update kernels. The `optimizer`
kernel argument is retained for binary compatibility but is no longer used.
Introduce Direct Feedback Alignment (DFA) backward pass with gradient clipping, feedback matrix initialization, and a dedicated backPropDfa method. Add optimizer snapshot/restore hooks (CaptureOptimizerSnapshot, RestoreOptimizerSnapshot, SetOptimizerForAllNeurons) to temporarily switch the entire network's optimizer for replay-only updates during pass 2, preserving the original optimizer state. Support all neuron types including dropout, deconv, LSTM, and softmax in the snapshot logic.
Eliminate the separate `AI_TOPOLOGY_PRESET` enum and input.
Fold the topology presets directly into `AI_CHOICE` as new combined values (MLP_3L, MLP_4L, CONV_2L, LSTM_2L, HYBRID_2L) plus `AI_NONE`.
Remove the `TopologyPreset` input variable and update default `AIType` assignments.
Update the market description to reflect the simplified single‑selector interface.
**Why:**
Users previously had to choose an AI architecture and a topology preset separately.
Now the UI shows one coherent selector that bundles architecture with its appropriate dense‑layer depth, reducing complexity and preventing mismatches.
In AI/Network.mqh, return early from InitDirectML during
tester/optimization/forward runs to prevent agent-side file-lock
failures caused by rapid stop/restart cycles accessing DLL imports.
In Expert/ExpertSignalAIBase.mqh, add MathIsValidNumber checks in
CalibratedConfidenceMagnitude and SignaledConfidence to safely handle
NaN values, and refactor ShutdownChartCleanup to accept a preserve
flag, avoiding unnecessary chart purges during tester runs for faster
shutdowns. Also add m_purgeChartOnDestruct member.
In AI/NeuronDirectML.mqh, clean up a minor comment formatting issue.
