forked from animatedread/Warrior_EA
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
AnimateDread
|
320f13253f |
feat: first-passage ladder + expectancy scan - price every geometry, not just the chosen one
Corrects the premise of the previous plan. Break-even is NOT a ceiling. If the model shifts the win probability on the bars it selects from p0 = m/(m+k) to p0 + d, then EV = (p0+d)*k - (1-p0-d)*m = d*(k+m) because p0*k - (1-p0)*m is zero by construction. The stop:target RATIO is expectancy-neutral - a punishing break-even is exactly repaid by the payoff - and only the real edge d and the TOTAL WIDTH (k+m) move EV. Width matters because the spread is charged once per trade however wide the barriers are, so a narrow barrier spends much of its own range on costs. DeriveBarrierGeometry's own comment already said the ratio buys nothing; the objective just never followed from it. Blocker this had to solve first: m_excUpCache/m_excDownCache hold only MAXIMUM travel each way, and a maximum cannot say which side was reached FIRST - so any geometry other than the walked one was undecidable on precisely the bars where both barriers were touched, ~28% of the sample. - BARRIER_LADDER: per bar, the first-touch AGE for 8 travel distances in each direction, filled during the walk the labels already run. Cursors keep it O(1) amortised per walked bar rather than 16 comparisons. Levels are travel FROM ENTRY, not barrier prices, so one ladder serves both directions and the spread is applied analytically when a level converts back to an SL/TP multiple - storing prices would need four ladders and bake today's spread into the cache. Sized, invalidated and validity-gated with the label caches. - ReportGeometryExpectancyScan: every ladder pair priced exactly off that cache - width in ATR and in SPREADS (cost efficiency, knowable without knowing d), break-even, both base rates, the share of bars resolved inside the horizon, and EV per unit of edge. Compares the widest resolvable pair against the quantile rule's pick. MEASUREMENT ONLY - the quantile rule still chooses. Nothing here can measure d, and width buys nothing if the wider target is less predictable. Base rates are printed beside each break-even because a persistent gap is DRIFT and must not be credited to the model. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
AnimateDread
|
0c85c54a5b |
fix: a restart no longer loses the measured geometry or the training window
Terminal restart, 22:25: all four resumed models sat on empty windows with enum 2:6 barriers. Three interlocking causes, all visible in one log excerpt: 1) THE PRE-SCAN WINDOW WAS SIZED BY THE SAVED WATERMARK. A resumed model's dtStudied sits at its last studied bar, so Bars(dtStudied, now) ~ 0 and the resumed-model MI pre-scan built a zero-bar "complete" label cache - logged as "Buy: 0 | Sell: 0 | Neutral: 0". Train()'s own era start RESETS dtStudied to the training-window rule before computing its window; the pre-scan did not. The rule is now factored into TrainWindowStart() and both use it. The scan also refuses to arm before SERIES_SYNCHRONIZED (it ran in the same second as OnInit), and deployed models keep their watermark - for them it gates inference recency, not a training window. 2) THE HORIZON LATCHED ON AN INDICATOR WARM-UP. ComputeBarrierHorizonBars ran against a ZigZag with 0 calculated legs, fell back, and EnsureBarrierHorizon latched fallback(32) x slMult x tpMult = 384 for the process lifetime. A leg-starved horizon is now PROVISIONAL: re-resolved on the next rebuild, the label cache wiped if it moved (labels from two horizons answer different questions), and the geometry deriver refuses to run from it - a pair derived over a warm-up window would get PINNED. 3) THE DERIVED GEOMETRY WAS NEVER PERSISTED. The .cfg is written at model creation and at weights-reset - both BEFORE era 0 derives - so the measured pair lived only in memory: every restart read back zeros, adopted nothing, fell back to the enum barriers, and the era-0-only gate meant a resumed model could NEVER re-derive. A full day of training on 3.33/1.62 resumed as 2:6. Now: the settled pair is pinned to the .cfg the moment derivation completes (one-shot, atomic write), and the derive gate accepts any model with no pinned pair, not just era 0 - mid-run stability is carried by m_geometryDerived itself, which never allows a second derivation. Both build variants compile 0 errors, 0 warnings. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> |
||
|
AnimateDread
|
9a7c37f334 |
fix: live trades now use the geometry the gate certifies; perf: BN kernels
Three changes, one theme: the trade placed, the trade graded, and the trade computed are now the same trade. 1) GEOMETRY WIRE (correctness, the ranked #1 open issue). The measured barrier pair reached the LABELS only - OpenParams still placed orders at the enum geometry (2*ATR/6*ATR), so the deploy gate certified "reaches 1.62*ATR before 3.33*ATR above break-even" about trades the EA never placed. Published via g_DerivedSlAtrMult/g_DerivedTpAtrMult (ConfidenceBridge, same same-tick contract as the confidence globals, because OpenParams runs on the root signal which has no pointer to the AI filter). Two writers: DeriveBarrierGeometry at era 0, and the .cfg adoption a deployed model takes. Overrides both legs and both Intelligent modes - the certificate is exact or it is nothing. TP is ATR-anchored like the label, NOT risk-relative, so a floor-widened stop cannot reshape the certified target. 2) BATCH NORM RUNS DEVICE-SIDE ON OPENCL. Four kernels in Network.cl - forward, hidden gradient, gamma/beta accumulate, gamma/beta apply - each a line-for-line transcription of the host implementation (NormalizeHost / HiddenGradHost / StepGammaBeta) including every NaN guard, clamp, and the exact moment-write ordering. The host copies remain the runtime for the DLL and pure-MQL5 tiers and the reference the kernels must match. Because this box has no OpenCL platform, the safety story is layered: - shim validation: kernels compiled as C and driven against a fp64 host transcription over NaN-poisoned stats, NaN gamma, over-clamp inputs, the frozen path, both optimizers, 3 batches - ALL PASS, worst normalized diff 0.132 vs tolerance 1.0 - in-situ self-check: each kernel is compared against its host twin ON FIRST USE on the real device (SelfCheckBn*), covering what the shim cannot - arg indices and buffer bindings. Any disagreement resyncs from the good copy, latches all BN kernels off process-wide, and training continues host-side. A transcription bug costs a warning and some speed, never a poisoned .nnw. - sync discipline: BatchOptions is now a CBufferDouble with explicit authority tracking (m_bnDeviceAuthoritative). Checkpoints/saves pull read-only; restores/loads/resets push; a mid-batch handover drains the device gamma/beta accumulator into the host arrays so no sample is lost. 3) SMALL FIXES. Apply-kernel build failure now latches the dispatch path at init (one warning instead of warning + failed Execute). Build tag bumped to win-scoring-gpu-v1 - first tag change since expectancy-stop-v1 despite five binary-changing commits. Both build variants compile 0 errors, 0 warnings. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> |
||
|
AnimateDread
|
5cef0947f4 |
fix: the deploy gate was benchmarking a win rate against a label frequency
The gate rests on an invariant stated at ExpertSignalAIBase.mqh:199 - under a
driftless walk P(touch +k before -m) is m/(m+k), and break-even for a k:m trade
is ALSO m/(m+k), so "beats chance" and "is profitable" are the same test.
That invariant needs reward >= risk, and the measured geometry no longer
satisfies it. With target 1.62*ATR and stop 3.33*ATR, break-even is 67.3%, but
both-won bars were stripped out of Buy and Sell so the label base rate read
37.5%. chancePrecPct is max(BuyTotal,SellTotal)/bars, so the gate was clearing
models nearly 30pp short of break-even: 42% "directional precision" is +4 sigma
against 37.5% and loses money on every single trade. Live since
|
||
|
AnimateDread
|
ce5265488e |
fix: both-won bars were labelled "do not trade" - resolve by first touch
Removing the min-reward:risk raise let the MEASURED geometry come back with the target NEARER than the stop (SP500 H1: target 1.62*ATR at q50 of favourable, stop 3.33*ATR at q75 of adverse). That reopened a branch the code called unreachable: price can reach +target and -target inside one horizon, winning in BOTH directions, and those bars fell through to Neutral. Neutral has only three producers, both-lost is unreachable (you cannot touch -3.33 without crossing -1.62 first, which wins the short), and timeouts logged at 1.0% of Neutral - so ~27% of ALL bars were being handed to the model as the abstain class when a trade either way would have collected its target. The cleanest positives in the sample, labelled "do not trade", while the fitted confidence threshold was being asked to find selectivity in what was left. Resolved by FIRST TOUCH: the target reached earlier is the trade that would have closed first. Same forward window, no extra lookahead. Same-bar ties stay Neutral - OHLC cannot order two touches, and unlike an intrabar stop tie there is no pessimistic side to fall to, so a guess would inject a coin-flip direction into the target. Also: - count both-won and its same-bar tie subset in the prebuild line, so the share is measured rather than inferred from arithmetic on a log line - scope the timeout counter to IS, matching the tally it is reported as a percentage OF; it was incremented over the whole scan and divided by an in-sample denominator - clear m_lastBarrierTimedOut at the top of the walk with the excursions, not at the bottom - the two early returns published the previous bar's verdict - mark the pass-1 label line PROVISIONAL. It prints the enum fallback because geometry can only be derived from excursions that do not exist yet, and it reads exactly like a config change that failed to take effect FORCES RETRAIN. Both build variants compile 0 errors, 0 warnings. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> |
||
|
AnimateDread
|
217b9bc9bf |
feat: remove Min_Risk_Reward_Ratio - a guess was overriding a measurement
The barrier geometry is derived from the instrument's own excursion distribution (stop at q75 of adverse travel, target at q50 of favourable), and then a 1:2 floor was applied on top, raising the target to twice whatever the stop happened to be. On SP500 H1 that pushed the target to 6.66*ATR, reached on 3.3% of bars inside the horizon - so the label became "almost never a win" and every topology was trained to predict an event that essentially does not occur. A measured target has to stay measured. The ratio never bought what it was believed to buy. A reward:risk floor does not create expectancy; it trades hit rate against payoff at a break-even the geometry already fixes - which this project has separately MEASURED (payoff 0.92 -> 5.72 with expectancy flat). What it did buy was two outages: four consecutive Market validation rejections for "no trading operations" when it rejected 100% of setups, and the label corruption above. Removed: - the input and the RISK_REWARD_RATIO enum (deleted, not left dangling - a live enum with no input behind it is the shape of the stale-.set incident that trained ~250 eras on the wrong target) - the forced target raise in the label geometry - the rrOK eligibility gate in the barrier-geometry scan, so every unclamped pairing now competes on the measurement alone. Clamping stays disqualifying for its own unrelated reason. - the reward < minRR*risk veto in OpenParams Kept: g_TradeRewardRiskRatio still computed and still bridged to Kelly sizing in MoneyIntelligent - the ratio as a SIZING input was always the sound use. Risk stays bounded where it actually is - account risk % and CRiskBudget. The low-reachability warning survives but is re-aimed: with nothing inflating the target, a target the market rarely reaches can only mean the horizon is truncating the excursions the geometry is derived from. Both build variants compile 0 errors / 0 warnings. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> |
||
|
AnimateDread
|
b3b7e7bceb |
fix: excursion window must not depend on the barrier it sizes
DIRECTION IS NOT THERE, and this run is what establishes it. Three symbols:
raw ASYMMETRY clears on all three (p=0.0199 / 0.0050 / 0.0050)
norm ASYMMETRY collapses on all three (p=0.3433 / 0.5075 / 0.2736),
USDCAD landing BELOW its own null
RANGE control strengthens to 3-5x its null everywhere
Divide sigma out and the apparent directional signal vanishes entirely. What
cleared was volatility leaking through an unnormalised difference. Note this
would have passed any replication test: three instruments at p=0.005 is exactly
the evidence one would accept before committing to a rebuild, and the confound
reproduces perfectly. Replication was never going to catch it - only the
normalisation could.
Two defects of mine, both surfaced by the same run.
1. THE GEOMETRY DERIVATION WAS DIVERGING, NOT CONVERGING. It produced a
14.57*ATR stop and a 29.14*ATR target that only 5.7% of bars ever reach.
Excursions were measured over the barrier horizon; the horizon scales with
the target; the target is a quantile of the excursions - so target ->
horizon -> excursions -> target ran away, and "settled" only because the
horizon ladder caps at 384 bars. A saturated runaway, which the iteration
guard could not catch because it watches for OSCILLATION.
Fixed at the root: excursions now accumulate only over m_swingMedianBars -
the UNSCALED median ZigZag leg, a property of the instrument that owes
nothing to the barrier. The barrier walk still runs the full horizon,
because that is how long the trade is held; only the MEASUREMENT used to
size the barrier is confined to a geometry-independent window.
(The Min_Risk_Reward_Ratio warning fired correctly and is what flagged it -
the diagnostic worked while the derivation behind it did not.)
2. THE CONFOUND VERDICT WAS UNREACHABLE. `sizeCleared && !asymCleared` was
tested first and is true whenever size clears - i.e. always - so the branch
that NAMES the volatility confound never printed; all three symbols showed
the generic size-not-direction message instead. Verdict chain rewritten with
the specific case first, and the dangling elses my first patch introduced
removed.
FORCES A FULL RETRAIN (the excursion window changes every derived barrier).
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
|
||
|
AnimateDread
|
32ffeb99f3 |
fix: normalise the asymmetry target - the raw one is confounded by volatility
Three symbols ran the excursion test. RANGE/UP/DOWN cleared on all three;
raw ASYMMETRY cleared on EURUSD and USDCAD at p=0.0050 and not on SP500
(p=0.1045). That looked like the first directional signal this project has
found. It probably is not, and the test as built could not tell.
(up-dn) IS NOT SCALE-FREE. If sigma is predictable - and RANGE clears at ~4x its
null on every instrument - and the directional part is symmetric noise eps, then
up-dn ~ sigma*eps, so a large sigma pushes the value into BOTH outer terciles. A
pure volatility predictor scores positive MI against a 3-bin (up-dn) while
carrying no directional information at all. Crucially that confound REPLICATES,
so reproducing on two instruments is not evidence against it - and the effect
sizes fit it: asymmetry runs 1.3-1.6x its null where RANGE runs ~4x, and carries
~0.1% of the target's entropy against RANGE's ~0.9%. That is the shape of a
leaked fraction of the volatility signal, not an independent one.
So add (up-dn)/(up+dn): bounded in [-1,+1], volatility divided out, and the only
target a directional claim may rest on. The verdict now separates the cases and
NAMES the confound when raw clears while normalised does not, instead of
reporting the raw line as a finding.
Two bugs of mine in the same block, both caught by output rather than review:
- The derived-geometry line had a MISORDERED argument list: it printed
"stop 25.00*ATR (q3 of adverse travel)" - the quantile percentage as the
multiple and the multiple as the quantile. Real values were 2.61 stop /
8.03 target. A 25*ATR stop is absurd on its face, which is why it was seen.
- THE STOP QUANTILE WAS BACKWARDS, and this one changes labels. It was 0.25
"so ordinary noise does not reach it", but q25 means 75% of bars EXCEED the
stop - hit three times in four. The printed reachability said exactly that
("stop on 75.0% of bars"). Now 0.75. A quantile is a threshold, not a rate.
This is the entire reason reachability is measured and printed rather than
assumed.
Also raises BARRIER_DERIVE_MAX_PASSES 3 -> 5: SP500 did not settle in 3 (stop
still moving ~14% per pass) while EURUSD and USDCAD converged on pass 2. And
bounds both quantile indices with MathMin(..., n-1) so q=1.0 cannot run off the
end of the sorted array.
The geometry from the previous run is NOT usable and the asymmetry result is
unresolved, not established. Both are decided by the next run.
FORCES A FULL RETRAIN (the stop quantile changes every label).
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
|
||
|
AnimateDread
|
a7701f032b |
feat: derive the ATR multiples from measured excursions - no hardcoded geometry
The barrier was still two constants. SL_Mode/TP_Mode left the Inputs tab in |
||
|
AnimateDread
|
2c78f3b90d |
diag: is "optimal SL/TP" learnable? Score the features against excursions
Proposed direction: train the net to predict entry/SL/TP that maximise return
and minimise drawdown, rather than to classify direction. Before rebuilding a
head, measure whether the target is learnable at all.
That question splits into two that behave nothing alike:
HOW FAR price travels (MFE/MAE) - essentially volatility, and volatility
clustering is about the most robust regularity in markets.
WHICH WAY it goes first (the asymmetry) - direction, which is what every
noise-floor verdict in this project has been about.
Expectancy comes ONLY from the second. The first buys position sizing and
drawdown control - worth having under prop-firm limits, but not an edge: exit
management on RANDOM entries already moved the payoff ratio 0.92 -> 5.72 with
expectancy FLAT.
Crucially this is NOT already answered. Every MI figure here scored the
triple-barrier label, i.e. one specific question at one fixed geometry. A
noise-floor result there says nothing about whether excursion MAGNITUDE is
learnable - different target, different answer.
Four targets, and the verdict is the CONTRAST, printed explicitly because the
dangerous misreading of "UP clears" is "we can predict profitable trades":
RANGE (up+dn) - realised volatility, included as a POSITIVE CONTROL that
SHOULD clear. Every prior verdict here lacked a control
expected to pass; a range target at the floor indicts the
measurement, not the market.
UP / DOWN - MFE / MAE.
ASYMMETRY - up-dn, the only one that can pay.
Collected inside the walk the label already does (one max, one min per bar).
The early-out when both barriers resolved is GONE: it would have truncated the
excursions at whichever bar tripped the last barrier, making the measurement a
function of the CURRENT SL/TP - the circularity this is trying to escape. The
loop was already bounded by the horizon, so only the average cost moves.
Discretised into 3 EQUAL-FREQUENCY bins, so every downstream piece (block
permutation, null, p-value) is reused unchanged. Equal-frequency because MFE is
fat-tailed and fixed-width bins would put nearly every row in bin 0; it also
pins H(Y) at ln(3)=1.099 for all four, making them comparable to each other and
to the barrier label's ~1.02 instead of confounded by class balance.
Two bugs fixed in this code before it ever ran, both of which would have
produced a plausible quiet wrong answer rather than an error:
- TripleBarrierLabel early-returns on invalid ATR/close BEFORE the point the
accumulators were reset, so one bar's excursions would be cached under
another bar's index. Cleared at the top now, ahead of every return.
- An unresolvable bar is still flagged as labelled but carries excursions of
exactly 0. Under equal-frequency binning a block of identical zeros drags
the lowest cut onto zero and a third of the sample lands in one
uninformative bin - a depressed score that reads as "not predictable", a
false negative in the direction that would wrongly kill the idea. Rows
where both excursions are zero are dropped; price cannot travel zero both
ways over a whole horizon.
Read-only diagnostic. No topology or label change: no retrain of its own.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
|
||
|
AnimateDread
|
168422ff7a |
fix(labels): the 128-bar horizon ceiling was truncating the shipped label
The corrected geometry scan exposed something bigger than the geometry
question it was asked. Every pairing from 2:6 upward came back CLAMPED -
including 2:6, the SHIPPED configuration.
First-passage time for a driftless walk leaving [-m,+k] goes as m*k, and
the measured swing median here is ~12 bars at m*k=1, so 2:6 wants ~144
bars and 3:10 wants ~360. The ladder stopped at 128. A clamped label
stops meaning "does the target come before the stop" and quietly becomes
"...within 128 bars", while the deployed EA holds until SL or TP with no
bar limit. So the target the models have been trained on all along was
not the strategy the EA executes, and the trades it silently reclassified
as Neutral were the SLOW WINNERS - precisely the ones a 1:3 barrier
exists to capture. Timeout share stayed ~0% throughout, which is why this
never showed up: the truncation lands in Neutral, not in the timeout
counter that was watching for it.
Ladder extended to 384 (12..128, 192, 256, 384) so every selectable
geometry gets an honest horizon. Cost is one embargo of at most 384 bars
out of ~38k.
Second fix, same class of error as the H(Y) one: the scan's "best
eligible" was 2:2, a 1:1 barrier, against a shipped Min_Risk_Reward_Ratio
of 1:2. Training four topologies on that target would have produced a
model whose every setup is rejected at the door - the exact failure
behind four consecutive Market rejections for "no trading operations".
Sub-minRR geometries are now ineligible and marked [<minRR], printed
rather than hidden.
Also drops the dense-depth tag from the display name ("Perceptron 3L" ->
"Perceptron"). Depth is derived, so it names nothing a user chose; the
config tag [PAI-0be2] already disambiguates concurrent charts and does it
for every input rather than one. Full topology still logged by "config -".
Compiles 0 errors / 0 warnings, standard and Market. Build tag
horizon-384-v1. Changes the LABEL for every geometry, so the next scan
supersedes the previous numbers - and a retrain is required before any
model trained under the truncated target means anything.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
|
||
|
AnimateDread
|
40af4a4b5b |
fix(labels): the geometry scan rewarded the labels it should reject
First run named 3:10 on all four charts, at 2.3x the configured 2:6. That answer was wrong and the fault was the ranking statistic. 3:10 wants a horizon of ~swingMedian*30 (~320 bars) and gets BARRIER_HORIZON_MAX. Clamped, most trades never resolve, the unresolved remainder all lands in Neutral, and H(Y) collapses. The old statistic divided the excess BY H(Y) - so a collapsing denominator made the most degenerate label look like the most predictable one. Every geometry from 2:6 upward was already showing the clamped h128, and the two widest scored highest, which is the fingerprint of the artefact rather than of signal. Two fixes: Rank on the raw excess in nats. Subtracting each geometry's OWN measured null already removes the class-balance bias, which is the only thing the normalisation was ever needed for. Disqualify clamped geometries outright rather than ranking them down. The deployed EA holds until SL or TP with no bar limit, so a truncated label trains the model on a question the strategy never asks. They are still printed, marked '!', so the disqualification is visible instead of a silent omission - and the scan now says so explicitly when nothing eligible is left, because "the limit is the feature set, not the target" is itself the finding in that case. The scan also reports each geometry's directional share and timeout share now. A label nobody can trade is not a candidate however well it scores, and that has to be visible in the same line as the score. Compiles 0 errors / 0 warnings. Build tag geometry-scan-v2. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> |
||
|
AnimateDread
|
f97ab9f1d6 |
feat(labels): measure which barrier is predictable at entry, don't guess
The alignment scan settled the shape of the problem: 4.7x more is knowable 5 bars into a 128-bar window than at the entry the model actually trades. A 6xATR target reached over 128 bars is decided overwhelmingly by what happens DURING the window, so whatever the entry state knows is buried under 128 bars of later noise. That is a property of the TARGET, and it is why four different architectures all landed on precision exactly equal to the base rate - no topology can undo it. So measure the target. For each SL/TP pairing a user can actually select, relabel the same sampled bars and score how much the SAME features say about THAT outcome at entry. Seconds, no training, no topology, and it runs on the diagnostic path that already exists. Ranked on excess over its OWN null as a share of its OWN H(Y), never on raw nats: each geometry has a different class balance, hence a different finite-sample bias and a different amount of information there to find, so raw MI would rank the most BALANCED label rather than the most PREDICTABLE one. The break-even win rate m/(m+k) is printed beside each so the ranking is read next to the bar the model must clear. Stated in the output because it is the easy thing to get wrong: chance precision EQUALS break-even at every geometry, so a tighter target does not hand you expectancy. It buys predictability - less noise piled on top of what the entry state knows - which is the one thing changing topology cannot do. Read-only by construction: it relabels a sampled copy via TripleBarrierLabel(), never writes the label cache (which belongs to the configured geometry), and restores the horizon and overrides it borrowed. The overrides apply only when BOTH are positive, so a half-set pair can never silently relabel a live run. Compiles 0 errors / 0 warnings, standard and Market. Build tag geometry-scan-v1. Redeploy only - no retrain to READ the ranking. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> |
||
|
AnimateDread
|
d7eea325fb |
refactor(ai): extract Layer.mqh and deduplicate AI config
- Moves CLayer neuron construction to AI/Impl/Layer.mqh to keep Network.mqh clean - Unifies four previously duplicated architecture initialisation blocks (MLP/CONV/LSTM/HYBRID) into a single shared function - Eliminates risk of behavioural drift where one architecture missed a setter, causing mismatched feature sets or targets |
||
|
AnimateDread
|
25813523d3 |
fix: refuse invalid SL/TP, fix the unreachable deploy floor, scale the horizon
Three defects found by reading the 2026-08-01 training logs, all of which
only became visible because the relabel made the numbers mean something.
1. A STALE ENUM TRAINED FOUR MODELS ON THE WRONG TARGET.
`OnInit: trade settings snapshot - SL_Mode=1 TP_Mode=-101`
-101 was TP_PREV_SWING, deleted from TAKE_PROFIT_MODE on 2026-07-31 in
|
||
|
AnimateDread
|
b4a704d309 |
feat(ai): triple-barrier labels replace exact-pivot ZigZag targets
The 31:1 class imbalance was self-inflicted by the TARGET, not a property of the market. Labelling only the exact bar where a ZigZag pivot confirms gave Buy 1164 / Sell 1164 / Neutral 35841, and every correction mechanism this codebase accumulated sits downstream of that one choice: the logit-adjusted loss and its range cap, the prior EMA, the +-3.0 output-bias seed, balanced-accuracy-then-precision selection with its coverage floor, the recall floor and its catch-22, the alternation gate, NMS, and the four oversampling designs that collapsed before them. The reference this engine is built on (references/neuronetworksbook.pdf ch. 3.1/3.3) also uses ZigZag, but targets the DIRECTION TO THE NEXT EXTREMUM on every bar - ~50/50 by construction, with no imbalance to correct at all. It never had this problem because it never asked "is this the pivot bar". Labels are now the triple barrier (Lopez de Prado ch. 3), using the EA's OWN SL_Mode/TP_Mode: does a trade opened at this bar's close reach its target before its stop, within a horizon. Buy = long resolves, Sell = short resolves, Neutral = neither. Consequences: - dir-precision in the era line stops being a proxy and becomes the win rate of the strategy under its own exit rules. - Expected balance ~25/25/50 at the shipped 1:3 (gambler's ruin), i.e. ~2:1 instead of 31:1. Measured and logged at the end of the prebuild. - Spread is charged on both legs, so it is a NET win rate. - Intrabar ambiguity resolves to the STOP. OHLC cannot order two touches inside one bar and the optimistic reading is how a backtested edge becomes a live loss. ZigZag stays as input features (EnableSwingContext) and now also supplies the vertical barrier: the horizon is the median confirmed leg length, snapped to a coarse ladder. Derived, not configured, and deliberately kept out of the filename fingerprint - a filename keyed on a measured quantity orphans a trained model the moment the measurement moves. Removed, because the premise died with the old target: - the alternation gate. Correct for pivot labels (a ZigZag cannot emit two same-type pivots in a row, so a repeat was provably a false fire), and wrong for barrier labels, which answer each bar independently. It also took its worst consequence with it: a one-sided model previously got ONE trade per backtest, a hard blocker on marketplace validation. - SignalClusterWindow now defaults off - it de-duplicated repeats that are now real trades. Kept as an opt-in display control. - LABEL_WINDOW_BARS, the pivot-widening pass, ConfirmedZigZagLabel. - the era-0 output-bias seed now needs a genuinely dominant class (0.70) rather than 0.40; at ~50% Neutral a +-3.0 seed is a distortion, not a correction. Also fixed, both found while wiring the above: 1. RefreshConvergedSignal sized its buffers from a date delta (Bars(sym, period, dtStudied, TimeCurrent())). dtStudied is a training watermark; in the tester it is loaded from a live-chart save AHEAD of the simulated date, so the interval inverted, Bars() returned ~0, and the buffer came out at exactly m_historyBars - deep enough for the OHLC window and far too shallow for the Donchian-50 / 20-bar-return / SMA extension behind it. Inference silently computed DIFFERENT features from the ones training learned on, live as well as in the tester. Now sized from what the feature builder actually needs. 2. The barrier horizon is resolved on the deployed path too. A deployed model never enters Train(), so it never reached the prebuild, and OnlineLearnStep reads the horizon as its confirmation delay - left at the fallback it would have backpropped bars whose barriers had not resolved. Silent lookahead in the one place that writes to a live model. SL_Mode/TP_Mode join the weights fingerprint: they define the labels now, so a model trained at 1:3 must never be silently reused at 1:1. This re-keys every pre-existing model by design - none were trained on this task. Inference census extended with the vote gate. LongCondition/ShortCondition open with a readiness check the refresh counters never see; in the tester it reduces to "the seeded _optcache.nnw must have LOADED", and if it did not, every vote is hard-zeroed while the model still answers Buy. The old three counters would have read that as "the model says Neutral" - false, and a completely different fix. This is the leading candidate for the zero-direction backtest and the census can now name it in one run. Both builds compile 0 errors / 0 warnings. Forces a full retrain. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> |
||
|
AnimateDread
|
397b0eac1f |
refactor(ai): nine class-imbalance inputs down to two
The imbalance section offered nine controls for one job. Audited against the
code, five of them did not do what their names said at the shipped defaults:
AILogitPriorStrength DEAD - Inference.mqh's post-hoc prior early-returns
whenever the adjusted loss is on, which is default.
OversampleParity DEAD in training - Training.mqh gated the replay loop
on !useLogitAdjustedLoss (correctly, citing Buda et
al. 2018). Live only in the online-learning path.
EnableMinorityReplay DEAD as replay. It survived ONLY as a focal-gamma
damper - "replay minority bars through pass-2
oversampling" was a focal-loss switch.
ConstrainReplay DEAD as a cap; it only chose damper 0.125 vs 0.25.
UseStaticPrior An exact duplicate of FreezePriorCalibration - the two
were OR'd together in the single place either is read.
So they were not five mechanisms fighting; they were one mechanism plus eight
knobs that mostly described machinery that no longer ran. That is worse than
a real conflict, because the log agreed with the names: the label-cache line
printed "reps up to 28x (90% parity) (seeding era 0's class-balance
oversampling)" on every run, describing an oversampling pass that had been
switched off. It is fixed here too - it cost this session a wrong diagnosis.
The one genuine redundancy was focal loss, running at gamma*0.125 alongside
the adjusted loss: two corrections on the same axis, the exact stacking
failure this file already cited Buda et al. for in two other places, damped
by a replay flag whose replay path was itself dead. Removed rather than
re-tuned. The plateau ladder is unaffected - its escape is the learning-rate
warm restart; the gamma anneal beside it only ever stepped toward zero.
WHAT REMAINS is logit-adjusted loss (Menon et al. 2021) plus a prior freeze:
LogitAdjustTau 0 = off; replaces the separate EnableLogitAdjusted-
Loss boolean, since a strength dial where 0 already
means off does not need an on/off switch beside it.
FreezePriorCalibration unchanged.
It is the only one of the six corrections with a consistency guarantee, and
it is consistent for exactly the balanced-error metric checkpoint selection
already ranks on - so the loss and the deploy decision optimize one thing.
The online continual-learning path keeps its own alpha-balanced focal weight,
now as constants pinned to the removed inputs' shipped defaults, so its
behaviour is unchanged. It legitimately needs its own correction:
ApplyLogitAdjustment() only runs inside a training run, so a deployed model
that was reloaded carries no logit offsets and would otherwise stream 31:1
data into itself uncorrected.
The weights-filename fingerprint is BYTE-IDENTICAL. The focal slot was a
double fed to a %d conversion and had always emitted a literal 0; the |MR:
segment is written as the constant its shipped defaults produced. Dropping
either would have re-keyed every model and forced a from-scratch retrain of
the one topology currently converged and trading.
Also removed as orphans: FOCAL_GAMMA_PRESET, MAX_OVERSAMPLE_REPLICAS,
OVERSAMPLE_PARITY_FRACTION, PLATEAU_GAMMA_STEP, and the now-unreachable
"neutralized by prior correction" diagnostic.
Both builds compile 0 errors, 0 warnings. No retrain forced.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
|
||
|
AnimateDread
|
2de93539d4 |
refactor: split CExpertSignalAIBase implementation by responsibility
ExpertSignalAIBase.mqh was 8216 lines: the class declaration followed by 87 method bodies covering training, labelling, feature extraction, persistence, chart drawing, online learning, the GA auto-tuner and inference, all in one file. Train() alone is 1492 lines; a change to arrow drawing meant scrolling past the era loop. Moved the bodies into Expert\AIBase\, included at the bottom of the original after the class declaration: Training.mqh 1607 era loop, plateau ladder, checkpoint select, deploy Features.mqh 1093 indicator creation + per-bar input feature vector ChartUI.mqh 634 arrows, arrow persistence, status panel, cleanup Persistence.mqh 492 .stats/.cfg sidecars, CPU-inference validation, copy OnlineLearning.mqh 461 live continual learning, EMA shadow, OOS simulator Labels.mqh 309 ZigZag pivot labels, async label-cache prebuild AutoTune.mqh 275 genetic tuner (population, crossover, halving) Inference.mqh 235 softmax, prior calibration, class priors ExpertSignalAIBase.mqh 8216 -> 3131 (declaration + topology build only) This is a pure relocation - verified mechanically, not by eye: HEAD's file reconstructed from the eight partials plus the surviving remainder is byte-identical to HEAD, span for span (scratchpad verify_split.py). No declaration moved, no signature changed, no code rewritten, so behaviour is unchanged by construction. Compiles 0 errors, 0 warnings. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> |