forked from mnbvc188199/Warrior_EA
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
AnimateDread
|
c393497fd6 |
fix(chart): display now reads era-end SNAPSHOTS - the live cache is wiped mid-era
Full-pipeline analysis after "threshold 30, attained often, nothing drawn,
still glued to buy". The log falsified the premise before any code did:
21:40:43 swept 4999, 794 voters, drew 491. Strongest 43.0% vs 30.0%
21:42:07 swept 4999, 0 voters, drew 0
21:51:30 swept 4999, 0 voters, drew 0
21:56:30 swept 4999, 922 voters, drew 382. Strongest 44.0% vs 30.0%
The arrows WERE drawn - 491 of them, then 382 - and then erased. ONE root
cause, three symptoms: every display path read m_arrowSignalCache, which is
wiped to sentinel at each era start and only complete again when pass 3
finishes. With eras at ~30s and a sweep at ~17s:
* ARROW FLICKER: a sweep landing mid-era found no voters anywhere, and its
else-branch deleted the arrow on every voteless bar - erasing the previous
sweep's entire output. The chart cycled populated -> blank -> populated;
the user kept catching the blank phase.
* READOUT GLUE: the newest-cache walk found only sentinel for ~90% of every
era and fell through to dPrevSignal - the frozen purge-band edge bar that
reads Buy.
|
||
|
AnimateDread
|
4f2d81a52e |
fix(chart): sweep hammered the news filter; peak was a fossil; neutrals invisible
Careful read of the 21:14 log window (user report: peak stuck at 50, label sticky, neutrals never shown). Three distinct defects, one commit because they share the two files. 1. 15,508 "CalendarValueHistory failed" lines in 68 SECONDS - ~230/second. The overlay sweep replayed Direction() on EVERY non-AI filter, including the news/session/risk-guard veto filters. The news filter calls CalendarValueHistory per evaluation and MT5's calendar cannot answer more than ~30 days back (the known calendar cliff), so every historical bar logged a failure - real wall-clock burned inside a sweep whose whole point is to stay cheap. Veto filters keep m_pattern_count at its 0 default (the same test UpdateSignalsWeights keys on): they cast no weighted vote, and a prohibition cannot be reconstructed faithfully anyway - it joins order validation in the cannot-replay family. Skipped. Compounding it: at era ~200 the four members complete a barrier round every ~20s while a full 5,000-bar sweep takes ~17s of slices - the sweep finished and instantly re-armed, forever, against arrow caches half-rebuilt mid-era. That is why the census's "had a voter" flapped 1299 -> 257 -> 1113 across three back-to-back sweeps. Re-arms now rate-limited to one per 5 minutes. 2. Peak 50 was a FOSSIL. m_votePeak never reset, so it still held a value attained under the 25/50/75/100 DEFAULT tier weights from the attach window before the first re-rank - unreachable ever since the weights became measured (pooled 27-32 in the same log). A ceiling nothing can reach reads as "the models are underperforming their own history", which is backwards: the history was priced in different money. The peak now resets at the same regime boundary as the census (StartFilteredOverlay), and the label shows max(live peak, census strongest-vote) - the census number is the actual answer to "can Min_Vote_Open ever be reached", measured over ~5,000 bars under the CURRENT weights. 3. Neutrals were invisible. The prospective count lumped Neutral-deciding models in with voters, so "4 model(s)" read identically whether all four voted or three sat flat. Now "2 vote/2 flat", and an all-neutral bar reads "VOTE flat ... 0 vote/4 flat" instead of "--" - the models answered, and the answer was Neutral. Expected values, from this log's own re-ranks (all four members' fires land in T3; tier weights 27-32; module weights 0.27-0.32): a unanimous-buy bar reads ~29-30%, mixed membership 28-34. The reported "stuck at buy 28, climbed to 30, flashes of sell, now 33.4" is those weights doing exactly what they should. The stickiness between moves is pass 2/2.5/3 - only pass 1 writes dPrevSignal, so the label holds the last pass-1 bar's decision for the remainder of each era. Display-only, and honest: it is the model's most recent output. NOT COMPILED - user compiles in MetaEditor. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
AnimateDread
|
b05b4f21d7 |
fix(chart): the vote readout was repainted once per bar, not once per timer tick
"Still stuck at 0" after |
||
|
AnimateDread
|
07aa01777c |
feat(chart): reconstruct the filtered view behind the handover point
Completes the filtered view from
|
||
|
AnimateDread
|
4858507146 |
feat(vote): thresholds become confidence percentages, on ONE scale everywhere
User request: "the entry/exit thresholds are manual numbers, I would like
them to be confidence percentages, so the current 20 would be only 20%
confidence in a profitable trade."
WHY 20 WAS EVER SENSIBLE. Under UseDatabaseRanking both factors of a filter's
contribution are win rates: the pattern weight is that pattern's measured win
rate (UpdateSignalsWeights -> ApplyPatternWeight) and m_weight is the filter's
average win rate over its patterns, /100. Dividing the sum by the VOTER COUNT
therefore produced a mean of PRODUCTS of two win rates - a genuinely
60%-accurate filter firing a 60% pattern scored 0.60 x 60 = 36. The number was
never on a probability scale, so its magnitude meant nothing on its own.
Dividing by Sum(m_weight) instead makes it a weighted MEAN of win rates, which
is a win rate: result = Sum(w_i*p_i)/Sum(w_i). Every voter at 60% now reads 60;
MACD's double-divergence pattern (weight 100) voting alone reads 100. m_weight
stops being a discount on the probability and becomes how much a filter's
opinion COUNTS - which is what a module weight should always have been.
Default Min_Vote_Open 20 -> 50: not a tightening, the same bar re-expressed.
ONE SCALE, EVERYWHERE - the part that made this bigger than a rescale. Three
other places compared against a 0..1 softmax confidence and would each have
become a fresh currency mismatch the moment the input changed meaning:
* the AI early-exit route (LiveSignedConfidence vs m_ai_exit_threshold) now
reads m_lastAiVote - the AI filters' own weighted mean, undiluted by the
classic side, which is the only reason that route exists - against the
same m_threshold_close the averaged vote uses. m_ai_exit_threshold is
retired rather than left dangling.
* m_oosDecisionSeries now carries the vote, not the confidence, so the exit
SIMULATION stops modelling a close rule the EA does not run.
* ExitPolicy() clamped anything > 1.0 to zero. Passing the unscaled input
through that would have silently switched vote exits off in the
simulation while live went on running them - found before it shipped;
the bound now tracks the scale.
LiveSignedConfidence() is deliberately untouched and still 0..1: MM sizing,
SL/TP scaling and the intelligent trailing want a model confidence, not a win
rate.
CALIBRATION CAVEAT, stated in the code where the claim is made: this is only a
real probability to the extent the pattern weights are. A pattern with fewer
than MIN_TRADES_FOR_WIN_RATE journaled trades keeps its DEFAULT weight - a
designed prior (25/50/75/100 for the AI tiers), not a measurement. Until the
signal DB fills, "60" means "the designed conviction of the patterns that
fired". Closing that gap is the next commit.
Also corrects VOTE_CLOSE_PRESETS' comment, which documented the two scales
this removes.
NOT COMPILED - user compiles in MetaEditor.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
||
|
AnimateDread
|
65a3e4e877 |
fix(chart): a purge that reports "zero leftovers" was only ever checking its own list
2026-08-17 21:58: all three charts hit "Abnormal termination" ~5.3 s into
OnDeinit with NO cleanup-timings line - the teardown was starved again. The
22:00 init purge then removed 993 / 1373 / 1557 stranded objects and reported
ZERO by-name leftovers on every chart, and the charts still came up with
duplicated panels. "Nothing matching our prefixes remains" and "the chart is
clean" are different statements and only the first was being made.
Three changes, in the order they matter:
1. WHY the teardown starved, and it is a gap in
|
||
|
AnimateDread
|
9bd7bd1b7f |
fix(reset): say what the reset actually did, per member and per file
The user reports "Delete & Reset Weights only wipes the first NN". I could not find a code path that skips ensemble members, and I am not going to assert one: the handler loops g_aiSignals[0..g_aiSignalCount), all four topologies register unconditionally in OnInit, and SetIdentity gives each its own State\<id>\ folder so the six deleted paths are genuinely distinct per member. What IS true is that the whole success path was SILENT - six FileDelete calls per member printing only on failure, and one chart-wide Alert - so a four-member reset and a one-member reset produce byte-identical output. The symptom could be neither confirmed nor refuted from a log. That is the defect I can fix today. - COMPILED <timestamp> (__DATETIME__) beside the build tag. The hand-edited tag had sat at scan-nofwd-v5 across a week of commits, so it could not answer the question it exists for. The compile stamp cannot be forgotten. Tag bumped to reset-census-v6. - RegistryLine() (public): ID, active file path, common/local, era, deployed vs training, ensemble index. The reset handler prints a numbered census of the whole registry BEFORE the confirm dialog. If that says 1 on an AI_HYBRID chart the fault is registration, not the reset - and RegisterAISignal already has a loud MAX_AI_SIGNALS message for exactly that. - The confirmation dialog now names the count, so a wrong registry is visible before anything is deleted rather than after. - ResetWeights prints one line per member: N deleted / N already absent / N FAILED, plus a per-suffix breakdown. "absent" on a member that should have had a .nnw is a completely different fault from "deleted"; they were identical. - ResetWeights' return value was discarded. A member whose BuildFreshTopology fails has had its files deleted and has no network - and the Alert still said "weights reset". Counted now, with an INCOMPLETE alert when they disagree. - Same for dbm.ResetDatabase(), whose bool was also dropped. The DB is one shared file for every signal on the chart, so there is nothing per-member to loop - the log now says that explicitly, since it is the question being asked. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
AnimateDread
|
ad80e0bb57 |
fix(shutdown): make ExitPolicy public, and stop every long loop the moment MT5 asks
Two things, one of which was a compile error.
1. ExitPolicy() was declared in the protected block but is pushed in from
Warrior_EA.mq5:770. Moved to public beside the other EA-facing setters.
2. Chart objects surviving OnDeinit. The 4,500 ms teardown budget is measured
from the STOP REQUEST, not from OnDeinit's first line, and OnDeinit cannot
begin until whatever is in flight returns - so a scan still running after
_StopFlag is raised does not delay the cleanup, it SPENDS it, and the purge
never gets its turn.
New CExpertSignalAIBase::ShutdownRequested() = IsStopped() || m_shutdownInProgress.
Deliberately NOT m_trainingStopRequested: that latches, and a latched flag
would permanently disable scans that must run again on the next Start.
Guarded, longest first:
- TuneIndicatorsByFilter - per candidate, restoring the OPERATOR's settings
on the way out (best[] is mutated in place; the tuner otherwise keeps the
last trial's parameters, which nothing chose).
- ReportBarrierGeometryScan - per pairing, breaking to ONE restore point so
m_barrierScanLiveLabels can never be left true (that makes ComputeLabelForBar
read the last candidate's multiples as the configured geometry).
- ReportFeatureLabelInformation / ReportExcursionInformation / lag profile -
nulls ABANDON rather than truncate: fewer draws is not a smaller null, it
is a wrong one, and p shifts toward significance. m_dirEvidence staying
false is the safe direction.
- SimulateExitPolicyOutcomes - zeroes its accumulators so the divergence line
is dropped instead of latching a partial expectancy as the run's only report.
- ReportGeometryExpectancyScan - per ladder rung.
- HttpGet - one choke point for up to a dozen blocking WebRequests per
first-pass Update(). An in-flight request cannot be cancelled; refusing to
start another is the whole remedy.
- PollTraining, OnChartEventHandler's study event, TuneIndicatorsAndTrain -
entry points, so a queued event cannot open an era during teardown.
TuneIndicatorsAndTrain's guard is the first statement, ahead of the
m_tuneFilterDone / g_ensembleChartTuneDone latches.
- OnTick / OnTimer / OnChartEvent.
Training's own bar loops already honoured this (pass 1 per bar, passes 2/2.5/3
yield on a 120 ms budget); the warm-up scans did not, and they are the longest
uninterruptible stretches the EA has.
StopTraining() is unchanged: the operator's Stop still finalises synchronously.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
||
|
AnimateDread
|
94019f363e |
feat(gate): grade OOS calls on the exit policy actually in force, and move vote combining out of the members and into the orchestrator
Option (a) from the exit-policy question: the certified number must be the traded number. Plus the modularity correction the user called for on |
||
|
AnimateDread
|
17f808e90d |
fix(ensemble): MAX_AI_SIGNALS was 3 - the ensemble creates 4, so CONVLSTM was silently dropped
AI_HYBRID enables PAI + CONV + LSTM + CONVLSTM and RegisterAISignal registers
them in exactly that order. MAX_AI_SIGNALS was 3, and the guard returned
silently, so the FOURTH - CONVLSTM - never entered g_aiSignals[].
Reported as "convlstm is not listening to the control panel buttons", which is
the visible tip. Everything in Warrior_EA.mq5 that reaches a model does so by
looping g_aiSignals[], so the dropped member also lost:
- every control panel button (pause/resume, stop/start, retrain, deploy,
save, load, reset weights)
- PollTraining() in OnTimer - no wall-clock training progress, so it only
advanced on ticks
- AutosaveWeightsIfDue() -> SaveWeightsNow()
- AltDataReload() on both the mapping-dialog and hourly-upkeep paths
- StartChartSignalRescan()/RescanPending() - the Show Signals sequence
- the All*/Any* aggregates (deployed/paused/stopped/complete), which were
therefore computed over 3 of 4 members and could report the ensemble
finished while CONVLSTM was still training
- OnDeinit's MarkShutdown(), ShutdownChartCleanup() and FlushTrainRun() -
so its arrows were stranded on the chart and its training run was never
flushed on shutdown
It stayed hidden because the model still trains and still votes: it lives in
the signal's own filter array, and it registers itself with the status panel
(ENSEMBLE_PANEL_MAX_MEMBERS is 6) rather than through g_aiSignals[]. So it
appeared on the panel, drew arrows and moved the vote while being unreachable
from every action and unsaveable on exit.
MAX_AI_SIGNALS 3 -> 5 (4 is today's true maximum; the spare slot means adding
META to a preset cannot reintroduce this - the array holds borrowed pointers,
so unused slots cost nothing).
RegisterAISignal now PRINTS on overflow instead of returning silently. A cap
that discards a model without saying so is a trapdoor, not a guard.
NOT COMPILED - user compiles.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
||
|
AnimateDread
|
c0c9f4a285 |
feat(target): withdraw the TrainingTarget option - barrier is the only live one
NOT COMPILED - user compiles.
The private build still DEFAULTED to TARGET_FRACTAL, so every fresh attach was
training the target adjudicated dead that morning (5,700 model-eras flat at -2pp,
best-of-243 p=0.17). The campaign closed; the default was never flipped back.
Rather than re-default it, the input is withdrawn entirely (user: "remove the
option if there is only one choice for now"). An input offering a single live
choice is worse than no input - it presents a dead option as supported, and an
operator picking it silently trains a model already known to carry nothing.
Direction models are now unconditionally triple-barrier.
Removed: the input, the TrainTargetFractal() call in the signal setup, and the
HoldToBarrier() exit-policy block (which existed only because the fractal vote
flips at swing-marker cadence, ~3-5 bars, far inside the barrier's travel time -
barrier-target models keep vote exits and always did, their label IS the vote's
horizon). Verified no code reference to TrainingTarget survives; the four
remaining mentions are comments.
Kept deliberately, so a rerun is a re-enable and not a rebuild: the TRAINING_TARGET
enum, the fractal label itself, its |TGT:FRA1 fingerprint token, its conditional
barrier-geometry derivation, HoldToBarrier()/m_holdToBarrier, and the campaign's
trained models on disk. Three lines bring it back; Inputs.mqh names them.
ALSO CORRECTS THE RECORD from
|
||
|
AnimateDread
|
b6736fd40b |
fix: the DB backfill could never run, and HEAD did not compile
Four defects in 64c5dd5/1a05e63, found by review + a baseline compile. Goals 1-8 of that session are unchanged; this makes 6 and 8 actually reachable. 1. HEAD DID NOT COMPILE - 6 errors. CControlPanel::Minimize/Maximize were declared `virtual bool ... override`, but CAppDialog declares both as `virtual void` (Controls\Dialog.mqh). errors 265 + 404 on each, plus 151 on `bool ok = CAppDialog::Minimize()`. Return type is void now; there was never a success flag to forward. Verified: 0 errors, 0 warnings. 2. THE BACKFILL COULD NEVER ADVANCE, and neither could the OOS continual simulation (that one has been dead since it was written). Both are armed at the instant convergence is declared, and both advance only from inside Train(), one chunk per call. But ScheduleTrainingIfNeeded's only per-tick ArmStudyEvent site sits in the `else` of a branch taken whenever m_trainingComplete is set and m_trainRunActive is clear - which is exactly the state FinalizeTrainRun() leaves behind one line before they are armed. Train() was never called again, so the walks sat at their start index forever: no "simulation complete" line, and not one row written to the DB this feature exists to fill. Only a manual Resume/Retrain unstuck them. Both flags now keep the model schedulable. 3. IN AI_HYBRID - the mode this ships in - the backfill was never even armed. Ensemble members deploy at Train() ENTRY and return immediately (so no era is wasted), which skips the era-end block the backfill was started from. All four members were a no-op for a second, independent reason. Armed on the ensemble deploy path too, from m_resumeBars/m_resumeOosCutoff. 4. RE-RUNS DUPLICATED ROWS. RegisterSignal inserts unconditionally - no key, no duplicate check - and m_dbBackfillDone is in-memory, so every later attach that retrained to convergence wrote a second full set of rows for the same bars. The ranking would count one bar once per model that ever deployed, weighting superseded opinions as heavily as the live one. A .dbfill marker stamps the deployed era; written only on completion (an interrupted walk redoes itself rather than ranking a partial window) and deleted with the other sidecars on reset-weights. Also: WarmBlocking's timeout was silent, which restored the exact silent pin failure it was added to prevent - it now says so in the journal, and returns true for "no reference pairs to wait for" so the warning stays rare enough to be read. Not addressed, needs a decision: the backfill scores the OOS window with the checkpoint that was SELECTED as best on that same window, then writes those win rates into the table filter weights rank on - the selection set consumed twice, undiscounted, while the deploy gate right next to it applies a family-wise correction for exactly that effect. The rows are also simulated triple-barrier outcomes at today's spread sharing a table with realised fills. The completion log line now states both plainly. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
AnimateDread
|
64c5dd55d3 | feat: implement one-shot pattern-database backfill and enhance accuracy tracking for ensemble models | ||
|
AnimateDread
|
1a05e632eb | fix(altdata): ensure alt-data is available before model initialization to prevent undersized models | ||
|
AnimateDread
|
b77e7b4766 |
fix(ensemble): responsive panel + synchronized eras + combined-vote accuracy
Four user-reported/requested items, one root cause chain:
1) DEAD CONTROL PANEL in AI_HYBRID mode. All members posted custom event
id 1 and handled id 1001, and CExpertCustom broadcasts every chart
event to every filter - so each posted event ran a train chunk in ALL
N members (N*N chunks per round) and the chart thread never idled
long enough to deliver clicks/drags. profiling.csv: 99.45% of time in
OnChartEventHandler. Fix: per-instance study-event ids
(STUDY_EVENT_ID_BASE + construction order, offset above the Controls
library's ON_* codes - id 1 was also ON_DBL_CLICK, so panel
double-clicks fired training chunks). ArmStudyEvent() is the single
post site; lost-event watchdog replaces the accidental
sibling-clears-my-flag rescue.
2) WARM-UP DUPLICATION. The auto-tune sweep is deterministic over
identical features/labels, and it ends in the full MI diagnostic
suite, which the MI-share gate never intercepted on the sweep path -
four members ran four identical ~36s sweep+report blocks. First
member publishes outcome (g_ensembleChartTuneDone/Installed/Settings);
the rest apply it and skip both.
3) DEINIT STRANDED PANEL+ARROWS (user repro 18:52). Root cause from the
log: the 4,500ms budget runs from MetaTrader's stop REQUEST - a heavy
autosave in flight ate it, OnDeinit got ~430ms and died in the first
member's arrow persist ("Abnormal termination" 432ms in). Fix: early
visible-UI sweep (native prefix deletes for status/panel/dialog)
right after ClearStatusLabel, and a fast path for still-training
models - their arrows are re-rendered every era, so they get one bulk
purge instead of scan+atomic-write in the death window.
4) ENSEMBLE FEATURES (user requests): era BARRIER - members advance era
by era together; a member ahead of the slowest still-training member
declines Train() calls and its chunk budget is donated
(TRAIN_TIME_BUDGET_MS = 120/activeTrainers, UI headroom constant).
COMBINED-VOTE OOS SCORE - each member's pass-3 scan contributes its
adjusted per-bar decision (0.0 on abstain) to a shared row buffer;
the last member to finish the era scores the averaged vote vs the
mirrored Min_Vote_Open against the same target-before-stop outcomes
members grade themselves on, publishing an "Ensemble vote" line on
the aggregated panel. Member headlines now carry their lifetime win
rate with break-even.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
|
||
|
AnimateDread
|
0a198fb95f |
feat(altdata): EIA wired, 24-instrument symbol catalog, mapping dialog for unknown symbols
EIA (user directive: "the NN might find patterns in it for both oil and regular symbols"). Weekly Petroleum Status Report via the v2 API - crude stocks ex-SPR, field production, refinery utilization - three features (1y percentile, 4w change, utilization) on EVERY catalog symbol, not just oil. EIA screened NULL on WTI's short 7y sample, so these ship as EXPLORATORY inputs: the deploy gate, not the screen, decides whether a model trained on them trades. Publication stamp observed+6d mirrors research/altdata/eia.py. Symbol handling was hardcoded to three if-blocks; it is now a catalog of 24 instruments x alias lists covering The5ers/FTMO/AvaTrade/Dukascopy/OANDA/IC Markets naming, with prefix matching for the broker suffix zoo (US500.cash, XAUUSDm, EURUSD.r). Adding an instrument is one AddSpec row. COT caches are named by CANONICAL so two brokers' names for one contract share a download. Unrecognised symbol -> a chart dialog (Panel\AltDataMapDialog.mqh, CAppDialog + dropdown) asks which instrument it is; the answer persists in symbol_map.cfg and "No alternative data" is a recorded choice, not a nag. Non-blocking by design: an unmapped symbol contributes 0 features and must never hold up a chart. Also: UrlEncodePart now escapes '%' - SoQL like-predicates use it as the wildcard and an unescaped one corrupts the query; docs/ gains the whitelist URLs, an API-key backup, and the catalog reference. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> |
||
|
AnimateDread
|
0788238c00 |
feat(inputs): unify ALL indicator periods under the tuner; EnableAltData input; AI-first defaults
- PeriodMA/MA_Type/PeriodRSI: input -> const seeds (closing the set: every
indicator parameter is now tuner-owned)
- Variables\TunedPeriods.mqh: chart-level tuned-period state. A gated
install writes TunedPeriods_{SYM}_{TF}.cfg; next attach reads it BEFORE
the DB fingerprint and classic-signal config, so classic votes, DB key,
and tuner seeds always describe the same indicators regardless of
classic/AI/hybrid use. Restart-grained adoption by design (no mid-run
handle churn); new periods re-key the signal DB (semantics rule).
- EnableAltData input in AI Input Features (consumption gate only;
collection keeps running); |ALT DB-fingerprint token; opt-out on an
alt-trained model correctly starts fresh via the width compare.
- Defaults: all four classic votes OFF (AI-first; WARRIOR_MARKET_BUILD
branches collapsed with the marketplace pivot), order-flow/Wyckoff NN
features OFF (alt data is the default information diet; toggles stay).
Compiles 0 errors / 0 warnings.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
|
||
|
AnimateDread
|
8657c4fa12 |
feat(altdata): EA-side self-sufficient alt-data collection (WebRequest + OnTimer)
System\AltDataFetch.mqh: the EA backfills missing alt-data history at
attach and keeps appending forward while deployed - online learning never
depends on an external process. CFTC Socrata API (no key, 2006->now, one
GET per symbol; ES name variants verified, max-OI dedupe) + FRED (VIXCLS/
DTWEXBGS, key from AltData\keys.txt). Identical publication stamps and
fixed a-priori transforms as research/altdata/export.py; rebuilds the
same {SYM}_D1.csv files, so Python and EA interoperate on one format.
OnTimer hook (30-min staleness check, in-memory compares when current;
never in tester - cache files serve there) + AltDataReload() on signals.
Classic-signal removal CANCELLED per user (vote experiment later).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
|
||
|
AnimateDread
|
609be10391 |
feat(ai): AI_HYBRID = ensemble preset (all NNs, one chart); conv+recurrent renamed AI_CONVLSTM
The user is right that no special combination logic is needed: the AI signals are ordinary voting filters, and the aggregate already has union semantics - abstaining filters do not dilute the average, so an ensemble chart trades whenever ANY deployed member clears the vote threshold and disagreeing members net out. What the ensemble preset actually adds: - AI_CHOICE value 4 renamed AI_CONVLSTM (the name says the front-end); enum VALUES stable, CSignalHYBRID class and State\HYBRID\ folder kept, so saved configs and trained models keep their identity. - New AI_HYBRID = 6: enables PAI+CONV+LSTM+CONVLSTM together on one chart - replaces four separate charts of the same symbol. Each member trains and self-gates independently; only certified members ever vote. - |ENS1 fingerprint token on every member, so an ensemble member's weight files can never collide with a solo model of identical settings on another chart of the same symbol (the duplicate-chart guard would otherwise correctly fight over one .nnw). - Private default AIType = AI_HYBRID: one D1 drop now yields every topology's gate verdict for that symbol. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> |
||
|
AnimateDread
|
7cc6e35adc |
fix(exits): hold-to-barrier policy for fractal-target charts - live trades now match the certificate
The first-ever family-wise gate pass (SP500 D1 PAI, +10.4pp, p=0.0081) certifies a win rate measured on HOLD-TO-RESOLUTION outcomes: entry, then the measured SL or TP decides. Live, three vote-driven exit routes could close earlier - the averaged-vote close, the AI early-exit route (both in CheckClosePosition), and CheckReverse - and the fractal target's vote flips at swing-marker cadence (~3-5 bars), far inside the barrier's typical travel time (median 7-8 D1 bars to target). The user observed exactly this: an opposite arrow near an entry, trade cut, price kept going. On a fractal-target chart with a live direction model, all three routes are now suppressed (m_holdToBarrier, set in InitializeSignal, loudly logged): positions run to their broker SL/TP. Risk guards and trailing are deliberately untouched - account protection is not signal opinion. Barrier-target models keep the vote exits: their label is the vote's own horizon, so for them the routes are semantically consistent. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> |
||
|
AnimateDread
|
61a8c42a9c |
feat(ai): TrainingTarget input - fractal-direction label for the direction models
User direction (2026-08-15): back to predicting swing turns, D1 charts, fractals over ZigZag pivots (their call - balances classes, matches the reference library target, and a 5-bar fractal confirms 2 bars after its extreme so labels resolve nearly to the present with no repaint embargo). - TRAINING_TARGET enum + TrainingTarget input: TARGET_BARRIER (Market default - existing models keep their meaning and fingerprints) or TARGET_FRACTAL (private default). - FractalDirectionLabel (Labels.mqh): per-bar 3-class label = direction from the bar close to the next confirmed strict 5-bar fractal extreme, costs charged in the same bid-series convention as the barrier label, Neutral when the move cannot clear max(2 spreads, 0.10 ATR) or on an outside bar (both-extreme bars are unorderable within OHLC). - The barrier walk still runs in full: measured SL/TP geometry, the expectancy scan, excursion caches and the era gate all keep scoring what a trade at the EA's own stop/target actually collected - only the TRAINING label changes. NOT the pre-b4a704d "is this bar the pivot" form; that target's 31:1 imbalance stays retired. - Fingerprint token |TGT:FRA1 so switching targets trains a separate model; AI_META unaffected (guarded setter). - Private defaults: AIType back to AI_HYBRID (direction topology needed) + TrainingTarget=TARGET_FRACTAL = drop-on-D1-chart workflow. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> |
||
|
AnimateDread
|
1bf3eba68a |
feat(meta): self-contained corpus - the META chart sweeps the real classic ladders over its own history
The user should not need a tester corpus run per symbol. Every pattern
condition in Signals\Signal{MA,RSI,MACD,Ichimoku}.mqh anchors its reads on
`int idx = StartIndex()` with zero hardcoded indices (verified), so a
name-hiding StartIndex override + EvalShift(i) on CExpertSignalCustom makes
the EXACT live ladder code answer "what would you have fired at bar i" -
the silent-divergence trap that justified the DB corpus does not exist on
this path, and neither do the GMT-offset ambiguity, the DB row caps, or
the wipe procedure.
- CExpertSignalCustom: m_evalShift + StartIndex()/EvalShift() +
SweepPrepare(bars) (deep-resizes the shared price series); the four
classic signal classes override SweepPrepare to deep-resize their own
indicator buffers.
- CSignalMETA::BuildCorpusBySweep: per bar x per source filter, run
Direction() shifted, harvest the per-side pattern slots + netVote into
the same corpus arrays the DB loader fills; entry=bar open so
MetaPrepareEra's resolution matches at offset +0 with zero price error.
DB corpus remains the fallback when classic filters are disabled.
- Warrior_EA.mq5: META gets the enabled classic filters as candidate
sources (family ids match the descriptor one-hot).
- UseDatabaseRanking default false -> true (user request): a META chart
journals + ranks out of the box.
Workflow per symbol is now: attach ONE chart with AIType=META (optionally
Meta_ExportDataset=true for the offline pool) - candidates, labels,
training and export all happen in place, ~10 seconds of sweep instead of a
tester run.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
|
||
|
AnimateDread
|
444909d0a3 |
feat: S2 meta-labeling head - binary trade-quality model over the classic-candidate corpus
The NN now has a target that is not per-bar direction (closed, best-of-999 p=1.0000): P(win | this journaled candidate, at the EA's own SL/TP, net of cost). One net for all 52 pattern-sides, AIType=AI_META. - NetForward.mqh: the host-side softmax+CE gradient generalized total==3 -> 2||3 on both backprop paths; a 2-class softmax IS a logistic head, and no compute backend changes. - SignalMETA.mqh (new): corpus loaded read-only from the LARGEST signal DB on disk (decoupled from the config fingerprint that burned four S1 runs); the GMT->server offset is measured PER ROW against entryPrice vs bar open (DST-immune, histogram logged); a window-span regime filter drops the pre-2017 daily-backfill rows; 31-feature setup descriptor appended at the input (26 one-hot + side + tanh netVote + SL/TP ATR + spread/ATR). - Training.mqh: candidate-queued pass 1, binary-target pass 2, per-candidate calibration (2.5) and OOS (3) walks. Counter mapping win->Buy / loss->Sell lets checkpoint selection, the edge floor, the plateau ladder and the family-wise deploy gate run UNCHANGED: precision reads as win rate among traded candidates, chance as the base win rate, recalls as sensitivity/ specificity. Era-end META line: coverage x (p - break-even) vs the null. - Labels are the side-conditional triple-barrier win caches - never the DB's stop-and-reverse outcome. Logit adjustment deliberately skipped (~40% base rate). Live inference + online learning guarded off until S3. - Fingerprint: conditional |TGT:META1; State\META\ folder + 2-output filename slot keep meta models fully separate from direction models. Compiles clean (0 errors, 0 warnings). S2 run = attach a chart with AIType=AI_META; S3 wires the votes via the per-side hooks. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> |
||
|
AnimateDread
|
68459208bf |
fix(meta): make the stale-DB corpus warning unmissable in the tester
The warning lived inside the VerboseMode-gated corpus report, so a forgotten wipe silently voided an entire 18-year corpus run - the outdated-row guard rejected the whole replay against leftover rows and the run appended 35 rows instead of building a corpus. The check now runs unconditionally at tester OnInit (MetaCorpusStaleCheck): 52 quiet one-row newest-key probes vs the test start, with a loud stop- wipe-rerun instruction when the DB is newer than the test. Absent tables probe quietly via FetchNewestTimeKey''s new quiet flag. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> |
||
|
AnimateDread
|
4507ea69a9 |
feat(meta): S1 - the signal DB becomes the meta-label training corpus
Implements stage S1 of Meta_Labeling_Design.md, superseding the original "training-time ladder sweep": the per-side journaling from 652bf81/195be20 already produces the exact candidate stream a sweep would compute - every pattern instance the live ladders fire, both sides, uncensored, with netVote and touchable entry price - so the corpus is READ from the DB instead of re-implementing 26 ladder conditions in training code. That eliminates the silent-divergence trap outright: the corpus is by construction identical to live behaviour. Accepted costs are documented in the module and the doc: coverage equals the populating backtest, and sampling is one candidate per fire-stretch (the right dedup for training anyway). - Expert\AIBase\MetaCorpus.mqh: CMetaCorpus reader (52 tables -> SMetaCandidate rows) + VerboseMode OnInit report: volume/closed/ S&R-win-rate per family, span, and the GMT->server bar-offset match table (offsets +0..+3h) that S2''s label plumbing pins to - measured, not assumed. - DB_MaxRowsPerTable input (default 1000 = old MAX_TABLE_ROWS): a corpus build raises it (e.g. 20000) so a 15-20 year backtest isn''t pruned; wired through CExpertSignalCustom::MaxTableRows(). - Report-only stage: nothing downstream consumes the corpus yet. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> |
||
|
AnimateDread
|
652bf81112 |
fix(db): per-side pattern journaling + versioned journaling semantics
The labelMatchesVote gate compared a single last-writer-wins label (LongCondition then ShortCondition) against the net vote sign, which structurally censored the pattern tables: a long event co-occurring with any short-side state model lost its label to the later writer and was dropped, while the mirrored short event journaled fine. Ichimoku models 0/3 and MA model 1 could not produce a row at all by construction (MA model 1 was "revived" in |
||
|
AnimateDread
|
36e8463310 | refactor: derive history bars for input sequences and update related configurations | ||
|
AnimateDread
|
77e8080cfe |
fix: four risk-layer holes a funded account would eventually find
1. The expectancy stop was stone dead at shipped defaults. Its only feed -
RecordTradeResult inside CTradeJournalManager::Update() - ran solely under
UseDatabaseRanking, which ships false, so the
|
||
|
AnimateDread
|
c5acc5a7a8 |
perf: pass 1 forward-passed ~40% of bars that a later pass redid anyway
Pass 1 already skipped its feedForward on QUEUED bars, because pass 2 redoes them. The same argument covers two more bands it was still forwarding: OOS window (30% of bars) - pass 3 re-forwards every one of them calibration band (~10% of bars) - pass 2.5 re-forwards every one of them All three passes derive their bounds from the same helpers and apply the identical eligibility test, so the bar sets are equal by construction, not by coincidence. Only the two purge bands and the ineligible edge bars are visited in pass 1 and nowhere else - those keep their forward pass. The scan's copy was never the one that survived. Its arrow-cache write was overwritten by pass 3's (with the thresholded, post-training decision), its status-label paint was transient, and its predicted-class tally measured last era's weights. Those tallies move to pass 2.5 and pass 3, on the raw argmax exactly as pass 1 and pass 2 count it, so the population behind the panel's "Predicted -> Buy/Sell/Neutral" line is unchanged and stays comparable with the "Actual" line beside it, which pass 1 still accumulates over every labelled bar. Verified unaffected by the cut: dPrevSignal and m_lastBarTime are both written last by bars 0/1, which are label-ineligible and therefore still forwarded, so FinalizeTrainRun's `dtStudied = m_lastBarTime` and Lifecycle's newBarPending sentinel read the same values as before. Correctness, not just speed: batch norm is UNFROZEN during pass 1 (passes 2.5 and 3 freeze it deliberately), so every scan-time forward on a held-out bar was advancing the BN running mean/variance from data the model is graded on. Those running statistics are inference-time model state. It is the mild, unsupervised kind of leakage - feature statistics, not labels - but it fed the weights pass 3 then scored, and it is now gone. Cost: ~40% of all bars lose one forward pass per era, ~16% of net time once pass 2's backward pass is weighted in. Per-dispatch, so it lands on every backend. Both variants compile 0 errors / 0 warnings. Build tag scan-nofwd-v5. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
AnimateDread
|
e2c959331f |
perf: the excursion head cost 3.6x era time - cut its dispatches ~250x
Measured on exc-race-v3: LSTM era 300s -> 1087s (net 272->748s, "other" 30->337s). My estimate had been "single-digit percent". The cost is per-DISPATCH, not per-FLOP, and therefore hits EVERY backend: the head is 19k weights and ~2.4 GFLOP an era - seconds of arithmetic - but ~48k forward/backward calls x several layer submits each, and its 760-wide layer exceeds the CPU DLL's inline threshold so each one pays a real handoff. The classifier's own net time tripled too, from contention with a second pool on an already-full box. Three changes, all backend-neutral because they remove submits rather than tune threads: SCORE ONLY DISJOINT WINDOWS (~64x). Adjacent bars share all but one bar of their horizon, so 16k consecutive bars were always ~250 independent observations - the full-sample tally was never worth more than the disjoint one, it just quoted an n that was ~64x too large. Dropping it costs nothing statistically and removes 63 of every 64 forward passes. The two parallel tallies collapse into one, which is also less code. The trailing ring still advances on every bar: it needs the outcome SEQUENCE, and that is array lookups, not a forward pass. TRAIN ON EVERY 4th PRIMARY BAR (4x). The target is low-dimensional and strongly autocorrelated - neighbouring bars carry near-identical excursion information - so per-bar training buys resolution the target does not have. Strided on ATTEMPTS, not acceptances, so a stretch of unlabelled bars cannot silently change the spacing. OWN TIMING COLUMN. The head's passes were landing in the era line's "other" bucket, which is how a 3.6x regression read as an unexplained jump in the one column nobody attributes. A cost that cannot be seen in the timing line cannot be traded off against anything. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
AnimateDread
|
345a672500 |
fix: purge every EA object namespace on init and after deinit teardown
Leftover objects survived deinit because the cleanup list had drifted. PurgeChart()'s own comment said it removed "our namespaced signal arrows plus the status-label objects" while the code removed arrows ONLY, and the panel prefix was swept at OnInit and nowhere else - so an ordinary deinit left the status line, and any panel straggler, on the chart. Three scattered call sites and a comment cannot be kept in step. There is now ONE list - WarriorChartPrefixes() - covering arrows, status label and panel, and one sweep, WarriorPurgeChartObjects(), used by every path. Add a prefix there when a new object family appears and every cleanup picks it up. Two call sites added: OnInit, before ANYTHING is drawn (including the status label it would otherwise delete). Chart objects live in the chart PROFILE, not in the EA, so they outlive the process: a deinit force-terminated at MetaTrader's ~4,500 ms budget, a crash, a terminal kill, or an .ex5 replaced while attached all strand objects no later deinit will ever own - and deleting the EA's files does not remove them, which is why they read as corruption. Arrows are included: LoadChartSignals restores them from their sidecar moments later and already opens with its own arrow sweep, so this only removes orphans the sidecar does not account for - the ones SaveChartSignals would otherwise ADOPT, since it rebuilds that sidecar by scanning the chart. OnDeinit, after ExtPanel.Destroy. Destroy walks an unbounded control tree and ClearStatusLabel clears text rather than guaranteeing object removal; either can leave a straggler and nothing looked afterwards. Bounded work - three prefix deletes and one object-list scan - so it respects the ordering rule that keeps the cheap visible cleanup ahead of the heavy save. Arrows excluded: ShutdownChartCleanup already persisted and removed them and re-deleting would race that write. The two are complementary: the deinit sweep closes the ordinary case, the OnInit purge closes the case where MetaTrader never let us finish. Only the second can help after a starved shutdown. Both sweeps rescan by name across EVERY object type and delete what the bulk call missed. ObjectsDeleteAll's return has already been observed disagreeing with a by-name scan of the same chart microseconds apart, and object commands are queued on the chart rather than applied inline, so a returned count is not evidence the objects are gone. Panel create site now uses WARRIOR_PANEL_PREFIX instead of a literal, so the name cannot drift away from the list that cleans it up. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
AnimateDread
|
4cfbb82634 |
feat: race the excursion head against a trailing-quantile incumbent
Beating a frozen global constant is the weakest admissible bar for replacing a global constant. The honest incumbent is a rolling rung frequency: it adapts to the volatility regime - exactly what the head claims to predict - and needs no model, no 760 inputs and no training. Implemented as a ring of per-bar outcome bitmasks (32 rungs fit one ulong), sized horizon + EXCURSION_TRAIL_WINDOW. The newest `horizon` entries are held back UNRESOLVED: a bar's rung outcomes are only known one horizon later, so using them would be lookahead and would flatter the incumbent into an opponent the head could never fairly beat. Pass 3 walks oldest-to-newest, so "pushed more than horizon bars ago" is exactly "resolved by now". Each push is O(rungs), not O(window). The head's decision-rung Brier is pro-rated to the trailing estimate's coverage before the ratio, since the incumbent only scores bars where its window is warm. This line is worth reading on its own, independently of the head: if the trailing quantile beats the global constant, that is a cheap risk-control win available with no machine learning at all - and it is the same number either way, so the run answers both questions in one pass. The ring is deliberately NOT reset per era - it estimates the market, not the era, and re-warming 500 bars every era would leave the incumbent unusable over the first chunk of every scoring pass, handing the head a free win on exactly those bars. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
AnimateDread
|
06d4785e39 |
fix: the excursion gate would have passed Stage 2 on an artifact I made
Second-opinion review killed the +4.2% far-rung result, correctly, and
the mechanism is my own bug. A head trained toward {0.05,0.9} converges
to 0.05+0.85p, so its bias is 0.05-0.15p: negative where p is near 1,
POSITIVE where p < 1/3, growing monotonically as the rung gets farther.
Against a baseline frozen at the IS rate, an upward-biased head scores
positive Brier skill whenever the OOS rate merely sits above the IS rate.
Predicted signature: huge negatives near, ~zero at p=1/3, growing
positives far. Observed: -82% ... -0.6% ... +1.2/+2.7/+4.2. The far rungs
were not the clean end of a distorted measurement, they were the other
face of the same artifact. Everything before
|
||
|
AnimateDread
|
25aca8367c |
fix: the excursion head was scored against a cap I gave it
ExcursionTargets built its 32 binary targets from the classifier's LABEL_SMOOTH_HIGH/LOW (0.9/0.05). That caps what the head can ever output at 0.9, and the near ladder rungs have base rates close to 1.0 - almost every bar travels 0.5 ATR inside a 64-bar horizon. The Brier comparison is then decided before the net learns anything: constant at 0.99 -> 0.99*(0.01)^2 + 0.01*(0.99)^2 = 0.0099 head at 0.90 -> 0.99*(0.10)^2 + 0.01*(0.90)^2 = 0.0180 skill -82% Which is what the first run reported at rung 0.50: PAI -61.8%, CONV -146%. A property of the target encoding, not of predictability. Smoothing earns its place on the 3-class head, where it stops one logit running away inside a softmax competition. There is no competition here and this head is scored on calibration, so it has to be free to say 0.99 when the answer is 0.99. Hard 1/0 is safe against the runaway smoothing guards: this is an MSE-on-sigmoid gradient (calcOutputGradients) whose (target - output) term vanishes as the output approaches the target, not the unbounded-logit cross-entropy the classifier uses. The far rungs, where the artifact is smallest, already showed positive skill on the two topologies with a sequence stage (LSTM 3.00:+1.2% 4.00:+2.7% 5.00:+4.2%, HYBRID similar), so the verdict was being decided by the most distorted end of the ladder. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
AnimateDread
|
f6150ee35b |
fix: cache only feature SUCCESSES - the cold-indicator poison came back through the guards ba13eef did not cover
|
||
|
AnimateDread
|
950b0fdab0 |
diag: name the cause when every feature window fails, and enforce the width contract
Era 0 stalls with "NOT ONE of 54681 scanned bars produced a usable feature window, windows ok=0 failed=54681" and nothing else. That line reads identically for a cold ATR, a conditionally-missing optional feature block and an out-of-range index, so it cannot be diagnosed without one restart per hypothesis. Two changes: 1. WIDTH CONTRACT in BufferTempData. Every enabled block must emit exactly m_neuronsCount values on EVERY bar. A block that emits its values on some bars and skips them on others (indicator, panel or series unavailable for that bar) does not merely shorten the window - it SHIFTS every feature after it into the wrong slot, and the net then trains on silently misaligned inputs that still look like a valid window to everything downstream. Now rejected, rolled back and reported once, naming the optional blocks (XA / SPR / swing context) as the ones carrying an availability test. Worth having independently of the current stall. 2. BuildFeatureWindow records WHICH lookback slot rejected and how much of the window was assembled, and the pass-1 stall report renders it: "slot 0 of 20 REJECTED (window had 0 of 760)" is an indicator warm-up or history-edge read; "every lookback bar ACCEPTED and the window was still short: 640 of 760" is a missing 6-value block. No behaviour change on a healthy run: the width check is an equality that already holds, and the diagnostics render only inside the total-failure branch. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
AnimateDread
|
2d28f6542b |
feat: excursion-size head (Stage 1, measurement only)
Direction is closed - normalised asymmetry fails on three instruments with a working positive control, and the classifier's own best-of-999 era-cap test agrees (+0.9pp = 1.48 sigma, family-wise p=1.0000). SIZE is a different question and RANGE clears at ~4x its null. Checked the denomination before building on that, since the source memo warns to: m_excUpCache holds (maxHigh - fill)/ATR, so "RANGE is predictable" is a claim about travel RELATIVE to current ATR, not a restatement of "ATR is autocorrelated". It is exactly the part a fixed multiple (stop 3.31*ATR, target 1.64*ATR) discards. A second small CNet, 760 -> 24 -> 32 sigmoid outputs = P(price reaches ladder rung k) upward and downward. Survival parameterisation rather than regressing the multiple, because it needs nothing new from CNet: sigmoid outputs and the per-neuron delta the `total != 3` branch already applies (a quantile head would need a linear activation and a pinball gradient in Network.mqh, Network.cl and the DirectML path, on a class four topologies share). Targets are free - m_ladderUpAt already records first-touch age per rung with 0 meaning never reached. Separate net, not extra outputs on the classifier: more outputs would change m_outputNeuronsCount, the .nnw shape and the fingerprint, and push the count off 3 - the exact condition backProp uses to select the joint softmax gradient the 3-class head depends on. The classifier is bit-for-bit unaffected and this is removable without trace. STAGE 1 PLACES NO ORDERS. It reports a Brier skill score against the constant per-rung base rate - the baseline a fixed ATR multiple already assumes - with both predictors fitted IS and evaluated OOS, so neither gets a look at the test set. Positive skill justifies Stage 2 (drive SL/TP and sizing off ExcursionQuantile, which is defined and deliberately uncalled). Zero or negative means ATR already carries everything and Stage 2 must not be built. Trains only on primary occurrences: the replay queue oversamples for CLASS balance, and a direction-balanced sample is a biased SIZE sample. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
AnimateDread
|
0c8b4dc30d |
fix: the deploy gate graded the un-thresholded model
coveragePct, dirPrecPct and the declustered TRADED tally were all computed from oPrevSignal - the RAW argmax - while the live order, the arrow and the panel all run on oDeploySignal, which is argmax AFTER the confidence threshold. The gate was certifying a strategy the EA does not trade. Invisible until now: the threshold sat at ~0.02, so the two populations were the same set. The held-out calibration slice ( |
||
|
AnimateDread
|
2189316c35 |
fix: the operating point was fitted on bars the net had memorized
FitDirConfThreshold harvested its margin histogram from pass 2's own
backprop samples. Pairing every fit against the same era's OOS result
shows what that measured:
PAI era 1 IS 25% cov @ 66.1% (-0.8pp) -> OOS 64% (-3pp) gap +2.1pp
PAI era 76 IS 90% cov @ 79.6% (+12.7pp) -> OOS 65% (-2pp) gap +14.6pp
LSTM era 9 IS 77% cov @ 81.6% (+14.6pp) -> OOS 63% (-4pp) gap +18.6pp
The gap grows monotonically while OOS stays flat, so within a handful of
eras the curve stops describing behaviour on unseen bars. That is fatal
here specifically, because the objective branches on the SIGN of
(p - break-even): the memorized curve reads +12pp at 95% coverage, so
coverage x (p - p0) correctly maximises coverage and returns ~0.02 - fire
on every bar. The "p < p0 -> get more selective" branch, which is the
actual regime and the entire point of
|
||
|
AnimateDread
|
d919a4aea2 |
feat: 10-bar decluster window + alternation on every signal consumer
SignalClusterWindow 3 -> 10 for all topologies. On H1 a 3-bar window
collapsed only the tightest runs and left visible clusters at every
turn; 10 bars is closer to the spacing of genuinely distinct setups.
ALTERNATION. Rule 1 only collapses a same-direction run INSIDE the
window; past it a second Buy is emitted with no Sell between, giving
Buy/Buy/Buy/Sell. With both directions tradeable that sequence is the
model re-entering a move it is already in rather than finding a new
one. The kept sequence must now alternate: the first signal passes,
and after that a direction passes only if the last KEPT signal was the
opposite one.
Added to ALL THREE consumers, with identical logic, because they must
agree:
- NmsLiveAccept -> the live trade
- pass 3's OOS replay -> the tally the deploy gate grades
- PruneDirectionalClusters -> the drawn history
A rule applied to only some of these certifies one strategy and trades
another - the same defect class as the geometry the gate certified
while OpenParams placed something else (
|
||
|
AnimateDread
|
ccfbc62561 |
fix: the recall gate was unsatisfiable and the LR decay was a spiral
Both made the run structurally unable to succeed, independently of any
signal in the data. Found by reading the 13:01 log.
RECALL GATE. m_objectiveMet required Buy, Sell AND Neutral OOS recall
each >= 40%. First-touch resolution (
|
||
|
AnimateDread
|
ece2154102 |
fix: flush the in-flight era on shutdown; sweep orphaned chart objects on attach
Chart objects live in the MT5 chart PROFILE, not in this EA's files. They survive a terminal restart, a recompile, and deleting every .nnw/.cfg/.stats/.arrows on disk. Only a deinit that RUNS TO COMPLETION removes them - and MetaTrader force-terminates OnDeinit at roughly 4,500 ms, so a run killed mid-cleanup orphans them permanently with no owner left to clean up after. That is the "deleted every file, recompiled, restarted, old arrows and a stale panel still there" report: nothing was wrong with the files and deleting them could not have helped. Both halves are fixed. STOP OVERRUNNING THE BUDGET. OnDeinit used to finalise the in-flight run (StopTraining -> FinalizeTrainRun: checkpoint restore, live-state re-seed) and then write two full nets per chart. On four charts that is the bulk of the budget, spent to preserve a PARTIAL era that was never scored, never checkpointed and never deployable. FlushTrainRun() discards it instead - drop the resumable bookkeeping, leave the net neutral (unfreeze BN, flush the batch, batch size 1), skip the save - and training resumes from the last completed era, which the era-end save and the periodic autosave have already put on disk. What is discarded is bounded by one era. A CONVERGED model keeps the old finalise-and-save path: its weights can carry online-learning updates made since the last era boundary, and for a deployed model no further era boundary is coming to persist them. MAKE CLEANUP SELF-HEALING. Every purge sat behind a branch - no model loaded, sidecar missing - so the common paths returned leaving whatever the previous instance stranded. LoadChartSignals now sweeps the arrow namespace unconditionally before restoring, so the post-init chart holds exactly what the sidecar holds whichever branch runs, and the panel gets the same treatment before Create() (CAppDialog namespaces its controls, so a killed Destroy strands the lot and the next attach draws a second panel on the corpse). Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
AnimateDread
|
1a5157befc |
fix: training could only advance one 120ms chunk per bar
ScheduleTrainingIfNeeded() armed the next Train() call only when dtStudied < lastBarDate. That watermark test is right for a CONVERGED model - one inference refresh per new bar - and wrong for a training run, because Train() is chunked: it does ~120ms of work and yields, needing thousands of calls to finish one era, and every one of those calls has to be armed from there. dtStudied is two incompatible things. Train() sets it to the training WINDOW START (~2008); FinalizeTrainRun() sets it to the last bar SCANNED (~now). So the moment any run finalized, the scheduler went silent until the next candle closed. On H1 that is one chunk per hour. The symptom was indistinguishable from a hang: no era lines, no heartbeats, not one of the six instrumented stall branches - because Train() was not being CALLED. The TRAIN STALL line that caught it reported runActive=Y only because m_trainRunActive had been set microseconds earlier in that same call, and eraResume=N proved no era was in flight. Two log bursts, 28 minutes apart, exactly one H1 bar. Before |
||
|
AnimateDread
|
9a7c37f334 |
fix: live trades now use the geometry the gate certifies; perf: BN kernels
Three changes, one theme: the trade placed, the trade graded, and the trade computed are now the same trade. 1) GEOMETRY WIRE (correctness, the ranked #1 open issue). The measured barrier pair reached the LABELS only - OpenParams still placed orders at the enum geometry (2*ATR/6*ATR), so the deploy gate certified "reaches 1.62*ATR before 3.33*ATR above break-even" about trades the EA never placed. Published via g_DerivedSlAtrMult/g_DerivedTpAtrMult (ConfidenceBridge, same same-tick contract as the confidence globals, because OpenParams runs on the root signal which has no pointer to the AI filter). Two writers: DeriveBarrierGeometry at era 0, and the .cfg adoption a deployed model takes. Overrides both legs and both Intelligent modes - the certificate is exact or it is nothing. TP is ATR-anchored like the label, NOT risk-relative, so a floor-widened stop cannot reshape the certified target. 2) BATCH NORM RUNS DEVICE-SIDE ON OPENCL. Four kernels in Network.cl - forward, hidden gradient, gamma/beta accumulate, gamma/beta apply - each a line-for-line transcription of the host implementation (NormalizeHost / HiddenGradHost / StepGammaBeta) including every NaN guard, clamp, and the exact moment-write ordering. The host copies remain the runtime for the DLL and pure-MQL5 tiers and the reference the kernels must match. Because this box has no OpenCL platform, the safety story is layered: - shim validation: kernels compiled as C and driven against a fp64 host transcription over NaN-poisoned stats, NaN gamma, over-clamp inputs, the frozen path, both optimizers, 3 batches - ALL PASS, worst normalized diff 0.132 vs tolerance 1.0 - in-situ self-check: each kernel is compared against its host twin ON FIRST USE on the real device (SelfCheckBn*), covering what the shim cannot - arg indices and buffer bindings. Any disagreement resyncs from the good copy, latches all BN kernels off process-wide, and training continues host-side. A transcription bug costs a warning and some speed, never a poisoned .nnw. - sync discipline: BatchOptions is now a CBufferDouble with explicit authority tracking (m_bnDeviceAuthoritative). Checkpoints/saves pull read-only; restores/loads/resets push; a mid-batch handover drains the device gamma/beta accumulator into the host arrays so no sample is lost. 3) SMALL FIXES. Apply-kernel build failure now latches the dispatch path at init (one warning instead of warning + failed Execute). Build tag bumped to win-scoring-gpu-v1 - first tag change since expectancy-stop-v1 despite five binary-changing commits. Both build variants compile 0 errors, 0 warnings. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> |
||
|
AnimateDread
|
217b9bc9bf |
feat: remove Min_Risk_Reward_Ratio - a guess was overriding a measurement
The barrier geometry is derived from the instrument's own excursion distribution (stop at q75 of adverse travel, target at q50 of favourable), and then a 1:2 floor was applied on top, raising the target to twice whatever the stop happened to be. On SP500 H1 that pushed the target to 6.66*ATR, reached on 3.3% of bars inside the horizon - so the label became "almost never a win" and every topology was trained to predict an event that essentially does not occur. A measured target has to stay measured. The ratio never bought what it was believed to buy. A reward:risk floor does not create expectancy; it trades hit rate against payoff at a break-even the geometry already fixes - which this project has separately MEASURED (payoff 0.92 -> 5.72 with expectancy flat). What it did buy was two outages: four consecutive Market validation rejections for "no trading operations" when it rejected 100% of setups, and the label corruption above. Removed: - the input and the RISK_REWARD_RATIO enum (deleted, not left dangling - a live enum with no input behind it is the shape of the stale-.set incident that trained ~250 eras on the wrong target) - the forced target raise in the label geometry - the rrOK eligibility gate in the barrier-geometry scan, so every unclamped pairing now competes on the measurement alone. Clamping stays disqualifying for its own unrelated reason. - the reward < minRR*risk veto in OpenParams Kept: g_TradeRewardRiskRatio still computed and still bridged to Kelly sizing in MoneyIntelligent - the ratio as a SIZING input was always the sound use. Risk stays bounded where it actually is - account risk % and CRiskBudget. The low-reachability warning survives but is re-aimed: with nothing inflating the target, a target the market rarely reaches can only mean the horizon is truncating the excursions the geometry is derived from. Both build variants compile 0 errors / 0 warnings. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> |
||
|
AnimateDread
|
371f8aaecd |
fix: the Adam second moment was never Adam - all four tiers
Root cause of the B=32 regression, and it predates F4 entirely. Every Adam
kernel stored v already square-rooted and then fed that stored value back in
as if it were the variance:
v_new = sqrt(b2 * v_old + (1 - b2) * g^2)
That recursion has a fixed point at v ~= b2 = 0.999 for ANY gradient below
unit scale, so the denominator stops tracking the gradient and Adam degrades
into plain SGD with lr = lt. Measured against the shipped WarriorCPU.dll
(batch_accum_check.cpp, TestOptimizerScaleInvariance), 4000 steps of a
constant gradient: 3285x less displacement at |g|=1e-5 than at |g|=1, where
a scale-invariant optimizer gives the same distance for both. After the fix
all six magnitudes read 1.199 and v tracks |g| exactly.
It hit conv/LSTM specifically because they sit behind a batch-norm with
running variance ~2.6e+05, so their gradients arrive divided by ~500 - deep
in the degraded regime - while the dense stack near the loss stayed in the
working one. In situ on SP500 H1: lstm1 dW/W 2.62/10.0/7.14% -> 0.024/0.022/
0.003%, conv1 decaying to 0.000% by era 30. NeuronBatchNorm.mqh already
squared v back for gamma/beta and its comment named the kernels as wrong,
which is exactly why gamma/beta kept training while the stages behind froze.
Persisted .nnw needs no migration - v keeps its std-dev meaning.
Also, the two ways F4 exposed it, both mine:
- No LR compensation for B fewer steps per era. sqrt(B) for adaptive methods
(Krizhevsky 2014; Granziol et al. 2022), applied once in
InitialEtaForOptimizer(). Linear scaling (Goyal et al. 2017) is for SGD.
- Plateau patience denominated in eras, so raising B made the ladder 32x more
impatient in its only unit. PAI converged at era 41 on ~49k updates where
the same config had been finding new bests at era 1028.
TrainPlateauPatienceEras() stretches it by the same sqrt(B).
TRAIN_BATCH_SIZE 32 -> 8 so the patience stretch stays affordable (8 -> 23
eras per stage, not 8 -> 45). Both helpers are identities at B=1.
Deploy gate: DEPLOY_MIN_SIDE_RECALL_PCT (10%) folded into tradeableOK. The
perceptron reported Sell:0% recall in all 41 eras, cleared the floor on Buy
alone at 36.6% vs 34% chance, deployed, and sprayed buy arrows. Folded into
the ranking key rather than checked at deploy time so a one-sided era cannot
become best-so-far in the first place.
Deinit: the arrow purge now runs BEFORE ExtPanel.Destroy(), an unbounded
CAppDialog teardown that sat ahead of it - the same ordering inversion the
rule there exists to prevent. CONV was force-terminated 4.8 s into OnDeinit
(vs ~1.1 s for the three that finished) having reached none of its cleanup,
so its arrows stayed on the chart. Steps are now timed in the log.
PurgeChart's verification rescan filtered on OBJ_ARROW, the same blind spot
as the bulk delete, so "persisted 10 ... cleared 0" passed silently. It now
walks every object type and reports the object counts when both are zero.
Both build variants compile 0 errors / 0 warnings; both DLLs rebuilt.
FORCES A RETRAIN (already forced by N1) and both DLLs must ship with the .ex5.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
|
||
|
AnimateDread
|
da54639996 |
feat: expectancy stop - halt when the measured result says the strategy loses
The daily (4%) and total (8%) rules bound how FAST an account can lose. Nothing
noticed WHETHER it was losing. A negative-expectancy signal traded at 1% inside
that envelope breaches no rule and still arrives at zero - it just takes longer,
with every limit green the whole way down. That is the realistic way this EA
destroys an account, and no existing guard could see it.
THE ARITHMETIC THIS ENFORCES. Expected value per trade is p*TP - (1-p)*SL - cost.
With no directional edge p equals SL/(SL+TP), which is also the break-even rate,
so the payoff terms cancel exactly and EV = -cost. Expected P&L is -(trades) x
cost: strictly negative, proportional to activity. Measured here: directional
precision 23-24% against a 25% break-even, flat across every confidence tier,
with 58 points of spread on SP500. Sizing, stop placement and trailing move
variance around that mean; none of them changes its sign.
So every closed position now reports its result in R (net profit over money
actually at risk) and the running mean is tested against zero. Above the
configured minimum sample, if mean + sigma*SE < 0, new entries stop.
- SIGNIFICANTLY below, not merely below. A run of losers is ordinary variance
even for a profitable system; halting on the raw mean would be the same
act-on-noise error the MI gates exist to prevent. Using the standard error
means a wide spread simply demands more trades before the rule can fire.
- NET of swap and commission (ResolveClose already sums all three). Deliberate
and load-bearing: when the edge is zero, cost IS the expectancy, so a gross
version would measure a strategy nobody can trade.
- Reported in R so symbols, lot sizes and balances share one scale and one
mean. Trades without a stop are not scored rather than assigned a guessed R.
- LATCHED across restarts, like the daily halt and for the same reason: a
latch a reattach clears is not a latch. Clearing it means deleting the risk
state file, deliberately, after looking at why.
State is appended to the risk file length-guarded, so files written before this
still load and start their sample at zero rather than misreading.
Defaults 40 trades / 2 sigma; ExpectancyMinTrades = 0 disables it.
This does not make the strategy profitable and is not meant to. It stops paying
tuition on one the results say is losing, and does it on measurement rather than
on a drawdown limit finally being reached.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
|
||
|
AnimateDread
|
b3b7e7bceb |
fix: excursion window must not depend on the barrier it sizes
DIRECTION IS NOT THERE, and this run is what establishes it. Three symbols:
raw ASYMMETRY clears on all three (p=0.0199 / 0.0050 / 0.0050)
norm ASYMMETRY collapses on all three (p=0.3433 / 0.5075 / 0.2736),
USDCAD landing BELOW its own null
RANGE control strengthens to 3-5x its null everywhere
Divide sigma out and the apparent directional signal vanishes entirely. What
cleared was volatility leaking through an unnormalised difference. Note this
would have passed any replication test: three instruments at p=0.005 is exactly
the evidence one would accept before committing to a rebuild, and the confound
reproduces perfectly. Replication was never going to catch it - only the
normalisation could.
Two defects of mine, both surfaced by the same run.
1. THE GEOMETRY DERIVATION WAS DIVERGING, NOT CONVERGING. It produced a
14.57*ATR stop and a 29.14*ATR target that only 5.7% of bars ever reach.
Excursions were measured over the barrier horizon; the horizon scales with
the target; the target is a quantile of the excursions - so target ->
horizon -> excursions -> target ran away, and "settled" only because the
horizon ladder caps at 384 bars. A saturated runaway, which the iteration
guard could not catch because it watches for OSCILLATION.
Fixed at the root: excursions now accumulate only over m_swingMedianBars -
the UNSCALED median ZigZag leg, a property of the instrument that owes
nothing to the barrier. The barrier walk still runs the full horizon,
because that is how long the trade is held; only the MEASUREMENT used to
size the barrier is confined to a geometry-independent window.
(The Min_Risk_Reward_Ratio warning fired correctly and is what flagged it -
the diagnostic worked while the derivation behind it did not.)
2. THE CONFOUND VERDICT WAS UNREACHABLE. `sizeCleared && !asymCleared` was
tested first and is true whenever size clears - i.e. always - so the branch
that NAMES the volatility confound never printed; all three symbols showed
the generic size-not-direction message instead. Verdict chain rewritten with
the specific case first, and the dangling elses my first patch introduced
removed.
FORCES A FULL RETRAIN (the excursion window changes every derived barrier).
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
|
||
|
AnimateDread
|
32ffeb99f3 |
fix: normalise the asymmetry target - the raw one is confounded by volatility
Three symbols ran the excursion test. RANGE/UP/DOWN cleared on all three;
raw ASYMMETRY cleared on EURUSD and USDCAD at p=0.0050 and not on SP500
(p=0.1045). That looked like the first directional signal this project has
found. It probably is not, and the test as built could not tell.
(up-dn) IS NOT SCALE-FREE. If sigma is predictable - and RANGE clears at ~4x its
null on every instrument - and the directional part is symmetric noise eps, then
up-dn ~ sigma*eps, so a large sigma pushes the value into BOTH outer terciles. A
pure volatility predictor scores positive MI against a 3-bin (up-dn) while
carrying no directional information at all. Crucially that confound REPLICATES,
so reproducing on two instruments is not evidence against it - and the effect
sizes fit it: asymmetry runs 1.3-1.6x its null where RANGE runs ~4x, and carries
~0.1% of the target's entropy against RANGE's ~0.9%. That is the shape of a
leaked fraction of the volatility signal, not an independent one.
So add (up-dn)/(up+dn): bounded in [-1,+1], volatility divided out, and the only
target a directional claim may rest on. The verdict now separates the cases and
NAMES the confound when raw clears while normalised does not, instead of
reporting the raw line as a finding.
Two bugs of mine in the same block, both caught by output rather than review:
- The derived-geometry line had a MISORDERED argument list: it printed
"stop 25.00*ATR (q3 of adverse travel)" - the quantile percentage as the
multiple and the multiple as the quantile. Real values were 2.61 stop /
8.03 target. A 25*ATR stop is absurd on its face, which is why it was seen.
- THE STOP QUANTILE WAS BACKWARDS, and this one changes labels. It was 0.25
"so ordinary noise does not reach it", but q25 means 75% of bars EXCEED the
stop - hit three times in four. The printed reachability said exactly that
("stop on 75.0% of bars"). Now 0.75. A quantile is a threshold, not a rate.
This is the entire reason reachability is measured and printed rather than
assumed.
Also raises BARRIER_DERIVE_MAX_PASSES 3 -> 5: SP500 did not settle in 3 (stop
still moving ~14% per pass) while EURUSD and USDCAD converged on pass 2. And
bounds both quantile indices with MathMin(..., n-1) so q=1.0 cannot run off the
end of the sorted array.
The geometry from the previous run is NOT usable and the asymmetry result is
unresolved, not established. Both are decided by the next run.
FORCES A FULL RETRAIN (the stop quantile changes every label).
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
|
||
|
AnimateDread
|
a7701f032b |
feat: derive the ATR multiples from measured excursions - no hardcoded geometry
The barrier was still two constants. SL_Mode/TP_Mode left the Inputs tab in |