mnbvc188199/Warrior_EA
Watch
1
0
Fork
You've already forked Warrior_EA
 1
forked from animatedread/Warrior_EA
Code Pull requests Activity
Warrior_EA/Expert/AIBase/AutoTune.mqh

777 lines
41 KiB
MQL5
Raw Permalink Normal View History

refactor: split CExpertSignalAIBase implementation by responsibility ExpertSignalAIBase.mqh was 8216 lines: the class declaration followed by 87 method bodies covering training, labelling, feature extraction, persistence, chart drawing, online learning, the GA auto-tuner and inference, all in one file. Train() alone is 1492 lines; a change to arrow drawing meant scrolling past the era loop. Moved the bodies into Expert\AIBase\, included at the bottom of the original after the class declaration: Training.mqh 1607 era loop, plateau ladder, checkpoint select, deploy Features.mqh 1093 indicator creation + per-bar input feature vector ChartUI.mqh 634 arrows, arrow persistence, status panel, cleanup Persistence.mqh 492 .stats/.cfg sidecars, CPU-inference validation, copy OnlineLearning.mqh 461 live continual learning, EMA shadow, OOS simulator Labels.mqh 309 ZigZag pivot labels, async label-cache prebuild AutoTune.mqh 275 genetic tuner (population, crossover, halving) Inference.mqh 235 softmax, prior calibration, class priors ExpertSignalAIBase.mqh 8216 -> 3131 (declaration + topology build only) This is a pure relocation - verified mechanically, not by eye: HEAD's file reconstructed from the eight partials plus the surviving remainder is byte-identical to HEAD, span for span (scratchpad verify_split.py). No declaration moved, no signature changed, no code rewritten, so behaviour is unchanged by construction. Compiles 0 errors, 0 warnings. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-29 00:42:45 -04:00
//+------------------------------------------------------------------+
//| Warrior_EA |
//| AnimateDread |
//| |
fix(autotune): MI scorer read an array nobody filled; add the permutation floor THE TUNER WAS A SILENT NO-OP. Every chart logged auto-tune complete - 17 candidate settings scored in ~139s, feature/label mutual information 0.0000 -> 0.0000 nats (no improvement) 0.0000 is not a weak result, it is a broken measurement: finite-sample MI is biased UPWARD, so even pure noise scores above zero. Cause: ScoreCurrentParamsByMI called BufferTempDataCompute(), which APPENDS the bar's features to TempData and never touches m_featureCache - only the caching wrapper BufferTempData() writes that array. It then read m_featureCache, which ReInitADIndicators had just invalidated. Every column came back constant, FeatureColumnMI returned 0 for all of them, and all 17 candidates tied at exactly zero. 139 s per chart to return the settings it started with. Now reads the values back out of TempData, where they actually land. And an exactly-zero best score is called out as a fault rather than reported as "no improvement", because that is what it is. ADDED: a PERMUTATION BASELINE, which is the diagnostic this project has been missing. MI's finite-sample bias is ~(bins-1)(classes-1)/(2n) nats - at these sample sizes the same order as any real edge in this domain - so a raw MI figure is uninterpretable on its own. Shuffling the labels destroys every genuine association while leaving sample size, binning and class proportions intact, so the score it produces IS this dataset's noise floor, measured rather than approximated. The log now reads feature/label information - X nats against a shuffled-label floor of Y and says outright whether the features carry usable information about the target. It needs no training, no topology and no convergence, so unlike every accuracy number in this codebase it cannot be confounded by an optimizer or an objective. If the score sits on the floor, no change of architecture can help - which is the question the last three days of zero-edge results have been circling. DEPLOY FLOOR: `dirPrecPct > chancePrecPct` passed anything above chance by any amount. At ~11,000 directional calls the standard error of the precision estimate is ~0.4pp, so that gate was accepting sub-one-sigma noise - the perceptron deployed at edge +0pp on 2026-08-01. Now requires EDGE_MIN_SIGMAS (2.0) standard errors above chance, computed from the actual call count, so the bar scales with the evidence instead of needing a hand-picked constant. Recorded with it, because it is why chance is the right reference at all: under a driftless random walk P(touch +k*ATR before -m*ATR) = m/(m+k), and the break-even win rate for a k:m reward:risk trade is ALSO m/(m+k). The label's own base rate IS the break-even rate, at every SL/TP setting. So "beats chance" and "is profitable" are the same test, and no choice of SL/TP can manufacture an edge - only prediction can. Both builds compile 0 errors / 0 warnings. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-01 13:05:50 -04:00
//| Filter-based indicator auto-tuner (mutual information scoring). |
refactor: split CExpertSignalAIBase implementation by responsibility ExpertSignalAIBase.mqh was 8216 lines: the class declaration followed by 87 method bodies covering training, labelling, feature extraction, persistence, chart drawing, online learning, the GA auto-tuner and inference, all in one file. Train() alone is 1492 lines; a change to arrow drawing meant scrolling past the era loop. Moved the bodies into Expert\AIBase\, included at the bottom of the original after the class declaration: Training.mqh 1607 era loop, plateau ladder, checkpoint select, deploy Features.mqh 1093 indicator creation + per-bar input feature vector ChartUI.mqh 634 arrows, arrow persistence, status panel, cleanup Persistence.mqh 492 .stats/.cfg sidecars, CPU-inference validation, copy OnlineLearning.mqh 461 live continual learning, EMA shadow, OOS simulator Labels.mqh 309 ZigZag pivot labels, async label-cache prebuild AutoTune.mqh 275 genetic tuner (population, crossover, halving) Inference.mqh 235 softmax, prior calibration, class priors ExpertSignalAIBase.mqh 8216 -> 3131 (declaration + topology build only) This is a pure relocation - verified mechanically, not by eye: HEAD's file reconstructed from the eight partials plus the surviving remainder is byte-identical to HEAD, span for span (scratchpad verify_split.py). No declaration moved, no signature changed, no code rewritten, so behaviour is unchanged by construction. Compiles 0 errors, 0 warnings. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-29 00:42:45 -04:00
//+------------------------------------------------------------------+
#ifndef WARRIOR_AIBASE_AUTOTUNE_MQH
#define WARRIOR_AIBASE_AUTOTUNE_MQH
refactor(comments): stdlib comment style across the remaining in-scope files Same pass as 0b06f8e, applied file by file: comment runs of 4+ lines compressed to their leading topic sentences, capped at 4 lines, whole sentences only. Warning sentences (NEVER / MUST / trap / would-have) survive the budget. Every file was checked the same way before committing: the list of non-comment lines is byte-identical to HEAD, and braces balance. No code was touched. Panel/, Enumerations/ and the already-terse System headers needed little or nothing - PooledGate, TradeChecks, BinomialStats and Random came through with no blocks over the threshold at all. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-22 00:25:52 -04:00
//--- ONCE-PER-CHART gate for the MI diagnostic suite on a multi-member ensemble. The first member
//--- to reach it runs it; the rest log one line and skip. Solo charts are untouched.
fix(ensemble): shared MI diagnostics + divided chunk budget - warm-up and panel responsiveness Two user-reported ensemble regressions, one cause each: - "getting ready is very long": every member ran the full MI diagnostic suite (headline MI, positive control, alignment, lag profile, geometry scan + winner test - ~200 permuted draws per line) on IDENTICAL features and labels, reporting the same numbers four times. First member runs it, the rest adopt with one log line. Documented caveat: if the geometry scan ever ADOPTS a winner under its gate (it never has), the adoption becomes donor-only and the gate must be revisited. - "panel not responsive": four members chunks queue back-to-back on the one chart thread - 4 x 120ms = 480ms worst-case click latency, the exact regime the 200ms note in Training.mqh already documents as broken. Ensemble members now use a 30ms chunk budget, restoring solo UI latency at slightly higher dispatch overhead. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-15 17:37:10 -04:00
bool g_ensembleChartMiReportDone = false;
feat(ensemble): per-NN inputs replace the preset selector - the meta head becomes the vote's gate User design (2026-08-19): 'remove the enum menu that selects neural networks... individual inputs for every NN just like classic signals... the META NN should be integrated into the voting decision pipeline when enabled... as a bonus meta labelling is applied to enabled NNs.' - AI_CHOICE is GONE (tombstoned per the stale-.set doctrine). Use_MLP/Use_CONV/Use_LSTM/ Use_CONVLSTM are ordinary bools like the classic votes; the ensemble arithmetic adapts to any subset because the consensus divisor is the enabled capable weight. Two or more enabled = ensemble (|ENS1 token + joint gate, exactly the old AI_HYBRID fingerprints, so existing weight files keep loading); one = the old solo preset; none = classic-only. - Use_MetaLabeling un-couples META from the direction NNs (the old selector made them mutually exclusive). S3 ships: CSignalMETA::LiveMetaGate scores each vote-cleared entry (shared window at bar 1 + proposal descriptor: side, net vote, live geometry, spread/ATR; pattern one-hot ZEROED - ranking, not calibrated probability, documented in the body) and vetoes below the cost-adjusted break-even. Entries only; fail-open everywhere, loudly. - COEXISTENCE HAZARDS closed: VoteCapableWeight()=0 and ProspectiveVote()=false for the meta target - solo-only until today, a trained META would otherwise sit in the consensus divisor as a permanent abstainer and shrink every vote by its module weight. - CERTIFIED == TRADED: the ensemble era verdict replays the identical veto through the same g_warriorMetaGate pointer over its OOS fired bars (bar re-resolved from the row's own time; fail-open counted as fires and reported: 'metaGate: N approved, M vetoed, K unscored'). The overlay deliberately does NOT replay it (veto-filter-in-replay class, calendar-cliff precedent) - documented at the sweep site. Solo charts' own gate does not model the veto - the standing solo-gate caveat, documented at the input. - DB continuity: the pattern/journal DB fingerprint's first slot was (int)AIType; DbLegacyAiSlot() maps every legacy-expressible config to its OLD value (new 2-3 member subsets get 100+bitmask, outside the legacy range) so no existing database re-keys. filterID becomes the enabled roster via one EnabledNNSummary(). - HUD: the meta line shows the gate (armed/(trn), last P vs BE, ok/veto tally); the armed/disarmed announcement fires on state change via one latch (MetaGateArmedNow), not only when an entry happens to be proposed. NOT COMPILED - user compiles in MetaEditor. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-19 13:01:02 -04:00
//--- ONCE-PER-CHART share of the indicator auto-tune SWEEP on a multi-member ensemble, same doctrine as
fix(ensemble): responsive panel + synchronized eras + combined-vote accuracy Four user-reported/requested items, one root cause chain: 1) DEAD CONTROL PANEL in AI_HYBRID mode. All members posted custom event id 1 and handled id 1001, and CExpertCustom broadcasts every chart event to every filter - so each posted event ran a train chunk in ALL N members (N*N chunks per round) and the chart thread never idled long enough to deliver clicks/drags. profiling.csv: 99.45% of time in OnChartEventHandler. Fix: per-instance study-event ids (STUDY_EVENT_ID_BASE + construction order, offset above the Controls library's ON_* codes - id 1 was also ON_DBL_CLICK, so panel double-clicks fired training chunks). ArmStudyEvent() is the single post site; lost-event watchdog replaces the accidental sibling-clears-my-flag rescue. 2) WARM-UP DUPLICATION. The auto-tune sweep is deterministic over identical features/labels, and it ends in the full MI diagnostic suite, which the MI-share gate never intercepted on the sweep path - four members ran four identical ~36s sweep+report blocks. First member publishes outcome (g_ensembleChartTuneDone/Installed/Settings); the rest apply it and skip both. 3) DEINIT STRANDED PANEL+ARROWS (user repro 18:52). Root cause from the log: the 4,500ms budget runs from MetaTrader's stop REQUEST - a heavy autosave in flight ate it, OnDeinit got ~430ms and died in the first member's arrow persist ("Abnormal termination" 432ms in). Fix: early visible-UI sweep (native prefix deletes for status/panel/dialog) right after ClearStatusLabel, and a fast path for still-training models - their arrows are re-rendered every era, so they get one bulk purge instead of scan+atomic-write in the death window. 4) ENSEMBLE FEATURES (user requests): era BARRIER - members advance era by era together; a member ahead of the slowest still-training member declines Train() calls and its chunk budget is donated (TRAIN_TIME_BUDGET_MS = 120/activeTrainers, UI headroom constant). COMBINED-VOTE OOS SCORE - each member's pass-3 scan contributes its adjusted per-bar decision (0.0 on abstain) to a shared row buffer; the last member to finish the era scores the averaged vote vs the mirrored Min_Vote_Open against the same target-before-stop outcomes members grade themselves on, publishing an "Ensemble vote" line on the aggregated panel. Member headlines now carry their lifetime win rate with break-even. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-16 19:06:04 -04:00
//--- the MI gate above: the sweep scores candidate indicator settings by feature/label MI, and every
//--- ensemble member holds identical indicators, identical cached features and identical labels, so all
//--- N sweeps are the same deterministic calculation (verified 2026-08-16 on SP500 H4: four members,
//--- byte-identical scores, spans and selection p). Worse, the sweep ends in the full MI diagnostic
//--- suite (ReportFeatureLabelInformation at its tail), which the MI gate above never intercepts on the
//--- sweep path - so each duplicate sweep also duplicated the ~200-draw permutation nulls, the slowest
//--- single block of "getting ready". The first member runs the sweep and publishes its outcome here;
//--- the rest apply the outcome (install the winner, or keep the configured settings the sweep restored)
//--- and skip both the sweep and the report. Same caveat as the MI gate: any winner ADOPTION is made by
//--- the donor and applied to every member via the flattened settings below, which is the consistent
//--- choice - members training on divergent feature vectors would not be an ensemble. Solo charts are
//--- untouched.
bool g_ensembleChartTuneDone = false;
bool g_ensembleChartTuneInstalled = false; // did the donor's sweep clear the family-wise gate and install?
double g_ensembleChartTuneSettings[]; // CADIndicatorTuner::Flatten() of the donor's final settings
refactor(comments): stdlib comment style across the remaining in-scope files Same pass as 0b06f8e, applied file by file: comment runs of 4+ lines compressed to their leading topic sentences, capped at 4 lines, whole sentences only. Warning sentences (NEVER / MUST / trap / would-have) survive the budget. Every file was checked the same way before committing: the list of non-comment lines is byte-identical to HEAD, and braces balance. No code was touched. Panel/, Enumerations/ and the already-terse System headers needed little or nothing - PooledGate, TradeChecks, BinomialStats and Random came through with no blocks over the threshold at all. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-22 00:25:52 -04:00
//--- THE SAME DOCTRINE, APPLIED TO THE BARRIER GEOMETRY - and it was missing, which broke the
//--- ensemble. That was harmless while the scan only PRINTED.
fix(geometry): the ensemble was training on TWO DIFFERENT TARGETS - propagate the adopted barrier MEASURED 2026-08-17 19:06 on USDJPY, in the fresh run: 19:06:38 LSTM adopting barrier geometry 2:10 ... geometry authority 19:06:40 LSTM triple-barrier labels - stop 2.00 target 10.00, horizon 256 19:06:44 PAI / CONV / HYB break-even 33.3%, mean label lifespan 19.2 bars 19:06:45 LSTM break-even 16.7%, mean label lifespan 81.4 bars One chart, four members, two targets. A "Buy" from LSTM meant "10 ATR before a 2 ATR stop within 256 bars"; a "Buy" from PAI meant "3.21 before 1.61 within 64". The orchestrator averages those votes and the joint gate certifies the average as though they answered one question. And g_DerivedSlAtrMult - which places the LIVE order - is a single global, so the stop actually sent was whichever member wrote last: the same last-writer-wins class of bug as the live-exit confidence. CAUSE, and it is mine. The geometry scan sits at the end of the MI chain, and that chain runs ONCE PER CHART (g_ensembleChartMiReportDone) - whichever member reaches it first measures and the rest skip. Harmless while the scan only PRINTED; 62a719f made it authoritative and turned a skipped report into a skipped DECISION. The indicator tuner already had this doctrine (g_ensembleChartTuneSettings); the geometry had no equivalent. - g_ensembleChartGeomAdopted/Sl/Tp/SlMode/TpMode: the donor publishes its pairing, the siblings adopt it in the MI-skip branch. Ordering is safe by construction - MQL5 is single-threaded per chart and the donor sets g_ensembleChartMiReportDone only after the chain (and so the adoption) returns, so any member taking the skip branch does so strictly afterwards. - ApplyAdoptedGeometry(): the eleven side effects an adopted pairing must carry - derived pair, legacy mode ints, g_Derived* live globals, .cfg rewrite, label cache invalidation, horizon unlatch - in ONE function, because there are now two callers and duplicating them is how the two paths drift. - Guarded on era 0 for the donor's own reason: relabelling a partly trained net moves the target out from under weights already fitted to the old one. STILL OPEN: dead MA handles were not eliminated by cb30360. They now appear at a different site (SP500 19:06:35, during "label prebuild", and on PAI - the member that RAN the sweep), so there is a second handle-churn path I have not found. Recovery works and the sharing diagnosis stands; the trigger is not only the tuner's adopt branch. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-17 19:35:45 -04:00
bool g_ensembleChartGeomAdopted = false; // did the donor's scan adopt a pairing the siblings must take?
double g_ensembleChartGeomSl = 0.0; // the DERIVED pair (the one authority - see BarrierMultiples)
double g_ensembleChartGeomTp = 0.0;
int g_ensembleChartGeomSlMode = 0; // legacy mode ints, kept in step for the fallback/fingerprint
int g_ensembleChartGeomTpMode = 0;
research: export the feature matrix and a raw OHLCV grid for offline work The bottleneck on this project has never been the modelling - it is that every hypothesis costs a compile, a deploy, an attach and a log read, and answers exactly one question. Days have gone into questions that are seconds of arithmetic once the data is in hand. Adds a RESEARCH-ONLY build, gated behind WARRIOR_EXPORT_FEATURES and never compiled into a shipped binary, which writes two things to Common\Files\Warrior_EA\Research\ and then does nothing at all: <symbol>_<tf>_features.csv - one row per bar: index, time, OHLC, ATR, and the m_neuronsCount feature values. Exactly what the network sees. The raw bars ride along on purpose: with OHLC and ATR offline, every barrier geometry, horizon and in-trade target is recomputable without MetaTrader in the loop. <symbol>_<tf>_rates.csv - raw OHLCV across a grid of 8 symbols x 5 timeframes. The 26 engineered features only exist for the attached chart (indicator handles bind to PERIOD_CURRENT); raw rates do not, so ONE attach yields the whole research grid. The bar time also makes session/hour/day-of-week derivable - the only inputs in play that are not a transform of the same OHLCV series. Safety, because this binary gets attached to a chart on a LIVE ACCOUNT to reach real history: - OnTick returns immediately, so Expert.OnTick() - the entire trading path - is unreachable regardless of the AlgoTrading toggle, the signal state or the inputs. Structurally incapable of sending an order, not merely unlikely to. - No config lock. It never trains and never saves a model, so it has nothing to protect against a concurrent chart - and taking the lock would make it refuse to start exactly when the config it wants to read is already open, which is when it is most useful. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-01 15:49:57 -04:00
//+------------------------------------------------------------------+
refactor(build): retire the WARRIOR_EXPORT_FEATURES compile flag Last surviving compile-time feature switch in the codebase - the same pattern already killed for the MARKET build and DirectML tier (02766b5): one build, configured at runtime like every other module (inputs + getters/setters, set in ConfigureAISignal during OnInit), not a second code path that only existed if someone remembered to define a macro before compiling. Replaced with `input bool ExportFeaturesOnly = false` (Variables/Inputs.mqh) and a plain m_exportFeaturesOnly member + setter, matching AutoTuneIndicators' exact shape. Four call sites converted from #ifdef to a runtime read of the same variable: - Warrior_EA.mq5 OnTick() - reads the input directly (this check has to stand before any per-signal object exists) - Topology.mqh's config-lock skip and ExportFeatureMatrix() call - read m_exportFeaturesOnly, now set by ConfigureAISignal before InitIndicators() runs (same init-order guarantee AutoTuneIndicators already relies on) - ExportFeatureMatrix()/ExportRawRates() declarations - always compiled now, called conditionally instead of not existing as symbols No change to what the flag does when off (the state of every build that exists today, since the macro was never defined anywhere in-repo) or when on; only how it's set. Verified: WARRIOR_EXPORT_FEATURES fully gone from every #ifdef/#endif in the tree; brace and ifdef/endif counts balance in every touched file; ConfigureAISignal runs before StepInitIndicators in OnInit's linear init chain, so the flag reaches InitNeuralNetwork() in time. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-08-23 19:14:55 -04:00
//| RESEARCH ONLY, called only when m_exportFeaturesOnly is set - see |
//| the declaration comment. |
research: export the feature matrix and a raw OHLCV grid for offline work The bottleneck on this project has never been the modelling - it is that every hypothesis costs a compile, a deploy, an attach and a log read, and answers exactly one question. Days have gone into questions that are seconds of arithmetic once the data is in hand. Adds a RESEARCH-ONLY build, gated behind WARRIOR_EXPORT_FEATURES and never compiled into a shipped binary, which writes two things to Common\Files\Warrior_EA\Research\ and then does nothing at all: <symbol>_<tf>_features.csv - one row per bar: index, time, OHLC, ATR, and the m_neuronsCount feature values. Exactly what the network sees. The raw bars ride along on purpose: with OHLC and ATR offline, every barrier geometry, horizon and in-trade target is recomputable without MetaTrader in the loop. <symbol>_<tf>_rates.csv - raw OHLCV across a grid of 8 symbols x 5 timeframes. The 26 engineered features only exist for the attached chart (indicator handles bind to PERIOD_CURRENT); raw rates do not, so ONE attach yields the whole research grid. The bar time also makes session/hour/day-of-week derivable - the only inputs in play that are not a transform of the same OHLCV series. Safety, because this binary gets attached to a chart on a LIVE ACCOUNT to reach real history: - OnTick returns immediately, so Expert.OnTick() - the entire trading path - is unreachable regardless of the AlgoTrading toggle, the signal state or the inputs. Structurally incapable of sending an order, not merely unlikely to. - No config lock. It never trains and never saves a model, so it has nothing to protect against a concurrent chart - and taking the lock would make it refuse to start exactly when the config it wants to read is already open, which is when it is most useful. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-01 15:49:57 -04:00
//+------------------------------------------------------------------+
void CExpertSignalAIBase::ExportFeatureMatrix(void)
{
if(MQLInfoInteger(MQL_OPTIMIZATION))
return;
int barsNow = Bars(m_symbol.Name(), PERIOD_CURRENT);
fix(depth): route EVERY ResizeBuffers call site through one indicator-depth gate 1dda479 clamped the training sweep. It left five other paths asking the indicators for a depth they cannot serve, and on a live account the quiet ones are worse than the stall was - a stalled chart is visible, a chart trading on a degraded feature window is not. ServableBars(want, context) is now the single gate, and all six go through it: training sweep clamp, floored at TRAIN_MIN_CLAMPED_BARS (below that a small positive BarsCalculated is warm-up, which m_coldSweepTick owns) label prebuild clamp - labels come from price/ADZigZag and would survive a capped MA, but ResizeBuffers sizes EVERY buffer and a failed CopyBuffer leaves m_MA EMPTY for the next reader, so this path could silently re-break the block Train()'s clamp just fixed live inference HOLD. Below `need` the swing block takes its degraded path and inference runs on a different feature distribution than the model was fitted on. This EA sizes real positions off that output, so no signal beats a mismatched one online learning HOLD, same reason and worse - this path WRITES to a live trading model, so a mismatched (features, label) pair is not a wrong arrow, it is a wrong weight update that compounds every bar chart rescan clamp - SIGNAL_RESCAN_LOOKBACK_BARS is 5000 and MT5's smallest "Max bars in chart" is also 5000, so this one is genuinely reachable; uncapped it repaints the window all-Neutral research export clamp before the emptiness test, so a capped symbol exports the depth it has rather than writing a CSV with a dead feature block - an artefact that looks complete and is silently wrong Both HOLDs are insurance, not expected states: `need` tops out near 1,152 bars (16 + 750 + 384 + 2) against a 5,000 floor on the terminal setting. They exist so the failure mode is unreachable rather than merely unlikely. Not changed: a genuinely SHORT price history still takes the old degraded path at every site. That is pre-existing behaviour and narrowing it would mute charts that trade today, so it stays a separate decision rather than a side effect of this fix. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-17 13:33:16 -04:00
//--- Clamp BEFORE the emptiness test, so a fully-capped symbol reports the depth it can actually
refactor(comments): stdlib comment style across the remaining in-scope files Same pass as 0b06f8e, applied file by file: comment runs of 4+ lines compressed to their leading topic sentences, capped at 4 lines, whole sentences only. Warning sentences (NEVER / MUST / trap / would-have) survive the budget. Every file was checked the same way before committing: the list of non-comment lines is byte-identical to HEAD, and braces balance. No code was touched. Panel/, Enumerations/ and the already-terse System headers needed little or nothing - PooledGate, TradeChecks, BinomialStats and Random came through with no blocks over the threshold at all. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-22 00:25:52 -04:00
//--- export rather than the price-series depth it cannot.
fix(depth): route EVERY ResizeBuffers call site through one indicator-depth gate 1dda479 clamped the training sweep. It left five other paths asking the indicators for a depth they cannot serve, and on a live account the quiet ones are worse than the stall was - a stalled chart is visible, a chart trading on a degraded feature window is not. ServableBars(want, context) is now the single gate, and all six go through it: training sweep clamp, floored at TRAIN_MIN_CLAMPED_BARS (below that a small positive BarsCalculated is warm-up, which m_coldSweepTick owns) label prebuild clamp - labels come from price/ADZigZag and would survive a capped MA, but ResizeBuffers sizes EVERY buffer and a failed CopyBuffer leaves m_MA EMPTY for the next reader, so this path could silently re-break the block Train()'s clamp just fixed live inference HOLD. Below `need` the swing block takes its degraded path and inference runs on a different feature distribution than the model was fitted on. This EA sizes real positions off that output, so no signal beats a mismatched one online learning HOLD, same reason and worse - this path WRITES to a live trading model, so a mismatched (features, label) pair is not a wrong arrow, it is a wrong weight update that compounds every bar chart rescan clamp - SIGNAL_RESCAN_LOOKBACK_BARS is 5000 and MT5's smallest "Max bars in chart" is also 5000, so this one is genuinely reachable; uncapped it repaints the window all-Neutral research export clamp before the emptiness test, so a capped symbol exports the depth it has rather than writing a CSV with a dead feature block - an artefact that looks complete and is silently wrong Both HOLDs are insurance, not expected states: `need` tops out near 1,152 bars (16 + 750 + 384 + 2) against a 5,000 floor on the terminal setting. They exist so the failure mode is unreachable rather than merely unlikely. Not changed: a genuinely SHORT price history still takes the old degraded path at every site. That is pre-existing behaviour and narrowing it would mute charts that trade today, so it stays a separate decision rather than a side effect of this fix. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-17 13:33:16 -04:00
barsNow = ServableBars(barsNow, "feature export");
research: export the feature matrix and a raw OHLCV grid for offline work The bottleneck on this project has never been the modelling - it is that every hypothesis costs a compile, a deploy, an attach and a log read, and answers exactly one question. Days have gone into questions that are seconds of arithmetic once the data is in hand. Adds a RESEARCH-ONLY build, gated behind WARRIOR_EXPORT_FEATURES and never compiled into a shipped binary, which writes two things to Common\Files\Warrior_EA\Research\ and then does nothing at all: <symbol>_<tf>_features.csv - one row per bar: index, time, OHLC, ATR, and the m_neuronsCount feature values. Exactly what the network sees. The raw bars ride along on purpose: with OHLC and ATR offline, every barrier geometry, horizon and in-trade target is recomputable without MetaTrader in the loop. <symbol>_<tf>_rates.csv - raw OHLCV across a grid of 8 symbols x 5 timeframes. The 26 engineered features only exist for the attached chart (indicator handles bind to PERIOD_CURRENT); raw rates do not, so ONE attach yields the whole research grid. The bar time also makes session/hour/day-of-week derivable - the only inputs in play that are not a transform of the same OHLCV series. Safety, because this binary gets attached to a chart on a LIVE ACCOUNT to reach real history: - OnTick returns immediately, so Expert.OnTick() - the entire trading path - is unreachable regardless of the AlgoTrading toggle, the signal state or the inputs. Structurally incapable of sending an order, not merely unlikely to. - No config lock. It never trains and never saves a model, so it has nothing to protect against a concurrent chart - and taking the lock would make it refuse to start exactly when the config it wants to read is already open, which is when it is most useful. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-01 15:49:57 -04:00
if(barsNow <= m_historyBars + 2)
{
Print(ID + ": EXPORT - only " + IntegerToString(barsNow) + " bars available, nothing to write");
return;
}
if(!ResizeBuffers(barsNow) || !RefreshData())
{
Print(ID + ": EXPORT - buffers not ready (" + IntegerToString(barsNow) + " bars), aborting");
return;
}
EnsureBarCachesCapacity(barsNow);
EnsureBarrierHorizon(barsNow);
string dir = eaName + "\\Research\\";
string fn = dir + m_symbol.Name() + "_" + IntegerToString(_Period) + "_features.csv";
refactor(stdlib): adopt Math\Stat for the deploy gate's normal tail; retire the b1/b2/lr/momentum macros The gate's NormalUpperTail was a hand-rolled Abramowitz & Stegun 26.2.17 approximation. Its own comment gave the reason - "drags a chain of headers behind it" - and that turned out to be one file: Math\Stat\Normal.mqh includes only Math.mqh, which includes nothing. Swapped for Cody's rational approximation in the library (~18 significant digits vs |error| < 7.5e-8). No past verdict changes: at the z the gate operates on, the difference is orders of magnitude below DEPLOY_FAMILY_WISE_ALPHA. Adopting it needed the four bare macros in AI\Network.mqh gone first. "#define b1 AdamBeta1" collides with an identifier in Math.mqh, so the include would have macro-expanded the library's own local and failed to compile - the same landmine that made the original author rename the approximation's coefficients to ntB1..ntB5 rather than use the reference's b1..b5. lr, b2 and momentum are the same class of hazard: single-token global macros in a 52k-line codebase. All four now resolve to the input names they always aliased, which is a pure textual identity - verified zero bare occurrences remain. Also: - SelectionSort over the buffered signals was O(n^2) with an O(n^2) count of StructToTime calls, because the comparison rebuilt both datetimes from the six int date fields every time. Now materialises the keys once and does an insertion sort; ArraySort cannot permute a struct array. IsEarlier goes with it, MakeDateTime becomes SignalTime. - Seven FileOpen sites lacked FILE_SHARE_READ|FILE_SHARE_WRITE, including AtomicWriteBegin, which stages every model save. All 43 sites now carry them - an exclusive open fails outright when another process holds the path, which here has meant a silently skipped save. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-19 19:31:36 -04:00
int h = FileOpen(fn, FILE_COMMON | FILE_WRITE | FILE_CSV | FILE_ANSI | FILE_SHARE_READ | FILE_SHARE_WRITE, ',');
research: export the feature matrix and a raw OHLCV grid for offline work The bottleneck on this project has never been the modelling - it is that every hypothesis costs a compile, a deploy, an attach and a log read, and answers exactly one question. Days have gone into questions that are seconds of arithmetic once the data is in hand. Adds a RESEARCH-ONLY build, gated behind WARRIOR_EXPORT_FEATURES and never compiled into a shipped binary, which writes two things to Common\Files\Warrior_EA\Research\ and then does nothing at all: <symbol>_<tf>_features.csv - one row per bar: index, time, OHLC, ATR, and the m_neuronsCount feature values. Exactly what the network sees. The raw bars ride along on purpose: with OHLC and ATR offline, every barrier geometry, horizon and in-trade target is recomputable without MetaTrader in the loop. <symbol>_<tf>_rates.csv - raw OHLCV across a grid of 8 symbols x 5 timeframes. The 26 engineered features only exist for the attached chart (indicator handles bind to PERIOD_CURRENT); raw rates do not, so ONE attach yields the whole research grid. The bar time also makes session/hour/day-of-week derivable - the only inputs in play that are not a transform of the same OHLCV series. Safety, because this binary gets attached to a chart on a LIVE ACCOUNT to reach real history: - OnTick returns immediately, so Expert.OnTick() - the entire trading path - is unreachable regardless of the AlgoTrading toggle, the signal state or the inputs. Structurally incapable of sending an order, not merely unlikely to. - No config lock. It never trains and never saves a model, so it has nothing to protect against a concurrent chart - and taking the lock would make it refuse to start exactly when the config it wants to read is already open, which is when it is most useful. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-01 15:49:57 -04:00
if(h == INVALID_HANDLE)
{
Print(ID + ": EXPORT - cannot open " + fn + ", error " + IntegerToString(GetLastError()));
return;
}
string header = "idx,time,open,high,low,close,atr";
for(int f = 0; f < m_neuronsCount; f++)
header += ",f" + IntegerToString(f);
FileWrite(h, header);
//--- Oldest first. The loop walks DOWN the series index, which is forward in time (higher index =
//--- older), so the file reads chronologically and Python can treat row order as time order.
int written = 0, skipped = 0;
uint t0 = GetTickCount();
for(int i = barsNow - 1; i >= 0; i--)
{
TempData.Clear();
if(!BufferTempData(i) || TempData.Total() < m_neuronsCount)
{
skipped++;
continue;
}
double atr = m_ATR.Main(i);
string row = IntegerToString(i) + "," + IntegerToString((long)m_Time.GetData(i)) + "," +
DoubleToString(m_Open.GetData(i), _Digits) + "," +
DoubleToString(m_High.GetData(i), _Digits) + "," +
DoubleToString(m_Low.GetData(i), _Digits) + "," +
DoubleToString(m_Close.GetData(i), _Digits) + "," +
DoubleToString(MathIsValidNumber(atr) ? atr : 0.0, _Digits);
for(int f = 0; f < m_neuronsCount; f++)
row += "," + DoubleToString(TempData.At(f), 8);
FileWrite(h, row);
written++;
}
TempData.Clear();
FileClose(h);
Print(ID + StringFormat(": EXPORT COMPLETE - %d rows x %d features -> Common\\Files\\%s "
"(%d bars skipped for missing features, %.1fs, horizon %d, spread %d points)",
written, m_neuronsCount, fn, skipped, (GetTickCount() - t0) / 1000.0,
m_barrierHorizonBars, (int)m_symbol.Spread()));
ExportRawRates();
}
//+------------------------------------------------------------------+
refactor(comments): box headers to stdlib length The //| box blocks were excluded from 0b06f8e and 5efdb48 and were what remained: 160 of them ran to 10+ lines, the longest to 88. Compressed to their leading topic sentences - 5 lines for a function header, 8 for a file header - keeping the box format and the standard MQL5 name/author lines verbatim. Verified at the BYTE level this time, across every in-scope file: the list of non-comment lines is byte-identical to HEAD and braces balance. The first check compared a locale-decoded 'git show' against a UTF-8 read and flagged 25 files that had not changed at all - every BOM and every non-ASCII line mismatched. 47,696 -> 40,665 lines in scope; comment share 38% -> 26%. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-22 00:30:14 -04:00
//| RESEARCH BUILD ONLY. Raw OHLCV for a GRID of symbols/timeframes, |
//| not just this chart's. |
research: export the feature matrix and a raw OHLCV grid for offline work The bottleneck on this project has never been the modelling - it is that every hypothesis costs a compile, a deploy, an attach and a log read, and answers exactly one question. Days have gone into questions that are seconds of arithmetic once the data is in hand. Adds a RESEARCH-ONLY build, gated behind WARRIOR_EXPORT_FEATURES and never compiled into a shipped binary, which writes two things to Common\Files\Warrior_EA\Research\ and then does nothing at all: <symbol>_<tf>_features.csv - one row per bar: index, time, OHLC, ATR, and the m_neuronsCount feature values. Exactly what the network sees. The raw bars ride along on purpose: with OHLC and ATR offline, every barrier geometry, horizon and in-trade target is recomputable without MetaTrader in the loop. <symbol>_<tf>_rates.csv - raw OHLCV across a grid of 8 symbols x 5 timeframes. The 26 engineered features only exist for the attached chart (indicator handles bind to PERIOD_CURRENT); raw rates do not, so ONE attach yields the whole research grid. The bar time also makes session/hour/day-of-week derivable - the only inputs in play that are not a transform of the same OHLCV series. Safety, because this binary gets attached to a chart on a LIVE ACCOUNT to reach real history: - OnTick returns immediately, so Expert.OnTick() - the entire trading path - is unreachable regardless of the AlgoTrading toggle, the signal state or the inputs. Structurally incapable of sending an order, not merely unlikely to. - No config lock. It never trains and never saves a model, so it has nothing to protect against a concurrent chart - and taking the lock would make it refuse to start exactly when the config it wants to read is already open, which is when it is most useful. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-01 15:49:57 -04:00
//+------------------------------------------------------------------+
void CExpertSignalAIBase::ExportRawRates(void)
{
string symbols[] = { "SP500", "USDJPY", "XAUUSD", "EURUSD", "GBPUSD", "US30", "NAS100", "BTCUSD" };
ENUM_TIMEFRAMES tfs[] = { PERIOD_M5, PERIOD_M15, PERIOD_H1, PERIOD_H4, PERIOD_D1 };
string dir = eaName + "\\Research\\";
int cells = 0, rowsTotal = 0;
for(int s = 0; s < ArraySize(symbols); s++)
{
//--- Skip silently rather than warn: the grid is deliberately broader than any one broker's symbol
//--- list, so an absent instrument is expected, not an error.
if(!SymbolSelect(symbols[s], true))
continue;
for(int p = 0; p < ArraySize(tfs); p++)
{
MqlRates r[];
ArraySetAsSeries(r, false); // oldest first, so file order is time order
int got = CopyRates(symbols[s], tfs[p], 0, 200000, r);
if(got <= 100)
continue;
string fn = dir + symbols[s] + "_" + IntegerToString((int)tfs[p]) + "_rates.csv";
refactor(stdlib): adopt Math\Stat for the deploy gate's normal tail; retire the b1/b2/lr/momentum macros The gate's NormalUpperTail was a hand-rolled Abramowitz & Stegun 26.2.17 approximation. Its own comment gave the reason - "drags a chain of headers behind it" - and that turned out to be one file: Math\Stat\Normal.mqh includes only Math.mqh, which includes nothing. Swapped for Cody's rational approximation in the library (~18 significant digits vs |error| < 7.5e-8). No past verdict changes: at the z the gate operates on, the difference is orders of magnitude below DEPLOY_FAMILY_WISE_ALPHA. Adopting it needed the four bare macros in AI\Network.mqh gone first. "#define b1 AdamBeta1" collides with an identifier in Math.mqh, so the include would have macro-expanded the library's own local and failed to compile - the same landmine that made the original author rename the approximation's coefficients to ntB1..ntB5 rather than use the reference's b1..b5. lr, b2 and momentum are the same class of hazard: single-token global macros in a 52k-line codebase. All four now resolve to the input names they always aliased, which is a pure textual identity - verified zero bare occurrences remain. Also: - SelectionSort over the buffered signals was O(n^2) with an O(n^2) count of StructToTime calls, because the comparison rebuilt both datetimes from the six int date fields every time. Now materialises the keys once and does an insertion sort; ArraySort cannot permute a struct array. IsEarlier goes with it, MakeDateTime becomes SignalTime. - Seven FileOpen sites lacked FILE_SHARE_READ|FILE_SHARE_WRITE, including AtomicWriteBegin, which stages every model save. All 43 sites now carry them - an exclusive open fails outright when another process holds the path, which here has meant a silently skipped save. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-19 19:31:36 -04:00
int h = FileOpen(fn, FILE_COMMON | FILE_WRITE | FILE_CSV | FILE_ANSI | FILE_SHARE_READ | FILE_SHARE_WRITE, ',');
research: export the feature matrix and a raw OHLCV grid for offline work The bottleneck on this project has never been the modelling - it is that every hypothesis costs a compile, a deploy, an attach and a log read, and answers exactly one question. Days have gone into questions that are seconds of arithmetic once the data is in hand. Adds a RESEARCH-ONLY build, gated behind WARRIOR_EXPORT_FEATURES and never compiled into a shipped binary, which writes two things to Common\Files\Warrior_EA\Research\ and then does nothing at all: <symbol>_<tf>_features.csv - one row per bar: index, time, OHLC, ATR, and the m_neuronsCount feature values. Exactly what the network sees. The raw bars ride along on purpose: with OHLC and ATR offline, every barrier geometry, horizon and in-trade target is recomputable without MetaTrader in the loop. <symbol>_<tf>_rates.csv - raw OHLCV across a grid of 8 symbols x 5 timeframes. The 26 engineered features only exist for the attached chart (indicator handles bind to PERIOD_CURRENT); raw rates do not, so ONE attach yields the whole research grid. The bar time also makes session/hour/day-of-week derivable - the only inputs in play that are not a transform of the same OHLCV series. Safety, because this binary gets attached to a chart on a LIVE ACCOUNT to reach real history: - OnTick returns immediately, so Expert.OnTick() - the entire trading path - is unreachable regardless of the AlgoTrading toggle, the signal state or the inputs. Structurally incapable of sending an order, not merely unlikely to. - No config lock. It never trains and never saves a model, so it has nothing to protect against a concurrent chart - and taking the lock would make it refuse to start exactly when the config it wants to read is already open, which is when it is most useful. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-01 15:49:57 -04:00
if(h == INVALID_HANDLE)
continue;
int dg = (int)SymbolInfoInteger(symbols[s], SYMBOL_DIGITS);
FileWrite(h, "time,open,high,low,close,tickvol,spread");
for(int i = 0; i < got; i++)
FileWrite(h, IntegerToString((long)r[i].time) + "," +
DoubleToString(r[i].open, dg) + "," + DoubleToString(r[i].high, dg) + "," +
DoubleToString(r[i].low, dg) + "," + DoubleToString(r[i].close, dg) + "," +
IntegerToString((long)r[i].tick_volume) + "," + IntegerToString(r[i].spread));
FileClose(h);
cells++;
rowsTotal += got;
Print(ID + StringFormat(": EXPORT rates - %s %s: %d bars", symbols[s],
EnumToString(tfs[p]), got));
}
}
Print(ID + StringFormat(": EXPORT RATES COMPLETE - %d cells, %d bars total, under Common\\Files\\%s",
cells, rowsTotal, dir));
}
refactor(comments): stdlib comment style across the remaining in-scope files Same pass as 0b06f8e, applied file by file: comment runs of 4+ lines compressed to their leading topic sentences, capped at 4 lines, whole sentences only. Warning sentences (NEVER / MUST / trap / would-have) survive the budget. Every file was checked the same way before committing: the list of non-comment lines is byte-identical to HEAD, and braces balance. No code was touched. Panel/, Enumerations/ and the already-terse System headers needed little or nothing - PooledGate, TradeChecks, BinomialStats and Random came through with no blocks over the threshold at all. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-22 00:25:52 -04:00
//--- The genetic + successive-halving helpers that used to live here (GaRungEras, GaExtract,
//--- GaStore, GaMutate, GaRandomCandidate, GaBlockCrossover, GaSortAliveByScoreDesc,
//--- GaBreedNextGeneration) were deleted on 2026-08-01 together with the search they served.
refactor: split CExpertSignalAIBase implementation by responsibility ExpertSignalAIBase.mqh was 8216 lines: the class declaration followed by 87 method bodies covering training, labelling, feature extraction, persistence, chart drawing, online learning, the GA auto-tuner and inference, all in one file. Train() alone is 1492 lines; a change to arrow drawing meant scrolling past the era loop. Moved the bodies into Expert\AIBase\, included at the bottom of the original after the class declaration: Training.mqh 1607 era loop, plateau ladder, checkpoint select, deploy Features.mqh 1093 indicator creation + per-bar input feature vector ChartUI.mqh 634 arrows, arrow persistence, status panel, cleanup Persistence.mqh 492 .stats/.cfg sidecars, CPU-inference validation, copy OnlineLearning.mqh 461 live continual learning, EMA shadow, OOS simulator Labels.mqh 309 ZigZag pivot labels, async label-cache prebuild AutoTune.mqh 275 genetic tuner (population, crossover, halving) Inference.mqh 235 softmax, prior calibration, class priors ExpertSignalAIBase.mqh 8216 -> 3131 (declaration + topology build only) This is a pure relocation - verified mechanically, not by eye: HEAD's file reconstructed from the eight partials plus the surviving remainder is byte-identical to HEAD, span for span (scratchpad verify_split.py). No declaration moved, no signature changed, no code rewritten, so behaviour is unchanged by construction. Compiles 0 errors, 0 warnings. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-29 00:42:45 -04:00
//+------------------------------------------------------------------+
refactor(comments): box headers to stdlib length The //| box blocks were excluded from 0b06f8e and 5efdb48 and were what remained: 160 of them ran to 10+ lines, the longest to 88. Compressed to their leading topic sentences - 5 lines for a function header, 8 for a file header - keeping the box format and the standard MQL5 name/author lines verbatim. Verified at the BYTE level this time, across every in-scope file: the list of non-comment lines is byte-identical to HEAD and braces balance. The first check compared a locale-decoded 'git show' against a UTF-8 read and flagged 25 files that had not changed at all - every BOM and every non-ASCII line mismatched. 47,696 -> 40,665 lines in scope; comment share 38% -> 26%. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-22 00:30:14 -04:00
//| MUTUAL INFORMATION between one cached feature column and the |
//| triple-barrier label, in nats, over a sample of in-sample bars. |
refactor: split CExpertSignalAIBase implementation by responsibility ExpertSignalAIBase.mqh was 8216 lines: the class declaration followed by 87 method bodies covering training, labelling, feature extraction, persistence, chart drawing, online learning, the GA auto-tuner and inference, all in one file. Train() alone is 1492 lines; a change to arrow drawing meant scrolling past the era loop. Moved the bodies into Expert\AIBase\, included at the bottom of the original after the class declaration: Training.mqh 1607 era loop, plateau ladder, checkpoint select, deploy Features.mqh 1093 indicator creation + per-bar input feature vector ChartUI.mqh 634 arrows, arrow persistence, status panel, cleanup Persistence.mqh 492 .stats/.cfg sidecars, CPU-inference validation, copy OnlineLearning.mqh 461 live continual learning, EMA shadow, OOS simulator Labels.mqh 309 ZigZag pivot labels, async label-cache prebuild AutoTune.mqh 275 genetic tuner (population, crossover, halving) Inference.mqh 235 softmax, prior calibration, class priors ExpertSignalAIBase.mqh 8216 -> 3131 (declaration + topology build only) This is a pure relocation - verified mechanically, not by eye: HEAD's file reconstructed from the eight partials plus the surviving remainder is byte-identical to HEAD, span for span (scratchpad verify_split.py). No declaration moved, no signature changed, no code rewritten, so behaviour is unchanged by construction. Compiles 0 errors, 0 warnings. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-29 00:42:45 -04:00
//+------------------------------------------------------------------+
perf(autotune): replace the genetic search with a filter score - hours to seconds MEASURED COST OF THE GA, which is what retired it. Per generation: rung 0: 8 cand x 3 seeds x 3 eras = 72 eras rung 1: 4 cand x 3 seeds x 8 eras = 96 rung 2: 2 cand x 3 seeds x 20 eras = 120 = 288 eras/generation x 4 generations = 1152 eras BEFORE the winner's real training began. Against the observed era times on SP500 H1: PAI 29.1 s/era -> 9.3 h (matches the observed 00:37 -> 09:22) CONV 41.3 s/era -> 13.2 h LSTM 150.4 s/era -> 48.1 h HYBRID 154.6 s/era -> 49.5 h Two days to tune is not a first-run experience, and it is the phase in which the panel goes quiet, which is what made it look like a hang. It also bought nothing. The space is 90 points (10 MA periods x 9 MA types), so 1152 evaluations revisited each point ~13 times; and rungs of 3 and 8 eras cannot separate two MA periods at all. The 2026-08-01 run proves it: every finalist scored 25.0-25.9% balanced accuracy - below the 33.3% one-class floor, i.e. indistinguishable noise - and the search then "deployed the winner" of that. THE ERROR WAS THE SCORING FUNCTION, not its constants. Using a full training run to choose a feature's period is a wrapper method paying wrapper prices for a decision that does not need one. The reference book does not do this: ch. 3.3 selects inputs by measuring each candidate indicator's CORRELATION with the target and dropping the ones with none, with no network involved. So: rank candidates by the MUTUAL INFORMATION between the resulting feature vector and the triple-barrier label. MI rather than correlation because the label is 3-class categorical and the features are not monotonically related to it. Equal-FREQUENCY binning (rank-based), because these features are ATR-normalised and heavy-tailed - fixed-width bins put nearly everything in one bucket and report ~0 information for a genuinely useful feature. Scoring is arithmetic over the feature cache, so it costs seconds and its cost is independent of topology: LSTM now tunes as fast as the MLP. Coordinate sweep, not product sweep - cost is the SUM of per-parameter candidate counts, so enabling every indicator stays affordable - with a second pass that breaks early once nothing moves. Sampling is IS-ONLY. Letting the OOS window influence which indicator settings ship would mean the holdout had been used for selection and had stopped being a holdout. HONEST LIMIT, recorded because it is the price: MI is marginal, so a parameter that only pays off in combination with another can be missed (Guyon & Elisseeff 2003, filter vs wrapper). Given the wrapper it replaces was ranking pure noise at 48 h a run, this is strictly better. Deleted with it: GaRungEras/GaExtract/GaStore/GaMutate/GaRandomCandidate/ GaBlockCrossover/GaSortAliveByScoreDesc/GaBreedNextGeneration, 14 m_ga* members, the GA_*/TUNE_POP_* constants, and ComputeTuneTrialBudget. AND m_evalMode/m_evalEraBudget, because nothing set them any more - 28 read sites all permanently inert. That is not a tidy-up: the `if (!m_evalMode)` guard on UpdateClassPriors is exactly what silently disabled the imbalance correction for entire runs two commits ago. Dead machinery that still reads like live machinery is this codebase's most expensive recurring bug, and leaving 28 more instances of it would have been indefensible. The panel's tuning-progress state goes too - tuning no longer takes long enough to need one. Both builds compile 0 errors / 0 warnings. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-01 10:29:31 -04:00
double CExpertSignalAIBase::FeatureColumnMI(const double &vals[], const int &labels[], int n)
refactor: split CExpertSignalAIBase implementation by responsibility ExpertSignalAIBase.mqh was 8216 lines: the class declaration followed by 87 method bodies covering training, labelling, feature extraction, persistence, chart drawing, online learning, the GA auto-tuner and inference, all in one file. Train() alone is 1492 lines; a change to arrow drawing meant scrolling past the era loop. Moved the bodies into Expert\AIBase\, included at the bottom of the original after the class declaration: Training.mqh 1607 era loop, plateau ladder, checkpoint select, deploy Features.mqh 1093 indicator creation + per-bar input feature vector ChartUI.mqh 634 arrows, arrow persistence, status panel, cleanup Persistence.mqh 492 .stats/.cfg sidecars, CPU-inference validation, copy OnlineLearning.mqh 461 live continual learning, EMA shadow, OOS simulator Labels.mqh 309 ZigZag pivot labels, async label-cache prebuild AutoTune.mqh 275 genetic tuner (population, crossover, halving) Inference.mqh 235 softmax, prior calibration, class priors ExpertSignalAIBase.mqh 8216 -> 3131 (declaration + topology build only) This is a pure relocation - verified mechanically, not by eye: HEAD's file reconstructed from the eight partials plus the surviving remainder is byte-identical to HEAD, span for span (scratchpad verify_split.py). No declaration moved, no signature changed, no code rewritten, so behaviour is unchanged by construction. Compiles 0 errors, 0 warnings. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-29 00:42:45 -04:00
{
perf(autotune): replace the genetic search with a filter score - hours to seconds MEASURED COST OF THE GA, which is what retired it. Per generation: rung 0: 8 cand x 3 seeds x 3 eras = 72 eras rung 1: 4 cand x 3 seeds x 8 eras = 96 rung 2: 2 cand x 3 seeds x 20 eras = 120 = 288 eras/generation x 4 generations = 1152 eras BEFORE the winner's real training began. Against the observed era times on SP500 H1: PAI 29.1 s/era -> 9.3 h (matches the observed 00:37 -> 09:22) CONV 41.3 s/era -> 13.2 h LSTM 150.4 s/era -> 48.1 h HYBRID 154.6 s/era -> 49.5 h Two days to tune is not a first-run experience, and it is the phase in which the panel goes quiet, which is what made it look like a hang. It also bought nothing. The space is 90 points (10 MA periods x 9 MA types), so 1152 evaluations revisited each point ~13 times; and rungs of 3 and 8 eras cannot separate two MA periods at all. The 2026-08-01 run proves it: every finalist scored 25.0-25.9% balanced accuracy - below the 33.3% one-class floor, i.e. indistinguishable noise - and the search then "deployed the winner" of that. THE ERROR WAS THE SCORING FUNCTION, not its constants. Using a full training run to choose a feature's period is a wrapper method paying wrapper prices for a decision that does not need one. The reference book does not do this: ch. 3.3 selects inputs by measuring each candidate indicator's CORRELATION with the target and dropping the ones with none, with no network involved. So: rank candidates by the MUTUAL INFORMATION between the resulting feature vector and the triple-barrier label. MI rather than correlation because the label is 3-class categorical and the features are not monotonically related to it. Equal-FREQUENCY binning (rank-based), because these features are ATR-normalised and heavy-tailed - fixed-width bins put nearly everything in one bucket and report ~0 information for a genuinely useful feature. Scoring is arithmetic over the feature cache, so it costs seconds and its cost is independent of topology: LSTM now tunes as fast as the MLP. Coordinate sweep, not product sweep - cost is the SUM of per-parameter candidate counts, so enabling every indicator stays affordable - with a second pass that breaks early once nothing moves. Sampling is IS-ONLY. Letting the OOS window influence which indicator settings ship would mean the holdout had been used for selection and had stopped being a holdout. HONEST LIMIT, recorded because it is the price: MI is marginal, so a parameter that only pays off in combination with another can be missed (Guyon & Elisseeff 2003, filter vs wrapper). Given the wrapper it replaces was ranking pure noise at 48 h a run, this is strictly better. Deleted with it: GaRungEras/GaExtract/GaStore/GaMutate/GaRandomCandidate/ GaBlockCrossover/GaSortAliveByScoreDesc/GaBreedNextGeneration, 14 m_ga* members, the GA_*/TUNE_POP_* constants, and ComputeTuneTrialBudget. AND m_evalMode/m_evalEraBudget, because nothing set them any more - 28 read sites all permanently inert. That is not a tidy-up: the `if (!m_evalMode)` guard on UpdateClassPriors is exactly what silently disabled the imbalance correction for entire runs two commits ago. Dead machinery that still reads like live machinery is this codebase's most expensive recurring bug, and leaving 28 more instances of it would have been indefensible. The panel's tuning-progress state goes too - tuning no longer takes long enough to need one. Both builds compile 0 errors / 0 warnings. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-01 10:29:31 -04:00
if(n < MI_MIN_SAMPLES)
return 0.0;
double sorted[];
ArrayResize(sorted, n);
ArrayCopy(sorted, vals, 0, 0, n);
ArraySort(sorted);
//--- A column that never varies carries no information; short-circuit so the log below is never
//--- reached with a degenerate single-bin histogram.
if(sorted[0] == sorted[n - 1])
return 0.0;
int joint[]; ArrayResize(joint, MI_BINS * 3); ArrayInitialize(joint, 0);
int px[]; ArrayResize(px, MI_BINS); ArrayInitialize(px, 0);
int py[]; ArrayResize(py, 3); ArrayInitialize(py, 0);
for(int i = 0; i < n; i++)
refactor: split CExpertSignalAIBase implementation by responsibility ExpertSignalAIBase.mqh was 8216 lines: the class declaration followed by 87 method bodies covering training, labelling, feature extraction, persistence, chart drawing, online learning, the GA auto-tuner and inference, all in one file. Train() alone is 1492 lines; a change to arrow drawing meant scrolling past the era loop. Moved the bodies into Expert\AIBase\, included at the bottom of the original after the class declaration: Training.mqh 1607 era loop, plateau ladder, checkpoint select, deploy Features.mqh 1093 indicator creation + per-bar input feature vector ChartUI.mqh 634 arrows, arrow persistence, status panel, cleanup Persistence.mqh 492 .stats/.cfg sidecars, CPU-inference validation, copy OnlineLearning.mqh 461 live continual learning, EMA shadow, OOS simulator Labels.mqh 309 ZigZag pivot labels, async label-cache prebuild AutoTune.mqh 275 genetic tuner (population, crossover, halving) Inference.mqh 235 softmax, prior calibration, class priors ExpertSignalAIBase.mqh 8216 -> 3131 (declaration + topology build only) This is a pure relocation - verified mechanically, not by eye: HEAD's file reconstructed from the eight partials plus the surviving remainder is byte-identical to HEAD, span for span (scratchpad verify_split.py). No declaration moved, no signature changed, no code rewritten, so behaviour is unchanged by construction. Compiles 0 errors, 0 warnings. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-29 00:42:45 -04:00
{
perf(autotune): replace the genetic search with a filter score - hours to seconds MEASURED COST OF THE GA, which is what retired it. Per generation: rung 0: 8 cand x 3 seeds x 3 eras = 72 eras rung 1: 4 cand x 3 seeds x 8 eras = 96 rung 2: 2 cand x 3 seeds x 20 eras = 120 = 288 eras/generation x 4 generations = 1152 eras BEFORE the winner's real training began. Against the observed era times on SP500 H1: PAI 29.1 s/era -> 9.3 h (matches the observed 00:37 -> 09:22) CONV 41.3 s/era -> 13.2 h LSTM 150.4 s/era -> 48.1 h HYBRID 154.6 s/era -> 49.5 h Two days to tune is not a first-run experience, and it is the phase in which the panel goes quiet, which is what made it look like a hang. It also bought nothing. The space is 90 points (10 MA periods x 9 MA types), so 1152 evaluations revisited each point ~13 times; and rungs of 3 and 8 eras cannot separate two MA periods at all. The 2026-08-01 run proves it: every finalist scored 25.0-25.9% balanced accuracy - below the 33.3% one-class floor, i.e. indistinguishable noise - and the search then "deployed the winner" of that. THE ERROR WAS THE SCORING FUNCTION, not its constants. Using a full training run to choose a feature's period is a wrapper method paying wrapper prices for a decision that does not need one. The reference book does not do this: ch. 3.3 selects inputs by measuring each candidate indicator's CORRELATION with the target and dropping the ones with none, with no network involved. So: rank candidates by the MUTUAL INFORMATION between the resulting feature vector and the triple-barrier label. MI rather than correlation because the label is 3-class categorical and the features are not monotonically related to it. Equal-FREQUENCY binning (rank-based), because these features are ATR-normalised and heavy-tailed - fixed-width bins put nearly everything in one bucket and report ~0 information for a genuinely useful feature. Scoring is arithmetic over the feature cache, so it costs seconds and its cost is independent of topology: LSTM now tunes as fast as the MLP. Coordinate sweep, not product sweep - cost is the SUM of per-parameter candidate counts, so enabling every indicator stays affordable - with a second pass that breaks early once nothing moves. Sampling is IS-ONLY. Letting the OOS window influence which indicator settings ship would mean the holdout had been used for selection and had stopped being a holdout. HONEST LIMIT, recorded because it is the price: MI is marginal, so a parameter that only pays off in combination with another can be missed (Guyon & Elisseeff 2003, filter vs wrapper). Given the wrapper it replaces was ranking pure noise at 48 h a run, this is strictly better. Deleted with it: GaRungEras/GaExtract/GaStore/GaMutate/GaRandomCandidate/ GaBlockCrossover/GaSortAliveByScoreDesc/GaBreedNextGeneration, 14 m_ga* members, the GA_*/TUNE_POP_* constants, and ComputeTuneTrialBudget. AND m_evalMode/m_evalEraBudget, because nothing set them any more - 28 read sites all permanently inert. That is not a tidy-up: the `if (!m_evalMode)` guard on UpdateClassPriors is exactly what silently disabled the imbalance correction for entire runs two commits ago. Dead machinery that still reads like live machinery is this codebase's most expensive recurring bug, and leaving 28 more instances of it would have been indefensible. The panel's tuning-progress state goes too - tuning no longer takes long enough to need one. Both builds compile 0 errors / 0 warnings. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-01 10:29:31 -04:00
//--- rank via binary search on the sorted copy; ties land in the same bin, which is correct
int lo = 0, hi = n - 1, rank = 0;
while(lo <= hi)
{
int mid = (lo + hi) / 2;
if(sorted[mid] < vals[i])
{
rank = mid + 1;
lo = mid + 1;
}
else
hi = mid - 1;
}
int bx = (int)((double)rank * MI_BINS / n);
if(bx >= MI_BINS)
bx = MI_BINS - 1;
int by = labels[i];
if(by < 0 || by > 2)
continue;
joint[bx * 3 + by]++;
px[bx]++;
py[by]++;
refactor: split CExpertSignalAIBase implementation by responsibility ExpertSignalAIBase.mqh was 8216 lines: the class declaration followed by 87 method bodies covering training, labelling, feature extraction, persistence, chart drawing, online learning, the GA auto-tuner and inference, all in one file. Train() alone is 1492 lines; a change to arrow drawing meant scrolling past the era loop. Moved the bodies into Expert\AIBase\, included at the bottom of the original after the class declaration: Training.mqh 1607 era loop, plateau ladder, checkpoint select, deploy Features.mqh 1093 indicator creation + per-bar input feature vector ChartUI.mqh 634 arrows, arrow persistence, status panel, cleanup Persistence.mqh 492 .stats/.cfg sidecars, CPU-inference validation, copy OnlineLearning.mqh 461 live continual learning, EMA shadow, OOS simulator Labels.mqh 309 ZigZag pivot labels, async label-cache prebuild AutoTune.mqh 275 genetic tuner (population, crossover, halving) Inference.mqh 235 softmax, prior calibration, class priors ExpertSignalAIBase.mqh 8216 -> 3131 (declaration + topology build only) This is a pure relocation - verified mechanically, not by eye: HEAD's file reconstructed from the eight partials plus the surviving remainder is byte-identical to HEAD, span for span (scratchpad verify_split.py). No declaration moved, no signature changed, no code rewritten, so behaviour is unchanged by construction. Compiles 0 errors, 0 warnings. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-29 00:42:45 -04:00
}
perf(autotune): replace the genetic search with a filter score - hours to seconds MEASURED COST OF THE GA, which is what retired it. Per generation: rung 0: 8 cand x 3 seeds x 3 eras = 72 eras rung 1: 4 cand x 3 seeds x 8 eras = 96 rung 2: 2 cand x 3 seeds x 20 eras = 120 = 288 eras/generation x 4 generations = 1152 eras BEFORE the winner's real training began. Against the observed era times on SP500 H1: PAI 29.1 s/era -> 9.3 h (matches the observed 00:37 -> 09:22) CONV 41.3 s/era -> 13.2 h LSTM 150.4 s/era -> 48.1 h HYBRID 154.6 s/era -> 49.5 h Two days to tune is not a first-run experience, and it is the phase in which the panel goes quiet, which is what made it look like a hang. It also bought nothing. The space is 90 points (10 MA periods x 9 MA types), so 1152 evaluations revisited each point ~13 times; and rungs of 3 and 8 eras cannot separate two MA periods at all. The 2026-08-01 run proves it: every finalist scored 25.0-25.9% balanced accuracy - below the 33.3% one-class floor, i.e. indistinguishable noise - and the search then "deployed the winner" of that. THE ERROR WAS THE SCORING FUNCTION, not its constants. Using a full training run to choose a feature's period is a wrapper method paying wrapper prices for a decision that does not need one. The reference book does not do this: ch. 3.3 selects inputs by measuring each candidate indicator's CORRELATION with the target and dropping the ones with none, with no network involved. So: rank candidates by the MUTUAL INFORMATION between the resulting feature vector and the triple-barrier label. MI rather than correlation because the label is 3-class categorical and the features are not monotonically related to it. Equal-FREQUENCY binning (rank-based), because these features are ATR-normalised and heavy-tailed - fixed-width bins put nearly everything in one bucket and report ~0 information for a genuinely useful feature. Scoring is arithmetic over the feature cache, so it costs seconds and its cost is independent of topology: LSTM now tunes as fast as the MLP. Coordinate sweep, not product sweep - cost is the SUM of per-parameter candidate counts, so enabling every indicator stays affordable - with a second pass that breaks early once nothing moves. Sampling is IS-ONLY. Letting the OOS window influence which indicator settings ship would mean the holdout had been used for selection and had stopped being a holdout. HONEST LIMIT, recorded because it is the price: MI is marginal, so a parameter that only pays off in combination with another can be missed (Guyon & Elisseeff 2003, filter vs wrapper). Given the wrapper it replaces was ranking pure noise at 48 h a run, this is strictly better. Deleted with it: GaRungEras/GaExtract/GaStore/GaMutate/GaRandomCandidate/ GaBlockCrossover/GaSortAliveByScoreDesc/GaBreedNextGeneration, 14 m_ga* members, the GA_*/TUNE_POP_* constants, and ComputeTuneTrialBudget. AND m_evalMode/m_evalEraBudget, because nothing set them any more - 28 read sites all permanently inert. That is not a tidy-up: the `if (!m_evalMode)` guard on UpdateClassPriors is exactly what silently disabled the imbalance correction for entire runs two commits ago. Dead machinery that still reads like live machinery is this codebase's most expensive recurring bug, and leaving 28 more instances of it would have been indefensible. The panel's tuning-progress state goes too - tuning no longer takes long enough to need one. Both builds compile 0 errors / 0 warnings. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-01 10:29:31 -04:00
double mi = 0.0;
for(int b = 0; b < MI_BINS; b++)
refactor: split CExpertSignalAIBase implementation by responsibility ExpertSignalAIBase.mqh was 8216 lines: the class declaration followed by 87 method bodies covering training, labelling, feature extraction, persistence, chart drawing, online learning, the GA auto-tuner and inference, all in one file. Train() alone is 1492 lines; a change to arrow drawing meant scrolling past the era loop. Moved the bodies into Expert\AIBase\, included at the bottom of the original after the class declaration: Training.mqh 1607 era loop, plateau ladder, checkpoint select, deploy Features.mqh 1093 indicator creation + per-bar input feature vector ChartUI.mqh 634 arrows, arrow persistence, status panel, cleanup Persistence.mqh 492 .stats/.cfg sidecars, CPU-inference validation, copy OnlineLearning.mqh 461 live continual learning, EMA shadow, OOS simulator Labels.mqh 309 ZigZag pivot labels, async label-cache prebuild AutoTune.mqh 275 genetic tuner (population, crossover, halving) Inference.mqh 235 softmax, prior calibration, class priors ExpertSignalAIBase.mqh 8216 -> 3131 (declaration + topology build only) This is a pure relocation - verified mechanically, not by eye: HEAD's file reconstructed from the eight partials plus the surviving remainder is byte-identical to HEAD, span for span (scratchpad verify_split.py). No declaration moved, no signature changed, no code rewritten, so behaviour is unchanged by construction. Compiles 0 errors, 0 warnings. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-29 00:42:45 -04:00
{
perf(autotune): replace the genetic search with a filter score - hours to seconds MEASURED COST OF THE GA, which is what retired it. Per generation: rung 0: 8 cand x 3 seeds x 3 eras = 72 eras rung 1: 4 cand x 3 seeds x 8 eras = 96 rung 2: 2 cand x 3 seeds x 20 eras = 120 = 288 eras/generation x 4 generations = 1152 eras BEFORE the winner's real training began. Against the observed era times on SP500 H1: PAI 29.1 s/era -> 9.3 h (matches the observed 00:37 -> 09:22) CONV 41.3 s/era -> 13.2 h LSTM 150.4 s/era -> 48.1 h HYBRID 154.6 s/era -> 49.5 h Two days to tune is not a first-run experience, and it is the phase in which the panel goes quiet, which is what made it look like a hang. It also bought nothing. The space is 90 points (10 MA periods x 9 MA types), so 1152 evaluations revisited each point ~13 times; and rungs of 3 and 8 eras cannot separate two MA periods at all. The 2026-08-01 run proves it: every finalist scored 25.0-25.9% balanced accuracy - below the 33.3% one-class floor, i.e. indistinguishable noise - and the search then "deployed the winner" of that. THE ERROR WAS THE SCORING FUNCTION, not its constants. Using a full training run to choose a feature's period is a wrapper method paying wrapper prices for a decision that does not need one. The reference book does not do this: ch. 3.3 selects inputs by measuring each candidate indicator's CORRELATION with the target and dropping the ones with none, with no network involved. So: rank candidates by the MUTUAL INFORMATION between the resulting feature vector and the triple-barrier label. MI rather than correlation because the label is 3-class categorical and the features are not monotonically related to it. Equal-FREQUENCY binning (rank-based), because these features are ATR-normalised and heavy-tailed - fixed-width bins put nearly everything in one bucket and report ~0 information for a genuinely useful feature. Scoring is arithmetic over the feature cache, so it costs seconds and its cost is independent of topology: LSTM now tunes as fast as the MLP. Coordinate sweep, not product sweep - cost is the SUM of per-parameter candidate counts, so enabling every indicator stays affordable - with a second pass that breaks early once nothing moves. Sampling is IS-ONLY. Letting the OOS window influence which indicator settings ship would mean the holdout had been used for selection and had stopped being a holdout. HONEST LIMIT, recorded because it is the price: MI is marginal, so a parameter that only pays off in combination with another can be missed (Guyon & Elisseeff 2003, filter vs wrapper). Given the wrapper it replaces was ranking pure noise at 48 h a run, this is strictly better. Deleted with it: GaRungEras/GaExtract/GaStore/GaMutate/GaRandomCandidate/ GaBlockCrossover/GaSortAliveByScoreDesc/GaBreedNextGeneration, 14 m_ga* members, the GA_*/TUNE_POP_* constants, and ComputeTuneTrialBudget. AND m_evalMode/m_evalEraBudget, because nothing set them any more - 28 read sites all permanently inert. That is not a tidy-up: the `if (!m_evalMode)` guard on UpdateClassPriors is exactly what silently disabled the imbalance correction for entire runs two commits ago. Dead machinery that still reads like live machinery is this codebase's most expensive recurring bug, and leaving 28 more instances of it would have been indefensible. The panel's tuning-progress state goes too - tuning no longer takes long enough to need one. Both builds compile 0 errors / 0 warnings. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-01 10:29:31 -04:00
if(px[b] <= 0)
continue;
for(int c = 0; c < 3; c++)
refactor: split CExpertSignalAIBase implementation by responsibility ExpertSignalAIBase.mqh was 8216 lines: the class declaration followed by 87 method bodies covering training, labelling, feature extraction, persistence, chart drawing, online learning, the GA auto-tuner and inference, all in one file. Train() alone is 1492 lines; a change to arrow drawing meant scrolling past the era loop. Moved the bodies into Expert\AIBase\, included at the bottom of the original after the class declaration: Training.mqh 1607 era loop, plateau ladder, checkpoint select, deploy Features.mqh 1093 indicator creation + per-bar input feature vector ChartUI.mqh 634 arrows, arrow persistence, status panel, cleanup Persistence.mqh 492 .stats/.cfg sidecars, CPU-inference validation, copy OnlineLearning.mqh 461 live continual learning, EMA shadow, OOS simulator Labels.mqh 309 ZigZag pivot labels, async label-cache prebuild AutoTune.mqh 275 genetic tuner (population, crossover, halving) Inference.mqh 235 softmax, prior calibration, class priors ExpertSignalAIBase.mqh 8216 -> 3131 (declaration + topology build only) This is a pure relocation - verified mechanically, not by eye: HEAD's file reconstructed from the eight partials plus the surviving remainder is byte-identical to HEAD, span for span (scratchpad verify_split.py). No declaration moved, no signature changed, no code rewritten, so behaviour is unchanged by construction. Compiles 0 errors, 0 warnings. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-29 00:42:45 -04:00
{
perf(autotune): replace the genetic search with a filter score - hours to seconds MEASURED COST OF THE GA, which is what retired it. Per generation: rung 0: 8 cand x 3 seeds x 3 eras = 72 eras rung 1: 4 cand x 3 seeds x 8 eras = 96 rung 2: 2 cand x 3 seeds x 20 eras = 120 = 288 eras/generation x 4 generations = 1152 eras BEFORE the winner's real training began. Against the observed era times on SP500 H1: PAI 29.1 s/era -> 9.3 h (matches the observed 00:37 -> 09:22) CONV 41.3 s/era -> 13.2 h LSTM 150.4 s/era -> 48.1 h HYBRID 154.6 s/era -> 49.5 h Two days to tune is not a first-run experience, and it is the phase in which the panel goes quiet, which is what made it look like a hang. It also bought nothing. The space is 90 points (10 MA periods x 9 MA types), so 1152 evaluations revisited each point ~13 times; and rungs of 3 and 8 eras cannot separate two MA periods at all. The 2026-08-01 run proves it: every finalist scored 25.0-25.9% balanced accuracy - below the 33.3% one-class floor, i.e. indistinguishable noise - and the search then "deployed the winner" of that. THE ERROR WAS THE SCORING FUNCTION, not its constants. Using a full training run to choose a feature's period is a wrapper method paying wrapper prices for a decision that does not need one. The reference book does not do this: ch. 3.3 selects inputs by measuring each candidate indicator's CORRELATION with the target and dropping the ones with none, with no network involved. So: rank candidates by the MUTUAL INFORMATION between the resulting feature vector and the triple-barrier label. MI rather than correlation because the label is 3-class categorical and the features are not monotonically related to it. Equal-FREQUENCY binning (rank-based), because these features are ATR-normalised and heavy-tailed - fixed-width bins put nearly everything in one bucket and report ~0 information for a genuinely useful feature. Scoring is arithmetic over the feature cache, so it costs seconds and its cost is independent of topology: LSTM now tunes as fast as the MLP. Coordinate sweep, not product sweep - cost is the SUM of per-parameter candidate counts, so enabling every indicator stays affordable - with a second pass that breaks early once nothing moves. Sampling is IS-ONLY. Letting the OOS window influence which indicator settings ship would mean the holdout had been used for selection and had stopped being a holdout. HONEST LIMIT, recorded because it is the price: MI is marginal, so a parameter that only pays off in combination with another can be missed (Guyon & Elisseeff 2003, filter vs wrapper). Given the wrapper it replaces was ranking pure noise at 48 h a run, this is strictly better. Deleted with it: GaRungEras/GaExtract/GaStore/GaMutate/GaRandomCandidate/ GaBlockCrossover/GaSortAliveByScoreDesc/GaBreedNextGeneration, 14 m_ga* members, the GA_*/TUNE_POP_* constants, and ComputeTuneTrialBudget. AND m_evalMode/m_evalEraBudget, because nothing set them any more - 28 read sites all permanently inert. That is not a tidy-up: the `if (!m_evalMode)` guard on UpdateClassPriors is exactly what silently disabled the imbalance correction for entire runs two commits ago. Dead machinery that still reads like live machinery is this codebase's most expensive recurring bug, and leaving 28 more instances of it would have been indefensible. The panel's tuning-progress state goes too - tuning no longer takes long enough to need one. Both builds compile 0 errors / 0 warnings. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-01 10:29:31 -04:00
int j = joint[b * 3 + c];
if(j <= 0 || py[c] <= 0)
continue;
double pxy = (double)j / n;
mi += pxy * MathLog(pxy / (((double)px[b] / n) * ((double)py[c] / n)));
refactor: split CExpertSignalAIBase implementation by responsibility ExpertSignalAIBase.mqh was 8216 lines: the class declaration followed by 87 method bodies covering training, labelling, feature extraction, persistence, chart drawing, online learning, the GA auto-tuner and inference, all in one file. Train() alone is 1492 lines; a change to arrow drawing meant scrolling past the era loop. Moved the bodies into Expert\AIBase\, included at the bottom of the original after the class declaration: Training.mqh 1607 era loop, plateau ladder, checkpoint select, deploy Features.mqh 1093 indicator creation + per-bar input feature vector ChartUI.mqh 634 arrows, arrow persistence, status panel, cleanup Persistence.mqh 492 .stats/.cfg sidecars, CPU-inference validation, copy OnlineLearning.mqh 461 live continual learning, EMA shadow, OOS simulator Labels.mqh 309 ZigZag pivot labels, async label-cache prebuild AutoTune.mqh 275 genetic tuner (population, crossover, halving) Inference.mqh 235 softmax, prior calibration, class priors ExpertSignalAIBase.mqh 8216 -> 3131 (declaration + topology build only) This is a pure relocation - verified mechanically, not by eye: HEAD's file reconstructed from the eight partials plus the surviving remainder is byte-identical to HEAD, span for span (scratchpad verify_split.py). No declaration moved, no signature changed, no code rewritten, so behaviour is unchanged by construction. Compiles 0 errors, 0 warnings. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-29 00:42:45 -04:00
}
}
perf(autotune): replace the genetic search with a filter score - hours to seconds MEASURED COST OF THE GA, which is what retired it. Per generation: rung 0: 8 cand x 3 seeds x 3 eras = 72 eras rung 1: 4 cand x 3 seeds x 8 eras = 96 rung 2: 2 cand x 3 seeds x 20 eras = 120 = 288 eras/generation x 4 generations = 1152 eras BEFORE the winner's real training began. Against the observed era times on SP500 H1: PAI 29.1 s/era -> 9.3 h (matches the observed 00:37 -> 09:22) CONV 41.3 s/era -> 13.2 h LSTM 150.4 s/era -> 48.1 h HYBRID 154.6 s/era -> 49.5 h Two days to tune is not a first-run experience, and it is the phase in which the panel goes quiet, which is what made it look like a hang. It also bought nothing. The space is 90 points (10 MA periods x 9 MA types), so 1152 evaluations revisited each point ~13 times; and rungs of 3 and 8 eras cannot separate two MA periods at all. The 2026-08-01 run proves it: every finalist scored 25.0-25.9% balanced accuracy - below the 33.3% one-class floor, i.e. indistinguishable noise - and the search then "deployed the winner" of that. THE ERROR WAS THE SCORING FUNCTION, not its constants. Using a full training run to choose a feature's period is a wrapper method paying wrapper prices for a decision that does not need one. The reference book does not do this: ch. 3.3 selects inputs by measuring each candidate indicator's CORRELATION with the target and dropping the ones with none, with no network involved. So: rank candidates by the MUTUAL INFORMATION between the resulting feature vector and the triple-barrier label. MI rather than correlation because the label is 3-class categorical and the features are not monotonically related to it. Equal-FREQUENCY binning (rank-based), because these features are ATR-normalised and heavy-tailed - fixed-width bins put nearly everything in one bucket and report ~0 information for a genuinely useful feature. Scoring is arithmetic over the feature cache, so it costs seconds and its cost is independent of topology: LSTM now tunes as fast as the MLP. Coordinate sweep, not product sweep - cost is the SUM of per-parameter candidate counts, so enabling every indicator stays affordable - with a second pass that breaks early once nothing moves. Sampling is IS-ONLY. Letting the OOS window influence which indicator settings ship would mean the holdout had been used for selection and had stopped being a holdout. HONEST LIMIT, recorded because it is the price: MI is marginal, so a parameter that only pays off in combination with another can be missed (Guyon & Elisseeff 2003, filter vs wrapper). Given the wrapper it replaces was ranking pure noise at 48 h a run, this is strictly better. Deleted with it: GaRungEras/GaExtract/GaStore/GaMutate/GaRandomCandidate/ GaBlockCrossover/GaSortAliveByScoreDesc/GaBreedNextGeneration, 14 m_ga* members, the GA_*/TUNE_POP_* constants, and ComputeTuneTrialBudget. AND m_evalMode/m_evalEraBudget, because nothing set them any more - 28 read sites all permanently inert. That is not a tidy-up: the `if (!m_evalMode)` guard on UpdateClassPriors is exactly what silently disabled the imbalance correction for entire runs two commits ago. Dead machinery that still reads like live machinery is this codebase's most expensive recurring bug, and leaving 28 more instances of it would have been indefensible. The panel's tuning-progress state goes too - tuning no longer takes long enough to need one. Both builds compile 0 errors / 0 warnings. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-01 10:29:31 -04:00
return (mi > 0.0) ? mi : 0.0;
refactor: split CExpertSignalAIBase implementation by responsibility ExpertSignalAIBase.mqh was 8216 lines: the class declaration followed by 87 method bodies covering training, labelling, feature extraction, persistence, chart drawing, online learning, the GA auto-tuner and inference, all in one file. Train() alone is 1492 lines; a change to arrow drawing meant scrolling past the era loop. Moved the bodies into Expert\AIBase\, included at the bottom of the original after the class declaration: Training.mqh 1607 era loop, plateau ladder, checkpoint select, deploy Features.mqh 1093 indicator creation + per-bar input feature vector ChartUI.mqh 634 arrows, arrow persistence, status panel, cleanup Persistence.mqh 492 .stats/.cfg sidecars, CPU-inference validation, copy OnlineLearning.mqh 461 live continual learning, EMA shadow, OOS simulator Labels.mqh 309 ZigZag pivot labels, async label-cache prebuild AutoTune.mqh 275 genetic tuner (population, crossover, halving) Inference.mqh 235 softmax, prior calibration, class priors ExpertSignalAIBase.mqh 8216 -> 3131 (declaration + topology build only) This is a pure relocation - verified mechanically, not by eye: HEAD's file reconstructed from the eight partials plus the surviving remainder is byte-identical to HEAD, span for span (scratchpad verify_split.py). No declaration moved, no signature changed, no code rewritten, so behaviour is unchanged by construction. Compiles 0 errors, 0 warnings. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-29 00:42:45 -04:00
}
//+------------------------------------------------------------------+
refactor(comments): box headers to stdlib length The //| box blocks were excluded from 0b06f8e and 5efdb48 and were what remained: 160 of them ran to 10+ lines, the longest to 88. Compressed to their leading topic sentences - 5 lines for a function header, 8 for a file header - keeping the box format and the standard MQL5 name/author lines verbatim. Verified at the BYTE level this time, across every in-scope file: the list of non-comment lines is byte-identical to HEAD and braces balance. The first check compared a locale-decoded 'git show' against a UTF-8 read and flagged 25 files that had not changed at all - every BOM and every non-ASCII line mismatched. 47,696 -> 40,665 lines in scope; comment share 38% -> 26%. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-22 00:30:14 -04:00
//| Scores the CURRENT indicator parameters by how much the |
//| resulting feature vector tells us about the label - the mean |
//| per-column mutual information over a stratified sample of in- |
//| sample bars. |
refactor: split CExpertSignalAIBase implementation by responsibility ExpertSignalAIBase.mqh was 8216 lines: the class declaration followed by 87 method bodies covering training, labelling, feature extraction, persistence, chart drawing, online learning, the GA auto-tuner and inference, all in one file. Train() alone is 1492 lines; a change to arrow drawing meant scrolling past the era loop. Moved the bodies into Expert\AIBase\, included at the bottom of the original after the class declaration: Training.mqh 1607 era loop, plateau ladder, checkpoint select, deploy Features.mqh 1093 indicator creation + per-bar input feature vector ChartUI.mqh 634 arrows, arrow persistence, status panel, cleanup Persistence.mqh 492 .stats/.cfg sidecars, CPU-inference validation, copy OnlineLearning.mqh 461 live continual learning, EMA shadow, OOS simulator Labels.mqh 309 ZigZag pivot labels, async label-cache prebuild AutoTune.mqh 275 genetic tuner (population, crossover, halving) Inference.mqh 235 softmax, prior calibration, class priors ExpertSignalAIBase.mqh 8216 -> 3131 (declaration + topology build only) This is a pure relocation - verified mechanically, not by eye: HEAD's file reconstructed from the eight partials plus the surviving remainder is byte-identical to HEAD, span for span (scratchpad verify_split.py). No declaration moved, no signature changed, no code rewritten, so behaviour is unchanged by construction. Compiles 0 errors, 0 warnings. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-29 00:42:45 -04:00
//+------------------------------------------------------------------+
diag: MI feature-lag profile - close the blind spot in every MI verdict so far BuildMiSample samples features from ONE bar. So every "MI is at the noise floor" result this codebase has produced - including yesterday's p=0.18 on SP500 H1 - described the ENTRY BAR's 31 features only, while the network is fed 20 bars of them. If information lived at lag 7 and not lag 0, the report would have said "no signal" while the model could still learn. The diagnostic we have been making decisions on had a blind spot exactly the width of the input vector. Adds a FEATURE-side offset to BuildMiSample, which is not the same thing as the existing labelBarOffset and is not interchangeable with it. Shifting the LABEL changes which trade is predicted, so at any non-zero offset the features sit inside the labelled window and the score is lookahead - that is precisely what the alignment scan measures and correctly reports (4.7x more knowable 5 bars into a 128-bar window). Shifting the FEATURES keeps the label pinned to the entry bar, so every row stays causal. ReportFeatureLagProfile() then scores k = 0..historyBars against the same block-permutation null and reports the deepest lag that clears it - the lookback the data supports, versus the 20 that was picked by hand and never measured. The null is redrawn PER LAG: finite-sample MI bias moves with the realised class counts and bin occupancy, and different rows survive the validity checks at each lag, so one shared floor would be right for lag 0 and wrong everywhere else. Draw count is reduced accordingly (40, not 200) since cost is draws x historyBars; this figure decides a lookback, never a trade. MiShiftPad now also covers historyBars, keeping the fixed-pad invariant that makes two builds comparable row by row. Read-only - no input, topology or label change, so no retrain. Both builds 0/0. Build tag lag-profile-v1. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-06 13:51:39 -04:00
int CExpertSignalAIBase::BuildMiSample(double &cols[], int &labels[], int labelBarOffset = 0,
diag: is "optimal SL/TP" learnable? Score the features against excursions Proposed direction: train the net to predict entry/SL/TP that maximise return and minimise drawdown, rather than to classify direction. Before rebuilding a head, measure whether the target is learnable at all. That question splits into two that behave nothing alike: HOW FAR price travels (MFE/MAE) - essentially volatility, and volatility clustering is about the most robust regularity in markets. WHICH WAY it goes first (the asymmetry) - direction, which is what every noise-floor verdict in this project has been about. Expectancy comes ONLY from the second. The first buys position sizing and drawdown control - worth having under prop-firm limits, but not an edge: exit management on RANDOM entries already moved the payoff ratio 0.92 -> 5.72 with expectancy FLAT. Crucially this is NOT already answered. Every MI figure here scored the triple-barrier label, i.e. one specific question at one fixed geometry. A noise-floor result there says nothing about whether excursion MAGNITUDE is learnable - different target, different answer. Four targets, and the verdict is the CONTRAST, printed explicitly because the dangerous misreading of "UP clears" is "we can predict profitable trades": RANGE (up+dn) - realised volatility, included as a POSITIVE CONTROL that SHOULD clear. Every prior verdict here lacked a control expected to pass; a range target at the floor indicts the measurement, not the market. UP / DOWN - MFE / MAE. ASYMMETRY - up-dn, the only one that can pay. Collected inside the walk the label already does (one max, one min per bar). The early-out when both barriers resolved is GONE: it would have truncated the excursions at whichever bar tripped the last barrier, making the measurement a function of the CURRENT SL/TP - the circularity this is trying to escape. The loop was already bounded by the horizon, so only the average cost moves. Discretised into 3 EQUAL-FREQUENCY bins, so every downstream piece (block permutation, null, p-value) is reused unchanged. Equal-frequency because MFE is fat-tailed and fixed-width bins would put nearly every row in bin 0; it also pins H(Y) at ln(3)=1.099 for all four, making them comparable to each other and to the barrier label's ~1.02 instead of confounded by class balance. Two bugs fixed in this code before it ever ran, both of which would have produced a plausible quiet wrong answer rather than an error: - TripleBarrierLabel early-returns on invalid ATR/close BEFORE the point the accumulators were reset, so one bar's excursions would be cached under another bar's index. Cleared at the top now, ahead of every return. - An unresolvable bar is still flagged as labelled but carries excursions of exactly 0. Under equal-frequency binning a block of identical zeros drags the lowest cut onto zero and a third of the sample lands in one uninformative bin - a depressed score that reads as "not predictable", a false negative in the direction that would wrongly kill the idea. Rows where both excursions are zero are dropped; price cannot travel zero both ways over a whole horizon. Read-only diagnostic. No topology or label change: no retrain of its own. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-07 10:22:41 -04:00
int featureBarOffset = 0, int target = MI_TARGET_BARRIER)
refactor: split CExpertSignalAIBase implementation by responsibility ExpertSignalAIBase.mqh was 8216 lines: the class declaration followed by 87 method bodies covering training, labelling, feature extraction, persistence, chart drawing, online learning, the GA auto-tuner and inference, all in one file. Train() alone is 1492 lines; a change to arrow drawing meant scrolling past the era loop. Moved the bodies into Expert\AIBase\, included at the bottom of the original after the class declaration: Training.mqh 1607 era loop, plateau ladder, checkpoint select, deploy Features.mqh 1093 indicator creation + per-bar input feature vector ChartUI.mqh 634 arrows, arrow persistence, status panel, cleanup Persistence.mqh 492 .stats/.cfg sidecars, CPU-inference validation, copy OnlineLearning.mqh 461 live continual learning, EMA shadow, OOS simulator Labels.mqh 309 ZigZag pivot labels, async label-cache prebuild AutoTune.mqh 275 genetic tuner (population, crossover, halving) Inference.mqh 235 softmax, prior calibration, class priors ExpertSignalAIBase.mqh 8216 -> 3131 (declaration + topology build only) This is a pure relocation - verified mechanically, not by eye: HEAD's file reconstructed from the eight partials plus the surviving remainder is byte-identical to HEAD, span for span (scratchpad verify_split.py). No declaration moved, no signature changed, no code rewritten, so behaviour is unchanged by construction. Compiles 0 errors, 0 warnings. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-29 00:42:45 -04:00
{
diag: is "optimal SL/TP" learnable? Score the features against excursions Proposed direction: train the net to predict entry/SL/TP that maximise return and minimise drawdown, rather than to classify direction. Before rebuilding a head, measure whether the target is learnable at all. That question splits into two that behave nothing alike: HOW FAR price travels (MFE/MAE) - essentially volatility, and volatility clustering is about the most robust regularity in markets. WHICH WAY it goes first (the asymmetry) - direction, which is what every noise-floor verdict in this project has been about. Expectancy comes ONLY from the second. The first buys position sizing and drawdown control - worth having under prop-firm limits, but not an edge: exit management on RANDOM entries already moved the payoff ratio 0.92 -> 5.72 with expectancy FLAT. Crucially this is NOT already answered. Every MI figure here scored the triple-barrier label, i.e. one specific question at one fixed geometry. A noise-floor result there says nothing about whether excursion MAGNITUDE is learnable - different target, different answer. Four targets, and the verdict is the CONTRAST, printed explicitly because the dangerous misreading of "UP clears" is "we can predict profitable trades": RANGE (up+dn) - realised volatility, included as a POSITIVE CONTROL that SHOULD clear. Every prior verdict here lacked a control expected to pass; a range target at the floor indicts the measurement, not the market. UP / DOWN - MFE / MAE. ASYMMETRY - up-dn, the only one that can pay. Collected inside the walk the label already does (one max, one min per bar). The early-out when both barriers resolved is GONE: it would have truncated the excursions at whichever bar tripped the last barrier, making the measurement a function of the CURRENT SL/TP - the circularity this is trying to escape. The loop was already bounded by the horizon, so only the average cost moves. Discretised into 3 EQUAL-FREQUENCY bins, so every downstream piece (block permutation, null, p-value) is reused unchanged. Equal-frequency because MFE is fat-tailed and fixed-width bins would put nearly every row in bin 0; it also pins H(Y) at ln(3)=1.099 for all four, making them comparable to each other and to the barrier label's ~1.02 instead of confounded by class balance. Two bugs fixed in this code before it ever ran, both of which would have produced a plausible quiet wrong answer rather than an error: - TripleBarrierLabel early-returns on invalid ATR/close BEFORE the point the accumulators were reset, so one bar's excursions would be cached under another bar's index. Cleared at the top now, ahead of every return. - An unresolvable bar is still flagged as labelled but carries excursions of exactly 0. Under equal-frequency binning a block of identical zeros drags the lowest cut onto zero and a third of the sample lands in one uninformative bin - a depressed score that reads as "not predictable", a false negative in the direction that would wrongly kill the idea. Rows where both excursions are zero are dropped; price cannot travel zero both ways over a whole horizon. Read-only diagnostic. No topology or label change: no retrain of its own. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-07 10:22:41 -04:00
//--- Continuous targets are collected raw here and discretised after the loop, because equal-frequency
//--- binning needs the whole sample's distribution before any one row can be assigned a bin.
double raw[];
bool continuousTarget = (target != MI_TARGET_BARRIER);
perf(autotune): replace the genetic search with a filter score - hours to seconds MEASURED COST OF THE GA, which is what retired it. Per generation: rung 0: 8 cand x 3 seeds x 3 eras = 72 eras rung 1: 4 cand x 3 seeds x 8 eras = 96 rung 2: 2 cand x 3 seeds x 20 eras = 120 = 288 eras/generation x 4 generations = 1152 eras BEFORE the winner's real training began. Against the observed era times on SP500 H1: PAI 29.1 s/era -> 9.3 h (matches the observed 00:37 -> 09:22) CONV 41.3 s/era -> 13.2 h LSTM 150.4 s/era -> 48.1 h HYBRID 154.6 s/era -> 49.5 h Two days to tune is not a first-run experience, and it is the phase in which the panel goes quiet, which is what made it look like a hang. It also bought nothing. The space is 90 points (10 MA periods x 9 MA types), so 1152 evaluations revisited each point ~13 times; and rungs of 3 and 8 eras cannot separate two MA periods at all. The 2026-08-01 run proves it: every finalist scored 25.0-25.9% balanced accuracy - below the 33.3% one-class floor, i.e. indistinguishable noise - and the search then "deployed the winner" of that. THE ERROR WAS THE SCORING FUNCTION, not its constants. Using a full training run to choose a feature's period is a wrapper method paying wrapper prices for a decision that does not need one. The reference book does not do this: ch. 3.3 selects inputs by measuring each candidate indicator's CORRELATION with the target and dropping the ones with none, with no network involved. So: rank candidates by the MUTUAL INFORMATION between the resulting feature vector and the triple-barrier label. MI rather than correlation because the label is 3-class categorical and the features are not monotonically related to it. Equal-FREQUENCY binning (rank-based), because these features are ATR-normalised and heavy-tailed - fixed-width bins put nearly everything in one bucket and report ~0 information for a genuinely useful feature. Scoring is arithmetic over the feature cache, so it costs seconds and its cost is independent of topology: LSTM now tunes as fast as the MLP. Coordinate sweep, not product sweep - cost is the SUM of per-parameter candidate counts, so enabling every indicator stays affordable - with a second pass that breaks early once nothing moves. Sampling is IS-ONLY. Letting the OOS window influence which indicator settings ship would mean the holdout had been used for selection and had stopped being a holdout. HONEST LIMIT, recorded because it is the price: MI is marginal, so a parameter that only pays off in combination with another can be missed (Guyon & Elisseeff 2003, filter vs wrapper). Given the wrapper it replaces was ranking pure noise at 48 h a run, this is strictly better. Deleted with it: GaRungEras/GaExtract/GaStore/GaMutate/GaRandomCandidate/ GaBlockCrossover/GaSortAliveByScoreDesc/GaBreedNextGeneration, 14 m_ga* members, the GA_*/TUNE_POP_* constants, and ComputeTuneTrialBudget. AND m_evalMode/m_evalEraBudget, because nothing set them any more - 28 read sites all permanently inert. That is not a tidy-up: the `if (!m_evalMode)` guard on UpdateClassPriors is exactly what silently disabled the imbalance correction for entire runs two commits ago. Dead machinery that still reads like live machinery is this codebase's most expensive recurring bug, and leaving 28 more instances of it would have been indefensible. The panel's tuning-progress state goes too - tuning no longer takes long enough to need one. Both builds compile 0 errors / 0 warnings. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-01 10:29:31 -04:00
int bars = m_labelCacheBars;
if(bars <= 0 || m_neuronsCount <= 0)
diag(autotune): five permutations was still a coin flip - use a real test The 5-draw z-score shipped an hour ago disproved itself on its first run. All four charts scored the IDENTICAL 0.00401 nats on identical features and identical labels - and reported z of +1.3, +2.0, +4.0 and +4.7. Two "AT THE NOISE FLOOR", two "a real association", same data. The entire swing came from estimating the null's spread from five draws, where the standard deviation of the standard-deviation estimate is ~35%: the denominator was noisier than the effect it was judging. Replaced with an empirical permutation test. 200 draws, p counted by rank with the +1/(B+1) correction (Phipson & Smyth 2010) so p is never reported as exactly zero - no normality assumption and no spread to estimate. The strongest single column is tested against the null distribution OF THE MAXIMUM, which corrects for scoring 26 features at once by construction and is far less conservative than Bonferroni. Affordable because BuildMiSample is now split out of ScoreCurrentParamsByMI and runs ONCE for the whole test - every draw reuses that sample and costs a relabel plus 26 histogram passes, not 2000 feature extractions. The coordinate sweep still calls the combined form, which is correct there: each candidate changes the indicator settings, so its features really do have to be re-extracted. The verdict line keeps both questions apart and prints both answers: the p-value for "is it real", the excess as a percentage of H(Y) for "is it big enough to trade". At n=2000 those can disagree, and collapsing them into one word is how a worthless effect gets called a discovery. Compiles 0 errors / 0 warnings. Build tag permtest-v1. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-01 13:45:46 -04:00
return -1;
perf(autotune): replace the genetic search with a filter score - hours to seconds MEASURED COST OF THE GA, which is what retired it. Per generation: rung 0: 8 cand x 3 seeds x 3 eras = 72 eras rung 1: 4 cand x 3 seeds x 8 eras = 96 rung 2: 2 cand x 3 seeds x 20 eras = 120 = 288 eras/generation x 4 generations = 1152 eras BEFORE the winner's real training began. Against the observed era times on SP500 H1: PAI 29.1 s/era -> 9.3 h (matches the observed 00:37 -> 09:22) CONV 41.3 s/era -> 13.2 h LSTM 150.4 s/era -> 48.1 h HYBRID 154.6 s/era -> 49.5 h Two days to tune is not a first-run experience, and it is the phase in which the panel goes quiet, which is what made it look like a hang. It also bought nothing. The space is 90 points (10 MA periods x 9 MA types), so 1152 evaluations revisited each point ~13 times; and rungs of 3 and 8 eras cannot separate two MA periods at all. The 2026-08-01 run proves it: every finalist scored 25.0-25.9% balanced accuracy - below the 33.3% one-class floor, i.e. indistinguishable noise - and the search then "deployed the winner" of that. THE ERROR WAS THE SCORING FUNCTION, not its constants. Using a full training run to choose a feature's period is a wrapper method paying wrapper prices for a decision that does not need one. The reference book does not do this: ch. 3.3 selects inputs by measuring each candidate indicator's CORRELATION with the target and dropping the ones with none, with no network involved. So: rank candidates by the MUTUAL INFORMATION between the resulting feature vector and the triple-barrier label. MI rather than correlation because the label is 3-class categorical and the features are not monotonically related to it. Equal-FREQUENCY binning (rank-based), because these features are ATR-normalised and heavy-tailed - fixed-width bins put nearly everything in one bucket and report ~0 information for a genuinely useful feature. Scoring is arithmetic over the feature cache, so it costs seconds and its cost is independent of topology: LSTM now tunes as fast as the MLP. Coordinate sweep, not product sweep - cost is the SUM of per-parameter candidate counts, so enabling every indicator stays affordable - with a second pass that breaks early once nothing moves. Sampling is IS-ONLY. Letting the OOS window influence which indicator settings ship would mean the holdout had been used for selection and had stopped being a holdout. HONEST LIMIT, recorded because it is the price: MI is marginal, so a parameter that only pays off in combination with another can be missed (Guyon & Elisseeff 2003, filter vs wrapper). Given the wrapper it replaces was ranking pure noise at 48 h a run, this is strictly better. Deleted with it: GaRungEras/GaExtract/GaStore/GaMutate/GaRandomCandidate/ GaBlockCrossover/GaSortAliveByScoreDesc/GaBreedNextGeneration, 14 m_ga* members, the GA_*/TUNE_POP_* constants, and ComputeTuneTrialBudget. AND m_evalMode/m_evalEraBudget, because nothing set them any more - 28 read sites all permanently inert. That is not a tidy-up: the `if (!m_evalMode)` guard on UpdateClassPriors is exactly what silently disabled the imbalance correction for entire runs two commits ago. Dead machinery that still reads like live machinery is this codebase's most expensive recurring bug, and leaving 28 more instances of it would have been indefensible. The panel's tuning-progress state goes too - tuning no longer takes long enough to need one. Both builds compile 0 errors / 0 warnings. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-01 10:29:31 -04:00
//--- Sample the IS region only. The OOS window must not influence which indicator settings ship, or
//--- the holdout has been used for selection and stops being a holdout at all.
int oosCutoff = (int)(MathMax(0, MathMin(100, m_oosSplitPct)) / 100.0
* MathMax(bars - MathMax(m_historyBars, 0), 0));
int lo = MathMax(oosCutoff, MathMax(m_barrierHorizonBars, 1) + 1);
int hi = bars - MathMax(m_historyBars, 0) - 1;
refactor(comments): stdlib comment style across the remaining in-scope files Same pass as 0b06f8e, applied file by file: comment runs of 4+ lines compressed to their leading topic sentences, capped at 4 lines, whole sentences only. Warning sentences (NEVER / MUST / trap / would-have) survive the budget. Every file was checked the same way before committing: the list of non-comment lines is byte-identical to HEAD, and braces balance. No code was touched. Panel/, Enumerations/ and the already-terse System headers needed little or nothing - PooledGate, TradeChecks, BinomialStats and Random came through with no blocks over the threshold at all. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-22 00:25:52 -04:00
//--- Keep the OFFSET label lookup inside the same bounds as the features, so a shifted scan
//--- measures a shift and not an edge effect. THE PAD IS FIXED, NOT |labelBarOffset|.
fix: correct MI sample alignment and improve BN weight diagnostic report The MI sample builder used `MathAbs(labelBarOffset)` as a padding, causing rows from offset and non-offset builds to be paired with a double shift. This broke the positive control, failed the 5× gate, and voided all reported mutual‑information figures. Replace with the fixed `MiShiftPad` constant to ensure builds enumerate the same set of bars and row-k alignment is preserved. Add `BatchOptionsTotal()` to `CNeuronBatchNormOCL` and split the packed BN weight array in the learning report into separate norms for the outgoing dense matrix, gamma, beta, running statistics, and Adam moment buffers. This turns an ambiguous single‑norm reading into precise diagnostics that distinguish weight divergence from scaling issues.
2026-08-02 08:12:47 -04:00
int shiftPad = MiShiftPad();
diag: MI feature-lag profile - close the blind spot in every MI verdict so far BuildMiSample samples features from ONE bar. So every "MI is at the noise floor" result this codebase has produced - including yesterday's p=0.18 on SP500 H1 - described the ENTRY BAR's 31 features only, while the network is fed 20 bars of them. If information lived at lag 7 and not lag 0, the report would have said "no signal" while the model could still learn. The diagnostic we have been making decisions on had a blind spot exactly the width of the input vector. Adds a FEATURE-side offset to BuildMiSample, which is not the same thing as the existing labelBarOffset and is not interchangeable with it. Shifting the LABEL changes which trade is predicted, so at any non-zero offset the features sit inside the labelled window and the score is lookahead - that is precisely what the alignment scan measures and correctly reports (4.7x more knowable 5 bars into a 128-bar window). Shifting the FEATURES keeps the label pinned to the entry bar, so every row stays causal. ReportFeatureLagProfile() then scores k = 0..historyBars against the same block-permutation null and reports the deepest lag that clears it - the lookback the data supports, versus the 20 that was picked by hand and never measured. The null is redrawn PER LAG: finite-sample MI bias moves with the realised class counts and bin occupancy, and different rows survive the validity checks at each lag, so one shared floor would be right for lag 0 and wrong everywhere else. Draw count is reduced accordingly (40, not 200) since cost is draws x historyBars; this figure decides a lookback, never a trade. MiShiftPad now also covers historyBars, keeping the fixed-pad invariant that makes two builds comparable row by row. Read-only - no input, topology or label change, so no retrain. Both builds 0/0. Build tag lag-profile-v1. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-06 13:51:39 -04:00
if(MathAbs(labelBarOffset) > shiftPad || MathAbs(featureBarOffset) > shiftPad)
fix: correct MI sample alignment and improve BN weight diagnostic report The MI sample builder used `MathAbs(labelBarOffset)` as a padding, causing rows from offset and non-offset builds to be paired with a double shift. This broke the positive control, failed the 5× gate, and voided all reported mutual‑information figures. Replace with the fixed `MiShiftPad` constant to ensure builds enumerate the same set of bars and row-k alignment is preserved. Add `BatchOptionsTotal()` to `CNeuronBatchNormOCL` and split the packed BN weight array in the learning report into separate norms for the outgoing dense matrix, gamma, beta, running statistics, and Adam moment buffers. This turns an ambiguous single‑norm reading into precise diagnostics that distinguish weight divergence from scaling issues.
2026-08-02 08:12:47 -04:00
return -1; // caller asked for a shift the pad does not cover
diag(autotune): a positive control, and a scan that separates "no signal" from "signal knocked out of step" Four architecturally different networks landed on the same precision - Buy 23-25% against a 25.4% base rate, Sell 19-22% against 22.0% - while making completely different calls (HYBRID votes Sell on 69% of bars, PAI on 41%). Precision equal to the base rate is what INDEPENDENCE looks like, and precision under independence is fixed by the label distribution, not by the architecture, so all four converging on it is arithmetic rather than coincidence. Accuracy meanwhile tracks coverage exactly as independence predicts (31.1/30.3/25.0 predicted vs 31.8/28.9/24.6 observed for PAI/CONV/HYB). But "no information in the data" and "information destroyed upstream of every topology" produce that identical picture, and the MI test alone cannot tell them apart either. Two additions: POSITIVE CONTROL. Three "measurements" in this codebase have turned out to be silent no-ops that produced plausible numbers - the MI scorer reading an array nobody filled, the eval-mode guard that switched off the imbalance correction, the alternation gate whose premise was never true. So the estimator now has to prove it responds to a signal known to be present before any floor reading is believed: the label of a neighbouring sample row, ~19 bars away and far inside the 128-bar barrier horizon, so the two outcome windows overlap heavily and MUST be associated. Same binning, same estimator. Near the floor => every MI figure is void. ALIGNMENT SCAN. Re-scores against the label taken from bar i+k for k in -5..+5. A peak at k != 0 is a feature/label misalignment - an off-by-one in the label index, a horizon applied to the wrong bar, a feature window that lags what it claims - which would destroy the information before any topology saw it and would look identical in every accuracy number this EA prints. A flat profile says the features simply do not carry this target. The sampled range is trimmed by |k| at both ends so a shift is measured rather than an edge effect, and both bars must carry a real label. Also: BuildMiSample publishes its stride instead of the report recomputing that arithmetic (it would drift), and the control sizes its buffers from its own sample count rather than the caller's. Compiles 0 errors / 0 warnings, standard and Market. Build tag mi-control-align-v1. Redeploy only - no retrain, no model deletion; the diagnostic runs on resumed models. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-01 14:12:10 -04:00
lo += shiftPad;
hi -= shiftPad;
perf(autotune): replace the genetic search with a filter score - hours to seconds MEASURED COST OF THE GA, which is what retired it. Per generation: rung 0: 8 cand x 3 seeds x 3 eras = 72 eras rung 1: 4 cand x 3 seeds x 8 eras = 96 rung 2: 2 cand x 3 seeds x 20 eras = 120 = 288 eras/generation x 4 generations = 1152 eras BEFORE the winner's real training began. Against the observed era times on SP500 H1: PAI 29.1 s/era -> 9.3 h (matches the observed 00:37 -> 09:22) CONV 41.3 s/era -> 13.2 h LSTM 150.4 s/era -> 48.1 h HYBRID 154.6 s/era -> 49.5 h Two days to tune is not a first-run experience, and it is the phase in which the panel goes quiet, which is what made it look like a hang. It also bought nothing. The space is 90 points (10 MA periods x 9 MA types), so 1152 evaluations revisited each point ~13 times; and rungs of 3 and 8 eras cannot separate two MA periods at all. The 2026-08-01 run proves it: every finalist scored 25.0-25.9% balanced accuracy - below the 33.3% one-class floor, i.e. indistinguishable noise - and the search then "deployed the winner" of that. THE ERROR WAS THE SCORING FUNCTION, not its constants. Using a full training run to choose a feature's period is a wrapper method paying wrapper prices for a decision that does not need one. The reference book does not do this: ch. 3.3 selects inputs by measuring each candidate indicator's CORRELATION with the target and dropping the ones with none, with no network involved. So: rank candidates by the MUTUAL INFORMATION between the resulting feature vector and the triple-barrier label. MI rather than correlation because the label is 3-class categorical and the features are not monotonically related to it. Equal-FREQUENCY binning (rank-based), because these features are ATR-normalised and heavy-tailed - fixed-width bins put nearly everything in one bucket and report ~0 information for a genuinely useful feature. Scoring is arithmetic over the feature cache, so it costs seconds and its cost is independent of topology: LSTM now tunes as fast as the MLP. Coordinate sweep, not product sweep - cost is the SUM of per-parameter candidate counts, so enabling every indicator stays affordable - with a second pass that breaks early once nothing moves. Sampling is IS-ONLY. Letting the OOS window influence which indicator settings ship would mean the holdout had been used for selection and had stopped being a holdout. HONEST LIMIT, recorded because it is the price: MI is marginal, so a parameter that only pays off in combination with another can be missed (Guyon & Elisseeff 2003, filter vs wrapper). Given the wrapper it replaces was ranking pure noise at 48 h a run, this is strictly better. Deleted with it: GaRungEras/GaExtract/GaStore/GaMutate/GaRandomCandidate/ GaBlockCrossover/GaSortAliveByScoreDesc/GaBreedNextGeneration, 14 m_ga* members, the GA_*/TUNE_POP_* constants, and ComputeTuneTrialBudget. AND m_evalMode/m_evalEraBudget, because nothing set them any more - 28 read sites all permanently inert. That is not a tidy-up: the `if (!m_evalMode)` guard on UpdateClassPriors is exactly what silently disabled the imbalance correction for entire runs two commits ago. Dead machinery that still reads like live machinery is this codebase's most expensive recurring bug, and leaving 28 more instances of it would have been indefensible. The panel's tuning-progress state goes too - tuning no longer takes long enough to need one. Both builds compile 0 errors / 0 warnings. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-01 10:29:31 -04:00
if(hi - lo < MI_MIN_SAMPLES)
diag(autotune): five permutations was still a coin flip - use a real test The 5-draw z-score shipped an hour ago disproved itself on its first run. All four charts scored the IDENTICAL 0.00401 nats on identical features and identical labels - and reported z of +1.3, +2.0, +4.0 and +4.7. Two "AT THE NOISE FLOOR", two "a real association", same data. The entire swing came from estimating the null's spread from five draws, where the standard deviation of the standard-deviation estimate is ~35%: the denominator was noisier than the effect it was judging. Replaced with an empirical permutation test. 200 draws, p counted by rank with the +1/(B+1) correction (Phipson & Smyth 2010) so p is never reported as exactly zero - no normality assumption and no spread to estimate. The strongest single column is tested against the null distribution OF THE MAXIMUM, which corrects for scoring 26 features at once by construction and is far less conservative than Bonferroni. Affordable because BuildMiSample is now split out of ScoreCurrentParamsByMI and runs ONCE for the whole test - every draw reuses that sample and costs a relabel plus 26 histogram passes, not 2000 feature extractions. The coordinate sweep still calls the combined form, which is correct there: each candidate changes the indicator settings, so its features really do have to be re-extracted. The verdict line keeps both questions apart and prints both answers: the p-value for "is it real", the excess as a percentage of H(Y) for "is it big enough to trade". At n=2000 those can disagree, and collapsing them into one word is how a worthless effect gets called a discovery. Compiles 0 errors / 0 warnings. Build tag permtest-v1. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-01 13:45:46 -04:00
return -1;
perf(autotune): replace the genetic search with a filter score - hours to seconds MEASURED COST OF THE GA, which is what retired it. Per generation: rung 0: 8 cand x 3 seeds x 3 eras = 72 eras rung 1: 4 cand x 3 seeds x 8 eras = 96 rung 2: 2 cand x 3 seeds x 20 eras = 120 = 288 eras/generation x 4 generations = 1152 eras BEFORE the winner's real training began. Against the observed era times on SP500 H1: PAI 29.1 s/era -> 9.3 h (matches the observed 00:37 -> 09:22) CONV 41.3 s/era -> 13.2 h LSTM 150.4 s/era -> 48.1 h HYBRID 154.6 s/era -> 49.5 h Two days to tune is not a first-run experience, and it is the phase in which the panel goes quiet, which is what made it look like a hang. It also bought nothing. The space is 90 points (10 MA periods x 9 MA types), so 1152 evaluations revisited each point ~13 times; and rungs of 3 and 8 eras cannot separate two MA periods at all. The 2026-08-01 run proves it: every finalist scored 25.0-25.9% balanced accuracy - below the 33.3% one-class floor, i.e. indistinguishable noise - and the search then "deployed the winner" of that. THE ERROR WAS THE SCORING FUNCTION, not its constants. Using a full training run to choose a feature's period is a wrapper method paying wrapper prices for a decision that does not need one. The reference book does not do this: ch. 3.3 selects inputs by measuring each candidate indicator's CORRELATION with the target and dropping the ones with none, with no network involved. So: rank candidates by the MUTUAL INFORMATION between the resulting feature vector and the triple-barrier label. MI rather than correlation because the label is 3-class categorical and the features are not monotonically related to it. Equal-FREQUENCY binning (rank-based), because these features are ATR-normalised and heavy-tailed - fixed-width bins put nearly everything in one bucket and report ~0 information for a genuinely useful feature. Scoring is arithmetic over the feature cache, so it costs seconds and its cost is independent of topology: LSTM now tunes as fast as the MLP. Coordinate sweep, not product sweep - cost is the SUM of per-parameter candidate counts, so enabling every indicator stays affordable - with a second pass that breaks early once nothing moves. Sampling is IS-ONLY. Letting the OOS window influence which indicator settings ship would mean the holdout had been used for selection and had stopped being a holdout. HONEST LIMIT, recorded because it is the price: MI is marginal, so a parameter that only pays off in combination with another can be missed (Guyon & Elisseeff 2003, filter vs wrapper). Given the wrapper it replaces was ranking pure noise at 48 h a run, this is strictly better. Deleted with it: GaRungEras/GaExtract/GaStore/GaMutate/GaRandomCandidate/ GaBlockCrossover/GaSortAliveByScoreDesc/GaBreedNextGeneration, 14 m_ga* members, the GA_*/TUNE_POP_* constants, and ComputeTuneTrialBudget. AND m_evalMode/m_evalEraBudget, because nothing set them any more - 28 read sites all permanently inert. That is not a tidy-up: the `if (!m_evalMode)` guard on UpdateClassPriors is exactly what silently disabled the imbalance correction for entire runs two commits ago. Dead machinery that still reads like live machinery is this codebase's most expensive recurring bug, and leaving 28 more instances of it would have been indefensible. The panel's tuning-progress state goes too - tuning no longer takes long enough to need one. Both builds compile 0 errors / 0 warnings. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-01 10:29:31 -04:00
int stride = (int)MathMax(1, (hi - lo) / MI_SAMPLE_BARS);
diag(autotune): a positive control, and a scan that separates "no signal" from "signal knocked out of step" Four architecturally different networks landed on the same precision - Buy 23-25% against a 25.4% base rate, Sell 19-22% against 22.0% - while making completely different calls (HYBRID votes Sell on 69% of bars, PAI on 41%). Precision equal to the base rate is what INDEPENDENCE looks like, and precision under independence is fixed by the label distribution, not by the architecture, so all four converging on it is arithmetic rather than coincidence. Accuracy meanwhile tracks coverage exactly as independence predicts (31.1/30.3/25.0 predicted vs 31.8/28.9/24.6 observed for PAI/CONV/HYB). But "no information in the data" and "information destroyed upstream of every topology" produce that identical picture, and the MI test alone cannot tell them apart either. Two additions: POSITIVE CONTROL. Three "measurements" in this codebase have turned out to be silent no-ops that produced plausible numbers - the MI scorer reading an array nobody filled, the eval-mode guard that switched off the imbalance correction, the alternation gate whose premise was never true. So the estimator now has to prove it responds to a signal known to be present before any floor reading is believed: the label of a neighbouring sample row, ~19 bars away and far inside the 128-bar barrier horizon, so the two outcome windows overlap heavily and MUST be associated. Same binning, same estimator. Near the floor => every MI figure is void. ALIGNMENT SCAN. Re-scores against the label taken from bar i+k for k in -5..+5. A peak at k != 0 is a feature/label misalignment - an off-by-one in the label index, a horizon applied to the wrong bar, a feature window that lags what it claims - which would destroy the information before any topology saw it and would look identical in every accuracy number this EA prints. A flat profile says the features simply do not carry this target. The sampled range is trimmed by |k| at both ends so a shift is measured rather than an edge effect, and both bars must carry a real label. Also: BuildMiSample publishes its stride instead of the report recomputing that arithmetic (it would drift), and the control sizes its buffers from its own sample count rather than the caller's. Compiles 0 errors / 0 warnings, standard and Market. Build tag mi-control-align-v1. Redeploy only - no retrain, no model deletion; the diagnostic runs on resumed models. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-01 14:12:10 -04:00
//--- Published so the positive control can say how many BARS apart two sample rows are without
//--- recomputing this arithmetic at the call site, where it would silently drift out of agreement.
m_miStrideBars = stride;
perf(autotune): replace the genetic search with a filter score - hours to seconds MEASURED COST OF THE GA, which is what retired it. Per generation: rung 0: 8 cand x 3 seeds x 3 eras = 72 eras rung 1: 4 cand x 3 seeds x 8 eras = 96 rung 2: 2 cand x 3 seeds x 20 eras = 120 = 288 eras/generation x 4 generations = 1152 eras BEFORE the winner's real training began. Against the observed era times on SP500 H1: PAI 29.1 s/era -> 9.3 h (matches the observed 00:37 -> 09:22) CONV 41.3 s/era -> 13.2 h LSTM 150.4 s/era -> 48.1 h HYBRID 154.6 s/era -> 49.5 h Two days to tune is not a first-run experience, and it is the phase in which the panel goes quiet, which is what made it look like a hang. It also bought nothing. The space is 90 points (10 MA periods x 9 MA types), so 1152 evaluations revisited each point ~13 times; and rungs of 3 and 8 eras cannot separate two MA periods at all. The 2026-08-01 run proves it: every finalist scored 25.0-25.9% balanced accuracy - below the 33.3% one-class floor, i.e. indistinguishable noise - and the search then "deployed the winner" of that. THE ERROR WAS THE SCORING FUNCTION, not its constants. Using a full training run to choose a feature's period is a wrapper method paying wrapper prices for a decision that does not need one. The reference book does not do this: ch. 3.3 selects inputs by measuring each candidate indicator's CORRELATION with the target and dropping the ones with none, with no network involved. So: rank candidates by the MUTUAL INFORMATION between the resulting feature vector and the triple-barrier label. MI rather than correlation because the label is 3-class categorical and the features are not monotonically related to it. Equal-FREQUENCY binning (rank-based), because these features are ATR-normalised and heavy-tailed - fixed-width bins put nearly everything in one bucket and report ~0 information for a genuinely useful feature. Scoring is arithmetic over the feature cache, so it costs seconds and its cost is independent of topology: LSTM now tunes as fast as the MLP. Coordinate sweep, not product sweep - cost is the SUM of per-parameter candidate counts, so enabling every indicator stays affordable - with a second pass that breaks early once nothing moves. Sampling is IS-ONLY. Letting the OOS window influence which indicator settings ship would mean the holdout had been used for selection and had stopped being a holdout. HONEST LIMIT, recorded because it is the price: MI is marginal, so a parameter that only pays off in combination with another can be missed (Guyon & Elisseeff 2003, filter vs wrapper). Given the wrapper it replaces was ranking pure noise at 48 h a run, this is strictly better. Deleted with it: GaRungEras/GaExtract/GaStore/GaMutate/GaRandomCandidate/ GaBlockCrossover/GaSortAliveByScoreDesc/GaBreedNextGeneration, 14 m_ga* members, the GA_*/TUNE_POP_* constants, and ComputeTuneTrialBudget. AND m_evalMode/m_evalEraBudget, because nothing set them any more - 28 read sites all permanently inert. That is not a tidy-up: the `if (!m_evalMode)` guard on UpdateClassPriors is exactly what silently disabled the imbalance correction for entire runs two commits ago. Dead machinery that still reads like live machinery is this codebase's most expensive recurring bug, and leaving 28 more instances of it would have been indefensible. The panel's tuning-progress state goes too - tuning no longer takes long enough to need one. Both builds compile 0 errors / 0 warnings. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-01 10:29:31 -04:00
int cap = (hi - lo) / stride + 1;
ArrayResize(cols, cap * m_neuronsCount);
ArrayResize(labels, cap);
diag: is "optimal SL/TP" learnable? Score the features against excursions Proposed direction: train the net to predict entry/SL/TP that maximise return and minimise drawdown, rather than to classify direction. Before rebuilding a head, measure whether the target is learnable at all. That question splits into two that behave nothing alike: HOW FAR price travels (MFE/MAE) - essentially volatility, and volatility clustering is about the most robust regularity in markets. WHICH WAY it goes first (the asymmetry) - direction, which is what every noise-floor verdict in this project has been about. Expectancy comes ONLY from the second. The first buys position sizing and drawdown control - worth having under prop-firm limits, but not an edge: exit management on RANDOM entries already moved the payoff ratio 0.92 -> 5.72 with expectancy FLAT. Crucially this is NOT already answered. Every MI figure here scored the triple-barrier label, i.e. one specific question at one fixed geometry. A noise-floor result there says nothing about whether excursion MAGNITUDE is learnable - different target, different answer. Four targets, and the verdict is the CONTRAST, printed explicitly because the dangerous misreading of "UP clears" is "we can predict profitable trades": RANGE (up+dn) - realised volatility, included as a POSITIVE CONTROL that SHOULD clear. Every prior verdict here lacked a control expected to pass; a range target at the floor indicts the measurement, not the market. UP / DOWN - MFE / MAE. ASYMMETRY - up-dn, the only one that can pay. Collected inside the walk the label already does (one max, one min per bar). The early-out when both barriers resolved is GONE: it would have truncated the excursions at whichever bar tripped the last barrier, making the measurement a function of the CURRENT SL/TP - the circularity this is trying to escape. The loop was already bounded by the horizon, so only the average cost moves. Discretised into 3 EQUAL-FREQUENCY bins, so every downstream piece (block permutation, null, p-value) is reused unchanged. Equal-frequency because MFE is fat-tailed and fixed-width bins would put nearly every row in bin 0; it also pins H(Y) at ln(3)=1.099 for all four, making them comparable to each other and to the barrier label's ~1.02 instead of confounded by class balance. Two bugs fixed in this code before it ever ran, both of which would have produced a plausible quiet wrong answer rather than an error: - TripleBarrierLabel early-returns on invalid ATR/close BEFORE the point the accumulators were reset, so one bar's excursions would be cached under another bar's index. Cleared at the top now, ahead of every return. - An unresolvable bar is still flagged as labelled but carries excursions of exactly 0. Under equal-frequency binning a block of identical zeros drags the lowest cut onto zero and a third of the sample lands in one uninformative bin - a depressed score that reads as "not predictable", a false negative in the direction that would wrongly kill the idea. Rows where both excursions are zero are dropped; price cannot travel zero both ways over a whole horizon. Read-only diagnostic. No topology or label change: no retrain of its own. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-07 10:22:41 -04:00
if(continuousTarget)
ArrayResize(raw, cap);
perf(autotune): replace the genetic search with a filter score - hours to seconds MEASURED COST OF THE GA, which is what retired it. Per generation: rung 0: 8 cand x 3 seeds x 3 eras = 72 eras rung 1: 4 cand x 3 seeds x 8 eras = 96 rung 2: 2 cand x 3 seeds x 20 eras = 120 = 288 eras/generation x 4 generations = 1152 eras BEFORE the winner's real training began. Against the observed era times on SP500 H1: PAI 29.1 s/era -> 9.3 h (matches the observed 00:37 -> 09:22) CONV 41.3 s/era -> 13.2 h LSTM 150.4 s/era -> 48.1 h HYBRID 154.6 s/era -> 49.5 h Two days to tune is not a first-run experience, and it is the phase in which the panel goes quiet, which is what made it look like a hang. It also bought nothing. The space is 90 points (10 MA periods x 9 MA types), so 1152 evaluations revisited each point ~13 times; and rungs of 3 and 8 eras cannot separate two MA periods at all. The 2026-08-01 run proves it: every finalist scored 25.0-25.9% balanced accuracy - below the 33.3% one-class floor, i.e. indistinguishable noise - and the search then "deployed the winner" of that. THE ERROR WAS THE SCORING FUNCTION, not its constants. Using a full training run to choose a feature's period is a wrapper method paying wrapper prices for a decision that does not need one. The reference book does not do this: ch. 3.3 selects inputs by measuring each candidate indicator's CORRELATION with the target and dropping the ones with none, with no network involved. So: rank candidates by the MUTUAL INFORMATION between the resulting feature vector and the triple-barrier label. MI rather than correlation because the label is 3-class categorical and the features are not monotonically related to it. Equal-FREQUENCY binning (rank-based), because these features are ATR-normalised and heavy-tailed - fixed-width bins put nearly everything in one bucket and report ~0 information for a genuinely useful feature. Scoring is arithmetic over the feature cache, so it costs seconds and its cost is independent of topology: LSTM now tunes as fast as the MLP. Coordinate sweep, not product sweep - cost is the SUM of per-parameter candidate counts, so enabling every indicator stays affordable - with a second pass that breaks early once nothing moves. Sampling is IS-ONLY. Letting the OOS window influence which indicator settings ship would mean the holdout had been used for selection and had stopped being a holdout. HONEST LIMIT, recorded because it is the price: MI is marginal, so a parameter that only pays off in combination with another can be missed (Guyon & Elisseeff 2003, filter vs wrapper). Given the wrapper it replaces was ranking pure noise at 48 h a run, this is strictly better. Deleted with it: GaRungEras/GaExtract/GaStore/GaMutate/GaRandomCandidate/ GaBlockCrossover/GaSortAliveByScoreDesc/GaBreedNextGeneration, 14 m_ga* members, the GA_*/TUNE_POP_* constants, and ComputeTuneTrialBudget. AND m_evalMode/m_evalEraBudget, because nothing set them any more - 28 read sites all permanently inert. That is not a tidy-up: the `if (!m_evalMode)` guard on UpdateClassPriors is exactly what silently disabled the imbalance correction for entire runs two commits ago. Dead machinery that still reads like live machinery is this codebase's most expensive recurring bug, and leaving 28 more instances of it would have been indefensible. The panel's tuning-progress state goes too - tuning no longer takes long enough to need one. Both builds compile 0 errors / 0 warnings. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-01 10:29:31 -04:00
int n = 0;
for(int i = lo; i < hi && n < cap; i += stride)
{
diag(autotune): a positive control, and a scan that separates "no signal" from "signal knocked out of step" Four architecturally different networks landed on the same precision - Buy 23-25% against a 25.4% base rate, Sell 19-22% against 22.0% - while making completely different calls (HYBRID votes Sell on 69% of bars, PAI on 41%). Precision equal to the base rate is what INDEPENDENCE looks like, and precision under independence is fixed by the label distribution, not by the architecture, so all four converging on it is arithmetic rather than coincidence. Accuracy meanwhile tracks coverage exactly as independence predicts (31.1/30.3/25.0 predicted vs 31.8/28.9/24.6 observed for PAI/CONV/HYB). But "no information in the data" and "information destroyed upstream of every topology" produce that identical picture, and the MI test alone cannot tell them apart either. Two additions: POSITIVE CONTROL. Three "measurements" in this codebase have turned out to be silent no-ops that produced plausible numbers - the MI scorer reading an array nobody filled, the eval-mode guard that switched off the imbalance correction, the alternation gate whose premise was never true. So the estimator now has to prove it responds to a signal known to be present before any floor reading is believed: the label of a neighbouring sample row, ~19 bars away and far inside the 128-bar barrier horizon, so the two outcome windows overlap heavily and MUST be associated. Same binning, same estimator. Near the floor => every MI figure is void. ALIGNMENT SCAN. Re-scores against the label taken from bar i+k for k in -5..+5. A peak at k != 0 is a feature/label misalignment - an off-by-one in the label index, a horizon applied to the wrong bar, a feature window that lags what it claims - which would destroy the information before any topology saw it and would look identical in every accuracy number this EA prints. A flat profile says the features simply do not carry this target. The sampled range is trimmed by |k| at both ends so a shift is measured rather than an edge effect, and both bars must carry a real label. Also: BuildMiSample publishes its stride instead of the report recomputing that arithmetic (it would drift), and the control sizes its buffers from its own sample count rather than the caller's. Compiles 0 errors / 0 warnings, standard and Market. Build tag mi-control-align-v1. Redeploy only - no retrain, no model deletion; the diagnostic runs on resumed models. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-01 14:12:10 -04:00
//--- Features come from bar i; the LABEL may be taken from a neighbouring bar (labelBarOffset != 0)
//--- so the caller can scan for a feature/label misalignment - see the alignment scan in
//--- ReportFeatureLabelInformation(). Both bars must carry a valid label for the row to count.
int li = i + labelBarOffset;
perf(autotune): replace the genetic search with a filter score - hours to seconds MEASURED COST OF THE GA, which is what retired it. Per generation: rung 0: 8 cand x 3 seeds x 3 eras = 72 eras rung 1: 4 cand x 3 seeds x 8 eras = 96 rung 2: 2 cand x 3 seeds x 20 eras = 120 = 288 eras/generation x 4 generations = 1152 eras BEFORE the winner's real training began. Against the observed era times on SP500 H1: PAI 29.1 s/era -> 9.3 h (matches the observed 00:37 -> 09:22) CONV 41.3 s/era -> 13.2 h LSTM 150.4 s/era -> 48.1 h HYBRID 154.6 s/era -> 49.5 h Two days to tune is not a first-run experience, and it is the phase in which the panel goes quiet, which is what made it look like a hang. It also bought nothing. The space is 90 points (10 MA periods x 9 MA types), so 1152 evaluations revisited each point ~13 times; and rungs of 3 and 8 eras cannot separate two MA periods at all. The 2026-08-01 run proves it: every finalist scored 25.0-25.9% balanced accuracy - below the 33.3% one-class floor, i.e. indistinguishable noise - and the search then "deployed the winner" of that. THE ERROR WAS THE SCORING FUNCTION, not its constants. Using a full training run to choose a feature's period is a wrapper method paying wrapper prices for a decision that does not need one. The reference book does not do this: ch. 3.3 selects inputs by measuring each candidate indicator's CORRELATION with the target and dropping the ones with none, with no network involved. So: rank candidates by the MUTUAL INFORMATION between the resulting feature vector and the triple-barrier label. MI rather than correlation because the label is 3-class categorical and the features are not monotonically related to it. Equal-FREQUENCY binning (rank-based), because these features are ATR-normalised and heavy-tailed - fixed-width bins put nearly everything in one bucket and report ~0 information for a genuinely useful feature. Scoring is arithmetic over the feature cache, so it costs seconds and its cost is independent of topology: LSTM now tunes as fast as the MLP. Coordinate sweep, not product sweep - cost is the SUM of per-parameter candidate counts, so enabling every indicator stays affordable - with a second pass that breaks early once nothing moves. Sampling is IS-ONLY. Letting the OOS window influence which indicator settings ship would mean the holdout had been used for selection and had stopped being a holdout. HONEST LIMIT, recorded because it is the price: MI is marginal, so a parameter that only pays off in combination with another can be missed (Guyon & Elisseeff 2003, filter vs wrapper). Given the wrapper it replaces was ranking pure noise at 48 h a run, this is strictly better. Deleted with it: GaRungEras/GaExtract/GaStore/GaMutate/GaRandomCandidate/ GaBlockCrossover/GaSortAliveByScoreDesc/GaBreedNextGeneration, 14 m_ga* members, the GA_*/TUNE_POP_* constants, and ComputeTuneTrialBudget. AND m_evalMode/m_evalEraBudget, because nothing set them any more - 28 read sites all permanently inert. That is not a tidy-up: the `if (!m_evalMode)` guard on UpdateClassPriors is exactly what silently disabled the imbalance correction for entire runs two commits ago. Dead machinery that still reads like live machinery is this codebase's most expensive recurring bug, and leaving 28 more instances of it would have been indefensible. The panel's tuning-progress state goes too - tuning no longer takes long enough to need one. Both builds compile 0 errors / 0 warnings. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-01 10:29:31 -04:00
if(i >= ArraySize(m_labelCacheHasValue) || !m_labelCacheHasValue[i])
continue;
feat(labels): measure which barrier is predictable at entry, don't guess The alignment scan settled the shape of the problem: 4.7x more is knowable 5 bars into a 128-bar window than at the entry the model actually trades. A 6xATR target reached over 128 bars is decided overwhelmingly by what happens DURING the window, so whatever the entry state knows is buried under 128 bars of later noise. That is a property of the TARGET, and it is why four different architectures all landed on precision exactly equal to the base rate - no topology can undo it. So measure the target. For each SL/TP pairing a user can actually select, relabel the same sampled bars and score how much the SAME features say about THAT outcome at entry. Seconds, no training, no topology, and it runs on the diagnostic path that already exists. Ranked on excess over its OWN null as a share of its OWN H(Y), never on raw nats: each geometry has a different class balance, hence a different finite-sample bias and a different amount of information there to find, so raw MI would rank the most BALANCED label rather than the most PREDICTABLE one. The break-even win rate m/(m+k) is printed beside each so the ranking is read next to the bar the model must clear. Stated in the output because it is the easy thing to get wrong: chance precision EQUALS break-even at every geometry, so a tighter target does not hand you expectancy. It buys predictability - less noise piled on top of what the entry state knows - which is the one thing changing topology cannot do. Read-only by construction: it relabels a sampled copy via TripleBarrierLabel(), never writes the label cache (which belongs to the configured geometry), and restores the horizon and overrides it borrowed. The overrides apply only when BOTH are positive, so a half-set pair can never silently relabel a live run. Compiles 0 errors / 0 warnings, standard and Market. Build tag geometry-scan-v1. Redeploy only - no retrain to READ the ranking. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-01 14:32:04 -04:00
//--- The geometry scan asks "what WOULD this label be under a different barrier?", which by
//--- definition is not in the cache. Compute it on the spot instead - the cache belongs to the
//--- configured geometry and a scan must never write to it.
if(!m_barrierScanLiveLabels && (li < 0 || li >= ArraySize(m_labelCacheHasValue) || !m_labelCacheHasValue[li]))
continue;
if(m_barrierScanLiveLabels && (li < MathMax(m_barrierHorizonBars, 1) || li >= bars))
diag(autotune): a positive control, and a scan that separates "no signal" from "signal knocked out of step" Four architecturally different networks landed on the same precision - Buy 23-25% against a 25.4% base rate, Sell 19-22% against 22.0% - while making completely different calls (HYBRID votes Sell on 69% of bars, PAI on 41%). Precision equal to the base rate is what INDEPENDENCE looks like, and precision under independence is fixed by the label distribution, not by the architecture, so all four converging on it is arithmetic rather than coincidence. Accuracy meanwhile tracks coverage exactly as independence predicts (31.1/30.3/25.0 predicted vs 31.8/28.9/24.6 observed for PAI/CONV/HYB). But "no information in the data" and "information destroyed upstream of every topology" produce that identical picture, and the MI test alone cannot tell them apart either. Two additions: POSITIVE CONTROL. Three "measurements" in this codebase have turned out to be silent no-ops that produced plausible numbers - the MI scorer reading an array nobody filled, the eval-mode guard that switched off the imbalance correction, the alternation gate whose premise was never true. So the estimator now has to prove it responds to a signal known to be present before any floor reading is believed: the label of a neighbouring sample row, ~19 bars away and far inside the 128-bar barrier horizon, so the two outcome windows overlap heavily and MUST be associated. Same binning, same estimator. Near the floor => every MI figure is void. ALIGNMENT SCAN. Re-scores against the label taken from bar i+k for k in -5..+5. A peak at k != 0 is a feature/label misalignment - an off-by-one in the label index, a horizon applied to the wrong bar, a feature window that lags what it claims - which would destroy the information before any topology saw it and would look identical in every accuracy number this EA prints. A flat profile says the features simply do not carry this target. The sampled range is trimmed by |k| at both ends so a shift is measured rather than an edge effect, and both bars must carry a real label. Also: BuildMiSample publishes its stride instead of the report recomputing that arithmetic (it would drift), and the control sizes its buffers from its own sample count rather than the caller's. Compiles 0 errors / 0 warnings, standard and Market. Build tag mi-control-align-v1. Redeploy only - no retrain, no model deletion; the diagnostic runs on resumed models. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-01 14:12:10 -04:00
continue;
refactor(comments): stdlib comment style across the remaining in-scope files Same pass as 0b06f8e, applied file by file: comment runs of 4+ lines compressed to their leading topic sentences, capped at 4 lines, whole sentences only. Warning sentences (NEVER / MUST / trap / would-have) survive the budget. Every file was checked the same way before committing: the list of non-comment lines is byte-identical to HEAD, and braces balance. No code was touched. Panel/, Enumerations/ and the already-terse System headers needed little or nothing - PooledGate, TradeChecks, BinomialStats and Random came through with no blocks over the threshold at all. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-22 00:25:52 -04:00
//--- BufferTempData(), NOT BufferTempDataCompute(). The Compute variant APPENDS the bar's
//--- features to TempData and never touches m_featureCache - only the caching wrapper writes
//--- that array.
fix(autotune): MI scorer read an array nobody filled; add the permutation floor THE TUNER WAS A SILENT NO-OP. Every chart logged auto-tune complete - 17 candidate settings scored in ~139s, feature/label mutual information 0.0000 -> 0.0000 nats (no improvement) 0.0000 is not a weak result, it is a broken measurement: finite-sample MI is biased UPWARD, so even pure noise scores above zero. Cause: ScoreCurrentParamsByMI called BufferTempDataCompute(), which APPENDS the bar's features to TempData and never touches m_featureCache - only the caching wrapper BufferTempData() writes that array. It then read m_featureCache, which ReInitADIndicators had just invalidated. Every column came back constant, FeatureColumnMI returned 0 for all of them, and all 17 candidates tied at exactly zero. 139 s per chart to return the settings it started with. Now reads the values back out of TempData, where they actually land. And an exactly-zero best score is called out as a fault rather than reported as "no improvement", because that is what it is. ADDED: a PERMUTATION BASELINE, which is the diagnostic this project has been missing. MI's finite-sample bias is ~(bins-1)(classes-1)/(2n) nats - at these sample sizes the same order as any real edge in this domain - so a raw MI figure is uninterpretable on its own. Shuffling the labels destroys every genuine association while leaving sample size, binning and class proportions intact, so the score it produces IS this dataset's noise floor, measured rather than approximated. The log now reads feature/label information - X nats against a shuffled-label floor of Y and says outright whether the features carry usable information about the target. It needs no training, no topology and no convergence, so unlike every accuracy number in this codebase it cannot be confounded by an optimizer or an objective. If the score sits on the floor, no change of architecture can help - which is the question the last three days of zero-edge results have been circling. DEPLOY FLOOR: `dirPrecPct > chancePrecPct` passed anything above chance by any amount. At ~11,000 directional calls the standard error of the precision estimate is ~0.4pp, so that gate was accepting sub-one-sigma noise - the perceptron deployed at edge +0pp on 2026-08-01. Now requires EDGE_MIN_SIGMAS (2.0) standard errors above chance, computed from the actual call count, so the bar scales with the evidence instead of needing a hand-picked constant. Recorded with it, because it is why chance is the right reference at all: under a driftless random walk P(touch +k*ATR before -m*ATR) = m/(m+k), and the break-even win rate for a k:m reward:risk trade is ALSO m/(m+k). The label's own base rate IS the break-even rate, at every SL/TP setting. So "beats chance" and "is profitable" are the same test, and no choice of SL/TP can manufacture an edge - only prediction can. Both builds compile 0 errors / 0 warnings. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-01 13:05:50 -04:00
TempData.Clear();
diag: MI feature-lag profile - close the blind spot in every MI verdict so far BuildMiSample samples features from ONE bar. So every "MI is at the noise floor" result this codebase has produced - including yesterday's p=0.18 on SP500 H1 - described the ENTRY BAR's 31 features only, while the network is fed 20 bars of them. If information lived at lag 7 and not lag 0, the report would have said "no signal" while the model could still learn. The diagnostic we have been making decisions on had a blind spot exactly the width of the input vector. Adds a FEATURE-side offset to BuildMiSample, which is not the same thing as the existing labelBarOffset and is not interchangeable with it. Shifting the LABEL changes which trade is predicted, so at any non-zero offset the features sit inside the labelled window and the score is lookahead - that is precisely what the alignment scan measures and correctly reports (4.7x more knowable 5 bars into a 128-bar window). Shifting the FEATURES keeps the label pinned to the entry bar, so every row stays causal. ReportFeatureLagProfile() then scores k = 0..historyBars against the same block-permutation null and reports the deepest lag that clears it - the lookback the data supports, versus the 20 that was picked by hand and never measured. The null is redrawn PER LAG: finite-sample MI bias moves with the realised class counts and bin occupancy, and different rows survive the validity checks at each lag, so one shared floor would be right for lag 0 and wrong everywhere else. Draw count is reduced accordingly (40, not 200) since cost is draws x historyBars; this figure decides a lookback, never a trade. MiShiftPad now also covers historyBars, keeping the fixed-pad invariant that makes two builds comparable row by row. Read-only - no input, topology or label change, so no retrain. Both builds 0/0. Build tag lag-profile-v1. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-06 13:51:39 -04:00
if(!BufferTempData(i + featureBarOffset) || TempData.Total() < m_neuronsCount)
perf(autotune): replace the genetic search with a filter score - hours to seconds MEASURED COST OF THE GA, which is what retired it. Per generation: rung 0: 8 cand x 3 seeds x 3 eras = 72 eras rung 1: 4 cand x 3 seeds x 8 eras = 96 rung 2: 2 cand x 3 seeds x 20 eras = 120 = 288 eras/generation x 4 generations = 1152 eras BEFORE the winner's real training began. Against the observed era times on SP500 H1: PAI 29.1 s/era -> 9.3 h (matches the observed 00:37 -> 09:22) CONV 41.3 s/era -> 13.2 h LSTM 150.4 s/era -> 48.1 h HYBRID 154.6 s/era -> 49.5 h Two days to tune is not a first-run experience, and it is the phase in which the panel goes quiet, which is what made it look like a hang. It also bought nothing. The space is 90 points (10 MA periods x 9 MA types), so 1152 evaluations revisited each point ~13 times; and rungs of 3 and 8 eras cannot separate two MA periods at all. The 2026-08-01 run proves it: every finalist scored 25.0-25.9% balanced accuracy - below the 33.3% one-class floor, i.e. indistinguishable noise - and the search then "deployed the winner" of that. THE ERROR WAS THE SCORING FUNCTION, not its constants. Using a full training run to choose a feature's period is a wrapper method paying wrapper prices for a decision that does not need one. The reference book does not do this: ch. 3.3 selects inputs by measuring each candidate indicator's CORRELATION with the target and dropping the ones with none, with no network involved. So: rank candidates by the MUTUAL INFORMATION between the resulting feature vector and the triple-barrier label. MI rather than correlation because the label is 3-class categorical and the features are not monotonically related to it. Equal-FREQUENCY binning (rank-based), because these features are ATR-normalised and heavy-tailed - fixed-width bins put nearly everything in one bucket and report ~0 information for a genuinely useful feature. Scoring is arithmetic over the feature cache, so it costs seconds and its cost is independent of topology: LSTM now tunes as fast as the MLP. Coordinate sweep, not product sweep - cost is the SUM of per-parameter candidate counts, so enabling every indicator stays affordable - with a second pass that breaks early once nothing moves. Sampling is IS-ONLY. Letting the OOS window influence which indicator settings ship would mean the holdout had been used for selection and had stopped being a holdout. HONEST LIMIT, recorded because it is the price: MI is marginal, so a parameter that only pays off in combination with another can be missed (Guyon & Elisseeff 2003, filter vs wrapper). Given the wrapper it replaces was ranking pure noise at 48 h a run, this is strictly better. Deleted with it: GaRungEras/GaExtract/GaStore/GaMutate/GaRandomCandidate/ GaBlockCrossover/GaSortAliveByScoreDesc/GaBreedNextGeneration, 14 m_ga* members, the GA_*/TUNE_POP_* constants, and ComputeTuneTrialBudget. AND m_evalMode/m_evalEraBudget, because nothing set them any more - 28 read sites all permanently inert. That is not a tidy-up: the `if (!m_evalMode)` guard on UpdateClassPriors is exactly what silently disabled the imbalance correction for entire runs two commits ago. Dead machinery that still reads like live machinery is this codebase's most expensive recurring bug, and leaving 28 more instances of it would have been indefensible. The panel's tuning-progress state goes too - tuning no longer takes long enough to need one. Both builds compile 0 errors / 0 warnings. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-01 10:29:31 -04:00
continue;
for(int f = 0; f < m_neuronsCount; f++)
fix(autotune): MI scorer read an array nobody filled; add the permutation floor THE TUNER WAS A SILENT NO-OP. Every chart logged auto-tune complete - 17 candidate settings scored in ~139s, feature/label mutual information 0.0000 -> 0.0000 nats (no improvement) 0.0000 is not a weak result, it is a broken measurement: finite-sample MI is biased UPWARD, so even pure noise scores above zero. Cause: ScoreCurrentParamsByMI called BufferTempDataCompute(), which APPENDS the bar's features to TempData and never touches m_featureCache - only the caching wrapper BufferTempData() writes that array. It then read m_featureCache, which ReInitADIndicators had just invalidated. Every column came back constant, FeatureColumnMI returned 0 for all of them, and all 17 candidates tied at exactly zero. 139 s per chart to return the settings it started with. Now reads the values back out of TempData, where they actually land. And an exactly-zero best score is called out as a fault rather than reported as "no improvement", because that is what it is. ADDED: a PERMUTATION BASELINE, which is the diagnostic this project has been missing. MI's finite-sample bias is ~(bins-1)(classes-1)/(2n) nats - at these sample sizes the same order as any real edge in this domain - so a raw MI figure is uninterpretable on its own. Shuffling the labels destroys every genuine association while leaving sample size, binning and class proportions intact, so the score it produces IS this dataset's noise floor, measured rather than approximated. The log now reads feature/label information - X nats against a shuffled-label floor of Y and says outright whether the features carry usable information about the target. It needs no training, no topology and no convergence, so unlike every accuracy number in this codebase it cannot be confounded by an optimizer or an objective. If the score sits on the floor, no change of architecture can help - which is the question the last three days of zero-edge results have been circling. DEPLOY FLOOR: `dirPrecPct > chancePrecPct` passed anything above chance by any amount. At ~11,000 directional calls the standard error of the precision estimate is ~0.4pp, so that gate was accepting sub-one-sigma noise - the perceptron deployed at edge +0pp on 2026-08-01. Now requires EDGE_MIN_SIGMAS (2.0) standard errors above chance, computed from the actual call count, so the bar scales with the evidence instead of needing a hand-picked constant. Recorded with it, because it is why chance is the right reference at all: under a driftless random walk P(touch +k*ATR before -m*ATR) = m/(m+k), and the break-even win rate for a k:m reward:risk trade is ALSO m/(m+k). The label's own base rate IS the break-even rate, at every SL/TP setting. So "beats chance" and "is profitable" are the same test, and no choice of SL/TP can manufacture an edge - only prediction can. Both builds compile 0 errors / 0 warnings. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-01 13:05:50 -04:00
cols[n * m_neuronsCount + f] = TempData.At(f);
diag: is "optimal SL/TP" learnable? Score the features against excursions Proposed direction: train the net to predict entry/SL/TP that maximise return and minimise drawdown, rather than to classify direction. Before rebuilding a head, measure whether the target is learnable at all. That question splits into two that behave nothing alike: HOW FAR price travels (MFE/MAE) - essentially volatility, and volatility clustering is about the most robust regularity in markets. WHICH WAY it goes first (the asymmetry) - direction, which is what every noise-floor verdict in this project has been about. Expectancy comes ONLY from the second. The first buys position sizing and drawdown control - worth having under prop-firm limits, but not an edge: exit management on RANDOM entries already moved the payoff ratio 0.92 -> 5.72 with expectancy FLAT. Crucially this is NOT already answered. Every MI figure here scored the triple-barrier label, i.e. one specific question at one fixed geometry. A noise-floor result there says nothing about whether excursion MAGNITUDE is learnable - different target, different answer. Four targets, and the verdict is the CONTRAST, printed explicitly because the dangerous misreading of "UP clears" is "we can predict profitable trades": RANGE (up+dn) - realised volatility, included as a POSITIVE CONTROL that SHOULD clear. Every prior verdict here lacked a control expected to pass; a range target at the floor indicts the measurement, not the market. UP / DOWN - MFE / MAE. ASYMMETRY - up-dn, the only one that can pay. Collected inside the walk the label already does (one max, one min per bar). The early-out when both barriers resolved is GONE: it would have truncated the excursions at whichever bar tripped the last barrier, making the measurement a function of the CURRENT SL/TP - the circularity this is trying to escape. The loop was already bounded by the horizon, so only the average cost moves. Discretised into 3 EQUAL-FREQUENCY bins, so every downstream piece (block permutation, null, p-value) is reused unchanged. Equal-frequency because MFE is fat-tailed and fixed-width bins would put nearly every row in bin 0; it also pins H(Y) at ln(3)=1.099 for all four, making them comparable to each other and to the barrier label's ~1.02 instead of confounded by class balance. Two bugs fixed in this code before it ever ran, both of which would have produced a plausible quiet wrong answer rather than an error: - TripleBarrierLabel early-returns on invalid ATR/close BEFORE the point the accumulators were reset, so one bar's excursions would be cached under another bar's index. Cleared at the top now, ahead of every return. - An unresolvable bar is still flagged as labelled but carries excursions of exactly 0. Under equal-frequency binning a block of identical zeros drags the lowest cut onto zero and a third of the sample lands in one uninformative bin - a depressed score that reads as "not predictable", a false negative in the direction that would wrongly kill the idea. Rows where both excursions are zero are dropped; price cannot travel zero both ways over a whole horizon. Read-only diagnostic. No topology or label change: no retrain of its own. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-07 10:22:41 -04:00
if(continuousTarget)
feat(labels): measure which barrier is predictable at entry, don't guess The alignment scan settled the shape of the problem: 4.7x more is knowable 5 bars into a 128-bar window than at the entry the model actually trades. A 6xATR target reached over 128 bars is decided overwhelmingly by what happens DURING the window, so whatever the entry state knows is buried under 128 bars of later noise. That is a property of the TARGET, and it is why four different architectures all landed on precision exactly equal to the base rate - no topology can undo it. So measure the target. For each SL/TP pairing a user can actually select, relabel the same sampled bars and score how much the SAME features say about THAT outcome at entry. Seconds, no training, no topology, and it runs on the diagnostic path that already exists. Ranked on excess over its OWN null as a share of its OWN H(Y), never on raw nats: each geometry has a different class balance, hence a different finite-sample bias and a different amount of information there to find, so raw MI would rank the most BALANCED label rather than the most PREDICTABLE one. The break-even win rate m/(m+k) is printed beside each so the ranking is read next to the bar the model must clear. Stated in the output because it is the easy thing to get wrong: chance precision EQUALS break-even at every geometry, so a tighter target does not hand you expectancy. It buys predictability - less noise piled on top of what the entry state knows - which is the one thing changing topology cannot do. Read-only by construction: it relabels a sampled copy via TripleBarrierLabel(), never writes the label cache (which belongs to the configured geometry), and restores the horizon and overrides it borrowed. The overrides apply only when BOTH are positive, so a half-set pair can never silently relabel a live run. Compiles 0 errors / 0 warnings, standard and Market. Build tag geometry-scan-v1. Redeploy only - no retrain to READ the ranking. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-01 14:32:04 -04:00
{
refactor(comments): stdlib comment style across the remaining in-scope files Same pass as 0b06f8e, applied file by file: comment runs of 4+ lines compressed to their leading topic sentences, capped at 4 lines, whole sentences only. Warning sentences (NEVER / MUST / trap / would-have) survive the budget. Every file was checked the same way before committing: the list of non-comment lines is byte-identical to HEAD, and braces balance. No code was touched. Panel/, Enumerations/ and the already-terse System headers needed little or nothing - PooledGate, TradeChecks, BinomialStats and Random came through with no blocks over the threshold at all. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-22 00:25:52 -04:00
//--- Excursions come from the cache only.
diag: is "optimal SL/TP" learnable? Score the features against excursions Proposed direction: train the net to predict entry/SL/TP that maximise return and minimise drawdown, rather than to classify direction. Before rebuilding a head, measure whether the target is learnable at all. That question splits into two that behave nothing alike: HOW FAR price travels (MFE/MAE) - essentially volatility, and volatility clustering is about the most robust regularity in markets. WHICH WAY it goes first (the asymmetry) - direction, which is what every noise-floor verdict in this project has been about. Expectancy comes ONLY from the second. The first buys position sizing and drawdown control - worth having under prop-firm limits, but not an edge: exit management on RANDOM entries already moved the payoff ratio 0.92 -> 5.72 with expectancy FLAT. Crucially this is NOT already answered. Every MI figure here scored the triple-barrier label, i.e. one specific question at one fixed geometry. A noise-floor result there says nothing about whether excursion MAGNITUDE is learnable - different target, different answer. Four targets, and the verdict is the CONTRAST, printed explicitly because the dangerous misreading of "UP clears" is "we can predict profitable trades": RANGE (up+dn) - realised volatility, included as a POSITIVE CONTROL that SHOULD clear. Every prior verdict here lacked a control expected to pass; a range target at the floor indicts the measurement, not the market. UP / DOWN - MFE / MAE. ASYMMETRY - up-dn, the only one that can pay. Collected inside the walk the label already does (one max, one min per bar). The early-out when both barriers resolved is GONE: it would have truncated the excursions at whichever bar tripped the last barrier, making the measurement a function of the CURRENT SL/TP - the circularity this is trying to escape. The loop was already bounded by the horizon, so only the average cost moves. Discretised into 3 EQUAL-FREQUENCY bins, so every downstream piece (block permutation, null, p-value) is reused unchanged. Equal-frequency because MFE is fat-tailed and fixed-width bins would put nearly every row in bin 0; it also pins H(Y) at ln(3)=1.099 for all four, making them comparable to each other and to the barrier label's ~1.02 instead of confounded by class balance. Two bugs fixed in this code before it ever ran, both of which would have produced a plausible quiet wrong answer rather than an error: - TripleBarrierLabel early-returns on invalid ATR/close BEFORE the point the accumulators were reset, so one bar's excursions would be cached under another bar's index. Cleared at the top now, ahead of every return. - An unresolvable bar is still flagged as labelled but carries excursions of exactly 0. Under equal-frequency binning a block of identical zeros drags the lowest cut onto zero and a third of the sample lands in one uninformative bin - a depressed score that reads as "not predictable", a false negative in the direction that would wrongly kill the idea. Rows where both excursions are zero are dropped; price cannot travel zero both ways over a whole horizon. Read-only diagnostic. No topology or label change: no retrain of its own. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-07 10:22:41 -04:00
if(li >= ArraySize(m_excUpCache))
continue;
double up = m_excUpCache[li];
double dn = m_excDownCache[li];
if(!MathIsValidNumber(up) || !MathIsValidNumber(dn))
continue;
refactor(comments): stdlib comment style across the remaining in-scope files Same pass as 0b06f8e, applied file by file: comment runs of 4+ lines compressed to their leading topic sentences, capped at 4 lines, whole sentences only. Warning sentences (NEVER / MUST / trap / would-have) survive the budget. Every file was checked the same way before committing: the list of non-comment lines is byte-identical to HEAD, and braces balance. No code was touched. Panel/, Enumerations/ and the already-terse System headers needed little or nothing - PooledGate, TradeChecks, BinomialStats and Random came through with no blocks over the threshold at all. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-22 00:25:52 -04:00
//--- A bar that TripleBarrierLabel() could not resolve (no valid ATR or close, typically
//--- the oldest bars) is still flagged as having a label, but its excursions were cleared
//--- to zero rather than measured.
diag: is "optimal SL/TP" learnable? Score the features against excursions Proposed direction: train the net to predict entry/SL/TP that maximise return and minimise drawdown, rather than to classify direction. Before rebuilding a head, measure whether the target is learnable at all. That question splits into two that behave nothing alike: HOW FAR price travels (MFE/MAE) - essentially volatility, and volatility clustering is about the most robust regularity in markets. WHICH WAY it goes first (the asymmetry) - direction, which is what every noise-floor verdict in this project has been about. Expectancy comes ONLY from the second. The first buys position sizing and drawdown control - worth having under prop-firm limits, but not an edge: exit management on RANDOM entries already moved the payoff ratio 0.92 -> 5.72 with expectancy FLAT. Crucially this is NOT already answered. Every MI figure here scored the triple-barrier label, i.e. one specific question at one fixed geometry. A noise-floor result there says nothing about whether excursion MAGNITUDE is learnable - different target, different answer. Four targets, and the verdict is the CONTRAST, printed explicitly because the dangerous misreading of "UP clears" is "we can predict profitable trades": RANGE (up+dn) - realised volatility, included as a POSITIVE CONTROL that SHOULD clear. Every prior verdict here lacked a control expected to pass; a range target at the floor indicts the measurement, not the market. UP / DOWN - MFE / MAE. ASYMMETRY - up-dn, the only one that can pay. Collected inside the walk the label already does (one max, one min per bar). The early-out when both barriers resolved is GONE: it would have truncated the excursions at whichever bar tripped the last barrier, making the measurement a function of the CURRENT SL/TP - the circularity this is trying to escape. The loop was already bounded by the horizon, so only the average cost moves. Discretised into 3 EQUAL-FREQUENCY bins, so every downstream piece (block permutation, null, p-value) is reused unchanged. Equal-frequency because MFE is fat-tailed and fixed-width bins would put nearly every row in bin 0; it also pins H(Y) at ln(3)=1.099 for all four, making them comparable to each other and to the barrier label's ~1.02 instead of confounded by class balance. Two bugs fixed in this code before it ever ran, both of which would have produced a plausible quiet wrong answer rather than an error: - TripleBarrierLabel early-returns on invalid ATR/close BEFORE the point the accumulators were reset, so one bar's excursions would be cached under another bar's index. Cleared at the top now, ahead of every return. - An unresolvable bar is still flagged as labelled but carries excursions of exactly 0. Under equal-frequency binning a block of identical zeros drags the lowest cut onto zero and a third of the sample lands in one uninformative bin - a depressed score that reads as "not predictable", a false negative in the direction that would wrongly kill the idea. Rows where both excursions are zero are dropped; price cannot travel zero both ways over a whole horizon. Read-only diagnostic. No topology or label change: no retrain of its own. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-07 10:22:41 -04:00
if(up <= 0.0 && dn <= 0.0)
continue;
if(target == MI_TARGET_EXC_UP)
raw[n] = up;
else
if(target == MI_TARGET_EXC_DOWN)
raw[n] = dn;
else
if(target == MI_TARGET_EXC_RANGE)
raw[n] = up + dn;
else
fix: normalise the asymmetry target - the raw one is confounded by volatility Three symbols ran the excursion test. RANGE/UP/DOWN cleared on all three; raw ASYMMETRY cleared on EURUSD and USDCAD at p=0.0050 and not on SP500 (p=0.1045). That looked like the first directional signal this project has found. It probably is not, and the test as built could not tell. (up-dn) IS NOT SCALE-FREE. If sigma is predictable - and RANGE clears at ~4x its null on every instrument - and the directional part is symmetric noise eps, then up-dn ~ sigma*eps, so a large sigma pushes the value into BOTH outer terciles. A pure volatility predictor scores positive MI against a 3-bin (up-dn) while carrying no directional information at all. Crucially that confound REPLICATES, so reproducing on two instruments is not evidence against it - and the effect sizes fit it: asymmetry runs 1.3-1.6x its null where RANGE runs ~4x, and carries ~0.1% of the target's entropy against RANGE's ~0.9%. That is the shape of a leaked fraction of the volatility signal, not an independent one. So add (up-dn)/(up+dn): bounded in [-1,+1], volatility divided out, and the only target a directional claim may rest on. The verdict now separates the cases and NAMES the confound when raw clears while normalised does not, instead of reporting the raw line as a finding. Two bugs of mine in the same block, both caught by output rather than review: - The derived-geometry line had a MISORDERED argument list: it printed "stop 25.00*ATR (q3 of adverse travel)" - the quantile percentage as the multiple and the multiple as the quantile. Real values were 2.61 stop / 8.03 target. A 25*ATR stop is absurd on its face, which is why it was seen. - THE STOP QUANTILE WAS BACKWARDS, and this one changes labels. It was 0.25 "so ordinary noise does not reach it", but q25 means 75% of bars EXCEED the stop - hit three times in four. The printed reachability said exactly that ("stop on 75.0% of bars"). Now 0.75. A quantile is a threshold, not a rate. This is the entire reason reachability is measured and printed rather than assumed. Also raises BARRIER_DERIVE_MAX_PASSES 3 -> 5: SP500 did not settle in 3 (stop still moving ~14% per pass) while EURUSD and USDCAD converged on pass 2. And bounds both quantile indices with MathMin(..., n-1) so q=1.0 cannot run off the end of the sorted array. The geometry from the previous run is NOT usable and the asymmetry result is unresolved, not established. Both are decided by the next run. FORCES A FULL RETRAIN (the stop quantile changes every label). Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-07 13:04:13 -04:00
if(target == MI_TARGET_EXC_ASYM)
raw[n] = up - dn;
else
{
//--- Scale-free asymmetry. The denominator is > 0 here because rows with both
//--- excursions zero were dropped above, so no guard is needed beyond that.
raw[n] = (up - dn) / (up + dn); // MI_TARGET_EXC_ASYM_NORM
}
diag: is "optimal SL/TP" learnable? Score the features against excursions Proposed direction: train the net to predict entry/SL/TP that maximise return and minimise drawdown, rather than to classify direction. Before rebuilding a head, measure whether the target is learnable at all. That question splits into two that behave nothing alike: HOW FAR price travels (MFE/MAE) - essentially volatility, and volatility clustering is about the most robust regularity in markets. WHICH WAY it goes first (the asymmetry) - direction, which is what every noise-floor verdict in this project has been about. Expectancy comes ONLY from the second. The first buys position sizing and drawdown control - worth having under prop-firm limits, but not an edge: exit management on RANDOM entries already moved the payoff ratio 0.92 -> 5.72 with expectancy FLAT. Crucially this is NOT already answered. Every MI figure here scored the triple-barrier label, i.e. one specific question at one fixed geometry. A noise-floor result there says nothing about whether excursion MAGNITUDE is learnable - different target, different answer. Four targets, and the verdict is the CONTRAST, printed explicitly because the dangerous misreading of "UP clears" is "we can predict profitable trades": RANGE (up+dn) - realised volatility, included as a POSITIVE CONTROL that SHOULD clear. Every prior verdict here lacked a control expected to pass; a range target at the floor indicts the measurement, not the market. UP / DOWN - MFE / MAE. ASYMMETRY - up-dn, the only one that can pay. Collected inside the walk the label already does (one max, one min per bar). The early-out when both barriers resolved is GONE: it would have truncated the excursions at whichever bar tripped the last barrier, making the measurement a function of the CURRENT SL/TP - the circularity this is trying to escape. The loop was already bounded by the horizon, so only the average cost moves. Discretised into 3 EQUAL-FREQUENCY bins, so every downstream piece (block permutation, null, p-value) is reused unchanged. Equal-frequency because MFE is fat-tailed and fixed-width bins would put nearly every row in bin 0; it also pins H(Y) at ln(3)=1.099 for all four, making them comparable to each other and to the barrier label's ~1.02 instead of confounded by class balance. Two bugs fixed in this code before it ever ran, both of which would have produced a plausible quiet wrong answer rather than an error: - TripleBarrierLabel early-returns on invalid ATR/close BEFORE the point the accumulators were reset, so one bar's excursions would be cached under another bar's index. Cleared at the top now, ahead of every return. - An unresolvable bar is still flagged as labelled but carries excursions of exactly 0. Under equal-frequency binning a block of identical zeros drags the lowest cut onto zero and a third of the sample lands in one uninformative bin - a depressed score that reads as "not predictable", a false negative in the direction that would wrongly kill the idea. Rows where both excursions are zero are dropped; price cannot travel zero both ways over a whole horizon. Read-only diagnostic. No topology or label change: no retrain of its own. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-07 10:22:41 -04:00
labels[n] = 0; // assigned below, once the distribution is known
feat(labels): measure which barrier is predictable at entry, don't guess The alignment scan settled the shape of the problem: 4.7x more is knowable 5 bars into a 128-bar window than at the entry the model actually trades. A 6xATR target reached over 128 bars is decided overwhelmingly by what happens DURING the window, so whatever the entry state knows is buried under 128 bars of later noise. That is a property of the TARGET, and it is why four different architectures all landed on precision exactly equal to the base rate - no topology can undo it. So measure the target. For each SL/TP pairing a user can actually select, relabel the same sampled bars and score how much the SAME features say about THAT outcome at entry. Seconds, no training, no topology, and it runs on the diagnostic path that already exists. Ranked on excess over its OWN null as a share of its OWN H(Y), never on raw nats: each geometry has a different class balance, hence a different finite-sample bias and a different amount of information there to find, so raw MI would rank the most BALANCED label rather than the most PREDICTABLE one. The break-even win rate m/(m+k) is printed beside each so the ranking is read next to the bar the model must clear. Stated in the output because it is the easy thing to get wrong: chance precision EQUALS break-even at every geometry, so a tighter target does not hand you expectancy. It buys predictability - less noise piled on top of what the entry state knows - which is the one thing changing topology cannot do. Read-only by construction: it relabels a sampled copy via TripleBarrierLabel(), never writes the label cache (which belongs to the configured geometry), and restores the horizon and overrides it borrowed. The overrides apply only when BOTH are positive, so a half-set pair can never silently relabel a live run. Compiles 0 errors / 0 warnings, standard and Market. Build tag geometry-scan-v1. Redeploy only - no retrain to READ the ranking. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-01 14:32:04 -04:00
}
else
diag: is "optimal SL/TP" learnable? Score the features against excursions Proposed direction: train the net to predict entry/SL/TP that maximise return and minimise drawdown, rather than to classify direction. Before rebuilding a head, measure whether the target is learnable at all. That question splits into two that behave nothing alike: HOW FAR price travels (MFE/MAE) - essentially volatility, and volatility clustering is about the most robust regularity in markets. WHICH WAY it goes first (the asymmetry) - direction, which is what every noise-floor verdict in this project has been about. Expectancy comes ONLY from the second. The first buys position sizing and drawdown control - worth having under prop-firm limits, but not an edge: exit management on RANDOM entries already moved the payoff ratio 0.92 -> 5.72 with expectancy FLAT. Crucially this is NOT already answered. Every MI figure here scored the triple-barrier label, i.e. one specific question at one fixed geometry. A noise-floor result there says nothing about whether excursion MAGNITUDE is learnable - different target, different answer. Four targets, and the verdict is the CONTRAST, printed explicitly because the dangerous misreading of "UP clears" is "we can predict profitable trades": RANGE (up+dn) - realised volatility, included as a POSITIVE CONTROL that SHOULD clear. Every prior verdict here lacked a control expected to pass; a range target at the floor indicts the measurement, not the market. UP / DOWN - MFE / MAE. ASYMMETRY - up-dn, the only one that can pay. Collected inside the walk the label already does (one max, one min per bar). The early-out when both barriers resolved is GONE: it would have truncated the excursions at whichever bar tripped the last barrier, making the measurement a function of the CURRENT SL/TP - the circularity this is trying to escape. The loop was already bounded by the horizon, so only the average cost moves. Discretised into 3 EQUAL-FREQUENCY bins, so every downstream piece (block permutation, null, p-value) is reused unchanged. Equal-frequency because MFE is fat-tailed and fixed-width bins would put nearly every row in bin 0; it also pins H(Y) at ln(3)=1.099 for all four, making them comparable to each other and to the barrier label's ~1.02 instead of confounded by class balance. Two bugs fixed in this code before it ever ran, both of which would have produced a plausible quiet wrong answer rather than an error: - TripleBarrierLabel early-returns on invalid ATR/close BEFORE the point the accumulators were reset, so one bar's excursions would be cached under another bar's index. Cleared at the top now, ahead of every return. - An unresolvable bar is still flagged as labelled but carries excursions of exactly 0. Under equal-frequency binning a block of identical zeros drags the lowest cut onto zero and a third of the sample lands in one uninformative bin - a depressed score that reads as "not predictable", a false negative in the direction that would wrongly kill the idea. Rows where both excursions are zero are dropped; price cannot travel zero both ways over a whole horizon. Read-only diagnostic. No topology or label change: no retrain of its own. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-07 10:22:41 -04:00
if(m_barrierScanLiveLabels)
{
ENUM_SIGNAL v = TripleBarrierLabel(li);
if(v == Neutral && m_lastBarrierTimedOut)
m_barrierScanTimeouts++;
labels[n] = (v == Buy) ? 0 : ((v == Sell) ? 1 : 2);
}
else
labels[n] = m_labelCacheBuy[li] ? 0 : (m_labelCacheSell[li] ? 1 : 2);
perf(autotune): replace the genetic search with a filter score - hours to seconds MEASURED COST OF THE GA, which is what retired it. Per generation: rung 0: 8 cand x 3 seeds x 3 eras = 72 eras rung 1: 4 cand x 3 seeds x 8 eras = 96 rung 2: 2 cand x 3 seeds x 20 eras = 120 = 288 eras/generation x 4 generations = 1152 eras BEFORE the winner's real training began. Against the observed era times on SP500 H1: PAI 29.1 s/era -> 9.3 h (matches the observed 00:37 -> 09:22) CONV 41.3 s/era -> 13.2 h LSTM 150.4 s/era -> 48.1 h HYBRID 154.6 s/era -> 49.5 h Two days to tune is not a first-run experience, and it is the phase in which the panel goes quiet, which is what made it look like a hang. It also bought nothing. The space is 90 points (10 MA periods x 9 MA types), so 1152 evaluations revisited each point ~13 times; and rungs of 3 and 8 eras cannot separate two MA periods at all. The 2026-08-01 run proves it: every finalist scored 25.0-25.9% balanced accuracy - below the 33.3% one-class floor, i.e. indistinguishable noise - and the search then "deployed the winner" of that. THE ERROR WAS THE SCORING FUNCTION, not its constants. Using a full training run to choose a feature's period is a wrapper method paying wrapper prices for a decision that does not need one. The reference book does not do this: ch. 3.3 selects inputs by measuring each candidate indicator's CORRELATION with the target and dropping the ones with none, with no network involved. So: rank candidates by the MUTUAL INFORMATION between the resulting feature vector and the triple-barrier label. MI rather than correlation because the label is 3-class categorical and the features are not monotonically related to it. Equal-FREQUENCY binning (rank-based), because these features are ATR-normalised and heavy-tailed - fixed-width bins put nearly everything in one bucket and report ~0 information for a genuinely useful feature. Scoring is arithmetic over the feature cache, so it costs seconds and its cost is independent of topology: LSTM now tunes as fast as the MLP. Coordinate sweep, not product sweep - cost is the SUM of per-parameter candidate counts, so enabling every indicator stays affordable - with a second pass that breaks early once nothing moves. Sampling is IS-ONLY. Letting the OOS window influence which indicator settings ship would mean the holdout had been used for selection and had stopped being a holdout. HONEST LIMIT, recorded because it is the price: MI is marginal, so a parameter that only pays off in combination with another can be missed (Guyon & Elisseeff 2003, filter vs wrapper). Given the wrapper it replaces was ranking pure noise at 48 h a run, this is strictly better. Deleted with it: GaRungEras/GaExtract/GaStore/GaMutate/GaRandomCandidate/ GaBlockCrossover/GaSortAliveByScoreDesc/GaBreedNextGeneration, 14 m_ga* members, the GA_*/TUNE_POP_* constants, and ComputeTuneTrialBudget. AND m_evalMode/m_evalEraBudget, because nothing set them any more - 28 read sites all permanently inert. That is not a tidy-up: the `if (!m_evalMode)` guard on UpdateClassPriors is exactly what silently disabled the imbalance correction for entire runs two commits ago. Dead machinery that still reads like live machinery is this codebase's most expensive recurring bug, and leaving 28 more instances of it would have been indefensible. The panel's tuning-progress state goes too - tuning no longer takes long enough to need one. Both builds compile 0 errors / 0 warnings. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-01 10:29:31 -04:00
n++;
}
fix(autotune): MI scorer read an array nobody filled; add the permutation floor THE TUNER WAS A SILENT NO-OP. Every chart logged auto-tune complete - 17 candidate settings scored in ~139s, feature/label mutual information 0.0000 -> 0.0000 nats (no improvement) 0.0000 is not a weak result, it is a broken measurement: finite-sample MI is biased UPWARD, so even pure noise scores above zero. Cause: ScoreCurrentParamsByMI called BufferTempDataCompute(), which APPENDS the bar's features to TempData and never touches m_featureCache - only the caching wrapper BufferTempData() writes that array. It then read m_featureCache, which ReInitADIndicators had just invalidated. Every column came back constant, FeatureColumnMI returned 0 for all of them, and all 17 candidates tied at exactly zero. 139 s per chart to return the settings it started with. Now reads the values back out of TempData, where they actually land. And an exactly-zero best score is called out as a fault rather than reported as "no improvement", because that is what it is. ADDED: a PERMUTATION BASELINE, which is the diagnostic this project has been missing. MI's finite-sample bias is ~(bins-1)(classes-1)/(2n) nats - at these sample sizes the same order as any real edge in this domain - so a raw MI figure is uninterpretable on its own. Shuffling the labels destroys every genuine association while leaving sample size, binning and class proportions intact, so the score it produces IS this dataset's noise floor, measured rather than approximated. The log now reads feature/label information - X nats against a shuffled-label floor of Y and says outright whether the features carry usable information about the target. It needs no training, no topology and no convergence, so unlike every accuracy number in this codebase it cannot be confounded by an optimizer or an objective. If the score sits on the floor, no change of architecture can help - which is the question the last three days of zero-edge results have been circling. DEPLOY FLOOR: `dirPrecPct > chancePrecPct` passed anything above chance by any amount. At ~11,000 directional calls the standard error of the precision estimate is ~0.4pp, so that gate was accepting sub-one-sigma noise - the perceptron deployed at edge +0pp on 2026-08-01. Now requires EDGE_MIN_SIGMAS (2.0) standard errors above chance, computed from the actual call count, so the bar scales with the evidence instead of needing a hand-picked constant. Recorded with it, because it is why chance is the right reference at all: under a driftless random walk P(touch +k*ATR before -m*ATR) = m/(m+k), and the break-even win rate for a k:m reward:risk trade is ALSO m/(m+k). The label's own base rate IS the break-even rate, at every SL/TP setting. So "beats chance" and "is profitable" are the same test, and no choice of SL/TP can manufacture an edge - only prediction can. Both builds compile 0 errors / 0 warnings. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-01 13:05:50 -04:00
TempData.Clear();
refactor(comments): stdlib comment style across the remaining in-scope files Same pass as 0b06f8e, applied file by file: comment runs of 4+ lines compressed to their leading topic sentences, capped at 4 lines, whole sentences only. Warning sentences (NEVER / MUST / trap / would-have) survive the budget. Every file was checked the same way before committing: the list of non-comment lines is byte-identical to HEAD, and braces balance. No code was touched. Panel/, Enumerations/ and the already-terse System headers needed little or nothing - PooledGate, TradeChecks, BinomialStats and Random came through with no blocks over the threshold at all. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-22 00:25:52 -04:00
//--- EQUAL-FREQUENCY DISCRETISATION into the same 3 classes FeatureColumnMI's joint table
//--- expects, so every downstream piece - the block permutation, the null, the p-value, the lag
//--- profile - works on a continuous target with no change at all.
diag: is "optimal SL/TP" learnable? Score the features against excursions Proposed direction: train the net to predict entry/SL/TP that maximise return and minimise drawdown, rather than to classify direction. Before rebuilding a head, measure whether the target is learnable at all. That question splits into two that behave nothing alike: HOW FAR price travels (MFE/MAE) - essentially volatility, and volatility clustering is about the most robust regularity in markets. WHICH WAY it goes first (the asymmetry) - direction, which is what every noise-floor verdict in this project has been about. Expectancy comes ONLY from the second. The first buys position sizing and drawdown control - worth having under prop-firm limits, but not an edge: exit management on RANDOM entries already moved the payoff ratio 0.92 -> 5.72 with expectancy FLAT. Crucially this is NOT already answered. Every MI figure here scored the triple-barrier label, i.e. one specific question at one fixed geometry. A noise-floor result there says nothing about whether excursion MAGNITUDE is learnable - different target, different answer. Four targets, and the verdict is the CONTRAST, printed explicitly because the dangerous misreading of "UP clears" is "we can predict profitable trades": RANGE (up+dn) - realised volatility, included as a POSITIVE CONTROL that SHOULD clear. Every prior verdict here lacked a control expected to pass; a range target at the floor indicts the measurement, not the market. UP / DOWN - MFE / MAE. ASYMMETRY - up-dn, the only one that can pay. Collected inside the walk the label already does (one max, one min per bar). The early-out when both barriers resolved is GONE: it would have truncated the excursions at whichever bar tripped the last barrier, making the measurement a function of the CURRENT SL/TP - the circularity this is trying to escape. The loop was already bounded by the horizon, so only the average cost moves. Discretised into 3 EQUAL-FREQUENCY bins, so every downstream piece (block permutation, null, p-value) is reused unchanged. Equal-frequency because MFE is fat-tailed and fixed-width bins would put nearly every row in bin 0; it also pins H(Y) at ln(3)=1.099 for all four, making them comparable to each other and to the barrier label's ~1.02 instead of confounded by class balance. Two bugs fixed in this code before it ever ran, both of which would have produced a plausible quiet wrong answer rather than an error: - TripleBarrierLabel early-returns on invalid ATR/close BEFORE the point the accumulators were reset, so one bar's excursions would be cached under another bar's index. Cleared at the top now, ahead of every return. - An unresolvable bar is still flagged as labelled but carries excursions of exactly 0. Under equal-frequency binning a block of identical zeros drags the lowest cut onto zero and a third of the sample lands in one uninformative bin - a depressed score that reads as "not predictable", a false negative in the direction that would wrongly kill the idea. Rows where both excursions are zero are dropped; price cannot travel zero both ways over a whole horizon. Read-only diagnostic. No topology or label change: no retrain of its own. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-07 10:22:41 -04:00
if(continuousTarget && n > 0)
{
refactor(stdlib): one quantile definition, from Math\Stat The codebase had THREE conventions for the same statistic. AltData took a true median; the barrier horizon and the derived input window took the upper of the two middle values; the MI terciles and the barrier stop ladder used nearest-rank indexing. All four now go through MathMedian / MathQuantile, which is R's type 7 and the library's one answer. System\AltData.mqh column median -> MathMedian (exact, no change) AIBase\Labels.mqh swing median -> MathMedian leg-range med -> MathMedian stop ladder -> MathQuantile, read in one call AIBase\Topology.mqh window median -> MathMedian AIBase\AutoTune.mqh MI terciles -> MathQuantile + MathMin/MathMax Signals\SignalSessionFilter DST last Sunday-> CDateTime::DaysInMonth() gaps[]/legs[] change from int to double so MathMedian can read them; the values are bar counts either way. VALUES MOVE. Even-sample medians shift by half a bin and the quantile reads interpolate, so the barrier geometry and the derived input window can land on different rungs - re-keying fingerprints and forcing a retrain. Accepted deliberately: stdlib consistency was the ask, and three private conventions for one statistic is what it buys out. Two YAGNI finds fell out of the ladder rewrite. MathQuantile sorts its own copy, so DeriveBarrierGeometry no longer sorts up[]/dn[] in place - which means upUnsorted[], a full array copy kept only to undo that sort, is gone. ArraySort(up) had no consumer needing order at all; it was pure work. The library call also gets a failure guard the hand-rolled indexing never needed but the ladder read does. Verified while here: Math\Stat\Math.mqh's MathAbs/MathMax/MathSqrt/MathPow and friends are ARRAY overloads, not scalar redefinitions, so pulling it into the translation unit shadows no builtin. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-19 20:16:03 -04:00
//--- EQUAL-FREQUENCY TERCILES off the library, so this and the barrier stop ladder share one
//--- quantile definition instead of the nearest-rank indexing each used to spell out.
double vals[];
ArrayResize(vals, n);
ArrayCopy(vals, raw, 0, 0, n);
double probs[2] = {1.0 / 3.0, 2.0 / 3.0};
double cuts[];
if(!MathQuantile(vals, probs, cuts))
return -1;
double cut1 = cuts[0];
double cut2 = cuts[1];
diag: is "optimal SL/TP" learnable? Score the features against excursions Proposed direction: train the net to predict entry/SL/TP that maximise return and minimise drawdown, rather than to classify direction. Before rebuilding a head, measure whether the target is learnable at all. That question splits into two that behave nothing alike: HOW FAR price travels (MFE/MAE) - essentially volatility, and volatility clustering is about the most robust regularity in markets. WHICH WAY it goes first (the asymmetry) - direction, which is what every noise-floor verdict in this project has been about. Expectancy comes ONLY from the second. The first buys position sizing and drawdown control - worth having under prop-firm limits, but not an edge: exit management on RANDOM entries already moved the payoff ratio 0.92 -> 5.72 with expectancy FLAT. Crucially this is NOT already answered. Every MI figure here scored the triple-barrier label, i.e. one specific question at one fixed geometry. A noise-floor result there says nothing about whether excursion MAGNITUDE is learnable - different target, different answer. Four targets, and the verdict is the CONTRAST, printed explicitly because the dangerous misreading of "UP clears" is "we can predict profitable trades": RANGE (up+dn) - realised volatility, included as a POSITIVE CONTROL that SHOULD clear. Every prior verdict here lacked a control expected to pass; a range target at the floor indicts the measurement, not the market. UP / DOWN - MFE / MAE. ASYMMETRY - up-dn, the only one that can pay. Collected inside the walk the label already does (one max, one min per bar). The early-out when both barriers resolved is GONE: it would have truncated the excursions at whichever bar tripped the last barrier, making the measurement a function of the CURRENT SL/TP - the circularity this is trying to escape. The loop was already bounded by the horizon, so only the average cost moves. Discretised into 3 EQUAL-FREQUENCY bins, so every downstream piece (block permutation, null, p-value) is reused unchanged. Equal-frequency because MFE is fat-tailed and fixed-width bins would put nearly every row in bin 0; it also pins H(Y) at ln(3)=1.099 for all four, making them comparable to each other and to the barrier label's ~1.02 instead of confounded by class balance. Two bugs fixed in this code before it ever ran, both of which would have produced a plausible quiet wrong answer rather than an error: - TripleBarrierLabel early-returns on invalid ATR/close BEFORE the point the accumulators were reset, so one bar's excursions would be cached under another bar's index. Cleared at the top now, ahead of every return. - An unresolvable bar is still flagged as labelled but carries excursions of exactly 0. Under equal-frequency binning a block of identical zeros drags the lowest cut onto zero and a third of the sample lands in one uninformative bin - a depressed score that reads as "not predictable", a false negative in the direction that would wrongly kill the idea. Rows where both excursions are zero are dropped; price cannot travel zero both ways over a whole horizon. Read-only diagnostic. No topology or label change: no retrain of its own. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-07 10:22:41 -04:00
//--- A degenerate target (every value identical, e.g. a cache that never filled) would land every
//--- row in one class and score a flat zero. Say so rather than reporting the zero as a finding.
refactor(stdlib): one quantile definition, from Math\Stat The codebase had THREE conventions for the same statistic. AltData took a true median; the barrier horizon and the derived input window took the upper of the two middle values; the MI terciles and the barrier stop ladder used nearest-rank indexing. All four now go through MathMedian / MathQuantile, which is R's type 7 and the library's one answer. System\AltData.mqh column median -> MathMedian (exact, no change) AIBase\Labels.mqh swing median -> MathMedian leg-range med -> MathMedian stop ladder -> MathQuantile, read in one call AIBase\Topology.mqh window median -> MathMedian AIBase\AutoTune.mqh MI terciles -> MathQuantile + MathMin/MathMax Signals\SignalSessionFilter DST last Sunday-> CDateTime::DaysInMonth() gaps[]/legs[] change from int to double so MathMedian can read them; the values are bar counts either way. VALUES MOVE. Even-sample medians shift by half a bin and the quantile reads interpolate, so the barrier geometry and the derived input window can land on different rungs - re-keying fingerprints and forcing a retrain. Accepted deliberately: stdlib consistency was the ask, and three private conventions for one statistic is what it buys out. Two YAGNI finds fell out of the ladder rewrite. MathQuantile sorts its own copy, so DeriveBarrierGeometry no longer sorts up[]/dn[] in place - which means upUnsorted[], a full array copy kept only to undo that sort, is gone. ArraySort(up) had no consumer needing order at all; it was pure work. The library call also gets a failure guard the hand-rolled indexing never needed but the ladder read does. Verified while here: Math\Stat\Math.mqh's MathAbs/MathMax/MathSqrt/MathPow and friends are ARRAY overloads, not scalar redefinitions, so pulling it into the translation unit shadows no builtin. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-19 20:16:03 -04:00
if(cut1 == cut2 && MathMin(vals) == MathMax(vals))
diag: is "optimal SL/TP" learnable? Score the features against excursions Proposed direction: train the net to predict entry/SL/TP that maximise return and minimise drawdown, rather than to classify direction. Before rebuilding a head, measure whether the target is learnable at all. That question splits into two that behave nothing alike: HOW FAR price travels (MFE/MAE) - essentially volatility, and volatility clustering is about the most robust regularity in markets. WHICH WAY it goes first (the asymmetry) - direction, which is what every noise-floor verdict in this project has been about. Expectancy comes ONLY from the second. The first buys position sizing and drawdown control - worth having under prop-firm limits, but not an edge: exit management on RANDOM entries already moved the payoff ratio 0.92 -> 5.72 with expectancy FLAT. Crucially this is NOT already answered. Every MI figure here scored the triple-barrier label, i.e. one specific question at one fixed geometry. A noise-floor result there says nothing about whether excursion MAGNITUDE is learnable - different target, different answer. Four targets, and the verdict is the CONTRAST, printed explicitly because the dangerous misreading of "UP clears" is "we can predict profitable trades": RANGE (up+dn) - realised volatility, included as a POSITIVE CONTROL that SHOULD clear. Every prior verdict here lacked a control expected to pass; a range target at the floor indicts the measurement, not the market. UP / DOWN - MFE / MAE. ASYMMETRY - up-dn, the only one that can pay. Collected inside the walk the label already does (one max, one min per bar). The early-out when both barriers resolved is GONE: it would have truncated the excursions at whichever bar tripped the last barrier, making the measurement a function of the CURRENT SL/TP - the circularity this is trying to escape. The loop was already bounded by the horizon, so only the average cost moves. Discretised into 3 EQUAL-FREQUENCY bins, so every downstream piece (block permutation, null, p-value) is reused unchanged. Equal-frequency because MFE is fat-tailed and fixed-width bins would put nearly every row in bin 0; it also pins H(Y) at ln(3)=1.099 for all four, making them comparable to each other and to the barrier label's ~1.02 instead of confounded by class balance. Two bugs fixed in this code before it ever ran, both of which would have produced a plausible quiet wrong answer rather than an error: - TripleBarrierLabel early-returns on invalid ATR/close BEFORE the point the accumulators were reset, so one bar's excursions would be cached under another bar's index. Cleared at the top now, ahead of every return. - An unresolvable bar is still flagged as labelled but carries excursions of exactly 0. Under equal-frequency binning a block of identical zeros drags the lowest cut onto zero and a third of the sample lands in one uninformative bin - a depressed score that reads as "not predictable", a false negative in the direction that would wrongly kill the idea. Rows where both excursions are zero are dropped; price cannot travel zero both ways over a whole horizon. Read-only diagnostic. No topology or label change: no retrain of its own. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-07 10:22:41 -04:00
{
Print(ID + ": MI excursion target " + IntegerToString(target) + " is CONSTANT across all "
+ IntegerToString(n) + " sampled bars - the excursion cache did not fill. Treating as "
"unusable rather than reporting its zero score as a measurement.");
return -1;
}
for(int q = 0; q < n; q++)
labels[q] = (raw[q] <= cut1) ? 0 : ((raw[q] <= cut2) ? 1 : 2);
}
diag(autotune): five permutations was still a coin flip - use a real test The 5-draw z-score shipped an hour ago disproved itself on its first run. All four charts scored the IDENTICAL 0.00401 nats on identical features and identical labels - and reported z of +1.3, +2.0, +4.0 and +4.7. Two "AT THE NOISE FLOOR", two "a real association", same data. The entire swing came from estimating the null's spread from five draws, where the standard deviation of the standard-deviation estimate is ~35%: the denominator was noisier than the effect it was judging. Replaced with an empirical permutation test. 200 draws, p counted by rank with the +1/(B+1) correction (Phipson & Smyth 2010) so p is never reported as exactly zero - no normality assumption and no spread to estimate. The strongest single column is tested against the null distribution OF THE MAXIMUM, which corrects for scoring 26 features at once by construction and is far less conservative than Bonferroni. Affordable because BuildMiSample is now split out of ScoreCurrentParamsByMI and runs ONCE for the whole test - every draw reuses that sample and costs a relabel plus 26 histogram passes, not 2000 feature extractions. The coordinate sweep still calls the combined form, which is correct there: each candidate changes the indicator settings, so its features really do have to be re-extracted. The verdict line keeps both questions apart and prints both answers: the p-value for "is it real", the excess as a percentage of H(Y) for "is it big enough to trade". At n=2000 those can disagree, and collapsing them into one word is how a worthless effect gets called a discovery. Compiles 0 errors / 0 warnings. Build tag permtest-v1. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-01 13:45:46 -04:00
return n;
}
//+------------------------------------------------------------------+
refactor(comments): box headers to stdlib length The //| box blocks were excluded from 0b06f8e and 5efdb48 and were what remained: 160 of them ran to 10+ lines, the longest to 88. Compressed to their leading topic sentences - 5 lines for a function header, 8 for a file header - keeping the box format and the standard MQL5 name/author lines verbatim. Verified at the BYTE level this time, across every in-scope file: the list of non-comment lines is byte-identical to HEAD and braces balance. The first check compared a locale-decoded 'git show' against a UTF-8 read and flagged 25 files that had not changed at all - every BOM and every non-ASCII line mismatched. 47,696 -> 40,665 lines in scope; comment share 38% -> 26%. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-22 00:30:14 -04:00
//| Score an already-extracted sample. Split out from the extraction |
//| above so the permutation test can reuse ONE sample across every |
//| draw: feature extraction dominates the cost, and re-running it |
//| per shuffle is what would have made a few hundred permutations |
//| unaffordable. |
diag(autotune): five permutations was still a coin flip - use a real test The 5-draw z-score shipped an hour ago disproved itself on its first run. All four charts scored the IDENTICAL 0.00401 nats on identical features and identical labels - and reported z of +1.3, +2.0, +4.0 and +4.7. Two "AT THE NOISE FLOOR", two "a real association", same data. The entire swing came from estimating the null's spread from five draws, where the standard deviation of the standard-deviation estimate is ~35%: the denominator was noisier than the effect it was judging. Replaced with an empirical permutation test. 200 draws, p counted by rank with the +1/(B+1) correction (Phipson & Smyth 2010) so p is never reported as exactly zero - no normality assumption and no spread to estimate. The strongest single column is tested against the null distribution OF THE MAXIMUM, which corrects for scoring 26 features at once by construction and is far less conservative than Bonferroni. Affordable because BuildMiSample is now split out of ScoreCurrentParamsByMI and runs ONCE for the whole test - every draw reuses that sample and costs a relabel plus 26 histogram passes, not 2000 feature extractions. The coordinate sweep still calls the combined form, which is correct there: each candidate changes the indicator settings, so its features really do have to be re-extracted. The verdict line keeps both questions apart and prints both answers: the p-value for "is it real", the excess as a percentage of H(Y) for "is it big enough to trade". At n=2000 those can disagree, and collapsing them into one word is how a worthless effect gets called a discovery. Compiles 0 errors / 0 warnings. Build tag permtest-v1. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-01 13:45:46 -04:00
//+------------------------------------------------------------------+
double CExpertSignalAIBase::ScoreMiSample(const double &cols[], int &labels[], int n, bool shuffleLabels)
feat(mi): per-column feature screen, and fix the block permutation it rides on CFeatureSelector keeps the per-column MI vector ScoreMiSample has always computed and thrown away. It is fed from inside the 200 draws ReportFeatureLabelInformation already performs, so the screen costs an array copy per draw and not one extra mutual-information computation. The keep-mask is cut on the single-step maxT (Westfall-Young) statistic - a column must beat the MAXIMUM of a null draw over all columns, which is strong family-wise control needing no Bonferroni factor, and is the same null of the maximum the headline verdict already trusts. The uncorrected per-comparison p is reported alongside it; the gap between the two counts IS the multiplicity correction, shown rather than described. Checked offline at 40 columns: 0/40 noise runs keep anything, where the uncorrected rule hands back ~2 columns per run, and a planted column is recovered 40/40. REPORT-ONLY. Nothing reads the mask. Pruning changes m_neuronsCount, which is in BuildModelFingerprint(), which invalidates every .nnw - that is a retrain across every chart and an operator's call to make after reading the report. Also fixes the block permutation, found while moving it. When blockRows did not divide n the short last block, drawn to a non-final slot, read past the end of the array; the read was clamped to labels[n-1], duplicating one label and truncating whichever block landed last. 18 of the 24 possible block orders on n=10/blockRows=3 altered the class counts. A duplicated label concentrates the class distribution, lowering H(Y) and so the null MI those draws can reach, so p-values leaned toward significance - the permissive direction, and m_dirEvidence is a deploy gate. Each block now contributes exactly its own length. The invariance the old comment asserted ("a permutation preserves the class counts - that invariance is itself a check on the shuffle") was never actually compared anywhere; BlockPermute now checks it and returns false, and all six shuffled call sites already guard on a negative return. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-23 18:45:00 -04:00
{
//--- Scratch the caller did not ask for. The two forms share ONE arithmetic so a caller reading the
//--- mean and a caller reading the columns can never be looking at two different measurements.
double perColumn[];
return ScoreMiSample(cols, labels, n, shuffleLabels, perColumn);
}
//+------------------------------------------------------------------+
//| See the declaration. perColumn[] comes back holding this sample's |
//| mutual information for every column, which is what the per-column |
//| screen accumulates its null from. |
//+------------------------------------------------------------------+
double CExpertSignalAIBase::ScoreMiSample(const double &cols[], int &labels[], int n, bool shuffleLabels,
double &perColumn[])
diag(autotune): five permutations was still a coin flip - use a real test The 5-draw z-score shipped an hour ago disproved itself on its first run. All four charts scored the IDENTICAL 0.00401 nats on identical features and identical labels - and reported z of +1.3, +2.0, +4.0 and +4.7. Two "AT THE NOISE FLOOR", two "a real association", same data. The entire swing came from estimating the null's spread from five draws, where the standard deviation of the standard-deviation estimate is ~35%: the denominator was noisier than the effect it was judging. Replaced with an empirical permutation test. 200 draws, p counted by rank with the +1/(B+1) correction (Phipson & Smyth 2010) so p is never reported as exactly zero - no normality assumption and no spread to estimate. The strongest single column is tested against the null distribution OF THE MAXIMUM, which corrects for scoring 26 features at once by construction and is far less conservative than Bonferroni. Affordable because BuildMiSample is now split out of ScoreCurrentParamsByMI and runs ONCE for the whole test - every draw reuses that sample and costs a relabel plus 26 histogram passes, not 2000 feature extractions. The coordinate sweep still calls the combined form, which is correct there: each candidate changes the indicator settings, so its features really do have to be re-extracted. The verdict line keeps both questions apart and prints both answers: the p-value for "is it real", the excess as a percentage of H(Y) for "is it big enough to trade". At n=2000 those can disagree, and collapsing them into one word is how a worthless effect gets called a discovery. Compiles 0 errors / 0 warnings. Build tag permtest-v1. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-01 13:45:46 -04:00
{
perf(autotune): replace the genetic search with a filter score - hours to seconds MEASURED COST OF THE GA, which is what retired it. Per generation: rung 0: 8 cand x 3 seeds x 3 eras = 72 eras rung 1: 4 cand x 3 seeds x 8 eras = 96 rung 2: 2 cand x 3 seeds x 20 eras = 120 = 288 eras/generation x 4 generations = 1152 eras BEFORE the winner's real training began. Against the observed era times on SP500 H1: PAI 29.1 s/era -> 9.3 h (matches the observed 00:37 -> 09:22) CONV 41.3 s/era -> 13.2 h LSTM 150.4 s/era -> 48.1 h HYBRID 154.6 s/era -> 49.5 h Two days to tune is not a first-run experience, and it is the phase in which the panel goes quiet, which is what made it look like a hang. It also bought nothing. The space is 90 points (10 MA periods x 9 MA types), so 1152 evaluations revisited each point ~13 times; and rungs of 3 and 8 eras cannot separate two MA periods at all. The 2026-08-01 run proves it: every finalist scored 25.0-25.9% balanced accuracy - below the 33.3% one-class floor, i.e. indistinguishable noise - and the search then "deployed the winner" of that. THE ERROR WAS THE SCORING FUNCTION, not its constants. Using a full training run to choose a feature's period is a wrapper method paying wrapper prices for a decision that does not need one. The reference book does not do this: ch. 3.3 selects inputs by measuring each candidate indicator's CORRELATION with the target and dropping the ones with none, with no network involved. So: rank candidates by the MUTUAL INFORMATION between the resulting feature vector and the triple-barrier label. MI rather than correlation because the label is 3-class categorical and the features are not monotonically related to it. Equal-FREQUENCY binning (rank-based), because these features are ATR-normalised and heavy-tailed - fixed-width bins put nearly everything in one bucket and report ~0 information for a genuinely useful feature. Scoring is arithmetic over the feature cache, so it costs seconds and its cost is independent of topology: LSTM now tunes as fast as the MLP. Coordinate sweep, not product sweep - cost is the SUM of per-parameter candidate counts, so enabling every indicator stays affordable - with a second pass that breaks early once nothing moves. Sampling is IS-ONLY. Letting the OOS window influence which indicator settings ship would mean the holdout had been used for selection and had stopped being a holdout. HONEST LIMIT, recorded because it is the price: MI is marginal, so a parameter that only pays off in combination with another can be missed (Guyon & Elisseeff 2003, filter vs wrapper). Given the wrapper it replaces was ranking pure noise at 48 h a run, this is strictly better. Deleted with it: GaRungEras/GaExtract/GaStore/GaMutate/GaRandomCandidate/ GaBlockCrossover/GaSortAliveByScoreDesc/GaBreedNextGeneration, 14 m_ga* members, the GA_*/TUNE_POP_* constants, and ComputeTuneTrialBudget. AND m_evalMode/m_evalEraBudget, because nothing set them any more - 28 read sites all permanently inert. That is not a tidy-up: the `if (!m_evalMode)` guard on UpdateClassPriors is exactly what silently disabled the imbalance correction for entire runs two commits ago. Dead machinery that still reads like live machinery is this codebase's most expensive recurring bug, and leaving 28 more instances of it would have been indefensible. The panel's tuning-progress state goes too - tuning no longer takes long enough to need one. Both builds compile 0 errors / 0 warnings. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-01 10:29:31 -04:00
if(n < MI_MIN_SAMPLES)
return -1.0;
refactor(comments): stdlib comment style across the remaining in-scope files Same pass as 0b06f8e, applied file by file: comment runs of 4+ lines compressed to their leading topic sentences, capped at 4 lines, whole sentences only. Warning sentences (NEVER / MUST / trap / would-have) survive the budget. Every file was checked the same way before committing: the list of non-comment lines is byte-identical to HEAD, and braces balance. No code was touched. Panel/, Enumerations/ and the already-terse System headers needed little or nothing - PooledGate, TradeChecks, BinomialStats and Random came through with no blocks over the threshold at all. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-22 00:25:52 -04:00
//--- PERMUTATION BASELINE. So a raw MI figure is uninterpretable on its own: 0.004 nats could be
//--- a genuine weak signal or could be pure noise.
//--- BLOCK permutation, not a free one, and the difference is the whole validity of the test. That
//--- was label autocorrelation leaking through an independence assumption, not an edge. It is Lopez
//--- de Prado ch.
fix(autotune): MI scorer read an array nobody filled; add the permutation floor THE TUNER WAS A SILENT NO-OP. Every chart logged auto-tune complete - 17 candidate settings scored in ~139s, feature/label mutual information 0.0000 -> 0.0000 nats (no improvement) 0.0000 is not a weak result, it is a broken measurement: finite-sample MI is biased UPWARD, so even pure noise scores above zero. Cause: ScoreCurrentParamsByMI called BufferTempDataCompute(), which APPENDS the bar's features to TempData and never touches m_featureCache - only the caching wrapper BufferTempData() writes that array. It then read m_featureCache, which ReInitADIndicators had just invalidated. Every column came back constant, FeatureColumnMI returned 0 for all of them, and all 17 candidates tied at exactly zero. 139 s per chart to return the settings it started with. Now reads the values back out of TempData, where they actually land. And an exactly-zero best score is called out as a fault rather than reported as "no improvement", because that is what it is. ADDED: a PERMUTATION BASELINE, which is the diagnostic this project has been missing. MI's finite-sample bias is ~(bins-1)(classes-1)/(2n) nats - at these sample sizes the same order as any real edge in this domain - so a raw MI figure is uninterpretable on its own. Shuffling the labels destroys every genuine association while leaving sample size, binning and class proportions intact, so the score it produces IS this dataset's noise floor, measured rather than approximated. The log now reads feature/label information - X nats against a shuffled-label floor of Y and says outright whether the features carry usable information about the target. It needs no training, no topology and no convergence, so unlike every accuracy number in this codebase it cannot be confounded by an optimizer or an objective. If the score sits on the floor, no change of architecture can help - which is the question the last three days of zero-edge results have been circling. DEPLOY FLOOR: `dirPrecPct > chancePrecPct` passed anything above chance by any amount. At ~11,000 directional calls the standard error of the precision estimate is ~0.4pp, so that gate was accepting sub-one-sigma noise - the perceptron deployed at edge +0pp on 2026-08-01. Now requires EDGE_MIN_SIGMAS (2.0) standard errors above chance, computed from the actual call count, so the bar scales with the evidence instead of needing a hand-picked constant. Recorded with it, because it is why chance is the right reference at all: under a driftless random walk P(touch +k*ATR before -m*ATR) = m/(m+k), and the break-even win rate for a k:m reward:risk trade is ALSO m/(m+k). The label's own base rate IS the break-even rate, at every SL/TP setting. So "beats chance" and "is profitable" are the same test, and no choice of SL/TP can manufacture an edge - only prediction can. Both builds compile 0 errors / 0 warnings. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-01 13:05:50 -04:00
if(shuffleLabels)
{
fix(diag): the symbol sweep was measuring its own sampling, not the market Twelve cells came back with higher-timeframe "signal" 5-9x anything on H1, at p=0.005. It was an artifact, and the sweep's own columns gave it away: excess tracked the sampling STRIDE almost monotonically, and the three D1 cells - stride collapsed to 1-5 bars against a 128-bar horizon, i.e. ~99% window overlap - were the three highest. Three flaws, all the same family: comparing numbers without the spread that belongs to them. 1. THE NULL ASSUMED INDEPENDENCE THE LABELS DO NOT HAVE. Triple-barrier labels overlap; two rows less than one horizon apart share most of their outcome window. A free Fisher-Yates shuffle destroys that dependence along with the association, making the null far narrower than the truth and handing out significance that isn't there - Lopez de Prado ch. 4 arriving through the back door of the significance test. Now permutes contiguous BLOCKS of at least one horizon, so the null keeps the autocorrelation and the p-value means what it says. It degrades honestly: severe overlap leaves few blocks, the null widens, nothing reaches significance. The block count is now printed, because THAT - not the row count - is the sample size a p-value rests on, and a warning fires under 30 blocks so "not significant" is not misread as "no signal" when it means "not enough independent history to tell". 2. THE POSITIVE CONTROL'S STRENGTH DEPENDED ON THE DATASET. It paired each row's label with the NEXT SAMPLE ROW's, whose distance is the stride - so on M5, where stride ran 160-717 bars against a 128-bar horizon, it was pairing two windows that never overlap. All three M5 cells duly reported a FAILED estimator and voided their own results with nothing wrong. A control whose strength varies with the cell cannot certify the cell. Now pinned to a quarter of the horizon, where ~75% overlap is guaranteed by construction. 3. THE LOOKAHEAD VERDICT HAD NO MARGIN. It flagged 7 of 12 cells on gaps of 0.00008-0.00040 nats against a measured null sd of ~0.00030 - noise, every one. Now requires 3 sd, the same discipline the deploy floor applies to precision. Compiles 0 errors / 0 warnings, standard and Market. Build tag blockperm-v1. Supersedes every number from the sweep. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-01 15:11:40 -04:00
int blockRows = (m_miStrideBars > 0)
? (int)MathCeil((double)MathMax(m_barrierHorizonBars, 1) / m_miStrideBars) : 1;
if(blockRows < 1)
blockRows = 1;
if(blockRows > n)
blockRows = n;
feat(mi): per-column feature screen, and fix the block permutation it rides on CFeatureSelector keeps the per-column MI vector ScoreMiSample has always computed and thrown away. It is fed from inside the 200 draws ReportFeatureLabelInformation already performs, so the screen costs an array copy per draw and not one extra mutual-information computation. The keep-mask is cut on the single-step maxT (Westfall-Young) statistic - a column must beat the MAXIMUM of a null draw over all columns, which is strong family-wise control needing no Bonferroni factor, and is the same null of the maximum the headline verdict already trusts. The uncorrected per-comparison p is reported alongside it; the gap between the two counts IS the multiplicity correction, shown rather than described. Checked offline at 40 columns: 0/40 noise runs keep anything, where the uncorrected rule hands back ~2 columns per run, and a planted column is recovered 40/40. REPORT-ONLY. Nothing reads the mask. Pruning changes m_neuronsCount, which is in BuildModelFingerprint(), which invalidates every .nnw - that is a retrain across every chart and an operator's call to make after reading the report. Also fixes the block permutation, found while moving it. When blockRows did not divide n the short last block, drawn to a non-final slot, read past the end of the array; the read was clamped to labels[n-1], duplicating one label and truncating whichever block landed last. 18 of the 24 possible block orders on n=10/blockRows=3 altered the class counts. A duplicated label concentrates the class distribution, lowering H(Y) and so the null MI those draws can reach, so p-values leaned toward significance - the permissive direction, and m_dirEvidence is a deploy gate. Each block now contributes exactly its own length. The invariance the old comment asserted ("a permutation preserves the class counts - that invariance is itself a check on the shuffle") was never actually compared anywhere; BlockPermute now checks it and returns false, and all six shuffled call sites already guard on a negative return. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-23 18:45:00 -04:00
//--- The permutation and its class-count invariance check both live in CFeatureSelector - see
//--- the ragged-tail note there for what the version written out here got wrong. A draw that
//--- fails the check is not scored: returning -1.0 makes the caller skip it, which shrinks the
//--- null by one draw rather than poisoning it with a sample that is not a permutation.
if(!CFeatureSelector::BlockPermute(labels, n, blockRows, m_miNullBlocks))
return -1.0;
fix(autotune): MI scorer read an array nobody filled; add the permutation floor THE TUNER WAS A SILENT NO-OP. Every chart logged auto-tune complete - 17 candidate settings scored in ~139s, feature/label mutual information 0.0000 -> 0.0000 nats (no improvement) 0.0000 is not a weak result, it is a broken measurement: finite-sample MI is biased UPWARD, so even pure noise scores above zero. Cause: ScoreCurrentParamsByMI called BufferTempDataCompute(), which APPENDS the bar's features to TempData and never touches m_featureCache - only the caching wrapper BufferTempData() writes that array. It then read m_featureCache, which ReInitADIndicators had just invalidated. Every column came back constant, FeatureColumnMI returned 0 for all of them, and all 17 candidates tied at exactly zero. 139 s per chart to return the settings it started with. Now reads the values back out of TempData, where they actually land. And an exactly-zero best score is called out as a fault rather than reported as "no improvement", because that is what it is. ADDED: a PERMUTATION BASELINE, which is the diagnostic this project has been missing. MI's finite-sample bias is ~(bins-1)(classes-1)/(2n) nats - at these sample sizes the same order as any real edge in this domain - so a raw MI figure is uninterpretable on its own. Shuffling the labels destroys every genuine association while leaving sample size, binning and class proportions intact, so the score it produces IS this dataset's noise floor, measured rather than approximated. The log now reads feature/label information - X nats against a shuffled-label floor of Y and says outright whether the features carry usable information about the target. It needs no training, no topology and no convergence, so unlike every accuracy number in this codebase it cannot be confounded by an optimizer or an objective. If the score sits on the floor, no change of architecture can help - which is the question the last three days of zero-edge results have been circling. DEPLOY FLOOR: `dirPrecPct > chancePrecPct` passed anything above chance by any amount. At ~11,000 directional calls the standard error of the precision estimate is ~0.4pp, so that gate was accepting sub-one-sigma noise - the perceptron deployed at edge +0pp on 2026-08-01. Now requires EDGE_MIN_SIGMAS (2.0) standard errors above chance, computed from the actual call count, so the bar scales with the evidence instead of needing a hand-picked constant. Recorded with it, because it is why chance is the right reference at all: under a driftless random walk P(touch +k*ATR before -m*ATR) = m/(m+k), and the break-even win rate for a k:m reward:risk trade is ALSO m/(m+k). The label's own base rate IS the break-even rate, at every SL/TP setting. So "beats chance" and "is profitable" are the same test, and no choice of SL/TP can manufacture an edge - only prediction can. Both builds compile 0 errors / 0 warnings. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-01 13:05:50 -04:00
}
diag(autotune): one label shuffle cannot settle the no-edge question The permutation baseline added in 018afb1 came back on all four charts as 0.00401 nats against floors of 0.00267 / 0.00298 / 0.00318 - three draws whose spread is as wide as the excess being judged, because one shuffle is one sample from the null, not the null. That is not enough to retire a topology on. Now MI_NOISE_PERMUTATIONS draws, reported as mean +/- sd with a z-score, plus two numbers the mean over 26 columns cannot express: - the STRONGEST single feature's MI, against its own shuffled value. One informative column among 25 useless ones is precisely the case the mean hides, and precisely the case worth finding. - the excess as a percentage of H(Y). At these sample sizes a z-score can be comfortably significant while the effect is worthless, so "is it real" and "is it big enough to matter" are asked separately and answered separately. The verdict line also now states the measure's limit every time rather than only when the news is bad: this is a MARGINAL, PER-BAR statistic and the network reads m_historyBars bars jointly, so it can prove signal exists but never that it does not. It rules out a per-feature edge - and therefore any indicator retuning - not an edge that lives in a combination or across time. Compiles 0 errors / 0 warnings, standard and Market. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-01 13:32:12 -04:00
//--- H(Y) over the sampled labels, so the caller can express MI as a fraction of the information the
feat(mi): per-column feature screen, and fix the block permutation it rides on CFeatureSelector keeps the per-column MI vector ScoreMiSample has always computed and thrown away. It is fed from inside the 200 draws ReportFeatureLabelInformation already performs, so the screen costs an array copy per draw and not one extra mutual-information computation. The keep-mask is cut on the single-step maxT (Westfall-Young) statistic - a column must beat the MAXIMUM of a null draw over all columns, which is strong family-wise control needing no Bonferroni factor, and is the same null of the maximum the headline verdict already trusts. The uncorrected per-comparison p is reported alongside it; the gap between the two counts IS the multiplicity correction, shown rather than described. Checked offline at 40 columns: 0/40 noise runs keep anything, where the uncorrected rule hands back ~2 columns per run, and a planted column is recovered 40/40. REPORT-ONLY. Nothing reads the mask. Pruning changes m_neuronsCount, which is in BuildModelFingerprint(), which invalidates every .nnw - that is a retrain across every chart and an operator's call to make after reading the report. Also fixes the block permutation, found while moving it. When blockRows did not divide n the short last block, drawn to a non-final slot, read past the end of the array; the read was clamped to labels[n-1], duplicating one label and truncating whichever block landed last. 18 of the 24 possible block orders on n=10/blockRows=3 altered the class counts. A duplicated label concentrates the class distribution, lowering H(Y) and so the null MI those draws can reach, so p-values leaned toward significance - the permissive direction, and m_dirEvidence is a deploy gate. Each block now contributes exactly its own length. The invariance the old comment asserted ("a permutation preserves the class counts - that invariance is itself a check on the shuffle") was never actually compared anywhere; BlockPermute now checks it and returns false, and all six shuffled call sites already guard on a negative return. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-23 18:45:00 -04:00
//--- label actually contains. Computed AFTER any shuffle, which leaves it unchanged by construction:
//--- a permutation preserves the class counts. That invariance is now genuinely CHECKED, inside
//--- BlockPermute, which returns false if it fails - this comment used to claim the invariance was
//--- "itself a check on the shuffle" while nothing anywhere compared the counts, and the shuffle it
//--- was vouching for had in fact been breaking it whenever blockRows did not divide n.
diag(autotune): one label shuffle cannot settle the no-edge question The permutation baseline added in 018afb1 came back on all four charts as 0.00401 nats against floors of 0.00267 / 0.00298 / 0.00318 - three draws whose spread is as wide as the excess being judged, because one shuffle is one sample from the null, not the null. That is not enough to retire a topology on. Now MI_NOISE_PERMUTATIONS draws, reported as mean +/- sd with a z-score, plus two numbers the mean over 26 columns cannot express: - the STRONGEST single feature's MI, against its own shuffled value. One informative column among 25 useless ones is precisely the case the mean hides, and precisely the case worth finding. - the excess as a percentage of H(Y). At these sample sizes a z-score can be comfortably significant while the effect is worthless, so "is it real" and "is it big enough to matter" are asked separately and answered separately. The verdict line also now states the measure's limit every time rather than only when the news is bad: this is a MARGINAL, PER-BAR statistic and the network reads m_historyBars bars jointly, so it can prove signal exists but never that it does not. It rules out a per-feature edge - and therefore any indicator retuning - not an edge that lives in a combination or across time. Compiles 0 errors / 0 warnings, standard and Market. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-01 13:32:12 -04:00
int classCount[3] = {0, 0, 0};
for(int k = 0; k < n; k++)
classCount[labels[k]]++;
m_miLabelEntropy = 0.0;
for(int c = 0; c < 3; c++)
{
if(classCount[c] <= 0)
continue;
double pc = (double)classCount[c] / n;
m_miLabelEntropy -= pc * MathLog(pc);
}
perf(autotune): replace the genetic search with a filter score - hours to seconds MEASURED COST OF THE GA, which is what retired it. Per generation: rung 0: 8 cand x 3 seeds x 3 eras = 72 eras rung 1: 4 cand x 3 seeds x 8 eras = 96 rung 2: 2 cand x 3 seeds x 20 eras = 120 = 288 eras/generation x 4 generations = 1152 eras BEFORE the winner's real training began. Against the observed era times on SP500 H1: PAI 29.1 s/era -> 9.3 h (matches the observed 00:37 -> 09:22) CONV 41.3 s/era -> 13.2 h LSTM 150.4 s/era -> 48.1 h HYBRID 154.6 s/era -> 49.5 h Two days to tune is not a first-run experience, and it is the phase in which the panel goes quiet, which is what made it look like a hang. It also bought nothing. The space is 90 points (10 MA periods x 9 MA types), so 1152 evaluations revisited each point ~13 times; and rungs of 3 and 8 eras cannot separate two MA periods at all. The 2026-08-01 run proves it: every finalist scored 25.0-25.9% balanced accuracy - below the 33.3% one-class floor, i.e. indistinguishable noise - and the search then "deployed the winner" of that. THE ERROR WAS THE SCORING FUNCTION, not its constants. Using a full training run to choose a feature's period is a wrapper method paying wrapper prices for a decision that does not need one. The reference book does not do this: ch. 3.3 selects inputs by measuring each candidate indicator's CORRELATION with the target and dropping the ones with none, with no network involved. So: rank candidates by the MUTUAL INFORMATION between the resulting feature vector and the triple-barrier label. MI rather than correlation because the label is 3-class categorical and the features are not monotonically related to it. Equal-FREQUENCY binning (rank-based), because these features are ATR-normalised and heavy-tailed - fixed-width bins put nearly everything in one bucket and report ~0 information for a genuinely useful feature. Scoring is arithmetic over the feature cache, so it costs seconds and its cost is independent of topology: LSTM now tunes as fast as the MLP. Coordinate sweep, not product sweep - cost is the SUM of per-parameter candidate counts, so enabling every indicator stays affordable - with a second pass that breaks early once nothing moves. Sampling is IS-ONLY. Letting the OOS window influence which indicator settings ship would mean the holdout had been used for selection and had stopped being a holdout. HONEST LIMIT, recorded because it is the price: MI is marginal, so a parameter that only pays off in combination with another can be missed (Guyon & Elisseeff 2003, filter vs wrapper). Given the wrapper it replaces was ranking pure noise at 48 h a run, this is strictly better. Deleted with it: GaRungEras/GaExtract/GaStore/GaMutate/GaRandomCandidate/ GaBlockCrossover/GaSortAliveByScoreDesc/GaBreedNextGeneration, 14 m_ga* members, the GA_*/TUNE_POP_* constants, and ComputeTuneTrialBudget. AND m_evalMode/m_evalEraBudget, because nothing set them any more - 28 read sites all permanently inert. That is not a tidy-up: the `if (!m_evalMode)` guard on UpdateClassPriors is exactly what silently disabled the imbalance correction for entire runs two commits ago. Dead machinery that still reads like live machinery is this codebase's most expensive recurring bug, and leaving 28 more instances of it would have been indefensible. The panel's tuning-progress state goes too - tuning no longer takes long enough to need one. Both builds compile 0 errors / 0 warnings. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-01 10:29:31 -04:00
double colVals[];
ArrayResize(colVals, n);
feat(mi): per-column feature screen, and fix the block permutation it rides on CFeatureSelector keeps the per-column MI vector ScoreMiSample has always computed and thrown away. It is fed from inside the 200 draws ReportFeatureLabelInformation already performs, so the screen costs an array copy per draw and not one extra mutual-information computation. The keep-mask is cut on the single-step maxT (Westfall-Young) statistic - a column must beat the MAXIMUM of a null draw over all columns, which is strong family-wise control needing no Bonferroni factor, and is the same null of the maximum the headline verdict already trusts. The uncorrected per-comparison p is reported alongside it; the gap between the two counts IS the multiplicity correction, shown rather than described. Checked offline at 40 columns: 0/40 noise runs keep anything, where the uncorrected rule hands back ~2 columns per run, and a planted column is recovered 40/40. REPORT-ONLY. Nothing reads the mask. Pruning changes m_neuronsCount, which is in BuildModelFingerprint(), which invalidates every .nnw - that is a retrain across every chart and an operator's call to make after reading the report. Also fixes the block permutation, found while moving it. When blockRows did not divide n the short last block, drawn to a non-final slot, read past the end of the array; the read was clamped to labels[n-1], duplicating one label and truncating whichever block landed last. 18 of the 24 possible block orders on n=10/blockRows=3 altered the class counts. A duplicated label concentrates the class distribution, lowering H(Y) and so the null MI those draws can reach, so p-values leaned toward significance - the permissive direction, and m_dirEvidence is a deploy gate. Each block now contributes exactly its own length. The invariance the old comment asserted ("a permutation preserves the class counts - that invariance is itself a check on the shuffle") was never actually compared anywhere; BlockPermute now checks it and returns false, and all six shuffled call sites already guard on a negative return. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-23 18:45:00 -04:00
//--- The per-column vector was previously computed and discarded, mean and max being all anyone
//--- kept. Keeping it is the whole of the per-column screen; it costs one array, not one MI.
ArrayResize(perColumn, m_neuronsCount);
perf(autotune): replace the genetic search with a filter score - hours to seconds MEASURED COST OF THE GA, which is what retired it. Per generation: rung 0: 8 cand x 3 seeds x 3 eras = 72 eras rung 1: 4 cand x 3 seeds x 8 eras = 96 rung 2: 2 cand x 3 seeds x 20 eras = 120 = 288 eras/generation x 4 generations = 1152 eras BEFORE the winner's real training began. Against the observed era times on SP500 H1: PAI 29.1 s/era -> 9.3 h (matches the observed 00:37 -> 09:22) CONV 41.3 s/era -> 13.2 h LSTM 150.4 s/era -> 48.1 h HYBRID 154.6 s/era -> 49.5 h Two days to tune is not a first-run experience, and it is the phase in which the panel goes quiet, which is what made it look like a hang. It also bought nothing. The space is 90 points (10 MA periods x 9 MA types), so 1152 evaluations revisited each point ~13 times; and rungs of 3 and 8 eras cannot separate two MA periods at all. The 2026-08-01 run proves it: every finalist scored 25.0-25.9% balanced accuracy - below the 33.3% one-class floor, i.e. indistinguishable noise - and the search then "deployed the winner" of that. THE ERROR WAS THE SCORING FUNCTION, not its constants. Using a full training run to choose a feature's period is a wrapper method paying wrapper prices for a decision that does not need one. The reference book does not do this: ch. 3.3 selects inputs by measuring each candidate indicator's CORRELATION with the target and dropping the ones with none, with no network involved. So: rank candidates by the MUTUAL INFORMATION between the resulting feature vector and the triple-barrier label. MI rather than correlation because the label is 3-class categorical and the features are not monotonically related to it. Equal-FREQUENCY binning (rank-based), because these features are ATR-normalised and heavy-tailed - fixed-width bins put nearly everything in one bucket and report ~0 information for a genuinely useful feature. Scoring is arithmetic over the feature cache, so it costs seconds and its cost is independent of topology: LSTM now tunes as fast as the MLP. Coordinate sweep, not product sweep - cost is the SUM of per-parameter candidate counts, so enabling every indicator stays affordable - with a second pass that breaks early once nothing moves. Sampling is IS-ONLY. Letting the OOS window influence which indicator settings ship would mean the holdout had been used for selection and had stopped being a holdout. HONEST LIMIT, recorded because it is the price: MI is marginal, so a parameter that only pays off in combination with another can be missed (Guyon & Elisseeff 2003, filter vs wrapper). Given the wrapper it replaces was ranking pure noise at 48 h a run, this is strictly better. Deleted with it: GaRungEras/GaExtract/GaStore/GaMutate/GaRandomCandidate/ GaBlockCrossover/GaSortAliveByScoreDesc/GaBreedNextGeneration, 14 m_ga* members, the GA_*/TUNE_POP_* constants, and ComputeTuneTrialBudget. AND m_evalMode/m_evalEraBudget, because nothing set them any more - 28 read sites all permanently inert. That is not a tidy-up: the `if (!m_evalMode)` guard on UpdateClassPriors is exactly what silently disabled the imbalance correction for entire runs two commits ago. Dead machinery that still reads like live machinery is this codebase's most expensive recurring bug, and leaving 28 more instances of it would have been indefensible. The panel's tuning-progress state goes too - tuning no longer takes long enough to need one. Both builds compile 0 errors / 0 warnings. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-01 10:29:31 -04:00
double total = 0.0;
diag(autotune): one label shuffle cannot settle the no-edge question The permutation baseline added in 018afb1 came back on all four charts as 0.00401 nats against floors of 0.00267 / 0.00298 / 0.00318 - three draws whose spread is as wide as the excess being judged, because one shuffle is one sample from the null, not the null. That is not enough to retire a topology on. Now MI_NOISE_PERMUTATIONS draws, reported as mean +/- sd with a z-score, plus two numbers the mean over 26 columns cannot express: - the STRONGEST single feature's MI, against its own shuffled value. One informative column among 25 useless ones is precisely the case the mean hides, and precisely the case worth finding. - the excess as a percentage of H(Y). At these sample sizes a z-score can be comfortably significant while the effect is worthless, so "is it real" and "is it big enough to matter" are asked separately and answered separately. The verdict line also now states the measure's limit every time rather than only when the news is bad: this is a MARGINAL, PER-BAR statistic and the network reads m_historyBars bars jointly, so it can prove signal exists but never that it does not. It rules out a per-feature edge - and therefore any indicator retuning - not an edge that lives in a combination or across time. Compiles 0 errors / 0 warnings, standard and Market. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-01 13:32:12 -04:00
m_miBestColumn = 0.0;
perf(autotune): replace the genetic search with a filter score - hours to seconds MEASURED COST OF THE GA, which is what retired it. Per generation: rung 0: 8 cand x 3 seeds x 3 eras = 72 eras rung 1: 4 cand x 3 seeds x 8 eras = 96 rung 2: 2 cand x 3 seeds x 20 eras = 120 = 288 eras/generation x 4 generations = 1152 eras BEFORE the winner's real training began. Against the observed era times on SP500 H1: PAI 29.1 s/era -> 9.3 h (matches the observed 00:37 -> 09:22) CONV 41.3 s/era -> 13.2 h LSTM 150.4 s/era -> 48.1 h HYBRID 154.6 s/era -> 49.5 h Two days to tune is not a first-run experience, and it is the phase in which the panel goes quiet, which is what made it look like a hang. It also bought nothing. The space is 90 points (10 MA periods x 9 MA types), so 1152 evaluations revisited each point ~13 times; and rungs of 3 and 8 eras cannot separate two MA periods at all. The 2026-08-01 run proves it: every finalist scored 25.0-25.9% balanced accuracy - below the 33.3% one-class floor, i.e. indistinguishable noise - and the search then "deployed the winner" of that. THE ERROR WAS THE SCORING FUNCTION, not its constants. Using a full training run to choose a feature's period is a wrapper method paying wrapper prices for a decision that does not need one. The reference book does not do this: ch. 3.3 selects inputs by measuring each candidate indicator's CORRELATION with the target and dropping the ones with none, with no network involved. So: rank candidates by the MUTUAL INFORMATION between the resulting feature vector and the triple-barrier label. MI rather than correlation because the label is 3-class categorical and the features are not monotonically related to it. Equal-FREQUENCY binning (rank-based), because these features are ATR-normalised and heavy-tailed - fixed-width bins put nearly everything in one bucket and report ~0 information for a genuinely useful feature. Scoring is arithmetic over the feature cache, so it costs seconds and its cost is independent of topology: LSTM now tunes as fast as the MLP. Coordinate sweep, not product sweep - cost is the SUM of per-parameter candidate counts, so enabling every indicator stays affordable - with a second pass that breaks early once nothing moves. Sampling is IS-ONLY. Letting the OOS window influence which indicator settings ship would mean the holdout had been used for selection and had stopped being a holdout. HONEST LIMIT, recorded because it is the price: MI is marginal, so a parameter that only pays off in combination with another can be missed (Guyon & Elisseeff 2003, filter vs wrapper). Given the wrapper it replaces was ranking pure noise at 48 h a run, this is strictly better. Deleted with it: GaRungEras/GaExtract/GaStore/GaMutate/GaRandomCandidate/ GaBlockCrossover/GaSortAliveByScoreDesc/GaBreedNextGeneration, 14 m_ga* members, the GA_*/TUNE_POP_* constants, and ComputeTuneTrialBudget. AND m_evalMode/m_evalEraBudget, because nothing set them any more - 28 read sites all permanently inert. That is not a tidy-up: the `if (!m_evalMode)` guard on UpdateClassPriors is exactly what silently disabled the imbalance correction for entire runs two commits ago. Dead machinery that still reads like live machinery is this codebase's most expensive recurring bug, and leaving 28 more instances of it would have been indefensible. The panel's tuning-progress state goes too - tuning no longer takes long enough to need one. Both builds compile 0 errors / 0 warnings. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-01 10:29:31 -04:00
for(int f = 0; f < m_neuronsCount; f++)
refactor: split CExpertSignalAIBase implementation by responsibility ExpertSignalAIBase.mqh was 8216 lines: the class declaration followed by 87 method bodies covering training, labelling, feature extraction, persistence, chart drawing, online learning, the GA auto-tuner and inference, all in one file. Train() alone is 1492 lines; a change to arrow drawing meant scrolling past the era loop. Moved the bodies into Expert\AIBase\, included at the bottom of the original after the class declaration: Training.mqh 1607 era loop, plateau ladder, checkpoint select, deploy Features.mqh 1093 indicator creation + per-bar input feature vector ChartUI.mqh 634 arrows, arrow persistence, status panel, cleanup Persistence.mqh 492 .stats/.cfg sidecars, CPU-inference validation, copy OnlineLearning.mqh 461 live continual learning, EMA shadow, OOS simulator Labels.mqh 309 ZigZag pivot labels, async label-cache prebuild AutoTune.mqh 275 genetic tuner (population, crossover, halving) Inference.mqh 235 softmax, prior calibration, class priors ExpertSignalAIBase.mqh 8216 -> 3131 (declaration + topology build only) This is a pure relocation - verified mechanically, not by eye: HEAD's file reconstructed from the eight partials plus the surviving remainder is byte-identical to HEAD, span for span (scratchpad verify_split.py). No declaration moved, no signature changed, no code rewritten, so behaviour is unchanged by construction. Compiles 0 errors, 0 warnings. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-29 00:42:45 -04:00
{
perf(autotune): replace the genetic search with a filter score - hours to seconds MEASURED COST OF THE GA, which is what retired it. Per generation: rung 0: 8 cand x 3 seeds x 3 eras = 72 eras rung 1: 4 cand x 3 seeds x 8 eras = 96 rung 2: 2 cand x 3 seeds x 20 eras = 120 = 288 eras/generation x 4 generations = 1152 eras BEFORE the winner's real training began. Against the observed era times on SP500 H1: PAI 29.1 s/era -> 9.3 h (matches the observed 00:37 -> 09:22) CONV 41.3 s/era -> 13.2 h LSTM 150.4 s/era -> 48.1 h HYBRID 154.6 s/era -> 49.5 h Two days to tune is not a first-run experience, and it is the phase in which the panel goes quiet, which is what made it look like a hang. It also bought nothing. The space is 90 points (10 MA periods x 9 MA types), so 1152 evaluations revisited each point ~13 times; and rungs of 3 and 8 eras cannot separate two MA periods at all. The 2026-08-01 run proves it: every finalist scored 25.0-25.9% balanced accuracy - below the 33.3% one-class floor, i.e. indistinguishable noise - and the search then "deployed the winner" of that. THE ERROR WAS THE SCORING FUNCTION, not its constants. Using a full training run to choose a feature's period is a wrapper method paying wrapper prices for a decision that does not need one. The reference book does not do this: ch. 3.3 selects inputs by measuring each candidate indicator's CORRELATION with the target and dropping the ones with none, with no network involved. So: rank candidates by the MUTUAL INFORMATION between the resulting feature vector and the triple-barrier label. MI rather than correlation because the label is 3-class categorical and the features are not monotonically related to it. Equal-FREQUENCY binning (rank-based), because these features are ATR-normalised and heavy-tailed - fixed-width bins put nearly everything in one bucket and report ~0 information for a genuinely useful feature. Scoring is arithmetic over the feature cache, so it costs seconds and its cost is independent of topology: LSTM now tunes as fast as the MLP. Coordinate sweep, not product sweep - cost is the SUM of per-parameter candidate counts, so enabling every indicator stays affordable - with a second pass that breaks early once nothing moves. Sampling is IS-ONLY. Letting the OOS window influence which indicator settings ship would mean the holdout had been used for selection and had stopped being a holdout. HONEST LIMIT, recorded because it is the price: MI is marginal, so a parameter that only pays off in combination with another can be missed (Guyon & Elisseeff 2003, filter vs wrapper). Given the wrapper it replaces was ranking pure noise at 48 h a run, this is strictly better. Deleted with it: GaRungEras/GaExtract/GaStore/GaMutate/GaRandomCandidate/ GaBlockCrossover/GaSortAliveByScoreDesc/GaBreedNextGeneration, 14 m_ga* members, the GA_*/TUNE_POP_* constants, and ComputeTuneTrialBudget. AND m_evalMode/m_evalEraBudget, because nothing set them any more - 28 read sites all permanently inert. That is not a tidy-up: the `if (!m_evalMode)` guard on UpdateClassPriors is exactly what silently disabled the imbalance correction for entire runs two commits ago. Dead machinery that still reads like live machinery is this codebase's most expensive recurring bug, and leaving 28 more instances of it would have been indefensible. The panel's tuning-progress state goes too - tuning no longer takes long enough to need one. Both builds compile 0 errors / 0 warnings. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-01 10:29:31 -04:00
for(int k = 0; k < n; k++)
colVals[k] = cols[k * m_neuronsCount + f];
diag(autotune): one label shuffle cannot settle the no-edge question The permutation baseline added in 018afb1 came back on all four charts as 0.00401 nats against floors of 0.00267 / 0.00298 / 0.00318 - three draws whose spread is as wide as the excess being judged, because one shuffle is one sample from the null, not the null. That is not enough to retire a topology on. Now MI_NOISE_PERMUTATIONS draws, reported as mean +/- sd with a z-score, plus two numbers the mean over 26 columns cannot express: - the STRONGEST single feature's MI, against its own shuffled value. One informative column among 25 useless ones is precisely the case the mean hides, and precisely the case worth finding. - the excess as a percentage of H(Y). At these sample sizes a z-score can be comfortably significant while the effect is worthless, so "is it real" and "is it big enough to matter" are asked separately and answered separately. The verdict line also now states the measure's limit every time rather than only when the news is bad: this is a MARGINAL, PER-BAR statistic and the network reads m_historyBars bars jointly, so it can prove signal exists but never that it does not. It rules out a per-feature edge - and therefore any indicator retuning - not an edge that lives in a combination or across time. Compiles 0 errors / 0 warnings, standard and Market. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-01 13:32:12 -04:00
double mi = FeatureColumnMI(colVals, labels, n);
feat(mi): per-column feature screen, and fix the block permutation it rides on CFeatureSelector keeps the per-column MI vector ScoreMiSample has always computed and thrown away. It is fed from inside the 200 draws ReportFeatureLabelInformation already performs, so the screen costs an array copy per draw and not one extra mutual-information computation. The keep-mask is cut on the single-step maxT (Westfall-Young) statistic - a column must beat the MAXIMUM of a null draw over all columns, which is strong family-wise control needing no Bonferroni factor, and is the same null of the maximum the headline verdict already trusts. The uncorrected per-comparison p is reported alongside it; the gap between the two counts IS the multiplicity correction, shown rather than described. Checked offline at 40 columns: 0/40 noise runs keep anything, where the uncorrected rule hands back ~2 columns per run, and a planted column is recovered 40/40. REPORT-ONLY. Nothing reads the mask. Pruning changes m_neuronsCount, which is in BuildModelFingerprint(), which invalidates every .nnw - that is a retrain across every chart and an operator's call to make after reading the report. Also fixes the block permutation, found while moving it. When blockRows did not divide n the short last block, drawn to a non-final slot, read past the end of the array; the read was clamped to labels[n-1], duplicating one label and truncating whichever block landed last. 18 of the 24 possible block orders on n=10/blockRows=3 altered the class counts. A duplicated label concentrates the class distribution, lowering H(Y) and so the null MI those draws can reach, so p-values leaned toward significance - the permissive direction, and m_dirEvidence is a deploy gate. Each block now contributes exactly its own length. The invariance the old comment asserted ("a permutation preserves the class counts - that invariance is itself a check on the shuffle") was never actually compared anywhere; BlockPermute now checks it and returns false, and all six shuffled call sites already guard on a negative return. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-23 18:45:00 -04:00
perColumn[f] = mi;
diag(autotune): one label shuffle cannot settle the no-edge question The permutation baseline added in 018afb1 came back on all four charts as 0.00401 nats against floors of 0.00267 / 0.00298 / 0.00318 - three draws whose spread is as wide as the excess being judged, because one shuffle is one sample from the null, not the null. That is not enough to retire a topology on. Now MI_NOISE_PERMUTATIONS draws, reported as mean +/- sd with a z-score, plus two numbers the mean over 26 columns cannot express: - the STRONGEST single feature's MI, against its own shuffled value. One informative column among 25 useless ones is precisely the case the mean hides, and precisely the case worth finding. - the excess as a percentage of H(Y). At these sample sizes a z-score can be comfortably significant while the effect is worthless, so "is it real" and "is it big enough to matter" are asked separately and answered separately. The verdict line also now states the measure's limit every time rather than only when the news is bad: this is a MARGINAL, PER-BAR statistic and the network reads m_historyBars bars jointly, so it can prove signal exists but never that it does not. It rules out a per-feature edge - and therefore any indicator retuning - not an edge that lives in a combination or across time. Compiles 0 errors / 0 warnings, standard and Market. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-01 13:32:12 -04:00
total += mi;
if(mi > m_miBestColumn)
m_miBestColumn = mi;
refactor: split CExpertSignalAIBase implementation by responsibility ExpertSignalAIBase.mqh was 8216 lines: the class declaration followed by 87 method bodies covering training, labelling, feature extraction, persistence, chart drawing, online learning, the GA auto-tuner and inference, all in one file. Train() alone is 1492 lines; a change to arrow drawing meant scrolling past the era loop. Moved the bodies into Expert\AIBase\, included at the bottom of the original after the class declaration: Training.mqh 1607 era loop, plateau ladder, checkpoint select, deploy Features.mqh 1093 indicator creation + per-bar input feature vector ChartUI.mqh 634 arrows, arrow persistence, status panel, cleanup Persistence.mqh 492 .stats/.cfg sidecars, CPU-inference validation, copy OnlineLearning.mqh 461 live continual learning, EMA shadow, OOS simulator Labels.mqh 309 ZigZag pivot labels, async label-cache prebuild AutoTune.mqh 275 genetic tuner (population, crossover, halving) Inference.mqh 235 softmax, prior calibration, class priors ExpertSignalAIBase.mqh 8216 -> 3131 (declaration + topology build only) This is a pure relocation - verified mechanically, not by eye: HEAD's file reconstructed from the eight partials plus the surviving remainder is byte-identical to HEAD, span for span (scratchpad verify_split.py). No declaration moved, no signature changed, no code rewritten, so behaviour is unchanged by construction. Compiles 0 errors, 0 warnings. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-29 00:42:45 -04:00
}
perf(autotune): replace the genetic search with a filter score - hours to seconds MEASURED COST OF THE GA, which is what retired it. Per generation: rung 0: 8 cand x 3 seeds x 3 eras = 72 eras rung 1: 4 cand x 3 seeds x 8 eras = 96 rung 2: 2 cand x 3 seeds x 20 eras = 120 = 288 eras/generation x 4 generations = 1152 eras BEFORE the winner's real training began. Against the observed era times on SP500 H1: PAI 29.1 s/era -> 9.3 h (matches the observed 00:37 -> 09:22) CONV 41.3 s/era -> 13.2 h LSTM 150.4 s/era -> 48.1 h HYBRID 154.6 s/era -> 49.5 h Two days to tune is not a first-run experience, and it is the phase in which the panel goes quiet, which is what made it look like a hang. It also bought nothing. The space is 90 points (10 MA periods x 9 MA types), so 1152 evaluations revisited each point ~13 times; and rungs of 3 and 8 eras cannot separate two MA periods at all. The 2026-08-01 run proves it: every finalist scored 25.0-25.9% balanced accuracy - below the 33.3% one-class floor, i.e. indistinguishable noise - and the search then "deployed the winner" of that. THE ERROR WAS THE SCORING FUNCTION, not its constants. Using a full training run to choose a feature's period is a wrapper method paying wrapper prices for a decision that does not need one. The reference book does not do this: ch. 3.3 selects inputs by measuring each candidate indicator's CORRELATION with the target and dropping the ones with none, with no network involved. So: rank candidates by the MUTUAL INFORMATION between the resulting feature vector and the triple-barrier label. MI rather than correlation because the label is 3-class categorical and the features are not monotonically related to it. Equal-FREQUENCY binning (rank-based), because these features are ATR-normalised and heavy-tailed - fixed-width bins put nearly everything in one bucket and report ~0 information for a genuinely useful feature. Scoring is arithmetic over the feature cache, so it costs seconds and its cost is independent of topology: LSTM now tunes as fast as the MLP. Coordinate sweep, not product sweep - cost is the SUM of per-parameter candidate counts, so enabling every indicator stays affordable - with a second pass that breaks early once nothing moves. Sampling is IS-ONLY. Letting the OOS window influence which indicator settings ship would mean the holdout had been used for selection and had stopped being a holdout. HONEST LIMIT, recorded because it is the price: MI is marginal, so a parameter that only pays off in combination with another can be missed (Guyon & Elisseeff 2003, filter vs wrapper). Given the wrapper it replaces was ranking pure noise at 48 h a run, this is strictly better. Deleted with it: GaRungEras/GaExtract/GaStore/GaMutate/GaRandomCandidate/ GaBlockCrossover/GaSortAliveByScoreDesc/GaBreedNextGeneration, 14 m_ga* members, the GA_*/TUNE_POP_* constants, and ComputeTuneTrialBudget. AND m_evalMode/m_evalEraBudget, because nothing set them any more - 28 read sites all permanently inert. That is not a tidy-up: the `if (!m_evalMode)` guard on UpdateClassPriors is exactly what silently disabled the imbalance correction for entire runs two commits ago. Dead machinery that still reads like live machinery is this codebase's most expensive recurring bug, and leaving 28 more instances of it would have been indefensible. The panel's tuning-progress state goes too - tuning no longer takes long enough to need one. Both builds compile 0 errors / 0 warnings. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-01 10:29:31 -04:00
return total / m_neuronsCount;
refactor: split CExpertSignalAIBase implementation by responsibility ExpertSignalAIBase.mqh was 8216 lines: the class declaration followed by 87 method bodies covering training, labelling, feature extraction, persistence, chart drawing, online learning, the GA auto-tuner and inference, all in one file. Train() alone is 1492 lines; a change to arrow drawing meant scrolling past the era loop. Moved the bodies into Expert\AIBase\, included at the bottom of the original after the class declaration: Training.mqh 1607 era loop, plateau ladder, checkpoint select, deploy Features.mqh 1093 indicator creation + per-bar input feature vector ChartUI.mqh 634 arrows, arrow persistence, status panel, cleanup Persistence.mqh 492 .stats/.cfg sidecars, CPU-inference validation, copy OnlineLearning.mqh 461 live continual learning, EMA shadow, OOS simulator Labels.mqh 309 ZigZag pivot labels, async label-cache prebuild AutoTune.mqh 275 genetic tuner (population, crossover, halving) Inference.mqh 235 softmax, prior calibration, class priors ExpertSignalAIBase.mqh 8216 -> 3131 (declaration + topology build only) This is a pure relocation - verified mechanically, not by eye: HEAD's file reconstructed from the eight partials plus the surviving remainder is byte-identical to HEAD, span for span (scratchpad verify_split.py). No declaration moved, no signature changed, no code rewritten, so behaviour is unchanged by construction. Compiles 0 errors, 0 warnings. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-29 00:42:45 -04:00
}
//+------------------------------------------------------------------+
diag(autotune): five permutations was still a coin flip - use a real test The 5-draw z-score shipped an hour ago disproved itself on its first run. All four charts scored the IDENTICAL 0.00401 nats on identical features and identical labels - and reported z of +1.3, +2.0, +4.0 and +4.7. Two "AT THE NOISE FLOOR", two "a real association", same data. The entire swing came from estimating the null's spread from five draws, where the standard deviation of the standard-deviation estimate is ~35%: the denominator was noisier than the effect it was judging. Replaced with an empirical permutation test. 200 draws, p counted by rank with the +1/(B+1) correction (Phipson & Smyth 2010) so p is never reported as exactly zero - no normality assumption and no spread to estimate. The strongest single column is tested against the null distribution OF THE MAXIMUM, which corrects for scoring 26 features at once by construction and is far less conservative than Bonferroni. Affordable because BuildMiSample is now split out of ScoreCurrentParamsByMI and runs ONCE for the whole test - every draw reuses that sample and costs a relabel plus 26 histogram passes, not 2000 feature extractions. The coordinate sweep still calls the combined form, which is correct there: each candidate changes the indicator settings, so its features really do have to be re-extracted. The verdict line keeps both questions apart and prints both answers: the p-value for "is it real", the excess as a percentage of H(Y) for "is it big enough to trade". At n=2000 those can disagree, and collapsing them into one word is how a worthless effect gets called a discovery. Compiles 0 errors / 0 warnings. Build tag permtest-v1. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-01 13:45:46 -04:00
//| Extract + score in one call - the form the coordinate sweep uses, |
//| where each candidate genuinely needs a fresh extraction because |
//| the indicator settings (and therefore the features) just changed. |
//+------------------------------------------------------------------+
double CExpertSignalAIBase::ScoreCurrentParamsByMI(bool shuffleLabels = false)
{
double cols[];
int labels[];
feat(tuner+altdata): tuner optimizes RANGE not direction; copy-paste whitelist UX on 4014 - MI_TUNE_TARGET = MI_TARGET_EXC_RANGE: the coordinate sweep scored candidates against the barrier label - measured noise - so it climbed a flat landscape and the gate rightly rejected every winner. It now selects indicator settings for MI vs realised RANGE (4x null, positive control), the channel the excursion head consumes these features for. Winner gate re-tests on the same target. Barrier-label report unchanged. - AltDataFetch 4014 handling: Alert popup + once-per-session walkthrough with the two whitelist URLs on their own journal lines (copy-paste ready); hourly-backoff retry instead of a permanent latch, so the whitelist fix takes effect without re-attaching. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-16 14:44:11 -04:00
//--- MI_TUNE_TARGET, not the barrier label - see the define's comment: the tuner selects
//--- indicator settings for the channel with measured signal (realised RANGE), not the one
//--- measured at the noise floor (direction).
int n = BuildMiSample(cols, labels, 0, 0, MI_TUNE_TARGET);
diag(autotune): five permutations was still a coin flip - use a real test The 5-draw z-score shipped an hour ago disproved itself on its first run. All four charts scored the IDENTICAL 0.00401 nats on identical features and identical labels - and reported z of +1.3, +2.0, +4.0 and +4.7. Two "AT THE NOISE FLOOR", two "a real association", same data. The entire swing came from estimating the null's spread from five draws, where the standard deviation of the standard-deviation estimate is ~35%: the denominator was noisier than the effect it was judging. Replaced with an empirical permutation test. 200 draws, p counted by rank with the +1/(B+1) correction (Phipson & Smyth 2010) so p is never reported as exactly zero - no normality assumption and no spread to estimate. The strongest single column is tested against the null distribution OF THE MAXIMUM, which corrects for scoring 26 features at once by construction and is far less conservative than Bonferroni. Affordable because BuildMiSample is now split out of ScoreCurrentParamsByMI and runs ONCE for the whole test - every draw reuses that sample and costs a relabel plus 26 histogram passes, not 2000 feature extractions. The coordinate sweep still calls the combined form, which is correct there: each candidate changes the indicator settings, so its features really do have to be re-extracted. The verdict line keeps both questions apart and prints both answers: the p-value for "is it real", the excess as a percentage of H(Y) for "is it big enough to trade". At n=2000 those can disagree, and collapsing them into one word is how a worthless effect gets called a discovery. Compiles 0 errors / 0 warnings. Build tag permtest-v1. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-01 13:45:46 -04:00
if(n < MI_MIN_SAMPLES)
return -1.0;
return ScoreMiSample(cols, labels, n, shuffleLabels);
}
//+------------------------------------------------------------------+
refactor(comments): box headers to stdlib length The //| box blocks were excluded from 0b06f8e and 5efdb48 and were what remained: 160 of them ran to 10+ lines, the longest to 88. Compressed to their leading topic sentences - 5 lines for a function header, 8 for a file header - keeping the box format and the standard MQL5 name/author lines verbatim. Verified at the BYTE level this time, across every in-scope file: the list of non-comment lines is byte-identical to HEAD and braces balance. The first check compared a locale-decoded 'git show' against a UTF-8 read and flagged 25 files that had not changed at all - every BOM and every non-ASCII line mismatched. 47,696 -> 40,665 lines in scope; comment share 38% -> 26%. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-22 00:30:14 -04:00
//| FILTER-BASED indicator tuning. Replaced the genetic + |
//| successive- halving search on 2026-08-01. |
refactor(inputs): 96 -> 70 inputs; remove two untested/unusable filter modules Every removal below is FINGERPRINT-NEUTRAL by construction: each retired input is pinned to the exact value it already shipped with, so running models keep their filenames and resume rather than restarting at era 0. Verified field by field against BuildConfigFingerprint. Removed as inputs, kept as pinned constants (the value was never a preference the user had a basis to change): - OutputNeuronsCount. The regression head predicts a continuous quantity the triple-barrier label does not contain; the target is an EVENT, so the right output is its probability. The regression code paths stay implemented and dormant - they cost nothing and removing them would touch every scoring path at once. - MinRecall. A safety floor, not a preference, and the only direction a user can move it is the harmful one: raising it past what the config reaches yields NO model, not a better one (observed repeatedly at 60). - SwingConfirmationBars. Stopped gating the labels with the relabel, but is STILL load-bearing for the swing-context input features - it is the ZigZag repainting embargo, and without it those 9 features read a leg the live bar could not have had yet. Pinned, not deleted. - MaxErasPerRun (runaway backstop, never reached in a healthy run), FreezePriorCalibration (unanswerable by a user; near-balanced labels make the priors stable anyway), VerboseMode (developer view, joins DebuggingMode), MACD/Ichimoku periods x6 (both indicators ship disabled, and as optimizer dimensions they are pure overfitting surface - the AI auto-tuner is the supported way to move them). - SignalClusterWindow -> 3, no longer an input. Barrier labels make consecutive setups real, which argued for 0; it is not 0 because on D1+ a 6-bar window spans over a week and two arrows a day apart on a weekly-scale move are one event. 3 splits it correctly by timeframe. - EnableOnlineLearning -> ON. Adapting to a changing market is what keeps a months-attached model from going stale, and the rolling-accuracy freeze is what makes it safe. See the caveat noted in the handoff: it had not been forward-tested on a live feed when this became default. Removed entirely: - Intraday Time Filter (5 inputs + Signals/SignalITF.mqh). Two of its five inputs were raw BITMASKS, which is an implementation detail exposed as a control. The job is covered three times over by things that are declarative or that learn: the session filter, the time-of-day/day-of-week input features (the network discovers which hours are good rather than being told), and the journal's time buckets. - Market Depth Filter (5 inputs + Signals/SignalMarketDepth.mqh, plus its OnInit probe and OnDeinit release). It needs real level-2 data that this broker - and most retail MT5 brokers - do not provide, so the module has never once executed against real data. Shipping four tuning dropdowns for an untested path is worse than shipping nothing: the only users who could enable it would be its first-ever testers, live. If DOM returns it should be a FEATURE fed to the network, not a rule-based veto with hand-tuned thresholds - imbalance is data. - IndicatorTuneTrials, replaced by ComputeTuneTrialBudget(). The useful budget depends on how many parameters are actually being searched, which depends on which features are enabled - so one number meant wildly different things run to run. The shipped 32 was ~10 candidates per dimension against one enabled indicator (wasteful: each costs GA_SEEDS full training runs) and under one per dimension against all nine (blind). Now population ~ 4 x active dimensions, clamped [8,64], with CADIndicatorTuner::ActiveDimensions() defined immediately above PerturbRandom() so the two cannot drift apart. - Six orphaned enums (TUNE_TRIALS_PRESET, DOM_*, ENTRY_HOUR_OF_DAY, TIME_FILTER_DAY_OF_WEEK), 81 lines. Other UX: - SL_ATR_x1 / TP_ATR_x3 now carry the "(classic)" default marker every other preset enum in the file already used. Nothing in the SL/TP dropdowns previously told a user which pair was the shipped default - which matters far more since the relabel, because those two define the labels and changing either forces a retrain. - Neural Network section moved directly ABOVE AI Input Features: choose the architecture, then choose what it sees. NN Optimizer / Performance stays last - the Adam/Sgd inputs are declared in AI/Network.mqh and render immediately after that divider. - News feature + window moved to the end of the AI feature list, below Wyckoff Bar Inversion. - Dropped "(0-100)" from Min vote to open - it is an enum, not a number. Both builds compile 0 errors / 0 warnings. No retrain forced. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-31 21:22:02 -04:00
//+------------------------------------------------------------------+
perf(autotune): replace the genetic search with a filter score - hours to seconds MEASURED COST OF THE GA, which is what retired it. Per generation: rung 0: 8 cand x 3 seeds x 3 eras = 72 eras rung 1: 4 cand x 3 seeds x 8 eras = 96 rung 2: 2 cand x 3 seeds x 20 eras = 120 = 288 eras/generation x 4 generations = 1152 eras BEFORE the winner's real training began. Against the observed era times on SP500 H1: PAI 29.1 s/era -> 9.3 h (matches the observed 00:37 -> 09:22) CONV 41.3 s/era -> 13.2 h LSTM 150.4 s/era -> 48.1 h HYBRID 154.6 s/era -> 49.5 h Two days to tune is not a first-run experience, and it is the phase in which the panel goes quiet, which is what made it look like a hang. It also bought nothing. The space is 90 points (10 MA periods x 9 MA types), so 1152 evaluations revisited each point ~13 times; and rungs of 3 and 8 eras cannot separate two MA periods at all. The 2026-08-01 run proves it: every finalist scored 25.0-25.9% balanced accuracy - below the 33.3% one-class floor, i.e. indistinguishable noise - and the search then "deployed the winner" of that. THE ERROR WAS THE SCORING FUNCTION, not its constants. Using a full training run to choose a feature's period is a wrapper method paying wrapper prices for a decision that does not need one. The reference book does not do this: ch. 3.3 selects inputs by measuring each candidate indicator's CORRELATION with the target and dropping the ones with none, with no network involved. So: rank candidates by the MUTUAL INFORMATION between the resulting feature vector and the triple-barrier label. MI rather than correlation because the label is 3-class categorical and the features are not monotonically related to it. Equal-FREQUENCY binning (rank-based), because these features are ATR-normalised and heavy-tailed - fixed-width bins put nearly everything in one bucket and report ~0 information for a genuinely useful feature. Scoring is arithmetic over the feature cache, so it costs seconds and its cost is independent of topology: LSTM now tunes as fast as the MLP. Coordinate sweep, not product sweep - cost is the SUM of per-parameter candidate counts, so enabling every indicator stays affordable - with a second pass that breaks early once nothing moves. Sampling is IS-ONLY. Letting the OOS window influence which indicator settings ship would mean the holdout had been used for selection and had stopped being a holdout. HONEST LIMIT, recorded because it is the price: MI is marginal, so a parameter that only pays off in combination with another can be missed (Guyon & Elisseeff 2003, filter vs wrapper). Given the wrapper it replaces was ranking pure noise at 48 h a run, this is strictly better. Deleted with it: GaRungEras/GaExtract/GaStore/GaMutate/GaRandomCandidate/ GaBlockCrossover/GaSortAliveByScoreDesc/GaBreedNextGeneration, 14 m_ga* members, the GA_*/TUNE_POP_* constants, and ComputeTuneTrialBudget. AND m_evalMode/m_evalEraBudget, because nothing set them any more - 28 read sites all permanently inert. That is not a tidy-up: the `if (!m_evalMode)` guard on UpdateClassPriors is exactly what silently disabled the imbalance correction for entire runs two commits ago. Dead machinery that still reads like live machinery is this codebase's most expensive recurring bug, and leaving 28 more instances of it would have been indefensible. The panel's tuning-progress state goes too - tuning no longer takes long enough to need one. Both builds compile 0 errors / 0 warnings. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-01 10:29:31 -04:00
void CExpertSignalAIBase::TuneIndicatorsByFilter(void)
refactor(inputs): 96 -> 70 inputs; remove two untested/unusable filter modules Every removal below is FINGERPRINT-NEUTRAL by construction: each retired input is pinned to the exact value it already shipped with, so running models keep their filenames and resume rather than restarting at era 0. Verified field by field against BuildConfigFingerprint. Removed as inputs, kept as pinned constants (the value was never a preference the user had a basis to change): - OutputNeuronsCount. The regression head predicts a continuous quantity the triple-barrier label does not contain; the target is an EVENT, so the right output is its probability. The regression code paths stay implemented and dormant - they cost nothing and removing them would touch every scoring path at once. - MinRecall. A safety floor, not a preference, and the only direction a user can move it is the harmful one: raising it past what the config reaches yields NO model, not a better one (observed repeatedly at 60). - SwingConfirmationBars. Stopped gating the labels with the relabel, but is STILL load-bearing for the swing-context input features - it is the ZigZag repainting embargo, and without it those 9 features read a leg the live bar could not have had yet. Pinned, not deleted. - MaxErasPerRun (runaway backstop, never reached in a healthy run), FreezePriorCalibration (unanswerable by a user; near-balanced labels make the priors stable anyway), VerboseMode (developer view, joins DebuggingMode), MACD/Ichimoku periods x6 (both indicators ship disabled, and as optimizer dimensions they are pure overfitting surface - the AI auto-tuner is the supported way to move them). - SignalClusterWindow -> 3, no longer an input. Barrier labels make consecutive setups real, which argued for 0; it is not 0 because on D1+ a 6-bar window spans over a week and two arrows a day apart on a weekly-scale move are one event. 3 splits it correctly by timeframe. - EnableOnlineLearning -> ON. Adapting to a changing market is what keeps a months-attached model from going stale, and the rolling-accuracy freeze is what makes it safe. See the caveat noted in the handoff: it had not been forward-tested on a live feed when this became default. Removed entirely: - Intraday Time Filter (5 inputs + Signals/SignalITF.mqh). Two of its five inputs were raw BITMASKS, which is an implementation detail exposed as a control. The job is covered three times over by things that are declarative or that learn: the session filter, the time-of-day/day-of-week input features (the network discovers which hours are good rather than being told), and the journal's time buckets. - Market Depth Filter (5 inputs + Signals/SignalMarketDepth.mqh, plus its OnInit probe and OnDeinit release). It needs real level-2 data that this broker - and most retail MT5 brokers - do not provide, so the module has never once executed against real data. Shipping four tuning dropdowns for an untested path is worse than shipping nothing: the only users who could enable it would be its first-ever testers, live. If DOM returns it should be a FEATURE fed to the network, not a rule-based veto with hand-tuned thresholds - imbalance is data. - IndicatorTuneTrials, replaced by ComputeTuneTrialBudget(). The useful budget depends on how many parameters are actually being searched, which depends on which features are enabled - so one number meant wildly different things run to run. The shipped 32 was ~10 candidates per dimension against one enabled indicator (wasteful: each costs GA_SEEDS full training runs) and under one per dimension against all nine (blind). Now population ~ 4 x active dimensions, clamped [8,64], with CADIndicatorTuner::ActiveDimensions() defined immediately above PerturbRandom() so the two cannot drift apart. - Six orphaned enums (TUNE_TRIALS_PRESET, DOM_*, ENTRY_HOUR_OF_DAY, TIME_FILTER_DAY_OF_WEEK), 81 lines. Other UX: - SL_ATR_x1 / TP_ATR_x3 now carry the "(classic)" default marker every other preset enum in the file already used. Nothing in the SL/TP dropdowns previously told a user which pair was the shipped default - which matters far more since the relabel, because those two define the labels and changing either forces a retrain. - Neural Network section moved directly ABOVE AI Input Features: choose the architecture, then choose what it sees. NN Optimizer / Performance stays last - the Adam/Sgd inputs are declared in AI/Network.mqh and render immediately after that divider. - News feature + window moved to the end of the AI feature list, below Wyckoff Bar Inversion. - Dropped "(0-100)" from Min vote to open - it is an enum, not a number. Both builds compile 0 errors / 0 warnings. No retrain forced. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-31 21:22:02 -04:00
{
perf(autotune): replace the genetic search with a filter score - hours to seconds MEASURED COST OF THE GA, which is what retired it. Per generation: rung 0: 8 cand x 3 seeds x 3 eras = 72 eras rung 1: 4 cand x 3 seeds x 8 eras = 96 rung 2: 2 cand x 3 seeds x 20 eras = 120 = 288 eras/generation x 4 generations = 1152 eras BEFORE the winner's real training began. Against the observed era times on SP500 H1: PAI 29.1 s/era -> 9.3 h (matches the observed 00:37 -> 09:22) CONV 41.3 s/era -> 13.2 h LSTM 150.4 s/era -> 48.1 h HYBRID 154.6 s/era -> 49.5 h Two days to tune is not a first-run experience, and it is the phase in which the panel goes quiet, which is what made it look like a hang. It also bought nothing. The space is 90 points (10 MA periods x 9 MA types), so 1152 evaluations revisited each point ~13 times; and rungs of 3 and 8 eras cannot separate two MA periods at all. The 2026-08-01 run proves it: every finalist scored 25.0-25.9% balanced accuracy - below the 33.3% one-class floor, i.e. indistinguishable noise - and the search then "deployed the winner" of that. THE ERROR WAS THE SCORING FUNCTION, not its constants. Using a full training run to choose a feature's period is a wrapper method paying wrapper prices for a decision that does not need one. The reference book does not do this: ch. 3.3 selects inputs by measuring each candidate indicator's CORRELATION with the target and dropping the ones with none, with no network involved. So: rank candidates by the MUTUAL INFORMATION between the resulting feature vector and the triple-barrier label. MI rather than correlation because the label is 3-class categorical and the features are not monotonically related to it. Equal-FREQUENCY binning (rank-based), because these features are ATR-normalised and heavy-tailed - fixed-width bins put nearly everything in one bucket and report ~0 information for a genuinely useful feature. Scoring is arithmetic over the feature cache, so it costs seconds and its cost is independent of topology: LSTM now tunes as fast as the MLP. Coordinate sweep, not product sweep - cost is the SUM of per-parameter candidate counts, so enabling every indicator stays affordable - with a second pass that breaks early once nothing moves. Sampling is IS-ONLY. Letting the OOS window influence which indicator settings ship would mean the holdout had been used for selection and had stopped being a holdout. HONEST LIMIT, recorded because it is the price: MI is marginal, so a parameter that only pays off in combination with another can be missed (Guyon & Elisseeff 2003, filter vs wrapper). Given the wrapper it replaces was ranking pure noise at 48 h a run, this is strictly better. Deleted with it: GaRungEras/GaExtract/GaStore/GaMutate/GaRandomCandidate/ GaBlockCrossover/GaSortAliveByScoreDesc/GaBreedNextGeneration, 14 m_ga* members, the GA_*/TUNE_POP_* constants, and ComputeTuneTrialBudget. AND m_evalMode/m_evalEraBudget, because nothing set them any more - 28 read sites all permanently inert. That is not a tidy-up: the `if (!m_evalMode)` guard on UpdateClassPriors is exactly what silently disabled the imbalance correction for entire runs two commits ago. Dead machinery that still reads like live machinery is this codebase's most expensive recurring bug, and leaving 28 more instances of it would have been indefensible. The panel's tuning-progress state goes too - tuning no longer takes long enough to need one. Both builds compile 0 errors / 0 warnings. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-01 10:29:31 -04:00
double best[];
m_indicatorTuner.Flatten(best);
double bestScore = ScoreCurrentParamsByMI();
if(bestScore < 0.0)
refactor: split CExpertSignalAIBase implementation by responsibility ExpertSignalAIBase.mqh was 8216 lines: the class declaration followed by 87 method bodies covering training, labelling, feature extraction, persistence, chart drawing, online learning, the GA auto-tuner and inference, all in one file. Train() alone is 1492 lines; a change to arrow drawing meant scrolling past the era loop. Moved the bodies into Expert\AIBase\, included at the bottom of the original after the class declaration: Training.mqh 1607 era loop, plateau ladder, checkpoint select, deploy Features.mqh 1093 indicator creation + per-bar input feature vector ChartUI.mqh 634 arrows, arrow persistence, status panel, cleanup Persistence.mqh 492 .stats/.cfg sidecars, CPU-inference validation, copy OnlineLearning.mqh 461 live continual learning, EMA shadow, OOS simulator Labels.mqh 309 ZigZag pivot labels, async label-cache prebuild AutoTune.mqh 275 genetic tuner (population, crossover, halving) Inference.mqh 235 softmax, prior calibration, class priors ExpertSignalAIBase.mqh 8216 -> 3131 (declaration + topology build only) This is a pure relocation - verified mechanically, not by eye: HEAD's file reconstructed from the eight partials plus the surviving remainder is byte-identical to HEAD, span for span (scratchpad verify_split.py). No declaration moved, no signature changed, no code rewritten, so behaviour is unchanged by construction. Compiles 0 errors, 0 warnings. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-29 00:42:45 -04:00
{
perf(autotune): replace the genetic search with a filter score - hours to seconds MEASURED COST OF THE GA, which is what retired it. Per generation: rung 0: 8 cand x 3 seeds x 3 eras = 72 eras rung 1: 4 cand x 3 seeds x 8 eras = 96 rung 2: 2 cand x 3 seeds x 20 eras = 120 = 288 eras/generation x 4 generations = 1152 eras BEFORE the winner's real training began. Against the observed era times on SP500 H1: PAI 29.1 s/era -> 9.3 h (matches the observed 00:37 -> 09:22) CONV 41.3 s/era -> 13.2 h LSTM 150.4 s/era -> 48.1 h HYBRID 154.6 s/era -> 49.5 h Two days to tune is not a first-run experience, and it is the phase in which the panel goes quiet, which is what made it look like a hang. It also bought nothing. The space is 90 points (10 MA periods x 9 MA types), so 1152 evaluations revisited each point ~13 times; and rungs of 3 and 8 eras cannot separate two MA periods at all. The 2026-08-01 run proves it: every finalist scored 25.0-25.9% balanced accuracy - below the 33.3% one-class floor, i.e. indistinguishable noise - and the search then "deployed the winner" of that. THE ERROR WAS THE SCORING FUNCTION, not its constants. Using a full training run to choose a feature's period is a wrapper method paying wrapper prices for a decision that does not need one. The reference book does not do this: ch. 3.3 selects inputs by measuring each candidate indicator's CORRELATION with the target and dropping the ones with none, with no network involved. So: rank candidates by the MUTUAL INFORMATION between the resulting feature vector and the triple-barrier label. MI rather than correlation because the label is 3-class categorical and the features are not monotonically related to it. Equal-FREQUENCY binning (rank-based), because these features are ATR-normalised and heavy-tailed - fixed-width bins put nearly everything in one bucket and report ~0 information for a genuinely useful feature. Scoring is arithmetic over the feature cache, so it costs seconds and its cost is independent of topology: LSTM now tunes as fast as the MLP. Coordinate sweep, not product sweep - cost is the SUM of per-parameter candidate counts, so enabling every indicator stays affordable - with a second pass that breaks early once nothing moves. Sampling is IS-ONLY. Letting the OOS window influence which indicator settings ship would mean the holdout had been used for selection and had stopped being a holdout. HONEST LIMIT, recorded because it is the price: MI is marginal, so a parameter that only pays off in combination with another can be missed (Guyon & Elisseeff 2003, filter vs wrapper). Given the wrapper it replaces was ranking pure noise at 48 h a run, this is strictly better. Deleted with it: GaRungEras/GaExtract/GaStore/GaMutate/GaRandomCandidate/ GaBlockCrossover/GaSortAliveByScoreDesc/GaBreedNextGeneration, 14 m_ga* members, the GA_*/TUNE_POP_* constants, and ComputeTuneTrialBudget. AND m_evalMode/m_evalEraBudget, because nothing set them any more - 28 read sites all permanently inert. That is not a tidy-up: the `if (!m_evalMode)` guard on UpdateClassPriors is exactly what silently disabled the imbalance correction for entire runs two commits ago. Dead machinery that still reads like live machinery is this codebase's most expensive recurring bug, and leaving 28 more instances of it would have been indefensible. The panel's tuning-progress state goes too - tuning no longer takes long enough to need one. Both builds compile 0 errors / 0 warnings. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-01 10:29:31 -04:00
Print(ID + ": auto-tune skipped - not enough labelled in-sample bars to score indicator settings");
refactor: split CExpertSignalAIBase implementation by responsibility ExpertSignalAIBase.mqh was 8216 lines: the class declaration followed by 87 method bodies covering training, labelling, feature extraction, persistence, chart drawing, online learning, the GA auto-tuner and inference, all in one file. Train() alone is 1492 lines; a change to arrow drawing meant scrolling past the era loop. Moved the bodies into Expert\AIBase\, included at the bottom of the original after the class declaration: Training.mqh 1607 era loop, plateau ladder, checkpoint select, deploy Features.mqh 1093 indicator creation + per-bar input feature vector ChartUI.mqh 634 arrows, arrow persistence, status panel, cleanup Persistence.mqh 492 .stats/.cfg sidecars, CPU-inference validation, copy OnlineLearning.mqh 461 live continual learning, EMA shadow, OOS simulator Labels.mqh 309 ZigZag pivot labels, async label-cache prebuild AutoTune.mqh 275 genetic tuner (population, crossover, halving) Inference.mqh 235 softmax, prior calibration, class priors ExpertSignalAIBase.mqh 8216 -> 3131 (declaration + topology build only) This is a pure relocation - verified mechanically, not by eye: HEAD's file reconstructed from the eight partials plus the surviving remainder is byte-identical to HEAD, span for span (scratchpad verify_split.py). No declaration moved, no signature changed, no code rewritten, so behaviour is unchanged by construction. Compiles 0 errors, 0 warnings. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-29 00:42:45 -04:00
return;
}
perf(autotune): replace the genetic search with a filter score - hours to seconds MEASURED COST OF THE GA, which is what retired it. Per generation: rung 0: 8 cand x 3 seeds x 3 eras = 72 eras rung 1: 4 cand x 3 seeds x 8 eras = 96 rung 2: 2 cand x 3 seeds x 20 eras = 120 = 288 eras/generation x 4 generations = 1152 eras BEFORE the winner's real training began. Against the observed era times on SP500 H1: PAI 29.1 s/era -> 9.3 h (matches the observed 00:37 -> 09:22) CONV 41.3 s/era -> 13.2 h LSTM 150.4 s/era -> 48.1 h HYBRID 154.6 s/era -> 49.5 h Two days to tune is not a first-run experience, and it is the phase in which the panel goes quiet, which is what made it look like a hang. It also bought nothing. The space is 90 points (10 MA periods x 9 MA types), so 1152 evaluations revisited each point ~13 times; and rungs of 3 and 8 eras cannot separate two MA periods at all. The 2026-08-01 run proves it: every finalist scored 25.0-25.9% balanced accuracy - below the 33.3% one-class floor, i.e. indistinguishable noise - and the search then "deployed the winner" of that. THE ERROR WAS THE SCORING FUNCTION, not its constants. Using a full training run to choose a feature's period is a wrapper method paying wrapper prices for a decision that does not need one. The reference book does not do this: ch. 3.3 selects inputs by measuring each candidate indicator's CORRELATION with the target and dropping the ones with none, with no network involved. So: rank candidates by the MUTUAL INFORMATION between the resulting feature vector and the triple-barrier label. MI rather than correlation because the label is 3-class categorical and the features are not monotonically related to it. Equal-FREQUENCY binning (rank-based), because these features are ATR-normalised and heavy-tailed - fixed-width bins put nearly everything in one bucket and report ~0 information for a genuinely useful feature. Scoring is arithmetic over the feature cache, so it costs seconds and its cost is independent of topology: LSTM now tunes as fast as the MLP. Coordinate sweep, not product sweep - cost is the SUM of per-parameter candidate counts, so enabling every indicator stays affordable - with a second pass that breaks early once nothing moves. Sampling is IS-ONLY. Letting the OOS window influence which indicator settings ship would mean the holdout had been used for selection and had stopped being a holdout. HONEST LIMIT, recorded because it is the price: MI is marginal, so a parameter that only pays off in combination with another can be missed (Guyon & Elisseeff 2003, filter vs wrapper). Given the wrapper it replaces was ranking pure noise at 48 h a run, this is strictly better. Deleted with it: GaRungEras/GaExtract/GaStore/GaMutate/GaRandomCandidate/ GaBlockCrossover/GaSortAliveByScoreDesc/GaBreedNextGeneration, 14 m_ga* members, the GA_*/TUNE_POP_* constants, and ComputeTuneTrialBudget. AND m_evalMode/m_evalEraBudget, because nothing set them any more - 28 read sites all permanently inert. That is not a tidy-up: the `if (!m_evalMode)` guard on UpdateClassPriors is exactly what silently disabled the imbalance correction for entire runs two commits ago. Dead machinery that still reads like live machinery is this codebase's most expensive recurring bug, and leaving 28 more instances of it would have been indefensible. The panel's tuning-progress state goes too - tuning no longer takes long enough to need one. Both builds compile 0 errors / 0 warnings. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-01 10:29:31 -04:00
double startScore = bestScore;
int evaluated = 0;
uint t0 = GetTickCount();
refactor(comments): stdlib comment style across the remaining in-scope files Same pass as 0b06f8e, applied file by file: comment runs of 4+ lines compressed to their leading topic sentences, capped at 4 lines, whole sentences only. Warning sentences (NEVER / MUST / trap / would-have) survive the budget. Every file was checked the same way before committing: the list of non-comment lines is byte-identical to HEAD, and braces balance. No code was touched. Panel/, Enumerations/ and the already-terse System headers needed little or nothing - PooledGate, TradeChecks, BinomialStats and Random came through with no blocks over the threshold at all. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-22 00:25:52 -04:00
//--- SPREAD OF THE CANDIDATE SCORES. Without it "no improvement" is ambiguous between two
//--- readings that want opposite responses: INERT (trial scores identical to the incumbent
//--- because the parameter change never reaches the scored features, so `sc > bestScore` can
//--- never fire) versus LIVE and genuinely finding nothing.
diag: report the indicator tuner's candidate spread - "no improvement" is ambiguous Auditing the other best-of-N scans after cccf94f turned up a third instance of the same pattern, and this one is worse than the two already fixed: the geometry scan and the lag profile PRINT a row, whereas TuneIndicatorsByFilter INSTALLS its winner (Unflatten + ReInitADIndicators) and the caller then calls BuildFreshTopology(), so an unguarded maximum changes the feature vector the network trains on. It has no null of any kind. But before adding one, the logs say something a noise-driven best-of-N cannot: 2026-08-05/06, four consecutive runs, 17 candidates each, every one "no improvement" with start and best identical to 4dp. The maximum of 17 draws from a noise distribution beats its incumbent about 94% of the time, so 4/4 is on the order of 1 in 100,000. Two readings fit and they want opposite responses: - INERT: trial scores come back identical to the incumbent because the parameter change never reaches the scored features (suspect the feature cache surviving ReInitADIndicators), so `sc > bestScore` can never fire. That is a dead code path, and gating it would be decorating a corpse. - LIVE and correctly finding nothing: then it needs the family-wise gate. The current log line cannot separate them, so add the number that can: the span of the candidate scores, with an explicit ZERO SPREAD callout naming the likely cause. Also widened the MI figures from 4dp to 5dp - at this scale 4dp rounds the entire effect away. No gate yet, deliberately: measure which failure this is, then fix that one. Read-only diagnostic. No input, topology or label change: no retrain. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-06 21:59:10 -04:00
double candMin = DBL_MAX, candMax = -DBL_MAX;
fix: make the indicator tuner actually measure, and gate what it installs ROOT CAUSE of the zero spread measured on SP500 H1 2026-08-07 (all 17 candidates returned exactly 0.00359 nats): the tune loop re-inits the indicators and then scores, with no RefreshData() between. ReInitADIndicators() does its part - Create() builds a NEW handle carrying the new parameters, and the feature cache is flagged stale so features really are recomputed. But BufferTempDataCompute() reads the CIndicatorBuffer objects, and only Refresh() copies data out of a handle into those. So every candidate was scored on values still held from the PREVIOUS handle. My earlier guess in the diagnostic ("suspect the feature cache") was wrong: the cache invalidation works. Two things land together, because neither is safe alone: 1. RefreshData() after the re-init, so a candidate is scored on its own features. 2. A SELECTION GATE on the install. bestScore is a MAXIMUM over candidates, and the maximum of N draws from a null beats its incumbent almost every time - so "it beat the incumbent" installs noise. This selector is the highest-stakes of the three found in this audit because it ACTS: it overwrites the user's configured indicator settings and forces BuildFreshTopology(), so the network then trains on whatever the noise picked. Fixing (1) without (2) would have made a dormant bug actively harmful. The gate draws the winner's own permutation null once, then corrects the p-value for having chosen it out of N with Sidak: p_family = 1 - (1-p)^N. Sidak rather than the max-of-N resample used by the geometry scan because each candidate here has a DIFFERENT feature set, so their draws cannot be pooled; Sidak needs only the one null. Exact under independence, mildly anti-conservative under positive dependence - stated in the comment rather than hidden. A rejected winner restores the configured settings, which best[] cannot do since the descent mutates it. Also reports the least-ready tunable handle's BarsCalculated(). IndicatorCreate() calculates asynchronously, so if the spread is STILL zero the handles simply are not done and the tuner needs to yield between candidates rather than score them back to back - a state machine like the label prebuild. That distinction is now readable from the log instead of requiring another guess. No input, topology or label change: no retrain. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-07 09:31:06 -04:00
int readyMin = INT_MAX;
//--- The configured settings, kept so a winner that fails the gate below can be handed back. best[] is
//--- mutated in place by the descent, so it cannot serve as the restore point.
double configured[];
ArrayCopy(configured, best);
perf(autotune): replace the genetic search with a filter score - hours to seconds MEASURED COST OF THE GA, which is what retired it. Per generation: rung 0: 8 cand x 3 seeds x 3 eras = 72 eras rung 1: 4 cand x 3 seeds x 8 eras = 96 rung 2: 2 cand x 3 seeds x 20 eras = 120 = 288 eras/generation x 4 generations = 1152 eras BEFORE the winner's real training began. Against the observed era times on SP500 H1: PAI 29.1 s/era -> 9.3 h (matches the observed 00:37 -> 09:22) CONV 41.3 s/era -> 13.2 h LSTM 150.4 s/era -> 48.1 h HYBRID 154.6 s/era -> 49.5 h Two days to tune is not a first-run experience, and it is the phase in which the panel goes quiet, which is what made it look like a hang. It also bought nothing. The space is 90 points (10 MA periods x 9 MA types), so 1152 evaluations revisited each point ~13 times; and rungs of 3 and 8 eras cannot separate two MA periods at all. The 2026-08-01 run proves it: every finalist scored 25.0-25.9% balanced accuracy - below the 33.3% one-class floor, i.e. indistinguishable noise - and the search then "deployed the winner" of that. THE ERROR WAS THE SCORING FUNCTION, not its constants. Using a full training run to choose a feature's period is a wrapper method paying wrapper prices for a decision that does not need one. The reference book does not do this: ch. 3.3 selects inputs by measuring each candidate indicator's CORRELATION with the target and dropping the ones with none, with no network involved. So: rank candidates by the MUTUAL INFORMATION between the resulting feature vector and the triple-barrier label. MI rather than correlation because the label is 3-class categorical and the features are not monotonically related to it. Equal-FREQUENCY binning (rank-based), because these features are ATR-normalised and heavy-tailed - fixed-width bins put nearly everything in one bucket and report ~0 information for a genuinely useful feature. Scoring is arithmetic over the feature cache, so it costs seconds and its cost is independent of topology: LSTM now tunes as fast as the MLP. Coordinate sweep, not product sweep - cost is the SUM of per-parameter candidate counts, so enabling every indicator stays affordable - with a second pass that breaks early once nothing moves. Sampling is IS-ONLY. Letting the OOS window influence which indicator settings ship would mean the holdout had been used for selection and had stopped being a holdout. HONEST LIMIT, recorded because it is the price: MI is marginal, so a parameter that only pays off in combination with another can be missed (Guyon & Elisseeff 2003, filter vs wrapper). Given the wrapper it replaces was ranking pure noise at 48 h a run, this is strictly better. Deleted with it: GaRungEras/GaExtract/GaStore/GaMutate/GaRandomCandidate/ GaBlockCrossover/GaSortAliveByScoreDesc/GaBreedNextGeneration, 14 m_ga* members, the GA_*/TUNE_POP_* constants, and ComputeTuneTrialBudget. AND m_evalMode/m_evalEraBudget, because nothing set them any more - 28 read sites all permanently inert. That is not a tidy-up: the `if (!m_evalMode)` guard on UpdateClassPriors is exactly what silently disabled the imbalance correction for entire runs two commits ago. Dead machinery that still reads like live machinery is this codebase's most expensive recurring bug, and leaving 28 more instances of it would have been indefensible. The panel's tuning-progress state goes too - tuning no longer takes long enough to need one. Both builds compile 0 errors / 0 warnings. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-01 10:29:31 -04:00
for(int pass = 0; pass < MI_TUNE_PASSES; pass++)
refactor: split CExpertSignalAIBase implementation by responsibility ExpertSignalAIBase.mqh was 8216 lines: the class declaration followed by 87 method bodies covering training, labelling, feature extraction, persistence, chart drawing, online learning, the GA auto-tuner and inference, all in one file. Train() alone is 1492 lines; a change to arrow drawing meant scrolling past the era loop. Moved the bodies into Expert\AIBase\, included at the bottom of the original after the class declaration: Training.mqh 1607 era loop, plateau ladder, checkpoint select, deploy Features.mqh 1093 indicator creation + per-bar input feature vector ChartUI.mqh 634 arrows, arrow persistence, status panel, cleanup Persistence.mqh 492 .stats/.cfg sidecars, CPU-inference validation, copy OnlineLearning.mqh 461 live continual learning, EMA shadow, OOS simulator Labels.mqh 309 ZigZag pivot labels, async label-cache prebuild AutoTune.mqh 275 genetic tuner (population, crossover, halving) Inference.mqh 235 softmax, prior calibration, class priors ExpertSignalAIBase.mqh 8216 -> 3131 (declaration + topology build only) This is a pure relocation - verified mechanically, not by eye: HEAD's file reconstructed from the eight partials plus the surviving remainder is byte-identical to HEAD, span for span (scratchpad verify_split.py). No declaration moved, no signature changed, no code rewritten, so behaviour is unchanged by construction. Compiles 0 errors, 0 warnings. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-29 00:42:45 -04:00
{
perf(autotune): replace the genetic search with a filter score - hours to seconds MEASURED COST OF THE GA, which is what retired it. Per generation: rung 0: 8 cand x 3 seeds x 3 eras = 72 eras rung 1: 4 cand x 3 seeds x 8 eras = 96 rung 2: 2 cand x 3 seeds x 20 eras = 120 = 288 eras/generation x 4 generations = 1152 eras BEFORE the winner's real training began. Against the observed era times on SP500 H1: PAI 29.1 s/era -> 9.3 h (matches the observed 00:37 -> 09:22) CONV 41.3 s/era -> 13.2 h LSTM 150.4 s/era -> 48.1 h HYBRID 154.6 s/era -> 49.5 h Two days to tune is not a first-run experience, and it is the phase in which the panel goes quiet, which is what made it look like a hang. It also bought nothing. The space is 90 points (10 MA periods x 9 MA types), so 1152 evaluations revisited each point ~13 times; and rungs of 3 and 8 eras cannot separate two MA periods at all. The 2026-08-01 run proves it: every finalist scored 25.0-25.9% balanced accuracy - below the 33.3% one-class floor, i.e. indistinguishable noise - and the search then "deployed the winner" of that. THE ERROR WAS THE SCORING FUNCTION, not its constants. Using a full training run to choose a feature's period is a wrapper method paying wrapper prices for a decision that does not need one. The reference book does not do this: ch. 3.3 selects inputs by measuring each candidate indicator's CORRELATION with the target and dropping the ones with none, with no network involved. So: rank candidates by the MUTUAL INFORMATION between the resulting feature vector and the triple-barrier label. MI rather than correlation because the label is 3-class categorical and the features are not monotonically related to it. Equal-FREQUENCY binning (rank-based), because these features are ATR-normalised and heavy-tailed - fixed-width bins put nearly everything in one bucket and report ~0 information for a genuinely useful feature. Scoring is arithmetic over the feature cache, so it costs seconds and its cost is independent of topology: LSTM now tunes as fast as the MLP. Coordinate sweep, not product sweep - cost is the SUM of per-parameter candidate counts, so enabling every indicator stays affordable - with a second pass that breaks early once nothing moves. Sampling is IS-ONLY. Letting the OOS window influence which indicator settings ship would mean the holdout had been used for selection and had stopped being a holdout. HONEST LIMIT, recorded because it is the price: MI is marginal, so a parameter that only pays off in combination with another can be missed (Guyon & Elisseeff 2003, filter vs wrapper). Given the wrapper it replaces was ranking pure noise at 48 h a run, this is strictly better. Deleted with it: GaRungEras/GaExtract/GaStore/GaMutate/GaRandomCandidate/ GaBlockCrossover/GaSortAliveByScoreDesc/GaBreedNextGeneration, 14 m_ga* members, the GA_*/TUNE_POP_* constants, and ComputeTuneTrialBudget. AND m_evalMode/m_evalEraBudget, because nothing set them any more - 28 read sites all permanently inert. That is not a tidy-up: the `if (!m_evalMode)` guard on UpdateClassPriors is exactly what silently disabled the imbalance correction for entire runs two commits ago. Dead machinery that still reads like live machinery is this codebase's most expensive recurring bug, and leaving 28 more instances of it would have been indefensible. The panel's tuning-progress state goes too - tuning no longer takes long enough to need one. Both builds compile 0 errors / 0 warnings. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-01 10:29:31 -04:00
bool improvedThisPass = false;
for(int p = 0; p < AD_TUNE_PARAM_COUNT; p++)
refactor: split CExpertSignalAIBase implementation by responsibility ExpertSignalAIBase.mqh was 8216 lines: the class declaration followed by 87 method bodies covering training, labelling, feature extraction, persistence, chart drawing, online learning, the GA auto-tuner and inference, all in one file. Train() alone is 1492 lines; a change to arrow drawing meant scrolling past the era loop. Moved the bodies into Expert\AIBase\, included at the bottom of the original after the class declaration: Training.mqh 1607 era loop, plateau ladder, checkpoint select, deploy Features.mqh 1093 indicator creation + per-bar input feature vector ChartUI.mqh 634 arrows, arrow persistence, status panel, cleanup Persistence.mqh 492 .stats/.cfg sidecars, CPU-inference validation, copy OnlineLearning.mqh 461 live continual learning, EMA shadow, OOS simulator Labels.mqh 309 ZigZag pivot labels, async label-cache prebuild AutoTune.mqh 275 genetic tuner (population, crossover, halving) Inference.mqh 235 softmax, prior calibration, class priors ExpertSignalAIBase.mqh 8216 -> 3131 (declaration + topology build only) This is a pure relocation - verified mechanically, not by eye: HEAD's file reconstructed from the eight partials plus the surviving remainder is byte-identical to HEAD, span for span (scratchpad verify_split.py). No declaration moved, no signature changed, no code rewritten, so behaviour is unchanged by construction. Compiles 0 errors, 0 warnings. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-29 00:42:45 -04:00
{
perf(autotune): replace the genetic search with a filter score - hours to seconds MEASURED COST OF THE GA, which is what retired it. Per generation: rung 0: 8 cand x 3 seeds x 3 eras = 72 eras rung 1: 4 cand x 3 seeds x 8 eras = 96 rung 2: 2 cand x 3 seeds x 20 eras = 120 = 288 eras/generation x 4 generations = 1152 eras BEFORE the winner's real training began. Against the observed era times on SP500 H1: PAI 29.1 s/era -> 9.3 h (matches the observed 00:37 -> 09:22) CONV 41.3 s/era -> 13.2 h LSTM 150.4 s/era -> 48.1 h HYBRID 154.6 s/era -> 49.5 h Two days to tune is not a first-run experience, and it is the phase in which the panel goes quiet, which is what made it look like a hang. It also bought nothing. The space is 90 points (10 MA periods x 9 MA types), so 1152 evaluations revisited each point ~13 times; and rungs of 3 and 8 eras cannot separate two MA periods at all. The 2026-08-01 run proves it: every finalist scored 25.0-25.9% balanced accuracy - below the 33.3% one-class floor, i.e. indistinguishable noise - and the search then "deployed the winner" of that. THE ERROR WAS THE SCORING FUNCTION, not its constants. Using a full training run to choose a feature's period is a wrapper method paying wrapper prices for a decision that does not need one. The reference book does not do this: ch. 3.3 selects inputs by measuring each candidate indicator's CORRELATION with the target and dropping the ones with none, with no network involved. So: rank candidates by the MUTUAL INFORMATION between the resulting feature vector and the triple-barrier label. MI rather than correlation because the label is 3-class categorical and the features are not monotonically related to it. Equal-FREQUENCY binning (rank-based), because these features are ATR-normalised and heavy-tailed - fixed-width bins put nearly everything in one bucket and report ~0 information for a genuinely useful feature. Scoring is arithmetic over the feature cache, so it costs seconds and its cost is independent of topology: LSTM now tunes as fast as the MLP. Coordinate sweep, not product sweep - cost is the SUM of per-parameter candidate counts, so enabling every indicator stays affordable - with a second pass that breaks early once nothing moves. Sampling is IS-ONLY. Letting the OOS window influence which indicator settings ship would mean the holdout had been used for selection and had stopped being a holdout. HONEST LIMIT, recorded because it is the price: MI is marginal, so a parameter that only pays off in combination with another can be missed (Guyon & Elisseeff 2003, filter vs wrapper). Given the wrapper it replaces was ranking pure noise at 48 h a run, this is strictly better. Deleted with it: GaRungEras/GaExtract/GaStore/GaMutate/GaRandomCandidate/ GaBlockCrossover/GaSortAliveByScoreDesc/GaBreedNextGeneration, 14 m_ga* members, the GA_*/TUNE_POP_* constants, and ComputeTuneTrialBudget. AND m_evalMode/m_evalEraBudget, because nothing set them any more - 28 read sites all permanently inert. That is not a tidy-up: the `if (!m_evalMode)` guard on UpdateClassPriors is exactly what silently disabled the imbalance correction for entire runs two commits ago. Dead machinery that still reads like live machinery is this codebase's most expensive recurring bug, and leaving 28 more instances of it would have been indefensible. The panel's tuning-progress state goes too - tuning no longer takes long enough to need one. Both builds compile 0 errors / 0 warnings. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-01 10:29:31 -04:00
//--- skip parameters whose indicator is switched off - they cannot affect the feature vector
int owner = m_indicatorTuner.ParamOwner(p);
bool on = (owner == 0 && m_useADCumulativeDelta) || (owner == 1 && m_useADShorteningOfThrust) ||
(owner == 2 && m_useADWyckoffEventStream) || (owner == 3 && m_useADWyckoffFailedStructure) ||
(owner == 4 && m_useADWyckoffSignificantBarInversion) || (owner == 5 && m_useMA) ||
(owner == 6 && m_useRSI) || (owner == 7 && m_useMACD) || (owner == 8 && m_useIchimoku);
if(!on)
continue;
double cands[];
int nc = m_indicatorTuner.ParamCandidates(p, cands);
double keep = best[p];
for(int c = 0; c < nc; c++)
{
fix(shutdown): make ExitPolicy public, and stop every long loop the moment MT5 asks Two things, one of which was a compile error. 1. ExitPolicy() was declared in the protected block but is pushed in from Warrior_EA.mq5:770. Moved to public beside the other EA-facing setters. 2. Chart objects surviving OnDeinit. The 4,500 ms teardown budget is measured from the STOP REQUEST, not from OnDeinit's first line, and OnDeinit cannot begin until whatever is in flight returns - so a scan still running after _StopFlag is raised does not delay the cleanup, it SPENDS it, and the purge never gets its turn. New CExpertSignalAIBase::ShutdownRequested() = IsStopped() || m_shutdownInProgress. Deliberately NOT m_trainingStopRequested: that latches, and a latched flag would permanently disable scans that must run again on the next Start. Guarded, longest first: - TuneIndicatorsByFilter - per candidate, restoring the OPERATOR's settings on the way out (best[] is mutated in place; the tuner otherwise keeps the last trial's parameters, which nothing chose). - ReportBarrierGeometryScan - per pairing, breaking to ONE restore point so m_barrierScanLiveLabels can never be left true (that makes ComputeLabelForBar read the last candidate's multiples as the configured geometry). - ReportFeatureLabelInformation / ReportExcursionInformation / lag profile - nulls ABANDON rather than truncate: fewer draws is not a smaller null, it is a wrong one, and p shifts toward significance. m_dirEvidence staying false is the safe direction. - SimulateExitPolicyOutcomes - zeroes its accumulators so the divergence line is dropped instead of latching a partial expectancy as the run's only report. - ReportGeometryExpectancyScan - per ladder rung. - HttpGet - one choke point for up to a dozen blocking WebRequests per first-pass Update(). An in-flight request cannot be cancelled; refusing to start another is the whole remedy. - PollTraining, OnChartEventHandler's study event, TuneIndicatorsAndTrain - entry points, so a queued event cannot open an era during teardown. TuneIndicatorsAndTrain's guard is the first statement, ahead of the m_tuneFilterDone / g_ensembleChartTuneDone latches. - OnTick / OnTimer / OnChartEvent. Training's own bar loops already honoured this (pass 1 per bar, passes 2/2.5/3 yield on a 120 ms budget); the warm-up scans did not, and they are the longest uninterruptible stretches the EA has. StopTraining() is unchanged: the operator's Stop still finalises synchronously. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-17 17:03:11 -04:00
//--- The longest uninterruptible stretch in the EA: every candidate re-creates handles,
//--- refreshes, and scores a full MI sample. Asked per candidate so a stop request costs at
//--- most one candidate rather than the rest of the descent - see ShutdownRequested().
if(ShutdownRequested())
{
refactor(comments): stdlib comment style across the remaining in-scope files Same pass as 0b06f8e, applied file by file: comment runs of 4+ lines compressed to their leading topic sentences, capped at 4 lines, whole sentences only. Warning sentences (NEVER / MUST / trap / would-have) survive the budget. Every file was checked the same way before committing: the list of non-comment lines is byte-identical to HEAD, and braces balance. No code was touched. Panel/, Enumerations/ and the already-terse System headers needed little or nothing - PooledGate, TradeChecks, BinomialStats and Random came through with no blocks over the threshold at all. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-22 00:25:52 -04:00
//--- Hand the OPERATOR's settings back before leaving. best[] is mutated in place by
//--- the descent and the tuner object currently carries the LAST TRIAL's parameters,
//--- which nothing chose and which the .cfg would otherwise persist as if it had
//--- been selected.
fix(shutdown): make ExitPolicy public, and stop every long loop the moment MT5 asks Two things, one of which was a compile error. 1. ExitPolicy() was declared in the protected block but is pushed in from Warrior_EA.mq5:770. Moved to public beside the other EA-facing setters. 2. Chart objects surviving OnDeinit. The 4,500 ms teardown budget is measured from the STOP REQUEST, not from OnDeinit's first line, and OnDeinit cannot begin until whatever is in flight returns - so a scan still running after _StopFlag is raised does not delay the cleanup, it SPENDS it, and the purge never gets its turn. New CExpertSignalAIBase::ShutdownRequested() = IsStopped() || m_shutdownInProgress. Deliberately NOT m_trainingStopRequested: that latches, and a latched flag would permanently disable scans that must run again on the next Start. Guarded, longest first: - TuneIndicatorsByFilter - per candidate, restoring the OPERATOR's settings on the way out (best[] is mutated in place; the tuner otherwise keeps the last trial's parameters, which nothing chose). - ReportBarrierGeometryScan - per pairing, breaking to ONE restore point so m_barrierScanLiveLabels can never be left true (that makes ComputeLabelForBar read the last candidate's multiples as the configured geometry). - ReportFeatureLabelInformation / ReportExcursionInformation / lag profile - nulls ABANDON rather than truncate: fewer draws is not a smaller null, it is a wrong one, and p shifts toward significance. m_dirEvidence staying false is the safe direction. - SimulateExitPolicyOutcomes - zeroes its accumulators so the divergence line is dropped instead of latching a partial expectancy as the run's only report. - ReportGeometryExpectancyScan - per ladder rung. - HttpGet - one choke point for up to a dozen blocking WebRequests per first-pass Update(). An in-flight request cannot be cancelled; refusing to start another is the whole remedy. - PollTraining, OnChartEventHandler's study event, TuneIndicatorsAndTrain - entry points, so a queued event cannot open an era during teardown. TuneIndicatorsAndTrain's guard is the first statement, ahead of the m_tuneFilterDone / g_ensembleChartTuneDone latches. - OnTick / OnTimer / OnChartEvent. Training's own bar loops already honoured this (pass 1 per bar, passes 2/2.5/3 yield on a 120 ms budget); the warm-up scans did not, and they are the longest uninterruptible stretches the EA has. StopTraining() is unchanged: the operator's Stop still finalises synchronously. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-17 17:03:11 -04:00
m_indicatorTuner.Unflatten(configured);
PrintFormat("%s: auto-tune ABANDONED after %d candidates - stop requested. Configured"
" indicator settings restored; nothing installed.", ID, evaluated);
return;
}
perf(autotune): replace the genetic search with a filter score - hours to seconds MEASURED COST OF THE GA, which is what retired it. Per generation: rung 0: 8 cand x 3 seeds x 3 eras = 72 eras rung 1: 4 cand x 3 seeds x 8 eras = 96 rung 2: 2 cand x 3 seeds x 20 eras = 120 = 288 eras/generation x 4 generations = 1152 eras BEFORE the winner's real training began. Against the observed era times on SP500 H1: PAI 29.1 s/era -> 9.3 h (matches the observed 00:37 -> 09:22) CONV 41.3 s/era -> 13.2 h LSTM 150.4 s/era -> 48.1 h HYBRID 154.6 s/era -> 49.5 h Two days to tune is not a first-run experience, and it is the phase in which the panel goes quiet, which is what made it look like a hang. It also bought nothing. The space is 90 points (10 MA periods x 9 MA types), so 1152 evaluations revisited each point ~13 times; and rungs of 3 and 8 eras cannot separate two MA periods at all. The 2026-08-01 run proves it: every finalist scored 25.0-25.9% balanced accuracy - below the 33.3% one-class floor, i.e. indistinguishable noise - and the search then "deployed the winner" of that. THE ERROR WAS THE SCORING FUNCTION, not its constants. Using a full training run to choose a feature's period is a wrapper method paying wrapper prices for a decision that does not need one. The reference book does not do this: ch. 3.3 selects inputs by measuring each candidate indicator's CORRELATION with the target and dropping the ones with none, with no network involved. So: rank candidates by the MUTUAL INFORMATION between the resulting feature vector and the triple-barrier label. MI rather than correlation because the label is 3-class categorical and the features are not monotonically related to it. Equal-FREQUENCY binning (rank-based), because these features are ATR-normalised and heavy-tailed - fixed-width bins put nearly everything in one bucket and report ~0 information for a genuinely useful feature. Scoring is arithmetic over the feature cache, so it costs seconds and its cost is independent of topology: LSTM now tunes as fast as the MLP. Coordinate sweep, not product sweep - cost is the SUM of per-parameter candidate counts, so enabling every indicator stays affordable - with a second pass that breaks early once nothing moves. Sampling is IS-ONLY. Letting the OOS window influence which indicator settings ship would mean the holdout had been used for selection and had stopped being a holdout. HONEST LIMIT, recorded because it is the price: MI is marginal, so a parameter that only pays off in combination with another can be missed (Guyon & Elisseeff 2003, filter vs wrapper). Given the wrapper it replaces was ranking pure noise at 48 h a run, this is strictly better. Deleted with it: GaRungEras/GaExtract/GaStore/GaMutate/GaRandomCandidate/ GaBlockCrossover/GaSortAliveByScoreDesc/GaBreedNextGeneration, 14 m_ga* members, the GA_*/TUNE_POP_* constants, and ComputeTuneTrialBudget. AND m_evalMode/m_evalEraBudget, because nothing set them any more - 28 read sites all permanently inert. That is not a tidy-up: the `if (!m_evalMode)` guard on UpdateClassPriors is exactly what silently disabled the imbalance correction for entire runs two commits ago. Dead machinery that still reads like live machinery is this codebase's most expensive recurring bug, and leaving 28 more instances of it would have been indefensible. The panel's tuning-progress state goes too - tuning no longer takes long enough to need one. Both builds compile 0 errors / 0 warnings. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-01 10:29:31 -04:00
if(cands[c] == keep)
continue; // already scored as the incumbent
double trial[];
ArrayCopy(trial, best);
trial[p] = cands[c];
m_indicatorTuner.Unflatten(trial);
ReInitADIndicators(m_indicatorsPtr); // also invalidates the feature cache (params changed)
refactor(comments): stdlib comment style across the remaining in-scope files Same pass as 0b06f8e, applied file by file: comment runs of 4+ lines compressed to their leading topic sentences, capped at 4 lines, whole sentences only. Warning sentences (NEVER / MUST / trap / would-have) survive the budget. Every file was checked the same way before committing: the list of non-comment lines is byte-identical to HEAD, and braces balance. No code was touched. Panel/, Enumerations/ and the already-terse System headers needed little or nothing - PooledGate, TradeChecks, BinomialStats and Random came through with no blocks over the threshold at all. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-22 00:25:52 -04:00
//--- REFRESH, or the re-init changes nothing that the scorer can see. Without this the
//--- buffers still hold values copied from the PREVIOUS handle, so every candidate is
//--- scored on identical features.
fix: make the indicator tuner actually measure, and gate what it installs ROOT CAUSE of the zero spread measured on SP500 H1 2026-08-07 (all 17 candidates returned exactly 0.00359 nats): the tune loop re-inits the indicators and then scores, with no RefreshData() between. ReInitADIndicators() does its part - Create() builds a NEW handle carrying the new parameters, and the feature cache is flagged stale so features really are recomputed. But BufferTempDataCompute() reads the CIndicatorBuffer objects, and only Refresh() copies data out of a handle into those. So every candidate was scored on values still held from the PREVIOUS handle. My earlier guess in the diagnostic ("suspect the feature cache") was wrong: the cache invalidation works. Two things land together, because neither is safe alone: 1. RefreshData() after the re-init, so a candidate is scored on its own features. 2. A SELECTION GATE on the install. bestScore is a MAXIMUM over candidates, and the maximum of N draws from a null beats its incumbent almost every time - so "it beat the incumbent" installs noise. This selector is the highest-stakes of the three found in this audit because it ACTS: it overwrites the user's configured indicator settings and forces BuildFreshTopology(), so the network then trains on whatever the noise picked. Fixing (1) without (2) would have made a dormant bug actively harmful. The gate draws the winner's own permutation null once, then corrects the p-value for having chosen it out of N with Sidak: p_family = 1 - (1-p)^N. Sidak rather than the max-of-N resample used by the geometry scan because each candidate here has a DIFFERENT feature set, so their draws cannot be pooled; Sidak needs only the one null. Exact under independence, mildly anti-conservative under positive dependence - stated in the comment rather than hidden. A rejected winner restores the configured settings, which best[] cannot do since the descent mutates it. Also reports the least-ready tunable handle's BarsCalculated(). IndicatorCreate() calculates asynchronously, so if the spread is STILL zero the handles simply are not done and the tuner needs to yield between candidates rather than score them back to back - a state machine like the label prebuild. That distinction is now readable from the log instead of requiring another guess. No input, topology or label change: no retrain. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-07 09:31:06 -04:00
RefreshData();
int ready = TunableBarsCalculated();
if(ready >= 0)
readyMin = (int)MathMin(readyMin, ready);
perf(autotune): replace the genetic search with a filter score - hours to seconds MEASURED COST OF THE GA, which is what retired it. Per generation: rung 0: 8 cand x 3 seeds x 3 eras = 72 eras rung 1: 4 cand x 3 seeds x 8 eras = 96 rung 2: 2 cand x 3 seeds x 20 eras = 120 = 288 eras/generation x 4 generations = 1152 eras BEFORE the winner's real training began. Against the observed era times on SP500 H1: PAI 29.1 s/era -> 9.3 h (matches the observed 00:37 -> 09:22) CONV 41.3 s/era -> 13.2 h LSTM 150.4 s/era -> 48.1 h HYBRID 154.6 s/era -> 49.5 h Two days to tune is not a first-run experience, and it is the phase in which the panel goes quiet, which is what made it look like a hang. It also bought nothing. The space is 90 points (10 MA periods x 9 MA types), so 1152 evaluations revisited each point ~13 times; and rungs of 3 and 8 eras cannot separate two MA periods at all. The 2026-08-01 run proves it: every finalist scored 25.0-25.9% balanced accuracy - below the 33.3% one-class floor, i.e. indistinguishable noise - and the search then "deployed the winner" of that. THE ERROR WAS THE SCORING FUNCTION, not its constants. Using a full training run to choose a feature's period is a wrapper method paying wrapper prices for a decision that does not need one. The reference book does not do this: ch. 3.3 selects inputs by measuring each candidate indicator's CORRELATION with the target and dropping the ones with none, with no network involved. So: rank candidates by the MUTUAL INFORMATION between the resulting feature vector and the triple-barrier label. MI rather than correlation because the label is 3-class categorical and the features are not monotonically related to it. Equal-FREQUENCY binning (rank-based), because these features are ATR-normalised and heavy-tailed - fixed-width bins put nearly everything in one bucket and report ~0 information for a genuinely useful feature. Scoring is arithmetic over the feature cache, so it costs seconds and its cost is independent of topology: LSTM now tunes as fast as the MLP. Coordinate sweep, not product sweep - cost is the SUM of per-parameter candidate counts, so enabling every indicator stays affordable - with a second pass that breaks early once nothing moves. Sampling is IS-ONLY. Letting the OOS window influence which indicator settings ship would mean the holdout had been used for selection and had stopped being a holdout. HONEST LIMIT, recorded because it is the price: MI is marginal, so a parameter that only pays off in combination with another can be missed (Guyon & Elisseeff 2003, filter vs wrapper). Given the wrapper it replaces was ranking pure noise at 48 h a run, this is strictly better. Deleted with it: GaRungEras/GaExtract/GaStore/GaMutate/GaRandomCandidate/ GaBlockCrossover/GaSortAliveByScoreDesc/GaBreedNextGeneration, 14 m_ga* members, the GA_*/TUNE_POP_* constants, and ComputeTuneTrialBudget. AND m_evalMode/m_evalEraBudget, because nothing set them any more - 28 read sites all permanently inert. That is not a tidy-up: the `if (!m_evalMode)` guard on UpdateClassPriors is exactly what silently disabled the imbalance correction for entire runs two commits ago. Dead machinery that still reads like live machinery is this codebase's most expensive recurring bug, and leaving 28 more instances of it would have been indefensible. The panel's tuning-progress state goes too - tuning no longer takes long enough to need one. Both builds compile 0 errors / 0 warnings. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-01 10:29:31 -04:00
double sc = ScoreCurrentParamsByMI();
evaluated++;
diag: report the indicator tuner's candidate spread - "no improvement" is ambiguous Auditing the other best-of-N scans after cccf94f turned up a third instance of the same pattern, and this one is worse than the two already fixed: the geometry scan and the lag profile PRINT a row, whereas TuneIndicatorsByFilter INSTALLS its winner (Unflatten + ReInitADIndicators) and the caller then calls BuildFreshTopology(), so an unguarded maximum changes the feature vector the network trains on. It has no null of any kind. But before adding one, the logs say something a noise-driven best-of-N cannot: 2026-08-05/06, four consecutive runs, 17 candidates each, every one "no improvement" with start and best identical to 4dp. The maximum of 17 draws from a noise distribution beats its incumbent about 94% of the time, so 4/4 is on the order of 1 in 100,000. Two readings fit and they want opposite responses: - INERT: trial scores come back identical to the incumbent because the parameter change never reaches the scored features (suspect the feature cache surviving ReInitADIndicators), so `sc > bestScore` can never fire. That is a dead code path, and gating it would be decorating a corpse. - LIVE and correctly finding nothing: then it needs the family-wise gate. The current log line cannot separate them, so add the number that can: the span of the candidate scores, with an explicit ZERO SPREAD callout naming the likely cause. Also widened the MI figures from 4dp to 5dp - at this scale 4dp rounds the entire effect away. No gate yet, deliberately: measure which failure this is, then fix that one. Read-only diagnostic. No input, topology or label change: no retrain. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-06 21:59:10 -04:00
if(sc >= 0.0)
{
candMin = MathMin(candMin, sc);
candMax = MathMax(candMax, sc);
}
perf(autotune): replace the genetic search with a filter score - hours to seconds MEASURED COST OF THE GA, which is what retired it. Per generation: rung 0: 8 cand x 3 seeds x 3 eras = 72 eras rung 1: 4 cand x 3 seeds x 8 eras = 96 rung 2: 2 cand x 3 seeds x 20 eras = 120 = 288 eras/generation x 4 generations = 1152 eras BEFORE the winner's real training began. Against the observed era times on SP500 H1: PAI 29.1 s/era -> 9.3 h (matches the observed 00:37 -> 09:22) CONV 41.3 s/era -> 13.2 h LSTM 150.4 s/era -> 48.1 h HYBRID 154.6 s/era -> 49.5 h Two days to tune is not a first-run experience, and it is the phase in which the panel goes quiet, which is what made it look like a hang. It also bought nothing. The space is 90 points (10 MA periods x 9 MA types), so 1152 evaluations revisited each point ~13 times; and rungs of 3 and 8 eras cannot separate two MA periods at all. The 2026-08-01 run proves it: every finalist scored 25.0-25.9% balanced accuracy - below the 33.3% one-class floor, i.e. indistinguishable noise - and the search then "deployed the winner" of that. THE ERROR WAS THE SCORING FUNCTION, not its constants. Using a full training run to choose a feature's period is a wrapper method paying wrapper prices for a decision that does not need one. The reference book does not do this: ch. 3.3 selects inputs by measuring each candidate indicator's CORRELATION with the target and dropping the ones with none, with no network involved. So: rank candidates by the MUTUAL INFORMATION between the resulting feature vector and the triple-barrier label. MI rather than correlation because the label is 3-class categorical and the features are not monotonically related to it. Equal-FREQUENCY binning (rank-based), because these features are ATR-normalised and heavy-tailed - fixed-width bins put nearly everything in one bucket and report ~0 information for a genuinely useful feature. Scoring is arithmetic over the feature cache, so it costs seconds and its cost is independent of topology: LSTM now tunes as fast as the MLP. Coordinate sweep, not product sweep - cost is the SUM of per-parameter candidate counts, so enabling every indicator stays affordable - with a second pass that breaks early once nothing moves. Sampling is IS-ONLY. Letting the OOS window influence which indicator settings ship would mean the holdout had been used for selection and had stopped being a holdout. HONEST LIMIT, recorded because it is the price: MI is marginal, so a parameter that only pays off in combination with another can be missed (Guyon & Elisseeff 2003, filter vs wrapper). Given the wrapper it replaces was ranking pure noise at 48 h a run, this is strictly better. Deleted with it: GaRungEras/GaExtract/GaStore/GaMutate/GaRandomCandidate/ GaBlockCrossover/GaSortAliveByScoreDesc/GaBreedNextGeneration, 14 m_ga* members, the GA_*/TUNE_POP_* constants, and ComputeTuneTrialBudget. AND m_evalMode/m_evalEraBudget, because nothing set them any more - 28 read sites all permanently inert. That is not a tidy-up: the `if (!m_evalMode)` guard on UpdateClassPriors is exactly what silently disabled the imbalance correction for entire runs two commits ago. Dead machinery that still reads like live machinery is this codebase's most expensive recurring bug, and leaving 28 more instances of it would have been indefensible. The panel's tuning-progress state goes too - tuning no longer takes long enough to need one. Both builds compile 0 errors / 0 warnings. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-01 10:29:31 -04:00
if(sc > bestScore)
{
bestScore = sc;
keep = cands[c];
improvedThisPass = true;
}
}
best[p] = keep;
refactor: split CExpertSignalAIBase implementation by responsibility ExpertSignalAIBase.mqh was 8216 lines: the class declaration followed by 87 method bodies covering training, labelling, feature extraction, persistence, chart drawing, online learning, the GA auto-tuner and inference, all in one file. Train() alone is 1492 lines; a change to arrow drawing meant scrolling past the era loop. Moved the bodies into Expert\AIBase\, included at the bottom of the original after the class declaration: Training.mqh 1607 era loop, plateau ladder, checkpoint select, deploy Features.mqh 1093 indicator creation + per-bar input feature vector ChartUI.mqh 634 arrows, arrow persistence, status panel, cleanup Persistence.mqh 492 .stats/.cfg sidecars, CPU-inference validation, copy OnlineLearning.mqh 461 live continual learning, EMA shadow, OOS simulator Labels.mqh 309 ZigZag pivot labels, async label-cache prebuild AutoTune.mqh 275 genetic tuner (population, crossover, halving) Inference.mqh 235 softmax, prior calibration, class priors ExpertSignalAIBase.mqh 8216 -> 3131 (declaration + topology build only) This is a pure relocation - verified mechanically, not by eye: HEAD's file reconstructed from the eight partials plus the surviving remainder is byte-identical to HEAD, span for span (scratchpad verify_split.py). No declaration moved, no signature changed, no code rewritten, so behaviour is unchanged by construction. Compiles 0 errors, 0 warnings. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-29 00:42:45 -04:00
}
perf(autotune): replace the genetic search with a filter score - hours to seconds MEASURED COST OF THE GA, which is what retired it. Per generation: rung 0: 8 cand x 3 seeds x 3 eras = 72 eras rung 1: 4 cand x 3 seeds x 8 eras = 96 rung 2: 2 cand x 3 seeds x 20 eras = 120 = 288 eras/generation x 4 generations = 1152 eras BEFORE the winner's real training began. Against the observed era times on SP500 H1: PAI 29.1 s/era -> 9.3 h (matches the observed 00:37 -> 09:22) CONV 41.3 s/era -> 13.2 h LSTM 150.4 s/era -> 48.1 h HYBRID 154.6 s/era -> 49.5 h Two days to tune is not a first-run experience, and it is the phase in which the panel goes quiet, which is what made it look like a hang. It also bought nothing. The space is 90 points (10 MA periods x 9 MA types), so 1152 evaluations revisited each point ~13 times; and rungs of 3 and 8 eras cannot separate two MA periods at all. The 2026-08-01 run proves it: every finalist scored 25.0-25.9% balanced accuracy - below the 33.3% one-class floor, i.e. indistinguishable noise - and the search then "deployed the winner" of that. THE ERROR WAS THE SCORING FUNCTION, not its constants. Using a full training run to choose a feature's period is a wrapper method paying wrapper prices for a decision that does not need one. The reference book does not do this: ch. 3.3 selects inputs by measuring each candidate indicator's CORRELATION with the target and dropping the ones with none, with no network involved. So: rank candidates by the MUTUAL INFORMATION between the resulting feature vector and the triple-barrier label. MI rather than correlation because the label is 3-class categorical and the features are not monotonically related to it. Equal-FREQUENCY binning (rank-based), because these features are ATR-normalised and heavy-tailed - fixed-width bins put nearly everything in one bucket and report ~0 information for a genuinely useful feature. Scoring is arithmetic over the feature cache, so it costs seconds and its cost is independent of topology: LSTM now tunes as fast as the MLP. Coordinate sweep, not product sweep - cost is the SUM of per-parameter candidate counts, so enabling every indicator stays affordable - with a second pass that breaks early once nothing moves. Sampling is IS-ONLY. Letting the OOS window influence which indicator settings ship would mean the holdout had been used for selection and had stopped being a holdout. HONEST LIMIT, recorded because it is the price: MI is marginal, so a parameter that only pays off in combination with another can be missed (Guyon & Elisseeff 2003, filter vs wrapper). Given the wrapper it replaces was ranking pure noise at 48 h a run, this is strictly better. Deleted with it: GaRungEras/GaExtract/GaStore/GaMutate/GaRandomCandidate/ GaBlockCrossover/GaSortAliveByScoreDesc/GaBreedNextGeneration, 14 m_ga* members, the GA_*/TUNE_POP_* constants, and ComputeTuneTrialBudget. AND m_evalMode/m_evalEraBudget, because nothing set them any more - 28 read sites all permanently inert. That is not a tidy-up: the `if (!m_evalMode)` guard on UpdateClassPriors is exactly what silently disabled the imbalance correction for entire runs two commits ago. Dead machinery that still reads like live machinery is this codebase's most expensive recurring bug, and leaving 28 more instances of it would have been indefensible. The panel's tuning-progress state goes too - tuning no longer takes long enough to need one. Both builds compile 0 errors / 0 warnings. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-01 10:29:31 -04:00
if(!improvedThisPass)
break; // coordinate descent has converged - further passes cannot move anything
refactor: split CExpertSignalAIBase implementation by responsibility ExpertSignalAIBase.mqh was 8216 lines: the class declaration followed by 87 method bodies covering training, labelling, feature extraction, persistence, chart drawing, online learning, the GA auto-tuner and inference, all in one file. Train() alone is 1492 lines; a change to arrow drawing meant scrolling past the era loop. Moved the bodies into Expert\AIBase\, included at the bottom of the original after the class declaration: Training.mqh 1607 era loop, plateau ladder, checkpoint select, deploy Features.mqh 1093 indicator creation + per-bar input feature vector ChartUI.mqh 634 arrows, arrow persistence, status panel, cleanup Persistence.mqh 492 .stats/.cfg sidecars, CPU-inference validation, copy OnlineLearning.mqh 461 live continual learning, EMA shadow, OOS simulator Labels.mqh 309 ZigZag pivot labels, async label-cache prebuild AutoTune.mqh 275 genetic tuner (population, crossover, halving) Inference.mqh 235 softmax, prior calibration, class priors ExpertSignalAIBase.mqh 8216 -> 3131 (declaration + topology build only) This is a pure relocation - verified mechanically, not by eye: HEAD's file reconstructed from the eight partials plus the surviving remainder is byte-identical to HEAD, span for span (scratchpad verify_split.py). No declaration moved, no signature changed, no code rewritten, so behaviour is unchanged by construction. Compiles 0 errors, 0 warnings. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-29 00:42:45 -04:00
}
refactor(comments): stdlib comment style across the remaining in-scope files Same pass as 0b06f8e, applied file by file: comment runs of 4+ lines compressed to their leading topic sentences, capped at 4 lines, whole sentences only. Warning sentences (NEVER / MUST / trap / would-have) survive the budget. Every file was checked the same way before committing: the list of non-comment lines is byte-identical to HEAD, and braces balance. No code was touched. Panel/, Enumerations/ and the already-terse System headers needed little or nothing - PooledGate, TradeChecks, BinomialStats and Random came through with no blocks over the threshold at all. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-22 00:25:52 -04:00
//--- SELECTION GATE. bestScore is a MAXIMUM over every candidate scored, so it carries the same
//--- defect the barrier-geometry winner test and the lag profile were fixed for: the maximum of
//--- N draws from a null sits well above any single draw, and installing on "it beat the
//--- incumbent" alone crowns noise.
fix: make the indicator tuner actually measure, and gate what it installs ROOT CAUSE of the zero spread measured on SP500 H1 2026-08-07 (all 17 candidates returned exactly 0.00359 nats): the tune loop re-inits the indicators and then scores, with no RefreshData() between. ReInitADIndicators() does its part - Create() builds a NEW handle carrying the new parameters, and the feature cache is flagged stale so features really are recomputed. But BufferTempDataCompute() reads the CIndicatorBuffer objects, and only Refresh() copies data out of a handle into those. So every candidate was scored on values still held from the PREVIOUS handle. My earlier guess in the diagnostic ("suspect the feature cache") was wrong: the cache invalidation works. Two things land together, because neither is safe alone: 1. RefreshData() after the re-init, so a candidate is scored on its own features. 2. A SELECTION GATE on the install. bestScore is a MAXIMUM over candidates, and the maximum of N draws from a null beats its incumbent almost every time - so "it beat the incumbent" installs noise. This selector is the highest-stakes of the three found in this audit because it ACTS: it overwrites the user's configured indicator settings and forces BuildFreshTopology(), so the network then trains on whatever the noise picked. Fixing (1) without (2) would have made a dormant bug actively harmful. The gate draws the winner's own permutation null once, then corrects the p-value for having chosen it out of N with Sidak: p_family = 1 - (1-p)^N. Sidak rather than the max-of-N resample used by the geometry scan because each candidate here has a DIFFERENT feature set, so their draws cannot be pooled; Sidak needs only the one null. Exact under independence, mildly anti-conservative under positive dependence - stated in the comment rather than hidden. A rejected winner restores the configured settings, which best[] cannot do since the descent mutates it. Also reports the least-ready tunable handle's BarsCalculated(). IndicatorCreate() calculates asynchronously, so if the spread is STILL zero the handles simply are not done and the tuner needs to yield between candidates rather than score them back to back - a state machine like the label prebuild. That distinction is now readable from the log instead of requiring another guess. No input, topology or label change: no retrain. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-07 09:31:06 -04:00
bool install = (bestScore > startScore);
double pFamily = 1.0;
int distinct = (int)MathMax(evaluated + 1, 1); // candidates scored, plus the incumbent
if(install)
{
double wc[];
int wl[];
feat(tuner+altdata): tuner optimizes RANGE not direction; copy-paste whitelist UX on 4014 - MI_TUNE_TARGET = MI_TARGET_EXC_RANGE: the coordinate sweep scored candidates against the barrier label - measured noise - so it climbed a flat landscape and the gate rightly rejected every winner. It now selects indicator settings for MI vs realised RANGE (4x null, positive control), the channel the excursion head consumes these features for. Winner gate re-tests on the same target. Barrier-label report unchanged. - AltDataFetch 4014 handling: Alert popup + once-per-session walkthrough with the two whitelist URLs on their own journal lines (copy-paste ready); hourly-backoff retry instead of a permanent latch, so the whitelist fix takes effect without re-attaching. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-16 14:44:11 -04:00
//--- same target as the sweep's scorer, or the gate would test the winner against a
//--- different question than the one it was selected on
int wn = BuildMiSample(wc, wl, 0, 0, MI_TUNE_TARGET);
fix: make the indicator tuner actually measure, and gate what it installs ROOT CAUSE of the zero spread measured on SP500 H1 2026-08-07 (all 17 candidates returned exactly 0.00359 nats): the tune loop re-inits the indicators and then scores, with no RefreshData() between. ReInitADIndicators() does its part - Create() builds a NEW handle carrying the new parameters, and the feature cache is flagged stale so features really are recomputed. But BufferTempDataCompute() reads the CIndicatorBuffer objects, and only Refresh() copies data out of a handle into those. So every candidate was scored on values still held from the PREVIOUS handle. My earlier guess in the diagnostic ("suspect the feature cache") was wrong: the cache invalidation works. Two things land together, because neither is safe alone: 1. RefreshData() after the re-init, so a candidate is scored on its own features. 2. A SELECTION GATE on the install. bestScore is a MAXIMUM over candidates, and the maximum of N draws from a null beats its incumbent almost every time - so "it beat the incumbent" installs noise. This selector is the highest-stakes of the three found in this audit because it ACTS: it overwrites the user's configured indicator settings and forces BuildFreshTopology(), so the network then trains on whatever the noise picked. Fixing (1) without (2) would have made a dormant bug actively harmful. The gate draws the winner's own permutation null once, then corrects the p-value for having chosen it out of N with Sidak: p_family = 1 - (1-p)^N. Sidak rather than the max-of-N resample used by the geometry scan because each candidate here has a DIFFERENT feature set, so their draws cannot be pooled; Sidak needs only the one null. Exact under independence, mildly anti-conservative under positive dependence - stated in the comment rather than hidden. A rejected winner restores the configured settings, which best[] cannot do since the descent mutates it. Also reports the least-ready tunable handle's BarsCalculated(). IndicatorCreate() calculates asynchronously, so if the spread is STILL zero the handles simply are not done and the tuner needs to yield between candidates rather than score them back to back - a state machine like the label prebuild. That distinction is now readable from the log instead of requiring another guess. No input, topology or label change: no retrain. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-07 09:31:06 -04:00
if(wn >= MI_MIN_SAMPLES)
{
double obs = ScoreMiSample(wc, wl, wn, false);
int atLeast = 0, draws = 0;
for(int s = 0; s < MI_NOISE_PERMUTATIONS; s++)
{
fix(shutdown): make ExitPolicy public, and stop every long loop the moment MT5 asks Two things, one of which was a compile error. 1. ExitPolicy() was declared in the protected block but is pushed in from Warrior_EA.mq5:770. Moved to public beside the other EA-facing setters. 2. Chart objects surviving OnDeinit. The 4,500 ms teardown budget is measured from the STOP REQUEST, not from OnDeinit's first line, and OnDeinit cannot begin until whatever is in flight returns - so a scan still running after _StopFlag is raised does not delay the cleanup, it SPENDS it, and the purge never gets its turn. New CExpertSignalAIBase::ShutdownRequested() = IsStopped() || m_shutdownInProgress. Deliberately NOT m_trainingStopRequested: that latches, and a latched flag would permanently disable scans that must run again on the next Start. Guarded, longest first: - TuneIndicatorsByFilter - per candidate, restoring the OPERATOR's settings on the way out (best[] is mutated in place; the tuner otherwise keeps the last trial's parameters, which nothing chose). - ReportBarrierGeometryScan - per pairing, breaking to ONE restore point so m_barrierScanLiveLabels can never be left true (that makes ComputeLabelForBar read the last candidate's multiples as the configured geometry). - ReportFeatureLabelInformation / ReportExcursionInformation / lag profile - nulls ABANDON rather than truncate: fewer draws is not a smaller null, it is a wrong one, and p shifts toward significance. m_dirEvidence staying false is the safe direction. - SimulateExitPolicyOutcomes - zeroes its accumulators so the divergence line is dropped instead of latching a partial expectancy as the run's only report. - ReportGeometryExpectancyScan - per ladder rung. - HttpGet - one choke point for up to a dozen blocking WebRequests per first-pass Update(). An in-flight request cannot be cancelled; refusing to start another is the whole remedy. - PollTraining, OnChartEventHandler's study event, TuneIndicatorsAndTrain - entry points, so a queued event cannot open an era during teardown. TuneIndicatorsAndTrain's guard is the first statement, ahead of the m_tuneFilterDone / g_ensembleChartTuneDone latches. - OnTick / OnTimer / OnChartEvent. Training's own bar loops already honoured this (pass 1 per bar, passes 2/2.5/3 yield on a 120 ms budget); the warm-up scans did not, and they are the longest uninterruptible stretches the EA has. StopTraining() is unchanged: the operator's Stop still finalises synchronously. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-17 17:03:11 -04:00
//--- A truncated null is not a smaller null, it is a WRONG one - fewer draws shifts p toward
//--- significance. So a stop here abandons the test entirely (draws stays 0, pFamily stays
//--- 1.0, install becomes false) rather than installing on a partial null.
if(ShutdownRequested())
{
draws = 0;
break;
}
fix: make the indicator tuner actually measure, and gate what it installs ROOT CAUSE of the zero spread measured on SP500 H1 2026-08-07 (all 17 candidates returned exactly 0.00359 nats): the tune loop re-inits the indicators and then scores, with no RefreshData() between. ReInitADIndicators() does its part - Create() builds a NEW handle carrying the new parameters, and the feature cache is flagged stale so features really are recomputed. But BufferTempDataCompute() reads the CIndicatorBuffer objects, and only Refresh() copies data out of a handle into those. So every candidate was scored on values still held from the PREVIOUS handle. My earlier guess in the diagnostic ("suspect the feature cache") was wrong: the cache invalidation works. Two things land together, because neither is safe alone: 1. RefreshData() after the re-init, so a candidate is scored on its own features. 2. A SELECTION GATE on the install. bestScore is a MAXIMUM over candidates, and the maximum of N draws from a null beats its incumbent almost every time - so "it beat the incumbent" installs noise. This selector is the highest-stakes of the three found in this audit because it ACTS: it overwrites the user's configured indicator settings and forces BuildFreshTopology(), so the network then trains on whatever the noise picked. Fixing (1) without (2) would have made a dormant bug actively harmful. The gate draws the winner's own permutation null once, then corrects the p-value for having chosen it out of N with Sidak: p_family = 1 - (1-p)^N. Sidak rather than the max-of-N resample used by the geometry scan because each candidate here has a DIFFERENT feature set, so their draws cannot be pooled; Sidak needs only the one null. Exact under independence, mildly anti-conservative under positive dependence - stated in the comment rather than hidden. A rejected winner restores the configured settings, which best[] cannot do since the descent mutates it. Also reports the least-ready tunable handle's BarsCalculated(). IndicatorCreate() calculates asynchronously, so if the spread is STILL zero the handles simply are not done and the tuner needs to yield between candidates rather than score them back to back - a state machine like the label prebuild. That distinction is now readable from the log instead of requiring another guess. No input, topology or label change: no retrain. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-07 09:31:06 -04:00
double d = ScoreMiSample(wc, wl, wn, true);
if(d < 0.0)
continue;
if(d >= obs)
atLeast++;
draws++;
}
if(draws > 0)
{
double pSingle = (double)(1 + atLeast) / (draws + 1);
pFamily = 1.0 - MathPow(1.0 - pSingle, (double)distinct);
}
}
install = (pFamily <= MI_TUNE_ALPHA);
}
if(!install)
{
ArrayCopy(best, configured);
bestScore = startScore;
}
perf(autotune): replace the genetic search with a filter score - hours to seconds MEASURED COST OF THE GA, which is what retired it. Per generation: rung 0: 8 cand x 3 seeds x 3 eras = 72 eras rung 1: 4 cand x 3 seeds x 8 eras = 96 rung 2: 2 cand x 3 seeds x 20 eras = 120 = 288 eras/generation x 4 generations = 1152 eras BEFORE the winner's real training began. Against the observed era times on SP500 H1: PAI 29.1 s/era -> 9.3 h (matches the observed 00:37 -> 09:22) CONV 41.3 s/era -> 13.2 h LSTM 150.4 s/era -> 48.1 h HYBRID 154.6 s/era -> 49.5 h Two days to tune is not a first-run experience, and it is the phase in which the panel goes quiet, which is what made it look like a hang. It also bought nothing. The space is 90 points (10 MA periods x 9 MA types), so 1152 evaluations revisited each point ~13 times; and rungs of 3 and 8 eras cannot separate two MA periods at all. The 2026-08-01 run proves it: every finalist scored 25.0-25.9% balanced accuracy - below the 33.3% one-class floor, i.e. indistinguishable noise - and the search then "deployed the winner" of that. THE ERROR WAS THE SCORING FUNCTION, not its constants. Using a full training run to choose a feature's period is a wrapper method paying wrapper prices for a decision that does not need one. The reference book does not do this: ch. 3.3 selects inputs by measuring each candidate indicator's CORRELATION with the target and dropping the ones with none, with no network involved. So: rank candidates by the MUTUAL INFORMATION between the resulting feature vector and the triple-barrier label. MI rather than correlation because the label is 3-class categorical and the features are not monotonically related to it. Equal-FREQUENCY binning (rank-based), because these features are ATR-normalised and heavy-tailed - fixed-width bins put nearly everything in one bucket and report ~0 information for a genuinely useful feature. Scoring is arithmetic over the feature cache, so it costs seconds and its cost is independent of topology: LSTM now tunes as fast as the MLP. Coordinate sweep, not product sweep - cost is the SUM of per-parameter candidate counts, so enabling every indicator stays affordable - with a second pass that breaks early once nothing moves. Sampling is IS-ONLY. Letting the OOS window influence which indicator settings ship would mean the holdout had been used for selection and had stopped being a holdout. HONEST LIMIT, recorded because it is the price: MI is marginal, so a parameter that only pays off in combination with another can be missed (Guyon & Elisseeff 2003, filter vs wrapper). Given the wrapper it replaces was ranking pure noise at 48 h a run, this is strictly better. Deleted with it: GaRungEras/GaExtract/GaStore/GaMutate/GaRandomCandidate/ GaBlockCrossover/GaSortAliveByScoreDesc/GaBreedNextGeneration, 14 m_ga* members, the GA_*/TUNE_POP_* constants, and ComputeTuneTrialBudget. AND m_evalMode/m_evalEraBudget, because nothing set them any more - 28 read sites all permanently inert. That is not a tidy-up: the `if (!m_evalMode)` guard on UpdateClassPriors is exactly what silently disabled the imbalance correction for entire runs two commits ago. Dead machinery that still reads like live machinery is this codebase's most expensive recurring bug, and leaving 28 more instances of it would have been indefensible. The panel's tuning-progress state goes too - tuning no longer takes long enough to need one. Both builds compile 0 errors / 0 warnings. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-01 10:29:31 -04:00
//--- install the winner and leave the indicators/feature cache consistent with it
m_indicatorTuner.Unflatten(best);
ReInitADIndicators(m_indicatorsPtr);
fix: make the indicator tuner actually measure, and gate what it installs ROOT CAUSE of the zero spread measured on SP500 H1 2026-08-07 (all 17 candidates returned exactly 0.00359 nats): the tune loop re-inits the indicators and then scores, with no RefreshData() between. ReInitADIndicators() does its part - Create() builds a NEW handle carrying the new parameters, and the feature cache is flagged stale so features really are recomputed. But BufferTempDataCompute() reads the CIndicatorBuffer objects, and only Refresh() copies data out of a handle into those. So every candidate was scored on values still held from the PREVIOUS handle. My earlier guess in the diagnostic ("suspect the feature cache") was wrong: the cache invalidation works. Two things land together, because neither is safe alone: 1. RefreshData() after the re-init, so a candidate is scored on its own features. 2. A SELECTION GATE on the install. bestScore is a MAXIMUM over candidates, and the maximum of N draws from a null beats its incumbent almost every time - so "it beat the incumbent" installs noise. This selector is the highest-stakes of the three found in this audit because it ACTS: it overwrites the user's configured indicator settings and forces BuildFreshTopology(), so the network then trains on whatever the noise picked. Fixing (1) without (2) would have made a dormant bug actively harmful. The gate draws the winner's own permutation null once, then corrects the p-value for having chosen it out of N with Sidak: p_family = 1 - (1-p)^N. Sidak rather than the max-of-N resample used by the geometry scan because each candidate here has a DIFFERENT feature set, so their draws cannot be pooled; Sidak needs only the one null. Exact under independence, mildly anti-conservative under positive dependence - stated in the comment rather than hidden. A rejected winner restores the configured settings, which best[] cannot do since the descent mutates it. Also reports the least-ready tunable handle's BarsCalculated(). IndicatorCreate() calculates asynchronously, so if the spread is STILL zero the handles simply are not done and the tuner needs to yield between candidates rather than score them back to back - a state machine like the label prebuild. That distinction is now readable from the log instead of requiring another guess. No input, topology or label change: no retrain. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-07 09:31:06 -04:00
RefreshData();
feat(inputs): unify ALL indicator periods under the tuner; EnableAltData input; AI-first defaults - PeriodMA/MA_Type/PeriodRSI: input -> const seeds (closing the set: every indicator parameter is now tuner-owned) - Variables\TunedPeriods.mqh: chart-level tuned-period state. A gated install writes TunedPeriods_{SYM}_{TF}.cfg; next attach reads it BEFORE the DB fingerprint and classic-signal config, so classic votes, DB key, and tuner seeds always describe the same indicators regardless of classic/AI/hybrid use. Restart-grained adoption by design (no mid-run handle churn); new periods re-key the signal DB (semantics rule). - EnableAltData input in AI Input Features (consumption gate only; collection keeps running); |ALT DB-fingerprint token; opt-out on an alt-trained model correctly starts fresh via the width compare. - Defaults: all four classic votes OFF (AI-first; WARRIOR_MARKET_BUILD branches collapsed with the marketplace pivot), order-flow/Wyckoff NN features OFF (alt data is the default information diet; toggles stay). Compiles 0 errors / 0 warnings. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-16 15:12:54 -04:00
//--- A gated INSTALL is chart-level news, not just this model's: persist the winning periods so
//--- the classic votes, the signal-DB key and every later tuner seed adopt them on the next
//--- attach (restart-grained - see Variables\TunedPeriods.mqh for why not mid-run).
if(install)
SaveTunedPeriods(m_indicatorTuner.maPeriod, m_indicatorTuner.maType, m_indicatorTuner.rsiPeriod,
m_indicatorTuner.macdFast, m_indicatorTuner.macdSlow, m_indicatorTuner.macdSignal,
m_indicatorTuner.ichiTenkan, m_indicatorTuner.ichiKijun, m_indicatorTuner.ichiSenkou);
diag: report the indicator tuner's candidate spread - "no improvement" is ambiguous Auditing the other best-of-N scans after cccf94f turned up a third instance of the same pattern, and this one is worse than the two already fixed: the geometry scan and the lag profile PRINT a row, whereas TuneIndicatorsByFilter INSTALLS its winner (Unflatten + ReInitADIndicators) and the caller then calls BuildFreshTopology(), so an unguarded maximum changes the feature vector the network trains on. It has no null of any kind. But before adding one, the logs say something a noise-driven best-of-N cannot: 2026-08-05/06, four consecutive runs, 17 candidates each, every one "no improvement" with start and best identical to 4dp. The maximum of 17 draws from a noise distribution beats its incumbent about 94% of the time, so 4/4 is on the order of 1 in 100,000. Two readings fit and they want opposite responses: - INERT: trial scores come back identical to the incumbent because the parameter change never reaches the scored features (suspect the feature cache surviving ReInitADIndicators), so `sc > bestScore` can never fire. That is a dead code path, and gating it would be decorating a corpse. - LIVE and correctly finding nothing: then it needs the family-wise gate. The current log line cannot separate them, so add the number that can: the span of the candidate scores, with an explicit ZERO SPREAD callout naming the likely cause. Also widened the MI figures from 4dp to 5dp - at this scale 4dp rounds the entire effect away. No gate yet, deliberately: measure which failure this is, then fix that one. Read-only diagnostic. No input, topology or label change: no retrain. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-06 21:59:10 -04:00
double candSpread = (evaluated > 0 && candMax >= candMin) ? (candMax - candMin) : 0.0;
diag(autotune): stop making the feature test cost a trained model The permutation test lived inside TuneIndicatorsByFilter, which is gated on era 0 - correctly, because re-running the SWEEP would change the input vector out from under weights already fitted to the old one. But the test itself reads cached features and writes nothing, so none of that applies to it, and the gate meant the only way to see the answer on a running model was to delete the model. Today that price was PAI's 45 trained eras and CONV's 31, spent to re-ask a read-only question. Split into ReportFeatureLabelInformation(), called from the sweep when it runs and directly when it does not - a resumed model, a disabled tuner, nothing tunable. Once per attach either way. Compiles 0 errors / 0 warnings. Build tag permtest-v2. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-01 14:01:32 -04:00
Print(ID + StringFormat(": auto-tune complete - %d candidate settings scored in %.1fs, "
diag: report the indicator tuner's candidate spread - "no improvement" is ambiguous Auditing the other best-of-N scans after cccf94f turned up a third instance of the same pattern, and this one is worse than the two already fixed: the geometry scan and the lag profile PRINT a row, whereas TuneIndicatorsByFilter INSTALLS its winner (Unflatten + ReInitADIndicators) and the caller then calls BuildFreshTopology(), so an unguarded maximum changes the feature vector the network trains on. It has no null of any kind. But before adding one, the logs say something a noise-driven best-of-N cannot: 2026-08-05/06, four consecutive runs, 17 candidates each, every one "no improvement" with start and best identical to 4dp. The maximum of 17 draws from a noise distribution beats its incumbent about 94% of the time, so 4/4 is on the order of 1 in 100,000. Two readings fit and they want opposite responses: - INERT: trial scores come back identical to the incumbent because the parameter change never reaches the scored features (suspect the feature cache surviving ReInitADIndicators), so `sc > bestScore` can never fire. That is a dead code path, and gating it would be decorating a corpse. - LIVE and correctly finding nothing: then it needs the family-wise gate. The current log line cannot separate them, so add the number that can: the span of the candidate scores, with an explicit ZERO SPREAD callout naming the likely cause. Also widened the MI figures from 4dp to 5dp - at this scale 4dp rounds the entire effect away. No gate yet, deliberately: measure which failure this is, then fix that one. Read-only diagnostic. No input, topology or label change: no retrain. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-06 21:59:10 -04:00
"feature/label mutual information %.5f -> %.5f nats%s | candidate scores span "
"%.5f (%.5f..%.5f)%s",
diag(autotune): stop making the feature test cost a trained model The permutation test lived inside TuneIndicatorsByFilter, which is gated on era 0 - correctly, because re-running the SWEEP would change the input vector out from under weights already fitted to the old one. But the test itself reads cached features and writes nothing, so none of that applies to it, and the gate meant the only way to see the answer on a running model was to delete the model. Today that price was PAI's 45 trained eras and CONV's 31, spent to re-ask a read-only question. Split into ReportFeatureLabelInformation(), called from the sweep when it runs and directly when it does not - a resumed model, a disabled tuner, nothing tunable. Once per attach either way. Compiles 0 errors / 0 warnings. Build tag permtest-v2. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-01 14:01:32 -04:00
evaluated, (GetTickCount() - t0) / 1000.0, startScore, bestScore,
diag: report the indicator tuner's candidate spread - "no improvement" is ambiguous Auditing the other best-of-N scans after cccf94f turned up a third instance of the same pattern, and this one is worse than the two already fixed: the geometry scan and the lag profile PRINT a row, whereas TuneIndicatorsByFilter INSTALLS its winner (Unflatten + ReInitADIndicators) and the caller then calls BuildFreshTopology(), so an unguarded maximum changes the feature vector the network trains on. It has no null of any kind. But before adding one, the logs say something a noise-driven best-of-N cannot: 2026-08-05/06, four consecutive runs, 17 candidates each, every one "no improvement" with start and best identical to 4dp. The maximum of 17 draws from a noise distribution beats its incumbent about 94% of the time, so 4/4 is on the order of 1 in 100,000. Two readings fit and they want opposite responses: - INERT: trial scores come back identical to the incumbent because the parameter change never reaches the scored features (suspect the feature cache surviving ReInitADIndicators), so `sc > bestScore` can never fire. That is a dead code path, and gating it would be decorating a corpse. - LIVE and correctly finding nothing: then it needs the family-wise gate. The current log line cannot separate them, so add the number that can: the span of the candidate scores, with an explicit ZERO SPREAD callout naming the likely cause. Also widened the MI figures from 4dp to 5dp - at this scale 4dp rounds the entire effect away. No gate yet, deliberately: measure which failure this is, then fix that one. Read-only diagnostic. No input, topology or label change: no retrain. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-06 21:59:10 -04:00
(bestScore <= startScore ? " (no improvement - keeping the configured settings)" : ""),
candSpread, (evaluated > 0 ? candMin : 0.0), (evaluated > 0 ? candMax : 0.0),
(evaluated > 0 && candSpread <= 0.0
fix: make the indicator tuner actually measure, and gate what it installs ROOT CAUSE of the zero spread measured on SP500 H1 2026-08-07 (all 17 candidates returned exactly 0.00359 nats): the tune loop re-inits the indicators and then scores, with no RefreshData() between. ReInitADIndicators() does its part - Create() builds a NEW handle carrying the new parameters, and the feature cache is flagged stale so features really are recomputed. But BufferTempDataCompute() reads the CIndicatorBuffer objects, and only Refresh() copies data out of a handle into those. So every candidate was scored on values still held from the PREVIOUS handle. My earlier guess in the diagnostic ("suspect the feature cache") was wrong: the cache invalidation works. Two things land together, because neither is safe alone: 1. RefreshData() after the re-init, so a candidate is scored on its own features. 2. A SELECTION GATE on the install. bestScore is a MAXIMUM over candidates, and the maximum of N draws from a null beats its incumbent almost every time - so "it beat the incumbent" installs noise. This selector is the highest-stakes of the three found in this audit because it ACTS: it overwrites the user's configured indicator settings and forces BuildFreshTopology(), so the network then trains on whatever the noise picked. Fixing (1) without (2) would have made a dormant bug actively harmful. The gate draws the winner's own permutation null once, then corrects the p-value for having chosen it out of N with Sidak: p_family = 1 - (1-p)^N. Sidak rather than the max-of-N resample used by the geometry scan because each candidate here has a DIFFERENT feature set, so their draws cannot be pooled; Sidak needs only the one null. Exact under independence, mildly anti-conservative under positive dependence - stated in the comment rather than hidden. A rejected winner restores the configured settings, which best[] cannot do since the descent mutates it. Also reports the least-ready tunable handle's BarsCalculated(). IndicatorCreate() calculates asynchronously, so if the spread is STILL zero the handles simply are not done and the tuner needs to yield between candidates rather than score them back to back - a state machine like the label prebuild. That distinction is now readable from the log instead of requiring another guess. No input, topology or label change: no retrain. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-07 09:31:06 -04:00
? StringFormat(" <-- ZERO SPREAD: every candidate scored identically, so the "
"parameter change is STILL not reaching the scored features even "
"with the post-re-init RefreshData(). Least-ready tunable handle "
"had %d bars calculated - if that is 0 or far below the study "
"window, the handles are simply not done calculating yet and the "
"tuner needs to yield between candidates rather than score them "
"back to back.", (readyMin == INT_MAX ? -1 : readyMin))
: StringFormat(" | winner %s (selection p=%.4f after correcting for %d "
"candidates, need <=%.2f)",
(install ? "INSTALLED" : "REJECTED - keeping the configured "
"settings, since the best of N noise draws beats its incumbent "
"almost every time"),
pFamily, distinct, MI_TUNE_ALPHA))));
refactor(comments): stdlib comment style across the remaining in-scope files Same pass as 0b06f8e, applied file by file: comment runs of 4+ lines compressed to their leading topic sentences, capped at 4 lines, whole sentences only. Warning sentences (NEVER / MUST / trap / would-have) survive the budget. Every file was checked the same way before committing: the list of non-comment lines is byte-identical to HEAD, and braces balance. No code was touched. Panel/, Enumerations/ and the already-terse System headers needed little or nothing - PooledGate, TradeChecks, BinomialStats and Random came through with no blocks over the threshold at all. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-22 00:25:52 -04:00
//--- An EXACTLY zero score is not a weak feature set, it is a broken measurement. Landing on
//--- 0.0000 means every column read back constant, which is what a feature-extraction fault
//--- looks like.
diag(autotune): stop making the feature test cost a trained model The permutation test lived inside TuneIndicatorsByFilter, which is gated on era 0 - correctly, because re-running the SWEEP would change the input vector out from under weights already fitted to the old one. But the test itself reads cached features and writes nothing, so none of that applies to it, and the gate meant the only way to see the answer on a running model was to delete the model. Today that price was PAI's 45 trained eras and CONV's 31, spent to re-ask a read-only question. Split into ReportFeatureLabelInformation(), called from the sweep when it runs and directly when it does not - a resumed model, a disabled tuner, nothing tunable. Once per attach either way. Compiles 0 errors / 0 warnings. Build tag permtest-v2. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-01 14:01:32 -04:00
if(bestScore <= 0.0)
Print(ID + ": WARNING - every candidate scored 0.0000 nats. Finite-sample bias alone should put "
"noise above zero, so this indicates the feature values are not being read, not that the "
"features are uninformative. Indicator settings left at their configured values.");
ReportFeatureLabelInformation();
}
//+------------------------------------------------------------------+
refactor(mi): split AutoTune.mqh - SEARCH vs MEASUREMENT Session C of the feature-selection/labeling refactor track. AutoTune.mqh was two responsibilities in one 1,584-line file: SEARCH (TuneIndicatorsByFilter, coordinate-descent over indicator settings) and MEASUREMENT (the "does this feature vector predict this label at all" evidence screen and its three sub-reports). Split along that seam into a new Expert/AIBase/FeatureScreen.mqh. Moved, verbatim (diffed byte-for-byte against the pre-split content - zero lines differ beyond the file-boundary comment headers): ReportFeatureLabel- Information, ReportExcursionInformation, ReportFeatureLagProfile, Report- BarrierGeometryScan, ApplyAdoptedGeometry. Stayed in AutoTune.mqh: the MI engine (FeatureColumnMI/BuildMiSample/ ScoreMiSample) both files call - a shared dependency used by two consumers is not itself a reason to split further; TuneIndicatorsByFilter; the export utilities; and TuneIndicatorsAndTrain, the entry point that decides which of the two branches a given model runs - it is the coordinator, not a member of either side. Still body-only method definitions of CExpertSignalAIBase, same as every other Expert\AIBase\*.mqh file - MQL5 has no partial classes, so this is a file-organisation move (legibility, SRP-per-file), not a coupling reduction. The include site says order between AIBase\*.mqh files is irrelevant, so the new include was added next to AutoTune.mqh's; the file-scope g_ensembleChart* globals both files reference stay declared in AutoTune.mqh's header, ahead of the new include either way. Verified: brace counts split exactly 105 -> 57+48; every one of the 14 function definitions HEAD had in AutoTune.mqh accounted for in exactly one of the two files, no duplicates; the WARRIOR_EXPORT_FEATURES ifdef/endif pair (unrelated, lines 35/147) untouched. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-08-23 19:08:36 -04:00
//| The MI evidence screen (ReportFeatureLabelInformation and its |
//| three sub-reports: lag profile, excursion learnability, barrier |
//| geometry scan + ApplyAdoptedGeometry) moved to FeatureScreen.mqh |
//| on 2026-08-23 - SEARCH (this file) vs MEASUREMENT (that one) are |
//| two responsibilities. FeatureColumnMI/BuildMiSample/ScoreMiSample |
//| above stay here: both files call them, and a shared dependency |
//| used by two consumers is not itself a reason to split further. |
diag(autotune): stop making the feature test cost a trained model The permutation test lived inside TuneIndicatorsByFilter, which is gated on era 0 - correctly, because re-running the SWEEP would change the input vector out from under weights already fitted to the old one. But the test itself reads cached features and writes nothing, so none of that applies to it, and the gate meant the only way to see the answer on a running model was to delete the model. Today that price was PAI's 45 trained eras and CONV's 31, spent to re-ask a read-only question. Split into ReportFeatureLabelInformation(), called from the sweep when it runs and directly when it does not - a resumed model, a disabled tuner, nothing tunable. Once per attach either way. Compiles 0 errors / 0 warnings. Build tag permtest-v2. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-01 14:01:32 -04:00
//+------------------------------------------------------------------+
perf(autotune): replace the genetic search with a filter score - hours to seconds MEASURED COST OF THE GA, which is what retired it. Per generation: rung 0: 8 cand x 3 seeds x 3 eras = 72 eras rung 1: 4 cand x 3 seeds x 8 eras = 96 rung 2: 2 cand x 3 seeds x 20 eras = 120 = 288 eras/generation x 4 generations = 1152 eras BEFORE the winner's real training began. Against the observed era times on SP500 H1: PAI 29.1 s/era -> 9.3 h (matches the observed 00:37 -> 09:22) CONV 41.3 s/era -> 13.2 h LSTM 150.4 s/era -> 48.1 h HYBRID 154.6 s/era -> 49.5 h Two days to tune is not a first-run experience, and it is the phase in which the panel goes quiet, which is what made it look like a hang. It also bought nothing. The space is 90 points (10 MA periods x 9 MA types), so 1152 evaluations revisited each point ~13 times; and rungs of 3 and 8 eras cannot separate two MA periods at all. The 2026-08-01 run proves it: every finalist scored 25.0-25.9% balanced accuracy - below the 33.3% one-class floor, i.e. indistinguishable noise - and the search then "deployed the winner" of that. THE ERROR WAS THE SCORING FUNCTION, not its constants. Using a full training run to choose a feature's period is a wrapper method paying wrapper prices for a decision that does not need one. The reference book does not do this: ch. 3.3 selects inputs by measuring each candidate indicator's CORRELATION with the target and dropping the ones with none, with no network involved. So: rank candidates by the MUTUAL INFORMATION between the resulting feature vector and the triple-barrier label. MI rather than correlation because the label is 3-class categorical and the features are not monotonically related to it. Equal-FREQUENCY binning (rank-based), because these features are ATR-normalised and heavy-tailed - fixed-width bins put nearly everything in one bucket and report ~0 information for a genuinely useful feature. Scoring is arithmetic over the feature cache, so it costs seconds and its cost is independent of topology: LSTM now tunes as fast as the MLP. Coordinate sweep, not product sweep - cost is the SUM of per-parameter candidate counts, so enabling every indicator stays affordable - with a second pass that breaks early once nothing moves. Sampling is IS-ONLY. Letting the OOS window influence which indicator settings ship would mean the holdout had been used for selection and had stopped being a holdout. HONEST LIMIT, recorded because it is the price: MI is marginal, so a parameter that only pays off in combination with another can be missed (Guyon & Elisseeff 2003, filter vs wrapper). Given the wrapper it replaces was ranking pure noise at 48 h a run, this is strictly better. Deleted with it: GaRungEras/GaExtract/GaStore/GaMutate/GaRandomCandidate/ GaBlockCrossover/GaSortAliveByScoreDesc/GaBreedNextGeneration, 14 m_ga* members, the GA_*/TUNE_POP_* constants, and ComputeTuneTrialBudget. AND m_evalMode/m_evalEraBudget, because nothing set them any more - 28 read sites all permanently inert. That is not a tidy-up: the `if (!m_evalMode)` guard on UpdateClassPriors is exactly what silently disabled the imbalance correction for entire runs two commits ago. Dead machinery that still reads like live machinery is this codebase's most expensive recurring bug, and leaving 28 more instances of it would have been indefensible. The panel's tuning-progress state goes too - tuning no longer takes long enough to need one. Both builds compile 0 errors / 0 warnings. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-01 10:29:31 -04:00
void CExpertSignalAIBase::TuneIndicatorsAndTrain(datetime StartTrainBar = 0)
{
fix(shutdown): make ExitPolicy public, and stop every long loop the moment MT5 asks Two things, one of which was a compile error. 1. ExitPolicy() was declared in the protected block but is pushed in from Warrior_EA.mq5:770. Moved to public beside the other EA-facing setters. 2. Chart objects surviving OnDeinit. The 4,500 ms teardown budget is measured from the STOP REQUEST, not from OnDeinit's first line, and OnDeinit cannot begin until whatever is in flight returns - so a scan still running after _StopFlag is raised does not delay the cleanup, it SPENDS it, and the purge never gets its turn. New CExpertSignalAIBase::ShutdownRequested() = IsStopped() || m_shutdownInProgress. Deliberately NOT m_trainingStopRequested: that latches, and a latched flag would permanently disable scans that must run again on the next Start. Guarded, longest first: - TuneIndicatorsByFilter - per candidate, restoring the OPERATOR's settings on the way out (best[] is mutated in place; the tuner otherwise keeps the last trial's parameters, which nothing chose). - ReportBarrierGeometryScan - per pairing, breaking to ONE restore point so m_barrierScanLiveLabels can never be left true (that makes ComputeLabelForBar read the last candidate's multiples as the configured geometry). - ReportFeatureLabelInformation / ReportExcursionInformation / lag profile - nulls ABANDON rather than truncate: fewer draws is not a smaller null, it is a wrong one, and p shifts toward significance. m_dirEvidence staying false is the safe direction. - SimulateExitPolicyOutcomes - zeroes its accumulators so the divergence line is dropped instead of latching a partial expectancy as the run's only report. - ReportGeometryExpectancyScan - per ladder rung. - HttpGet - one choke point for up to a dozen blocking WebRequests per first-pass Update(). An in-flight request cannot be cancelled; refusing to start another is the whole remedy. - PollTraining, OnChartEventHandler's study event, TuneIndicatorsAndTrain - entry points, so a queued event cannot open an era during teardown. TuneIndicatorsAndTrain's guard is the first statement, ahead of the m_tuneFilterDone / g_ensembleChartTuneDone latches. - OnTick / OnTimer / OnChartEvent. Training's own bar loops already honoured this (pass 1 per bar, passes 2/2.5/3 yield on a 120 ms budget); the warm-up scans did not, and they are the longest uninterruptible stretches the EA has. StopTraining() is unchanged: the operator's Stop still finalises synchronously. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-17 17:03:11 -04:00
//--- FIRST STATEMENT IN THE WHOLE TRAINING ENTRY POINT, ahead of every latch below it (m_tuneFilterDone,
//--- g_ensembleChartTuneDone) so a stop cannot mark a sweep as "already run" without running it. The
//--- individual scans yield on ShutdownRequested() as well; this simply refuses to start the chain.
if(ShutdownRequested())
return;
fix: prebuild and era sized different windows; diag: Train() names its branch TWO things, one incident. 1) THE BUG I SHIPPED IN 0c85c54. m_tuneStartTrainBar is declared, initialised to 0, and NEVER ASSIGNED - the assignment existed before the God-class split and the split dropped it, leaving a dead member. Harmless while nothing read it; a real defect the moment 0c85c54 made StartLabelCachePrebuild() reset dtStudied from it. Train() then computed the window as max(StartTrainBar, floor) while the prebuild computed max(0, floor), where StartTrainBar is the non-zero datetime OnChartEventHandler passes through from the "New Bar" event. The two therefore disagreed about `bars`, so EnsureBarCachesCapacity() saw a changed size at era start, wiped the caches, and re-armed a full 38k-bar prebuild - instead of training. Restored the assignment so both sides evaluate the identical expression. 2) THE REASON IT TOOK ALL NIGHT TO FIND. Train() is a state machine with six early-return branches above the era loop and every one of them is silent. Four charts burned a core each for 15 minutes with an empty journal: the pass heartbeats (694b756) proved the era loop was never reached, no prebuild completion line appeared either, and nothing external can see inside a single MQL5 thread - per-thread CPU says "busy", file writes say nothing, and the VPS has no debugger. That is an undiagnosable state, and it is the thing to fix, not just the bug of the day. ReportTrainStall() now names the branch Train() is taking whenever no era has completed for 3 minutes, at most once a minute per signal, with the state that decides the branch: run/prebuild/simOos/resume flags, era, dtStudied, and - for the cache-invalidation branch specifically - BOTH bar counts, since two sizings disagreeing is exactly what re-arms the prebuild forever. Silent on a healthy run: an era completing resets the clock. Both build variants compile 0 errors, 0 warnings. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-10 07:32:08 -04:00
//--- Publish the caller's window anchor so StartLabelCachePrebuild() sizes its window with the SAME
refactor(comments): stdlib comment style across the remaining in-scope files Same pass as 0b06f8e, applied file by file: comment runs of 4+ lines compressed to their leading topic sentences, capped at 4 lines, whole sentences only. Warning sentences (NEVER / MUST / trap / would-have) survive the budget. Every file was checked the same way before committing: the list of non-comment lines is byte-identical to HEAD, and braces balance. No code was touched. Panel/, Enumerations/ and the already-terse System headers needed little or nothing - PooledGate, TradeChecks, BinomialStats and Random came through with no blocks over the threshold at all. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-22 00:25:52 -04:00
//--- expression Train() uses.
fix: prebuild and era sized different windows; diag: Train() names its branch TWO things, one incident. 1) THE BUG I SHIPPED IN 0c85c54. m_tuneStartTrainBar is declared, initialised to 0, and NEVER ASSIGNED - the assignment existed before the God-class split and the split dropped it, leaving a dead member. Harmless while nothing read it; a real defect the moment 0c85c54 made StartLabelCachePrebuild() reset dtStudied from it. Train() then computed the window as max(StartTrainBar, floor) while the prebuild computed max(0, floor), where StartTrainBar is the non-zero datetime OnChartEventHandler passes through from the "New Bar" event. The two therefore disagreed about `bars`, so EnsureBarCachesCapacity() saw a changed size at era start, wiped the caches, and re-armed a full 38k-bar prebuild - instead of training. Restored the assignment so both sides evaluate the identical expression. 2) THE REASON IT TOOK ALL NIGHT TO FIND. Train() is a state machine with six early-return branches above the era loop and every one of them is silent. Four charts burned a core each for 15 minutes with an empty journal: the pass heartbeats (694b756) proved the era loop was never reached, no prebuild completion line appeared either, and nothing external can see inside a single MQL5 thread - per-thread CPU says "busy", file writes say nothing, and the VPS has no debugger. That is an undiagnosable state, and it is the thing to fix, not just the bug of the day. ReportTrainStall() now names the branch Train() is taking whenever no era has completed for 3 minutes, at most once a minute per signal, with the state that decides the branch: run/prebuild/simOos/resume flags, era, dtStudied, and - for the cache-invalidation branch specifically - BOTH bar counts, since two sizings disagreeing is exactly what re-arms the prebuild forever. Silent on a healthy run: an era completing resets the clock. Both build variants compile 0 errors, 0 warnings. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-10 07:32:08 -04:00
m_tuneStartTrainBar = StartTrainBar;
perf(autotune): replace the genetic search with a filter score - hours to seconds MEASURED COST OF THE GA, which is what retired it. Per generation: rung 0: 8 cand x 3 seeds x 3 eras = 72 eras rung 1: 4 cand x 3 seeds x 8 eras = 96 rung 2: 2 cand x 3 seeds x 20 eras = 120 = 288 eras/generation x 4 generations = 1152 eras BEFORE the winner's real training began. Against the observed era times on SP500 H1: PAI 29.1 s/era -> 9.3 h (matches the observed 00:37 -> 09:22) CONV 41.3 s/era -> 13.2 h LSTM 150.4 s/era -> 48.1 h HYBRID 154.6 s/era -> 49.5 h Two days to tune is not a first-run experience, and it is the phase in which the panel goes quiet, which is what made it look like a hang. It also bought nothing. The space is 90 points (10 MA periods x 9 MA types), so 1152 evaluations revisited each point ~13 times; and rungs of 3 and 8 eras cannot separate two MA periods at all. The 2026-08-01 run proves it: every finalist scored 25.0-25.9% balanced accuracy - below the 33.3% one-class floor, i.e. indistinguishable noise - and the search then "deployed the winner" of that. THE ERROR WAS THE SCORING FUNCTION, not its constants. Using a full training run to choose a feature's period is a wrapper method paying wrapper prices for a decision that does not need one. The reference book does not do this: ch. 3.3 selects inputs by measuring each candidate indicator's CORRELATION with the target and dropping the ones with none, with no network involved. So: rank candidates by the MUTUAL INFORMATION between the resulting feature vector and the triple-barrier label. MI rather than correlation because the label is 3-class categorical and the features are not monotonically related to it. Equal-FREQUENCY binning (rank-based), because these features are ATR-normalised and heavy-tailed - fixed-width bins put nearly everything in one bucket and report ~0 information for a genuinely useful feature. Scoring is arithmetic over the feature cache, so it costs seconds and its cost is independent of topology: LSTM now tunes as fast as the MLP. Coordinate sweep, not product sweep - cost is the SUM of per-parameter candidate counts, so enabling every indicator stays affordable - with a second pass that breaks early once nothing moves. Sampling is IS-ONLY. Letting the OOS window influence which indicator settings ship would mean the holdout had been used for selection and had stopped being a holdout. HONEST LIMIT, recorded because it is the price: MI is marginal, so a parameter that only pays off in combination with another can be missed (Guyon & Elisseeff 2003, filter vs wrapper). Given the wrapper it replaces was ranking pure noise at 48 h a run, this is strictly better. Deleted with it: GaRungEras/GaExtract/GaStore/GaMutate/GaRandomCandidate/ GaBlockCrossover/GaSortAliveByScoreDesc/GaBreedNextGeneration, 14 m_ga* members, the GA_*/TUNE_POP_* constants, and ComputeTuneTrialBudget. AND m_evalMode/m_evalEraBudget, because nothing set them any more - 28 read sites all permanently inert. That is not a tidy-up: the `if (!m_evalMode)` guard on UpdateClassPriors is exactly what silently disabled the imbalance correction for entire runs two commits ago. Dead machinery that still reads like live machinery is this codebase's most expensive recurring bug, and leaving 28 more instances of it would have been indefensible. The panel's tuning-progress state goes too - tuning no longer takes long enough to need one. Both builds compile 0 errors / 0 warnings. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-01 10:29:31 -04:00
bool anyTunable = (m_useADCumulativeDelta || m_useADShorteningOfThrust || m_useADWyckoffEventStream ||
m_useADWyckoffFailedStructure || m_useADWyckoffSignificantBarInversion ||
m_useMA || m_useRSI || m_useMACD || m_useIchimoku);
//--- Tune once per fresh model, before any weight has been trained. Gated on m_labelCachePrebuilt
//--- because the score needs labels, and on era 0 because re-tuning a partly-trained network would
//--- change its inputs out from under weights already fitted to the old ones.
if(m_autoTuneIndicators && anyTunable && !m_tuneFilterDone && m_labelCachePrebuilt && m_eraCount == 0)
refactor: split CExpertSignalAIBase implementation by responsibility ExpertSignalAIBase.mqh was 8216 lines: the class declaration followed by 87 method bodies covering training, labelling, feature extraction, persistence, chart drawing, online learning, the GA auto-tuner and inference, all in one file. Train() alone is 1492 lines; a change to arrow drawing meant scrolling past the era loop. Moved the bodies into Expert\AIBase\, included at the bottom of the original after the class declaration: Training.mqh 1607 era loop, plateau ladder, checkpoint select, deploy Features.mqh 1093 indicator creation + per-bar input feature vector ChartUI.mqh 634 arrows, arrow persistence, status panel, cleanup Persistence.mqh 492 .stats/.cfg sidecars, CPU-inference validation, copy OnlineLearning.mqh 461 live continual learning, EMA shadow, OOS simulator Labels.mqh 309 ZigZag pivot labels, async label-cache prebuild AutoTune.mqh 275 genetic tuner (population, crossover, halving) Inference.mqh 235 softmax, prior calibration, class priors ExpertSignalAIBase.mqh 8216 -> 3131 (declaration + topology build only) This is a pure relocation - verified mechanically, not by eye: HEAD's file reconstructed from the eight partials plus the surviving remainder is byte-identical to HEAD, span for span (scratchpad verify_split.py). No declaration moved, no signature changed, no code rewritten, so behaviour is unchanged by construction. Compiles 0 errors, 0 warnings. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-29 00:42:45 -04:00
{
perf(autotune): replace the genetic search with a filter score - hours to seconds MEASURED COST OF THE GA, which is what retired it. Per generation: rung 0: 8 cand x 3 seeds x 3 eras = 72 eras rung 1: 4 cand x 3 seeds x 8 eras = 96 rung 2: 2 cand x 3 seeds x 20 eras = 120 = 288 eras/generation x 4 generations = 1152 eras BEFORE the winner's real training began. Against the observed era times on SP500 H1: PAI 29.1 s/era -> 9.3 h (matches the observed 00:37 -> 09:22) CONV 41.3 s/era -> 13.2 h LSTM 150.4 s/era -> 48.1 h HYBRID 154.6 s/era -> 49.5 h Two days to tune is not a first-run experience, and it is the phase in which the panel goes quiet, which is what made it look like a hang. It also bought nothing. The space is 90 points (10 MA periods x 9 MA types), so 1152 evaluations revisited each point ~13 times; and rungs of 3 and 8 eras cannot separate two MA periods at all. The 2026-08-01 run proves it: every finalist scored 25.0-25.9% balanced accuracy - below the 33.3% one-class floor, i.e. indistinguishable noise - and the search then "deployed the winner" of that. THE ERROR WAS THE SCORING FUNCTION, not its constants. Using a full training run to choose a feature's period is a wrapper method paying wrapper prices for a decision that does not need one. The reference book does not do this: ch. 3.3 selects inputs by measuring each candidate indicator's CORRELATION with the target and dropping the ones with none, with no network involved. So: rank candidates by the MUTUAL INFORMATION between the resulting feature vector and the triple-barrier label. MI rather than correlation because the label is 3-class categorical and the features are not monotonically related to it. Equal-FREQUENCY binning (rank-based), because these features are ATR-normalised and heavy-tailed - fixed-width bins put nearly everything in one bucket and report ~0 information for a genuinely useful feature. Scoring is arithmetic over the feature cache, so it costs seconds and its cost is independent of topology: LSTM now tunes as fast as the MLP. Coordinate sweep, not product sweep - cost is the SUM of per-parameter candidate counts, so enabling every indicator stays affordable - with a second pass that breaks early once nothing moves. Sampling is IS-ONLY. Letting the OOS window influence which indicator settings ship would mean the holdout had been used for selection and had stopped being a holdout. HONEST LIMIT, recorded because it is the price: MI is marginal, so a parameter that only pays off in combination with another can be missed (Guyon & Elisseeff 2003, filter vs wrapper). Given the wrapper it replaces was ranking pure noise at 48 h a run, this is strictly better. Deleted with it: GaRungEras/GaExtract/GaStore/GaMutate/GaRandomCandidate/ GaBlockCrossover/GaSortAliveByScoreDesc/GaBreedNextGeneration, 14 m_ga* members, the GA_*/TUNE_POP_* constants, and ComputeTuneTrialBudget. AND m_evalMode/m_evalEraBudget, because nothing set them any more - 28 read sites all permanently inert. That is not a tidy-up: the `if (!m_evalMode)` guard on UpdateClassPriors is exactly what silently disabled the imbalance correction for entire runs two commits ago. Dead machinery that still reads like live machinery is this codebase's most expensive recurring bug, and leaving 28 more instances of it would have been indefensible. The panel's tuning-progress state goes too - tuning no longer takes long enough to need one. Both builds compile 0 errors / 0 warnings. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-01 10:29:31 -04:00
m_tuneFilterDone = true;
fix(ensemble): responsive panel + synchronized eras + combined-vote accuracy Four user-reported/requested items, one root cause chain: 1) DEAD CONTROL PANEL in AI_HYBRID mode. All members posted custom event id 1 and handled id 1001, and CExpertCustom broadcasts every chart event to every filter - so each posted event ran a train chunk in ALL N members (N*N chunks per round) and the chart thread never idled long enough to deliver clicks/drags. profiling.csv: 99.45% of time in OnChartEventHandler. Fix: per-instance study-event ids (STUDY_EVENT_ID_BASE + construction order, offset above the Controls library's ON_* codes - id 1 was also ON_DBL_CLICK, so panel double-clicks fired training chunks). ArmStudyEvent() is the single post site; lost-event watchdog replaces the accidental sibling-clears-my-flag rescue. 2) WARM-UP DUPLICATION. The auto-tune sweep is deterministic over identical features/labels, and it ends in the full MI diagnostic suite, which the MI-share gate never intercepted on the sweep path - four members ran four identical ~36s sweep+report blocks. First member publishes outcome (g_ensembleChartTuneDone/Installed/Settings); the rest apply it and skip both. 3) DEINIT STRANDED PANEL+ARROWS (user repro 18:52). Root cause from the log: the 4,500ms budget runs from MetaTrader's stop REQUEST - a heavy autosave in flight ate it, OnDeinit got ~430ms and died in the first member's arrow persist ("Abnormal termination" 432ms in). Fix: early visible-UI sweep (native prefix deletes for status/panel/dialog) right after ClearStatusLabel, and a fast path for still-training models - their arrows are re-rendered every era, so they get one bulk purge instead of scan+atomic-write in the death window. 4) ENSEMBLE FEATURES (user requests): era BARRIER - members advance era by era together; a member ahead of the slowest still-training member declines Train() calls and its chunk budget is donated (TRAIN_TIME_BUDGET_MS = 120/activeTrainers, UI headroom constant). COMBINED-VOTE OOS SCORE - each member's pass-3 scan contributes its adjusted per-bar decision (0.0 on abstain) to a shared row buffer; the last member to finish the era scores the averaged vote vs the mirrored Min_Vote_Open against the same target-before-stop outcomes members grade themselves on, publishing an "Ensemble vote" line on the aggregated panel. Member headlines now carry their lifetime win rate with break-even. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-16 19:06:04 -04:00
if(m_ensembleMember && g_ensembleChartTuneDone)
{
refactor(comments): stdlib comment style across the remaining in-scope files Same pass as 0b06f8e, applied file by file: comment runs of 4+ lines compressed to their leading topic sentences, capped at 4 lines, whole sentences only. Warning sentences (NEVER / MUST / trap / would-have) survive the budget. Every file was checked the same way before committing: the list of non-comment lines is byte-identical to HEAD, and braces balance. No code was touched. Panel/, Enumerations/ and the already-terse System headers needed little or nothing - PooledGate, TradeChecks, BinomialStats and Random came through with no blocks over the threshold at all. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-22 00:25:52 -04:00
//--- Another member on this chart already ran the identical sweep - apply its outcome
//--- instead of recomputing it (see g_ensembleChartTuneDone at the top of this file). SAY
//--- IT ON THE PANEL, not only in the journal.
fix(handles): the MA handle was SHARED, and a rejected sweep freed it for everyone else ROOT CAUSE of the six-session "silent block failure", measured rather than inferred. All TWELVE dead-handle recoveries in today's log report the SAME handle number - MA=-1(h13) - across two charts and all four members. It was never four handles. It was one. MT5 refcounts indicator requests, so four ensemble members asking for the same iMA on the same symbol/period share a single handle. TuneIndicatorsByFilter creates and drops ~35 of them scoring candidates; the sweep's runner ends up holding a live handle while its siblings still hold a number the terminal has already freed. Timeline, twice, to the millisecond: USDJPY 18:10:03 PAI: auto-tune complete 18:10:29.864/.910/.953 CONV/LSTM/HYB: "already ran ... REJECTED" 18:10:30.057/.065/.074 all three: MA=-1(h13), sweep bars all rejected XAUUSD 18:10:11 -> 18:10:42.19/.23/.27 -> 18:10:42.334 identical, same ~100ms The adopt branch re-initialised indicators only `if(g_ensembleChartTuneInstalled)` - exactly backwards. A REJECTED sweep churns just as many handles, and every one of the twelve recoveries followed a rejection. Parameters are still adopted only on an install; the HANDLES are now rebuilt either way. Four creations per chart. RepairDeadIndicatorHandles stays - it is cause-agnostic and it is what made this diagnosable. This removes the cause it was recovering from. Also, consistency of the warm-up status (user-reported: "only one nn will say scoring indicators, which leaves some doubt about what is going on"): - the sweeping member now says it is scoring "for the whole chart", so three idle rows read as the design rather than a stall; - the two adopt branches (tuner and MI) publish to the panel instead of only printing, so every row accounts for itself; - the MI suite publishes before it runs. It is the longest stretch of the whole warm-up - MI, lag profile, excursion targets, geometry scan, each with its own permutation null - and it published nothing at all, so during most of the warm-up the panel's last word described a step that had already finished. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-17 18:16:28 -04:00
PublishStatus(ID + " : adopting the chart's tuned indicators...");
//--- THE PARAMETERS are adopted only when a winner was installed...
fix(ensemble): responsive panel + synchronized eras + combined-vote accuracy Four user-reported/requested items, one root cause chain: 1) DEAD CONTROL PANEL in AI_HYBRID mode. All members posted custom event id 1 and handled id 1001, and CExpertCustom broadcasts every chart event to every filter - so each posted event ran a train chunk in ALL N members (N*N chunks per round) and the chart thread never idled long enough to deliver clicks/drags. profiling.csv: 99.45% of time in OnChartEventHandler. Fix: per-instance study-event ids (STUDY_EVENT_ID_BASE + construction order, offset above the Controls library's ON_* codes - id 1 was also ON_DBL_CLICK, so panel double-clicks fired training chunks). ArmStudyEvent() is the single post site; lost-event watchdog replaces the accidental sibling-clears-my-flag rescue. 2) WARM-UP DUPLICATION. The auto-tune sweep is deterministic over identical features/labels, and it ends in the full MI diagnostic suite, which the MI-share gate never intercepted on the sweep path - four members ran four identical ~36s sweep+report blocks. First member publishes outcome (g_ensembleChartTuneDone/Installed/Settings); the rest apply it and skip both. 3) DEINIT STRANDED PANEL+ARROWS (user repro 18:52). Root cause from the log: the 4,500ms budget runs from MetaTrader's stop REQUEST - a heavy autosave in flight ate it, OnDeinit got ~430ms and died in the first member's arrow persist ("Abnormal termination" 432ms in). Fix: early visible-UI sweep (native prefix deletes for status/panel/dialog) right after ClearStatusLabel, and a fast path for still-training models - their arrows are re-rendered every era, so they get one bulk purge instead of scan+atomic-write in the death window. 4) ENSEMBLE FEATURES (user requests): era BARRIER - members advance era by era together; a member ahead of the slowest still-training member declines Train() calls and its chunk budget is donated (TRAIN_TIME_BUDGET_MS = 120/activeTrainers, UI headroom constant). COMBINED-VOTE OOS SCORE - each member's pass-3 scan contributes its adjusted per-bar decision (0.0 on abstain) to a shared row buffer; the last member to finish the era scores the averaged vote vs the mirrored Min_Vote_Open against the same target-before-stop outcomes members grade themselves on, publishing an "Ensemble vote" line on the aggregated panel. Member headlines now carry their lifetime win rate with break-even. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-16 19:06:04 -04:00
if(g_ensembleChartTuneInstalled)
m_indicatorTuner.Unflatten(g_ensembleChartTuneSettings);
refactor(comments): stdlib comment style across the remaining in-scope files Same pass as 0b06f8e, applied file by file: comment runs of 4+ lines compressed to their leading topic sentences, capped at 4 lines, whole sentences only. Warning sentences (NEVER / MUST / trap / would-have) survive the budget. Every file was checked the same way before committing: the list of non-comment lines is byte-identical to HEAD, and braces balance. No code was touched. Panel/, Enumerations/ and the already-terse System headers needed little or nothing - PooledGate, TradeChecks, BinomialStats and Random came through with no blocks over the threshold at all. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-22 00:25:52 -04:00
//--- ...but the HANDLES must be rebuilt EITHER WAY, and that is not a tidiness point - it
//--- is the cause of the "silent block failure" that cost six sessions. That is the whole
//--- finding: it was never four handles, it was ONE.
fix(handles): the MA handle was SHARED, and a rejected sweep freed it for everyone else ROOT CAUSE of the six-session "silent block failure", measured rather than inferred. All TWELVE dead-handle recoveries in today's log report the SAME handle number - MA=-1(h13) - across two charts and all four members. It was never four handles. It was one. MT5 refcounts indicator requests, so four ensemble members asking for the same iMA on the same symbol/period share a single handle. TuneIndicatorsByFilter creates and drops ~35 of them scoring candidates; the sweep's runner ends up holding a live handle while its siblings still hold a number the terminal has already freed. Timeline, twice, to the millisecond: USDJPY 18:10:03 PAI: auto-tune complete 18:10:29.864/.910/.953 CONV/LSTM/HYB: "already ran ... REJECTED" 18:10:30.057/.065/.074 all three: MA=-1(h13), sweep bars all rejected XAUUSD 18:10:11 -> 18:10:42.19/.23/.27 -> 18:10:42.334 identical, same ~100ms The adopt branch re-initialised indicators only `if(g_ensembleChartTuneInstalled)` - exactly backwards. A REJECTED sweep churns just as many handles, and every one of the twelve recoveries followed a rejection. Parameters are still adopted only on an install; the HANDLES are now rebuilt either way. Four creations per chart. RepairDeadIndicatorHandles stays - it is cause-agnostic and it is what made this diagnosable. This removes the cause it was recovering from. Also, consistency of the warm-up status (user-reported: "only one nn will say scoring indicators, which leaves some doubt about what is going on"): - the sweeping member now says it is scoring "for the whole chart", so three idle rows read as the design rather than a stall; - the two adopt branches (tuner and MI) publish to the panel instead of only printing, so every row accounts for itself; - the MI suite publishes before it runs. It is the longest stretch of the whole warm-up - MI, lag profile, excursion targets, geometry scan, each with its own permutation null - and it published nothing at all, so during most of the warm-up the panel's last word described a step that had already finished. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-17 18:16:28 -04:00
ReInitADIndicators(m_indicatorsPtr);
RefreshData();
fix(ensemble): responsive panel + synchronized eras + combined-vote accuracy Four user-reported/requested items, one root cause chain: 1) DEAD CONTROL PANEL in AI_HYBRID mode. All members posted custom event id 1 and handled id 1001, and CExpertCustom broadcasts every chart event to every filter - so each posted event ran a train chunk in ALL N members (N*N chunks per round) and the chart thread never idled long enough to deliver clicks/drags. profiling.csv: 99.45% of time in OnChartEventHandler. Fix: per-instance study-event ids (STUDY_EVENT_ID_BASE + construction order, offset above the Controls library's ON_* codes - id 1 was also ON_DBL_CLICK, so panel double-clicks fired training chunks). ArmStudyEvent() is the single post site; lost-event watchdog replaces the accidental sibling-clears-my-flag rescue. 2) WARM-UP DUPLICATION. The auto-tune sweep is deterministic over identical features/labels, and it ends in the full MI diagnostic suite, which the MI-share gate never intercepted on the sweep path - four members ran four identical ~36s sweep+report blocks. First member publishes outcome (g_ensembleChartTuneDone/Installed/Settings); the rest apply it and skip both. 3) DEINIT STRANDED PANEL+ARROWS (user repro 18:52). Root cause from the log: the 4,500ms budget runs from MetaTrader's stop REQUEST - a heavy autosave in flight ate it, OnDeinit got ~430ms and died in the first member's arrow persist ("Abnormal termination" 432ms in). Fix: early visible-UI sweep (native prefix deletes for status/panel/dialog) right after ClearStatusLabel, and a fast path for still-training models - their arrows are re-rendered every era, so they get one bulk purge instead of scan+atomic-write in the death window. 4) ENSEMBLE FEATURES (user requests): era BARRIER - members advance era by era together; a member ahead of the slowest still-training member declines Train() calls and its chunk budget is donated (TRAIN_TIME_BUDGET_MS = 120/activeTrainers, UI headroom constant). COMBINED-VOTE OOS SCORE - each member's pass-3 scan contributes its adjusted per-bar decision (0.0 on abstain) to a shared row buffer; the last member to finish the era scores the averaged vote vs the mirrored Min_Vote_Open against the same target-before-stop outcomes members grade themselves on, publishing an "Ensemble vote" line on the aggregated panel. Member headlines now carry their lifetime win rate with break-even. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-16 19:06:04 -04:00
Print(ID + ": indicator auto-tune already ran on this chart - same indicators, same features, "
"same labels, same answer. " +
(g_ensembleChartTuneInstalled
? "Adopting the installed winner so every member trains on the same feature vector."
: "Keeping the configured settings (the sweep's winner was rejected by the selection gate).") +
" The first member's auto-tune report above is this model's too.");
}
else
{
refactor(comments): stdlib comment style across the remaining in-scope files Same pass as 0b06f8e, applied file by file: comment runs of 4+ lines compressed to their leading topic sentences, capped at 4 lines, whole sentences only. Warning sentences (NEVER / MUST / trap / would-have) survive the budget. Every file was checked the same way before committing: the list of non-comment lines is byte-identical to HEAD, and braces balance. No code was touched. Panel/, Enumerations/ and the already-terse System headers needed little or nothing - PooledGate, TradeChecks, BinomialStats and Random came through with no blocks over the threshold at all. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-22 00:25:52 -04:00
//--- Names the SCOPE, because the scope is what the other rows' silence means.
fix(handles): the MA handle was SHARED, and a rejected sweep freed it for everyone else ROOT CAUSE of the six-session "silent block failure", measured rather than inferred. All TWELVE dead-handle recoveries in today's log report the SAME handle number - MA=-1(h13) - across two charts and all four members. It was never four handles. It was one. MT5 refcounts indicator requests, so four ensemble members asking for the same iMA on the same symbol/period share a single handle. TuneIndicatorsByFilter creates and drops ~35 of them scoring candidates; the sweep's runner ends up holding a live handle while its siblings still hold a number the terminal has already freed. Timeline, twice, to the millisecond: USDJPY 18:10:03 PAI: auto-tune complete 18:10:29.864/.910/.953 CONV/LSTM/HYB: "already ran ... REJECTED" 18:10:30.057/.065/.074 all three: MA=-1(h13), sweep bars all rejected XAUUSD 18:10:11 -> 18:10:42.19/.23/.27 -> 18:10:42.334 identical, same ~100ms The adopt branch re-initialised indicators only `if(g_ensembleChartTuneInstalled)` - exactly backwards. A REJECTED sweep churns just as many handles, and every one of the twelve recoveries followed a rejection. Parameters are still adopted only on an install; the HANDLES are now rebuilt either way. Four creations per chart. RepairDeadIndicatorHandles stays - it is cause-agnostic and it is what made this diagnosable. This removes the cause it was recovering from. Also, consistency of the warm-up status (user-reported: "only one nn will say scoring indicators, which leaves some doubt about what is going on"): - the sweeping member now says it is scoring "for the whole chart", so three idle rows read as the design rather than a stall; - the two adopt branches (tuner and MI) publish to the panel instead of only printing, so every row accounts for itself; - the MI suite publishes before it runs. It is the longest stretch of the whole warm-up - MI, lag profile, excursion targets, geometry scan, each with its own permutation null - and it published nothing at all, so during most of the warm-up the panel's last word described a step that had already finished. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-17 18:16:28 -04:00
PublishStatus(ID + (m_ensembleMember
? " : scoring indicator settings for the whole chart..."
: " : scoring indicator settings..."));
fix(ensemble): responsive panel + synchronized eras + combined-vote accuracy Four user-reported/requested items, one root cause chain: 1) DEAD CONTROL PANEL in AI_HYBRID mode. All members posted custom event id 1 and handled id 1001, and CExpertCustom broadcasts every chart event to every filter - so each posted event ran a train chunk in ALL N members (N*N chunks per round) and the chart thread never idled long enough to deliver clicks/drags. profiling.csv: 99.45% of time in OnChartEventHandler. Fix: per-instance study-event ids (STUDY_EVENT_ID_BASE + construction order, offset above the Controls library's ON_* codes - id 1 was also ON_DBL_CLICK, so panel double-clicks fired training chunks). ArmStudyEvent() is the single post site; lost-event watchdog replaces the accidental sibling-clears-my-flag rescue. 2) WARM-UP DUPLICATION. The auto-tune sweep is deterministic over identical features/labels, and it ends in the full MI diagnostic suite, which the MI-share gate never intercepted on the sweep path - four members ran four identical ~36s sweep+report blocks. First member publishes outcome (g_ensembleChartTuneDone/Installed/Settings); the rest apply it and skip both. 3) DEINIT STRANDED PANEL+ARROWS (user repro 18:52). Root cause from the log: the 4,500ms budget runs from MetaTrader's stop REQUEST - a heavy autosave in flight ate it, OnDeinit got ~430ms and died in the first member's arrow persist ("Abnormal termination" 432ms in). Fix: early visible-UI sweep (native prefix deletes for status/panel/dialog) right after ClearStatusLabel, and a fast path for still-training models - their arrows are re-rendered every era, so they get one bulk purge instead of scan+atomic-write in the death window. 4) ENSEMBLE FEATURES (user requests): era BARRIER - members advance era by era together; a member ahead of the slowest still-training member declines Train() calls and its chunk budget is donated (TRAIN_TIME_BUDGET_MS = 120/activeTrainers, UI headroom constant). COMBINED-VOTE OOS SCORE - each member's pass-3 scan contributes its adjusted per-bar decision (0.0 on abstain) to a shared row buffer; the last member to finish the era scores the averaged vote vs the mirrored Min_Vote_Open against the same target-before-stop outcomes members grade themselves on, publishing an "Ensemble vote" line on the aggregated panel. Member headlines now carry their lifetime win rate with break-even. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-16 19:06:04 -04:00
//--- Snapshot the configured settings first: "did the sweep install?" is answered by comparing
//--- against the final settings, since a rejected winner is restored to exactly these values.
double tuneCfgBefore[];
m_indicatorTuner.Flatten(tuneCfgBefore);
TuneIndicatorsByFilter();
if(m_ensembleMember)
{
m_indicatorTuner.Flatten(g_ensembleChartTuneSettings);
g_ensembleChartTuneInstalled = false;
for(int tp = 0; tp < ArraySize(tuneCfgBefore); tp++)
if(g_ensembleChartTuneSettings[tp] != tuneCfgBefore[tp])
{
g_ensembleChartTuneInstalled = true;
break;
}
g_ensembleChartTuneDone = true;
//--- The sweep ends in ReportFeatureLabelInformation(), so the chart-level MI report is
//--- done too - mark it, or every other member would rerun the ~200-draw nulls the MI
//--- gate below exists to save.
if(m_miReportDone)
g_ensembleChartMiReportDone = true;
}
}
perf(autotune): replace the genetic search with a filter score - hours to seconds MEASURED COST OF THE GA, which is what retired it. Per generation: rung 0: 8 cand x 3 seeds x 3 eras = 72 eras rung 1: 4 cand x 3 seeds x 8 eras = 96 rung 2: 2 cand x 3 seeds x 20 eras = 120 = 288 eras/generation x 4 generations = 1152 eras BEFORE the winner's real training began. Against the observed era times on SP500 H1: PAI 29.1 s/era -> 9.3 h (matches the observed 00:37 -> 09:22) CONV 41.3 s/era -> 13.2 h LSTM 150.4 s/era -> 48.1 h HYBRID 154.6 s/era -> 49.5 h Two days to tune is not a first-run experience, and it is the phase in which the panel goes quiet, which is what made it look like a hang. It also bought nothing. The space is 90 points (10 MA periods x 9 MA types), so 1152 evaluations revisited each point ~13 times; and rungs of 3 and 8 eras cannot separate two MA periods at all. The 2026-08-01 run proves it: every finalist scored 25.0-25.9% balanced accuracy - below the 33.3% one-class floor, i.e. indistinguishable noise - and the search then "deployed the winner" of that. THE ERROR WAS THE SCORING FUNCTION, not its constants. Using a full training run to choose a feature's period is a wrapper method paying wrapper prices for a decision that does not need one. The reference book does not do this: ch. 3.3 selects inputs by measuring each candidate indicator's CORRELATION with the target and dropping the ones with none, with no network involved. So: rank candidates by the MUTUAL INFORMATION between the resulting feature vector and the triple-barrier label. MI rather than correlation because the label is 3-class categorical and the features are not monotonically related to it. Equal-FREQUENCY binning (rank-based), because these features are ATR-normalised and heavy-tailed - fixed-width bins put nearly everything in one bucket and report ~0 information for a genuinely useful feature. Scoring is arithmetic over the feature cache, so it costs seconds and its cost is independent of topology: LSTM now tunes as fast as the MLP. Coordinate sweep, not product sweep - cost is the SUM of per-parameter candidate counts, so enabling every indicator stays affordable - with a second pass that breaks early once nothing moves. Sampling is IS-ONLY. Letting the OOS window influence which indicator settings ship would mean the holdout had been used for selection and had stopped being a holdout. HONEST LIMIT, recorded because it is the price: MI is marginal, so a parameter that only pays off in combination with another can be missed (Guyon & Elisseeff 2003, filter vs wrapper). Given the wrapper it replaces was ranking pure noise at 48 h a run, this is strictly better. Deleted with it: GaRungEras/GaExtract/GaStore/GaMutate/GaRandomCandidate/ GaBlockCrossover/GaSortAliveByScoreDesc/GaBreedNextGeneration, 14 m_ga* members, the GA_*/TUNE_POP_* constants, and ComputeTuneTrialBudget. AND m_evalMode/m_evalEraBudget, because nothing set them any more - 28 read sites all permanently inert. That is not a tidy-up: the `if (!m_evalMode)` guard on UpdateClassPriors is exactly what silently disabled the imbalance correction for entire runs two commits ago. Dead machinery that still reads like live machinery is this codebase's most expensive recurring bug, and leaving 28 more instances of it would have been indefensible. The panel's tuning-progress state goes too - tuning no longer takes long enough to need one. Both builds compile 0 errors / 0 warnings. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-01 10:29:31 -04:00
//--- the winning parameters change the input vector, so the network must start from scratch on it
BuildFreshTopology();
refactor: split CExpertSignalAIBase implementation by responsibility ExpertSignalAIBase.mqh was 8216 lines: the class declaration followed by 87 method bodies covering training, labelling, feature extraction, persistence, chart drawing, online learning, the GA auto-tuner and inference, all in one file. Train() alone is 1492 lines; a change to arrow drawing meant scrolling past the era loop. Moved the bodies into Expert\AIBase\, included at the bottom of the original after the class declaration: Training.mqh 1607 era loop, plateau ladder, checkpoint select, deploy Features.mqh 1093 indicator creation + per-bar input feature vector ChartUI.mqh 634 arrows, arrow persistence, status panel, cleanup Persistence.mqh 492 .stats/.cfg sidecars, CPU-inference validation, copy OnlineLearning.mqh 461 live continual learning, EMA shadow, OOS simulator Labels.mqh 309 ZigZag pivot labels, async label-cache prebuild AutoTune.mqh 275 genetic tuner (population, crossover, halving) Inference.mqh 235 softmax, prior calibration, class priors ExpertSignalAIBase.mqh 8216 -> 3131 (declaration + topology build only) This is a pure relocation - verified mechanically, not by eye: HEAD's file reconstructed from the eight partials plus the surviving remainder is byte-identical to HEAD, span for span (scratchpad verify_split.py). No declaration moved, no signature changed, no code rewritten, so behaviour is unchanged by construction. Compiles 0 errors, 0 warnings. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-29 00:42:45 -04:00
}
refactor(comments): stdlib comment style across the remaining in-scope files Same pass as 0b06f8e, applied file by file: comment runs of 4+ lines compressed to their leading topic sentences, capped at 4 lines, whole sentences only. Warning sentences (NEVER / MUST / trap / would-have) survive the budget. Every file was checked the same way before committing: the list of non-comment lines is byte-identical to HEAD, and braces balance. No code was touched. Panel/, Enumerations/ and the already-terse System headers needed little or nothing - PooledGate, TradeChecks, BinomialStats and Random came through with no blocks over the threshold at all. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-22 00:25:52 -04:00
//--- The DIAGNOSTIC half runs even when the sweep does not: on a resumed model, on one whose
//--- tuner is switched off, and on one with nothing tunable.
fix: MI diagnostics never ran on a resumed model - the stated intent was never achieved A comment above the diagnostic branch says it "runs even when the sweep does not: on a resumed model ... tying it to that gate meant the only way to see the answer on a running model was to delete the model." It does not. Moving the diagnostic out of the tuner's gate left it behind m_labelCachePrebuilt, which has the same effect: the eager label pre-scan runs only on a FRESH start, because a net loaded from disk labels lazily per bar. So on a resumed model the flag is false forever and the whole MI block - headline, positive control, alignment scan, lag profile, geometry scan, winner test, and the auto-tune line - silently never runs. Measured on SP500 H1 2026-08-07: attached at era 271, still nothing by era 314, zero MI lines in the day's log, and the only "label cache pre-built" entry predates the attach. It also explains the shape of every capture on 08-05/06: each one came directly after a weights reset. The situation the comment was written to eliminate is exactly the situation that persisted. So drive the pre-scan when it is the only thing missing. Safe on a trained net: its one fresh-net side effect, pushing the output-layer bias toward the dominant class, is already gated on m_eraCount == 0, and the advance gate in Train() sits ABOVE if(!m_trainRunActive), so the era loop keeps its state - training pauses for the scan (~1s at 38k bars) and continues from where it was, not from 0. Announced only on a start that actually armed, since StartLabelCachePrebuild() returns unarmed when history is not ready and is retried per bar event. NOT sampled from the lazily-filled cache instead: BuildMiSample skips bars with no cached label, so that would score whichever subset training happened to have visited - a biased subsample presented as a measurement, which is the failure this diagnostic exists to catch. Also corrects a claim in 0d58923's comment. It argued four consecutive "no improvement" runs were ~1-in-100,000 evidence the indicator tuner is inert, by multiplying 5.6% across four runs. They are not independent trials: the MI scorer is deterministic and all four covered nearly the same bars, so an incumbent that is the maximum on this data is the maximum on every run. One ~1-in-18 observation with three correlated repeats, ~5.6% - unremarkable. The same independence assumption that made the uncorrected lag profile star four lags. The candidate-spread line stands: it settles inert-vs-live directly. No input, topology or label change: no retrain. Training in flight stays valid. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-07 09:08:01 -04:00
else if(!m_miReportDone && !m_labelCachePrebuilt && !m_labelPrebuildActive)
{
//--- Announce only on a start that actually took. StartLabelCachePrebuild() returns without arming
//--- if the buffers/history are not ready yet and is simply retried on the next call, so printing
//--- unconditionally would repeat the line once per bar event until it succeeds.
StartLabelCachePrebuild();
refactor(comments): stdlib comment style across the remaining in-scope files Same pass as 0b06f8e, applied file by file: comment runs of 4+ lines compressed to their leading topic sentences, capped at 4 lines, whole sentences only. Warning sentences (NEVER / MUST / trap / would-have) survive the budget. Every file was checked the same way before committing: the list of non-comment lines is byte-identical to HEAD, and braces balance. No code was touched. Panel/, Enumerations/ and the already-terse System headers needed little or nothing - PooledGate, TradeChecks, BinomialStats and Random came through with no blocks over the threshold at all. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-22 00:25:52 -04:00
//--- Says WHICH case this is rather than asserting the resumed one. A diagnostic that
//--- misreports its own trigger is worse than one that says nothing, because it gets quoted
//--- back as evidence.
fix: MI diagnostics never ran on a resumed model - the stated intent was never achieved A comment above the diagnostic branch says it "runs even when the sweep does not: on a resumed model ... tying it to that gate meant the only way to see the answer on a running model was to delete the model." It does not. Moving the diagnostic out of the tuner's gate left it behind m_labelCachePrebuilt, which has the same effect: the eager label pre-scan runs only on a FRESH start, because a net loaded from disk labels lazily per bar. So on a resumed model the flag is false forever and the whole MI block - headline, positive control, alignment scan, lag profile, geometry scan, winner test, and the auto-tune line - silently never runs. Measured on SP500 H1 2026-08-07: attached at era 271, still nothing by era 314, zero MI lines in the day's log, and the only "label cache pre-built" entry predates the attach. It also explains the shape of every capture on 08-05/06: each one came directly after a weights reset. The situation the comment was written to eliminate is exactly the situation that persisted. So drive the pre-scan when it is the only thing missing. Safe on a trained net: its one fresh-net side effect, pushing the output-layer bias toward the dominant class, is already gated on m_eraCount == 0, and the advance gate in Train() sits ABOVE if(!m_trainRunActive), so the era loop keeps its state - training pauses for the scan (~1s at 38k bars) and continues from where it was, not from 0. Announced only on a start that actually armed, since StartLabelCachePrebuild() returns unarmed when history is not ready and is retried per bar event. NOT sampled from the lazily-filled cache instead: BuildMiSample skips bars with no cached label, so that would score whichever subset training happened to have visited - a biased subsample presented as a measurement, which is the failure this diagnostic exists to catch. Also corrects a claim in 0d58923's comment. It argued four consecutive "no improvement" runs were ~1-in-100,000 evidence the indicator tuner is inert, by multiplying 5.6% across four runs. They are not independent trials: the MI scorer is deterministic and all four covered nearly the same bars, so an incumbent that is the maximum on this data is the maximum on every run. One ~1-in-18 observation with three correlated repeats, ~5.6% - unremarkable. The same independence assumption that made the uncorrected lag profile star four lags. The candidate-spread line stands: it settles inert-vs-live directly. No input, topology or label change: no retrain. Training in flight stays valid. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-07 09:08:01 -04:00
if(m_labelPrebuildActive)
feat: derive the ATR multiples from measured excursions - no hardcoded geometry The barrier was still two constants. SL_Mode/TP_Mode left the Inputs tab in 3482b6c, but the fallback was a hardcoded 2:6 and the geometry scan only ever chose from a hardcoded grid {2,3} x {2,3,4,6,8,10}. Picking the least-bad of eleven guesses is not deriving anything. WHY THE SCAN WAS THE WRONG INSTRUMENT, now measurable rather than argued. It ranks pairings by how predictable their OUTCOME is - a question about direction. The excursion test (2c78f3b) ran on SP500 H1 and direction is the one thing absent: ASYMMETRY p=0.0846, against RANGE/UP/DOWN all at p=0.0050, with RANGE scoring 0.01345 vs a 0.00343 null - 4x, where the barrier label sits at 1.01x. Hence the scan failing its own gate on every run, and its "winner" wandering 2:8 -> 3:8 -> 2:8 -> 2:4 across four runs of the same data. Excursion SIZE is strongly measurable, so derive the geometry from that instead. stop = q25 of measured ADVERSE travel (ordinary noise does not reach it) target = q50 of measured FAVOURABLE travel (reached ~half the time, by construction, inside the horizon) Continuous, in ATR units, superseding the enum multiples. Reachability ("target on X% of bars, stop on Y%") and the implied break-even are printed so the choice is auditable rather than trusted. FIXED-POINT ITERATION, not one-shot. ComputeBarrierHorizonBars scales the horizon with the target (first-passage time grows with the band) and the excursions are measured OVER the horizon, so target -> horizon -> excursions -> target is a real loop - deriving once sizes the target from travel measured under the PREVIOUS horizon. Re-measures until the multiples move <5%, capped at 3 passes, and says so if it does not settle. Does NOT create expectancy, and the log says as much: chance precision equals break-even at every geometry (m/(m+k) on both sides). It buys a target the market reaches and a stop that survives noise. Where Min_Risk_Reward_Ratio forces a target the market rarely reaches, it WARNS rather than overriding - the ratio is the user's risk policy, so the honest move is to state its cost. That is the collision that once rejected 100% of setups. Pinned in the .cfg as doubles appended AFTER this morning's two ints, so .cfg files written earlier today still load (their length guard finds no doubles) and a model that carries them was trained on them and never re-derives. Also fixes a message from e5ceed6 that claimed "this model resumed from disk" unconditionally - it printed above a "seeding era 0" line on a brand-new model, because the branch fires whenever the cache is not built, which is equally true before a fresh model's first prebuild. A diagnostic that misreports its own trigger is worse than one that says nothing: it gets quoted back as evidence. FORCES A FULL RETRAIN (labels change). Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-07 12:06:25 -04:00
Print(ID + (m_modelLoadedFromDisk
? ": MI diagnostics need a complete label cache and this model resumed from disk "
"(labels are filled lazily, so the cache covers only the bars training has "
"visited) - running the one-time pre-scan now, then the report. Training resumes "
"where it left off."
: ": MI diagnostics need a complete label cache and this model has not built one yet "
"- running the pre-scan now, then the report."));
fix: MI diagnostics never ran on a resumed model - the stated intent was never achieved A comment above the diagnostic branch says it "runs even when the sweep does not: on a resumed model ... tying it to that gate meant the only way to see the answer on a running model was to delete the model." It does not. Moving the diagnostic out of the tuner's gate left it behind m_labelCachePrebuilt, which has the same effect: the eager label pre-scan runs only on a FRESH start, because a net loaded from disk labels lazily per bar. So on a resumed model the flag is false forever and the whole MI block - headline, positive control, alignment scan, lag profile, geometry scan, winner test, and the auto-tune line - silently never runs. Measured on SP500 H1 2026-08-07: attached at era 271, still nothing by era 314, zero MI lines in the day's log, and the only "label cache pre-built" entry predates the attach. It also explains the shape of every capture on 08-05/06: each one came directly after a weights reset. The situation the comment was written to eliminate is exactly the situation that persisted. So drive the pre-scan when it is the only thing missing. Safe on a trained net: its one fresh-net side effect, pushing the output-layer bias toward the dominant class, is already gated on m_eraCount == 0, and the advance gate in Train() sits ABOVE if(!m_trainRunActive), so the era loop keeps its state - training pauses for the scan (~1s at 38k bars) and continues from where it was, not from 0. Announced only on a start that actually armed, since StartLabelCachePrebuild() returns unarmed when history is not ready and is retried per bar event. NOT sampled from the lazily-filled cache instead: BuildMiSample skips bars with no cached label, so that would score whichever subset training happened to have visited - a biased subsample presented as a measurement, which is the failure this diagnostic exists to catch. Also corrects a claim in 0d58923's comment. It argued four consecutive "no improvement" runs were ~1-in-100,000 evidence the indicator tuner is inert, by multiplying 5.6% across four runs. They are not independent trials: the MI scorer is deterministic and all four covered nearly the same bars, so an incumbent that is the maximum on this data is the maximum on every run. One ~1-in-18 observation with three correlated repeats, ~5.6% - unremarkable. The same independence assumption that made the uncorrected lag profile star four lags. The candidate-spread line stands: it settles inert-vs-live directly. No input, topology or label change: no retrain. Training in flight stays valid. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-07 09:08:01 -04:00
}
diag(autotune): stop making the feature test cost a trained model The permutation test lived inside TuneIndicatorsByFilter, which is gated on era 0 - correctly, because re-running the SWEEP would change the input vector out from under weights already fitted to the old one. But the test itself reads cached features and writes nothing, so none of that applies to it, and the gate meant the only way to see the answer on a running model was to delete the model. Today that price was PAI's 45 trained eras and CONV's 31, spent to re-ask a read-only question. Split into ReportFeatureLabelInformation(), called from the sweep when it runs and directly when it does not - a resumed model, a disabled tuner, nothing tunable. Once per attach either way. Compiles 0 errors / 0 warnings. Build tag permtest-v2. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-01 14:01:32 -04:00
else if(!m_miReportDone && m_labelCachePrebuilt)
Add new research scripts for trading strategy analysis - Implemented sqx_audit.py to audit StrategyQuant X trade lists, focusing on performance metrics and cost analysis. - Created sqx_portfolio.py to evaluate portfolio performance based on uncorrelated components and their impact on risk and return. - Developed swing.py to analyze cost ratios across different holding periods and assess swing trading structures. - Introduced test_management.py to investigate the effectiveness of exit rules on random entries and their impact on expectancy.
2026-08-02 12:25:20 -04:00
{
//--- WAIT FOR THE CROSS-ASSET PANEL. It is part of the feature vector but it is built inside
refactor(comments): stdlib comment style across the remaining in-scope files Same pass as 0b06f8e, applied file by file: comment runs of 4+ lines compressed to their leading topic sentences, capped at 4 lines, whole sentences only. Warning sentences (NEVER / MUST / trap / would-have) survive the budget. Every file was checked the same way before committing: the list of non-comment lines is byte-identical to HEAD, and braces balance. No code was touched. Panel/, Enumerations/ and the already-terse System headers needed little or nothing - PooledGate, TradeChecks, BinomialStats and Random came through with no blocks over the threshold at all. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-22 00:25:52 -04:00
//--- Train(), so on a fresh run this diagnostic would otherwise describe a NARROWER vector
//--- than the one training goes on to use.
fix(ensemble): shared MI diagnostics + divided chunk budget - warm-up and panel responsiveness Two user-reported ensemble regressions, one cause each: - "getting ready is very long": every member ran the full MI diagnostic suite (headline MI, positive control, alignment, lag profile, geometry scan + winner test - ~200 permuted draws per line) on IDENTICAL features and labels, reporting the same numbers four times. First member runs it, the rest adopt with one log line. Documented caveat: if the geometry scan ever ADOPTS a winner under its gate (it never has), the adoption becomes donor-only and the gate must be revisited. - "panel not responsive": four members chunks queue back-to-back on the one chart thread - 4 x 120ms = 480ms worst-case click latency, the exact regime the 200ms note in Training.mqh already documents as broken. Ensemble members now use a 30ms chunk budget, restoring solo UI latency at slightly higher dispatch overhead. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-15 17:37:10 -04:00
if(m_ensembleMember && g_ensembleChartMiReportDone)
{
//--- see g_ensembleChartMiReportDone at the top of this file
m_miReportDone = true;
fix(handles): the MA handle was SHARED, and a rejected sweep freed it for everyone else ROOT CAUSE of the six-session "silent block failure", measured rather than inferred. All TWELVE dead-handle recoveries in today's log report the SAME handle number - MA=-1(h13) - across two charts and all four members. It was never four handles. It was one. MT5 refcounts indicator requests, so four ensemble members asking for the same iMA on the same symbol/period share a single handle. TuneIndicatorsByFilter creates and drops ~35 of them scoring candidates; the sweep's runner ends up holding a live handle while its siblings still hold a number the terminal has already freed. Timeline, twice, to the millisecond: USDJPY 18:10:03 PAI: auto-tune complete 18:10:29.864/.910/.953 CONV/LSTM/HYB: "already ran ... REJECTED" 18:10:30.057/.065/.074 all three: MA=-1(h13), sweep bars all rejected XAUUSD 18:10:11 -> 18:10:42.19/.23/.27 -> 18:10:42.334 identical, same ~100ms The adopt branch re-initialised indicators only `if(g_ensembleChartTuneInstalled)` - exactly backwards. A REJECTED sweep churns just as many handles, and every one of the twelve recoveries followed a rejection. Parameters are still adopted only on an install; the HANDLES are now rebuilt either way. Four creations per chart. RepairDeadIndicatorHandles stays - it is cause-agnostic and it is what made this diagnosable. This removes the cause it was recovering from. Also, consistency of the warm-up status (user-reported: "only one nn will say scoring indicators, which leaves some doubt about what is going on"): - the sweeping member now says it is scoring "for the whole chart", so three idle rows read as the design rather than a stall; - the two adopt branches (tuner and MI) publish to the panel instead of only printing, so every row accounts for itself; - the MI suite publishes before it runs. It is the longest stretch of the whole warm-up - MI, lag profile, excursion targets, geometry scan, each with its own permutation null - and it published nothing at all, so during most of the warm-up the panel's last word described a step that had already finished. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-17 18:16:28 -04:00
//--- Same reasoning as the tuner's adopt branch above: published, not just printed, so the row
//--- says why it is not repeating the measurement.
PublishStatus(ID + " : reusing the chart's information report...");
fix(ensemble): shared MI diagnostics + divided chunk budget - warm-up and panel responsiveness Two user-reported ensemble regressions, one cause each: - "getting ready is very long": every member ran the full MI diagnostic suite (headline MI, positive control, alignment, lag profile, geometry scan + winner test - ~200 permuted draws per line) on IDENTICAL features and labels, reporting the same numbers four times. First member runs it, the rest adopt with one log line. Documented caveat: if the geometry scan ever ADOPTS a winner under its gate (it never has), the adoption becomes donor-only and the gate must be revisited. - "panel not responsive": four members chunks queue back-to-back on the one chart thread - 4 x 120ms = 480ms worst-case click latency, the exact regime the 200ms note in Training.mqh already documents as broken. Ensemble members now use a 30ms chunk budget, restoring solo UI latency at slightly higher dispatch overhead. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-15 17:37:10 -04:00
Print(ID + ": MI diagnostics already measured by another ensemble member on this chart - "
"same features, same labels, same answer. Skipped (saves the slowest part of the "
"ensemble's warm-up; the first member's report above is this model's too).");
refactor(comments): stdlib comment style across the remaining in-scope files Same pass as 0b06f8e, applied file by file: comment runs of 4+ lines compressed to their leading topic sentences, capped at 4 lines, whole sentences only. Warning sentences (NEVER / MUST / trap / would-have) survive the budget. Every file was checked the same way before committing: the list of non-comment lines is byte-identical to HEAD, and braces balance. No code was touched. Panel/, Enumerations/ and the already-terse System headers needed little or nothing - PooledGate, TradeChecks, BinomialStats and Random came through with no blocks over the threshold at all. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-22 00:25:52 -04:00
//--- ...BUT THE GEOMETRY IS NOT A REPORT, IT IS A DECISION, and skipping the chain that
//--- makes it is not the same as declining it.
fix(geometry): the ensemble was training on TWO DIFFERENT TARGETS - propagate the adopted barrier MEASURED 2026-08-17 19:06 on USDJPY, in the fresh run: 19:06:38 LSTM adopting barrier geometry 2:10 ... geometry authority 19:06:40 LSTM triple-barrier labels - stop 2.00 target 10.00, horizon 256 19:06:44 PAI / CONV / HYB break-even 33.3%, mean label lifespan 19.2 bars 19:06:45 LSTM break-even 16.7%, mean label lifespan 81.4 bars One chart, four members, two targets. A "Buy" from LSTM meant "10 ATR before a 2 ATR stop within 256 bars"; a "Buy" from PAI meant "3.21 before 1.61 within 64". The orchestrator averages those votes and the joint gate certifies the average as though they answered one question. And g_DerivedSlAtrMult - which places the LIVE order - is a single global, so the stop actually sent was whichever member wrote last: the same last-writer-wins class of bug as the live-exit confidence. CAUSE, and it is mine. The geometry scan sits at the end of the MI chain, and that chain runs ONCE PER CHART (g_ensembleChartMiReportDone) - whichever member reaches it first measures and the rest skip. Harmless while the scan only PRINTED; 62a719f made it authoritative and turned a skipped report into a skipped DECISION. The indicator tuner already had this doctrine (g_ensembleChartTuneSettings); the geometry had no equivalent. - g_ensembleChartGeomAdopted/Sl/Tp/SlMode/TpMode: the donor publishes its pairing, the siblings adopt it in the MI-skip branch. Ordering is safe by construction - MQL5 is single-threaded per chart and the donor sets g_ensembleChartMiReportDone only after the chain (and so the adoption) returns, so any member taking the skip branch does so strictly afterwards. - ApplyAdoptedGeometry(): the eleven side effects an adopted pairing must carry - derived pair, legacy mode ints, g_Derived* live globals, .cfg rewrite, label cache invalidation, horizon unlatch - in ONE function, because there are now two callers and duplicating them is how the two paths drift. - Guarded on era 0 for the donor's own reason: relabelling a partly trained net moves the target out from under weights already fitted to the old one. STILL OPEN: dead MA handles were not eliminated by cb30360. They now appear at a different site (SP500 19:06:35, during "label prebuild", and on PAI - the member that RAN the sweep), so there is a second handle-churn path I have not found. Recovery works and the sharing diagnosis stands; the trigger is not only the tuner's adopt branch. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-17 19:35:45 -04:00
if(g_ensembleChartGeomAdopted && m_eraCount == 0 && g_ensembleChartGeomSl > 0.0
&& g_ensembleChartGeomTp > 0.0
&& (m_derivedSlMult != g_ensembleChartGeomSl || m_derivedTpMult != g_ensembleChartGeomTp))
{
PrintFormat("%s: adopting the barrier geometry the chart's scan chose - %.2f*ATR / %.2f*ATR."
" This member never ran the scan (the MI chain runs once per chart), and keeping"
" its own derived pair would put this ensemble's members on DIFFERENT targets"
" while the orchestrator averages their votes as one.",
ID, g_ensembleChartGeomSl, g_ensembleChartGeomTp);
ApplyAdoptedGeometry(g_ensembleChartGeomSl, g_ensembleChartGeomTp,
g_ensembleChartGeomSlMode, g_ensembleChartGeomTpMode);
}
fix(ensemble): shared MI diagnostics + divided chunk budget - warm-up and panel responsiveness Two user-reported ensemble regressions, one cause each: - "getting ready is very long": every member ran the full MI diagnostic suite (headline MI, positive control, alignment, lag profile, geometry scan + winner test - ~200 permuted draws per line) on IDENTICAL features and labels, reporting the same numbers four times. First member runs it, the rest adopt with one log line. Documented caveat: if the geometry scan ever ADOPTS a winner under its gate (it never has), the adoption becomes donor-only and the gate must be revisited. - "panel not responsive": four members chunks queue back-to-back on the one chart thread - 4 x 120ms = 480ms worst-case click latency, the exact regime the 200ms note in Training.mqh already documents as broken. Ensemble members now use a 30ms chunk budget, restoring solo UI latency at slightly higher dispatch overhead. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-15 17:37:10 -04:00
}
Add new research scripts for trading strategy analysis - Implemented sqx_audit.py to audit StrategyQuant X trade lists, focusing on performance metrics and cost analysis. - Created sqx_portfolio.py to evaluate portfolio performance based on uncorrelated components and their impact on risk and return. - Developed swing.py to analyze cost ratios across different holding periods and assess swing trading structures. - Introduced test_management.py to investigate the effectiveness of exit rules on random entries and their impact on expectancy.
2026-08-02 12:25:20 -04:00
else
fix(ensemble): shared MI diagnostics + divided chunk budget - warm-up and panel responsiveness Two user-reported ensemble regressions, one cause each: - "getting ready is very long": every member ran the full MI diagnostic suite (headline MI, positive control, alignment, lag profile, geometry scan + winner test - ~200 permuted draws per line) on IDENTICAL features and labels, reporting the same numbers four times. First member runs it, the rest adopt with one log line. Documented caveat: if the geometry scan ever ADOPTS a winner under its gate (it never has), the adoption becomes donor-only and the gate must be revisited. - "panel not responsive": four members chunks queue back-to-back on the one chart thread - 4 x 120ms = 480ms worst-case click latency, the exact regime the 200ms note in Training.mqh already documents as broken. Ensemble members now use a 30ms chunk budget, restoring solo UI latency at slightly higher dispatch overhead. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-15 17:37:10 -04:00
if(m_crossAsset.IsReady() || m_miReportDeferrals >= MI_REPORT_MAX_DEFERRALS)
{
refactor(comments): stdlib comment style across the remaining in-scope files Same pass as 0b06f8e, applied file by file: comment runs of 4+ lines compressed to their leading topic sentences, capped at 4 lines, whole sentences only. Warning sentences (NEVER / MUST / trap / would-have) survive the budget. Every file was checked the same way before committing: the list of non-comment lines is byte-identical to HEAD, and braces balance. No code was touched. Panel/, Enumerations/ and the already-terse System headers needed little or nothing - PooledGate, TradeChecks, BinomialStats and Random came through with no blocks over the threshold at all. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-22 00:25:52 -04:00
//--- THE LONGEST SINGLE STRETCH OF THE WARM-UP - the MI suite, the lag profile, the
//--- excursion targets and the geometry scan, each with its own few-hundred-draw
//--- permutation null - and until now it published NOTHING.
fix(handles): the MA handle was SHARED, and a rejected sweep freed it for everyone else ROOT CAUSE of the six-session "silent block failure", measured rather than inferred. All TWELVE dead-handle recoveries in today's log report the SAME handle number - MA=-1(h13) - across two charts and all four members. It was never four handles. It was one. MT5 refcounts indicator requests, so four ensemble members asking for the same iMA on the same symbol/period share a single handle. TuneIndicatorsByFilter creates and drops ~35 of them scoring candidates; the sweep's runner ends up holding a live handle while its siblings still hold a number the terminal has already freed. Timeline, twice, to the millisecond: USDJPY 18:10:03 PAI: auto-tune complete 18:10:29.864/.910/.953 CONV/LSTM/HYB: "already ran ... REJECTED" 18:10:30.057/.065/.074 all three: MA=-1(h13), sweep bars all rejected XAUUSD 18:10:11 -> 18:10:42.19/.23/.27 -> 18:10:42.334 identical, same ~100ms The adopt branch re-initialised indicators only `if(g_ensembleChartTuneInstalled)` - exactly backwards. A REJECTED sweep churns just as many handles, and every one of the twelve recoveries followed a rejection. Parameters are still adopted only on an install; the HANDLES are now rebuilt either way. Four creations per chart. RepairDeadIndicatorHandles stays - it is cause-agnostic and it is what made this diagnosable. This removes the cause it was recovering from. Also, consistency of the warm-up status (user-reported: "only one nn will say scoring indicators, which leaves some doubt about what is going on"): - the sweeping member now says it is scoring "for the whole chart", so three idle rows read as the design rather than a stall; - the two adopt branches (tuner and MI) publish to the panel instead of only printing, so every row accounts for itself; - the MI suite publishes before it runs. It is the longest stretch of the whole warm-up - MI, lag profile, excursion targets, geometry scan, each with its own permutation null - and it published nothing at all, so during most of the warm-up the panel's last word described a step that had already finished. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-17 18:16:28 -04:00
PublishStatus(ID + (m_ensembleMember
? " : measuring feature/label information for the whole chart..."
: " : measuring feature/label information..."));
fix(ensemble): shared MI diagnostics + divided chunk budget - warm-up and panel responsiveness Two user-reported ensemble regressions, one cause each: - "getting ready is very long": every member ran the full MI diagnostic suite (headline MI, positive control, alignment, lag profile, geometry scan + winner test - ~200 permuted draws per line) on IDENTICAL features and labels, reporting the same numbers four times. First member runs it, the rest adopt with one log line. Documented caveat: if the geometry scan ever ADOPTS a winner under its gate (it never has), the adoption becomes donor-only and the gate must be revisited. - "panel not responsive": four members chunks queue back-to-back on the one chart thread - 4 x 120ms = 480ms worst-case click latency, the exact regime the 200ms note in Training.mqh already documents as broken. Ensemble members now use a 30ms chunk budget, restoring solo UI latency at slightly higher dispatch overhead. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-15 17:37:10 -04:00
ReportFeatureLabelInformation();
if(m_ensembleMember && m_miReportDone)
g_ensembleChartMiReportDone = true;
}
else
m_miReportDeferrals++;
Add new research scripts for trading strategy analysis - Implemented sqx_audit.py to audit StrategyQuant X trade lists, focusing on performance metrics and cost analysis. - Created sqx_portfolio.py to evaluate portfolio performance based on uncorrelated components and their impact on risk and return. - Developed swing.py to analyze cost ratios across different holding periods and assess swing trading structures. - Introduced test_management.py to investigate the effectiveness of exit rules on random entries and their impact on expectancy.
2026-08-02 12:25:20 -04:00
}
perf(autotune): replace the genetic search with a filter score - hours to seconds MEASURED COST OF THE GA, which is what retired it. Per generation: rung 0: 8 cand x 3 seeds x 3 eras = 72 eras rung 1: 4 cand x 3 seeds x 8 eras = 96 rung 2: 2 cand x 3 seeds x 20 eras = 120 = 288 eras/generation x 4 generations = 1152 eras BEFORE the winner's real training began. Against the observed era times on SP500 H1: PAI 29.1 s/era -> 9.3 h (matches the observed 00:37 -> 09:22) CONV 41.3 s/era -> 13.2 h LSTM 150.4 s/era -> 48.1 h HYBRID 154.6 s/era -> 49.5 h Two days to tune is not a first-run experience, and it is the phase in which the panel goes quiet, which is what made it look like a hang. It also bought nothing. The space is 90 points (10 MA periods x 9 MA types), so 1152 evaluations revisited each point ~13 times; and rungs of 3 and 8 eras cannot separate two MA periods at all. The 2026-08-01 run proves it: every finalist scored 25.0-25.9% balanced accuracy - below the 33.3% one-class floor, i.e. indistinguishable noise - and the search then "deployed the winner" of that. THE ERROR WAS THE SCORING FUNCTION, not its constants. Using a full training run to choose a feature's period is a wrapper method paying wrapper prices for a decision that does not need one. The reference book does not do this: ch. 3.3 selects inputs by measuring each candidate indicator's CORRELATION with the target and dropping the ones with none, with no network involved. So: rank candidates by the MUTUAL INFORMATION between the resulting feature vector and the triple-barrier label. MI rather than correlation because the label is 3-class categorical and the features are not monotonically related to it. Equal-FREQUENCY binning (rank-based), because these features are ATR-normalised and heavy-tailed - fixed-width bins put nearly everything in one bucket and report ~0 information for a genuinely useful feature. Scoring is arithmetic over the feature cache, so it costs seconds and its cost is independent of topology: LSTM now tunes as fast as the MLP. Coordinate sweep, not product sweep - cost is the SUM of per-parameter candidate counts, so enabling every indicator stays affordable - with a second pass that breaks early once nothing moves. Sampling is IS-ONLY. Letting the OOS window influence which indicator settings ship would mean the holdout had been used for selection and had stopped being a holdout. HONEST LIMIT, recorded because it is the price: MI is marginal, so a parameter that only pays off in combination with another can be missed (Guyon & Elisseeff 2003, filter vs wrapper). Given the wrapper it replaces was ranking pure noise at 48 h a run, this is strictly better. Deleted with it: GaRungEras/GaExtract/GaStore/GaMutate/GaRandomCandidate/ GaBlockCrossover/GaSortAliveByScoreDesc/GaBreedNextGeneration, 14 m_ga* members, the GA_*/TUNE_POP_* constants, and ComputeTuneTrialBudget. AND m_evalMode/m_evalEraBudget, because nothing set them any more - 28 read sites all permanently inert. That is not a tidy-up: the `if (!m_evalMode)` guard on UpdateClassPriors is exactly what silently disabled the imbalance correction for entire runs two commits ago. Dead machinery that still reads like live machinery is this codebase's most expensive recurring bug, and leaving 28 more instances of it would have been indefensible. The panel's tuning-progress state goes too - tuning no longer takes long enough to need one. Both builds compile 0 errors / 0 warnings. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-01 10:29:31 -04:00
Train(StartTrainBar);
refactor: split CExpertSignalAIBase implementation by responsibility ExpertSignalAIBase.mqh was 8216 lines: the class declaration followed by 87 method bodies covering training, labelling, feature extraction, persistence, chart drawing, online learning, the GA auto-tuner and inference, all in one file. Train() alone is 1492 lines; a change to arrow drawing meant scrolling past the era loop. Moved the bodies into Expert\AIBase\, included at the bottom of the original after the class declaration: Training.mqh 1607 era loop, plateau ladder, checkpoint select, deploy Features.mqh 1093 indicator creation + per-bar input feature vector ChartUI.mqh 634 arrows, arrow persistence, status panel, cleanup Persistence.mqh 492 .stats/.cfg sidecars, CPU-inference validation, copy OnlineLearning.mqh 461 live continual learning, EMA shadow, OOS simulator Labels.mqh 309 ZigZag pivot labels, async label-cache prebuild AutoTune.mqh 275 genetic tuner (population, crossover, halving) Inference.mqh 235 softmax, prior calibration, class priors ExpertSignalAIBase.mqh 8216 -> 3131 (declaration + topology build only) This is a pure relocation - verified mechanically, not by eye: HEAD's file reconstructed from the eight partials plus the surviving remainder is byte-identical to HEAD, span for span (scratchpad verify_split.py). No declaration moved, no signature changed, no code rewritten, so behaviour is unchanged by construction. Compiles 0 errors, 0 warnings. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-07-29 00:42:45 -04:00
}
#endif // WARRIOR_AIBASE_AUTOTUNE_MQH
Copy permalink